2024-09-16T00:11:48.340 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-16T00:11:48.366 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T00:11:48.455 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906392 branch: squid description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/direct_io}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7906392' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules name: teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 mon warn on pool no app: false osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request sha1: 8921c8ef947807b717db041d85f0761b7914d5fc subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt dry_run: true install: ceph: flavor: default sha1: 8921c8ef947807b717db041d85f0761b7914d5fc kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: ad35db06e8924912f01e32b35576297c0f1352b4 owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 3393 sha1: 8921c8ef947807b717db041d85f0761b7914d5fc sleep_before_teardown: 0 subset: 294/512 suite: fs suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: ad35db06e8924912f01e32b35576297c0f1352b4 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi038.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE0Qsbs3puj5B/ijHDgr5OMChtKtYyN04nx26iYX/JenSYmA8Va53I9lfEDN5CMWwQjdsBAnHy7NnuQ6kMiBDLM= smithi073.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCJqT2Gr38GRJAqpJ178sadxM+QS7RSQTkzAeCaTODqKqC4j8ImWMHlomYTNt6bJ5ePy7uqYuitBh2R8/L64Mz0= smithi122.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB6ftzdiCVttC6kXsw0CQqCp2bRs5ZvXdoIzfjoIgyfwD6K8fWL56yH1owUektAnCWJRvFyzkpATjKDoPj8+amc= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - quiescer: null - workunit: clients: all: - direct_io teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-15_21:24:03 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-16T00:11:48.456 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa; will attempt to use it 2024-09-16T00:11:48.456 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks 2024-09-16T00:11:48.456 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-16T00:11:48.458 INFO:teuthology.task.internal:Checking packages... 2024-09-16T00:11:48.482 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '8921c8ef947807b717db041d85f0761b7914d5fc' 2024-09-16T00:11:48.483 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-16T00:11:48.483 INFO:teuthology.packaging:ref: None 2024-09-16T00:11:48.483 INFO:teuthology.packaging:tag: None 2024-09-16T00:11:48.483 INFO:teuthology.packaging:branch: squid 2024-09-16T00:11:48.483 INFO:teuthology.packaging:sha1: 8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:11:48.483 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-09-16T00:11:48.689 INFO:teuthology.task.internal:Found packages for ceph version 19.1.1-328.gad35db06 2024-09-16T00:11:48.690 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-16T00:11:48.700 INFO:teuthology.task.internal:no buildpackages task found 2024-09-16T00:11:48.700 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-16T00:11:48.847 INFO:teuthology.task.internal:Saving configuration 2024-09-16T00:11:48.869 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-16T00:11:48.880 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-16T00:11:48.905 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi038.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906392', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 00:06:44.475866', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBE0Qsbs3puj5B/ijHDgr5OMChtKtYyN04nx26iYX/JenSYmA8Va53I9lfEDN5CMWwQjdsBAnHy7NnuQ6kMiBDLM='} 2024-09-16T00:11:48.927 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi073.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906392', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 00:06:44.477072', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCJqT2Gr38GRJAqpJ178sadxM+QS7RSQTkzAeCaTODqKqC4j8ImWMHlomYTNt6bJ5ePy7uqYuitBh2R8/L64Mz0='} 2024-09-16T00:11:48.946 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi122.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906392', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-16 00:06:44.474491', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB6ftzdiCVttC6kXsw0CQqCp2bRs5ZvXdoIzfjoIgyfwD6K8fWL56yH1owUektAnCWJRvFyzkpATjKDoPj8+amc='} 2024-09-16T00:11:48.947 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-16T00:11:48.957 INFO:teuthology.task.internal:roles: ubuntu@smithi038.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-16T00:11:48.957 INFO:teuthology.task.internal:roles: ubuntu@smithi073.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-16T00:11:48.957 INFO:teuthology.task.internal:roles: ubuntu@smithi122.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-16T00:11:48.957 INFO:teuthology.run_tasks:Running task console_log... 2024-09-16T00:11:49.205 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f022ee4bbe0>, signals=[15]) 2024-09-16T00:11:49.206 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-16T00:11:49.213 INFO:teuthology.task.internal:Opening connections... 2024-09-16T00:11:49.213 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi038.front.sepia.ceph.com 2024-09-16T00:11:49.215 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:11:49.296 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi073.front.sepia.ceph.com 2024-09-16T00:11:49.296 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:11:49.374 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi122.front.sepia.ceph.com 2024-09-16T00:11:49.375 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:11:49.449 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-16T00:11:49.454 DEBUG:teuthology.orchestra.run.smithi038:> uname -m 2024-09-16T00:11:49.473 INFO:teuthology.orchestra.run.smithi038.stdout:x86_64 2024-09-16T00:11:49.473 DEBUG:teuthology.orchestra.run.smithi038:> cat /etc/os-release 2024-09-16T00:11:49.531 INFO:teuthology.orchestra.run.smithi038.stdout:NAME="CentOS Stream" 2024-09-16T00:11:49.531 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION="9" 2024-09-16T00:11:49.531 INFO:teuthology.orchestra.run.smithi038.stdout:ID="centos" 2024-09-16T00:11:49.532 INFO:teuthology.orchestra.run.smithi038.stdout:ID_LIKE="rhel fedora" 2024-09-16T00:11:49.532 INFO:teuthology.orchestra.run.smithi038.stdout:VERSION_ID="9" 2024-09-16T00:11:49.532 INFO:teuthology.orchestra.run.smithi038.stdout:PLATFORM_ID="platform:el9" 2024-09-16T00:11:49.532 INFO:teuthology.orchestra.run.smithi038.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T00:11:49.532 INFO:teuthology.orchestra.run.smithi038.stdout:ANSI_COLOR="0;31" 2024-09-16T00:11:49.532 INFO:teuthology.orchestra.run.smithi038.stdout:LOGO="fedora-logo-icon" 2024-09-16T00:11:49.532 INFO:teuthology.orchestra.run.smithi038.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T00:11:49.532 INFO:teuthology.orchestra.run.smithi038.stdout:HOME_URL="https://centos.org/" 2024-09-16T00:11:49.532 INFO:teuthology.orchestra.run.smithi038.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T00:11:49.532 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T00:11:49.532 INFO:teuthology.orchestra.run.smithi038.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T00:11:49.533 INFO:teuthology.lock.ops:Updating smithi038.front.sepia.ceph.com on lock server 2024-09-16T00:11:49.558 DEBUG:teuthology.orchestra.run.smithi073:> uname -m 2024-09-16T00:11:49.577 INFO:teuthology.orchestra.run.smithi073.stdout:x86_64 2024-09-16T00:11:49.577 DEBUG:teuthology.orchestra.run.smithi073:> cat /etc/os-release 2024-09-16T00:11:49.634 INFO:teuthology.orchestra.run.smithi073.stdout:NAME="CentOS Stream" 2024-09-16T00:11:49.635 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION="9" 2024-09-16T00:11:49.635 INFO:teuthology.orchestra.run.smithi073.stdout:ID="centos" 2024-09-16T00:11:49.635 INFO:teuthology.orchestra.run.smithi073.stdout:ID_LIKE="rhel fedora" 2024-09-16T00:11:49.635 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION_ID="9" 2024-09-16T00:11:49.635 INFO:teuthology.orchestra.run.smithi073.stdout:PLATFORM_ID="platform:el9" 2024-09-16T00:11:49.635 INFO:teuthology.orchestra.run.smithi073.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T00:11:49.635 INFO:teuthology.orchestra.run.smithi073.stdout:ANSI_COLOR="0;31" 2024-09-16T00:11:49.635 INFO:teuthology.orchestra.run.smithi073.stdout:LOGO="fedora-logo-icon" 2024-09-16T00:11:49.635 INFO:teuthology.orchestra.run.smithi073.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T00:11:49.635 INFO:teuthology.orchestra.run.smithi073.stdout:HOME_URL="https://centos.org/" 2024-09-16T00:11:49.635 INFO:teuthology.orchestra.run.smithi073.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T00:11:49.635 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T00:11:49.635 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T00:11:49.636 INFO:teuthology.lock.ops:Updating smithi073.front.sepia.ceph.com on lock server 2024-09-16T00:11:49.657 DEBUG:teuthology.orchestra.run.smithi122:> uname -m 2024-09-16T00:11:49.675 INFO:teuthology.orchestra.run.smithi122.stdout:x86_64 2024-09-16T00:11:49.676 DEBUG:teuthology.orchestra.run.smithi122:> cat /etc/os-release 2024-09-16T00:11:49.733 INFO:teuthology.orchestra.run.smithi122.stdout:NAME="CentOS Stream" 2024-09-16T00:11:49.733 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION="9" 2024-09-16T00:11:49.734 INFO:teuthology.orchestra.run.smithi122.stdout:ID="centos" 2024-09-16T00:11:49.734 INFO:teuthology.orchestra.run.smithi122.stdout:ID_LIKE="rhel fedora" 2024-09-16T00:11:49.734 INFO:teuthology.orchestra.run.smithi122.stdout:VERSION_ID="9" 2024-09-16T00:11:49.734 INFO:teuthology.orchestra.run.smithi122.stdout:PLATFORM_ID="platform:el9" 2024-09-16T00:11:49.734 INFO:teuthology.orchestra.run.smithi122.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-16T00:11:49.734 INFO:teuthology.orchestra.run.smithi122.stdout:ANSI_COLOR="0;31" 2024-09-16T00:11:49.734 INFO:teuthology.orchestra.run.smithi122.stdout:LOGO="fedora-logo-icon" 2024-09-16T00:11:49.734 INFO:teuthology.orchestra.run.smithi122.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-16T00:11:49.734 INFO:teuthology.orchestra.run.smithi122.stdout:HOME_URL="https://centos.org/" 2024-09-16T00:11:49.734 INFO:teuthology.orchestra.run.smithi122.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-16T00:11:49.734 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-16T00:11:49.734 INFO:teuthology.orchestra.run.smithi122.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-16T00:11:49.735 INFO:teuthology.lock.ops:Updating smithi122.front.sepia.ceph.com on lock server 2024-09-16T00:11:49.757 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-16T00:11:49.770 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-16T00:11:49.778 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-16T00:11:49.779 DEBUG:teuthology.orchestra.run.smithi038:> test '!' -e /home/ubuntu/cephtest 2024-09-16T00:11:49.781 DEBUG:teuthology.orchestra.run.smithi073:> test '!' -e /home/ubuntu/cephtest 2024-09-16T00:11:49.784 DEBUG:teuthology.orchestra.run.smithi122:> test '!' -e /home/ubuntu/cephtest 2024-09-16T00:11:49.801 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-16T00:11:49.807 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-16T00:11:49.807 DEBUG:teuthology.orchestra.run.smithi038:> test -z $(ls -A /var/lib/ceph) 2024-09-16T00:11:49.839 DEBUG:teuthology.orchestra.run.smithi073:> test -z $(ls -A /var/lib/ceph) 2024-09-16T00:11:49.842 DEBUG:teuthology.orchestra.run.smithi122:> test -z $(ls -A /var/lib/ceph) 2024-09-16T00:11:49.864 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-16T00:11:49.937 INFO:teuthology.run_tasks:Running task kernel... 2024-09-16T00:11:49.953 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-16T00:11:49.953 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-16T00:11:49.953 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-16T00:11:49.954 DEBUG:teuthology.orchestra.run.smithi038:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T00:11:49.971 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:11:49.971 DEBUG:teuthology.orchestra.run.smithi038:> uname -r 2024-09-16T00:11:50.029 INFO:teuthology.orchestra.run.smithi038.stdout:5.14.0-503.el9.x86_64 2024-09-16T00:11:50.029 INFO:teuthology.task.kernel:Running kernel on smithi038: 5.14.0-503.el9.x86_64 2024-09-16T00:11:50.029 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum install -y kernel 2024-09-16T00:11:52.253 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.2 MB 00:00 2024-09-16T00:11:54.329 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - AppStream 22 MB/s | 20 MB 00:00 2024-09-16T00:12:00.055 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - CRB 3.7 MB/s | 6.5 MB 00:01 2024-09-16T00:12:01.614 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - Extras packages 70 kB/s | 19 kB 00:00 2024-09-16T00:12:02.635 INFO:teuthology.orchestra.run.smithi038.stdout:Extra Packages for Enterprise Linux 25 MB/s | 23 MB 00:00 2024-09-16T00:12:08.291 INFO:teuthology.orchestra.run.smithi038.stdout:lab-extras 9.2 kB/s | 1.7 kB 00:00 2024-09-16T00:12:10.365 INFO:teuthology.orchestra.run.smithi038.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-16T00:12:10.365 INFO:teuthology.orchestra.run.smithi038.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-16T00:12:10.457 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-09-16T00:12:10.473 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-09-16T00:12:10.473 INFO:teuthology.orchestra.run.smithi038.stdout: Package Architecture Version Repository Size 2024-09-16T00:12:10.473 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-09-16T00:12:10.473 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-09-16T00:12:10.473 INFO:teuthology.orchestra.run.smithi038.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-16T00:12:10.473 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2024-09-16T00:12:10.473 INFO:teuthology.orchestra.run.smithi038.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-16T00:12:10.473 INFO:teuthology.orchestra.run.smithi038.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-16T00:12:10.473 INFO:teuthology.orchestra.run.smithi038.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-16T00:12:10.473 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:12:10.474 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-09-16T00:12:10.474 INFO:teuthology.orchestra.run.smithi038.stdout:================================================================================ 2024-09-16T00:12:10.474 INFO:teuthology.orchestra.run.smithi038.stdout:Install 4 Packages 2024-09-16T00:12:10.474 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:12:10.474 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 79 M 2024-09-16T00:12:10.474 INFO:teuthology.orchestra.run.smithi038.stdout:Installed size: 126 M 2024-09-16T00:12:10.474 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-09-16T00:12:10.610 INFO:teuthology.orchestra.run.smithi038.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 200 kB/s | 23 kB 00:00 2024-09-16T00:12:11.118 INFO:teuthology.orchestra.run.smithi038.stdout:(2/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 55 MB/s | 35 MB 00:00 2024-09-16T00:12:11.360 INFO:teuthology.orchestra.run.smithi038.stdout:(3/4): kernel-core-5.14.0-505.el9.x86_64.rpm 18 MB/s | 16 MB 00:00 2024-09-16T00:12:11.777 INFO:teuthology.orchestra.run.smithi038.stdout:(4/4): kernel-modules-core-5.14.0-505.el9.x86_6 24 MB/s | 29 MB 00:01 2024-09-16T00:12:11.777 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:12:11.777 INFO:teuthology.orchestra.run.smithi038.stdout:Total 60 MB/s | 79 MB 00:01 2024-09-16T00:12:12.494 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-09-16T00:12:12.884 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-09-16T00:12:12.884 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-09-16T00:12:13.865 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-09-16T00:12:13.865 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-09-16T00:12:16.447 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-09-16T00:12:17.131 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-16T00:12:17.239 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T00:12:20.034 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T00:12:20.158 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T00:12:25.281 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T00:12:25.326 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T00:12:30.468 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T00:13:09.574 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T00:13:09.616 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-16T00:13:09.981 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-16T00:13:09.982 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-16T00:13:09.982 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-16T00:13:09.982 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-16T00:13:10.824 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-16T00:13:10.825 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:13:10.825 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-09-16T00:13:10.825 INFO:teuthology.orchestra.run.smithi038.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-16T00:13:10.825 INFO:teuthology.orchestra.run.smithi038.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-16T00:13:10.825 INFO:teuthology.orchestra.run.smithi038.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-16T00:13:10.825 INFO:teuthology.orchestra.run.smithi038.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-16T00:13:10.825 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:13:10.825 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-09-16T00:13:11.199 DEBUG:teuthology.orchestra.run.smithi038:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T00:13:11.283 INFO:teuthology.orchestra.run.smithi038.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T00:13:11.283 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T00:13:11.283 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-16T00:13:11.283 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-16T00:13:11.283 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-16T00:13:11.283 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-16T00:13:11.283 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi038.front.sepia.ceph.com, path=None, version=distro) 2024-09-16T00:13:11.284 DEBUG:teuthology.orchestra.run.smithi038:> rpm -q kernel | sort -rV | head -n 1 2024-09-16T00:13:11.342 INFO:teuthology.orchestra.run.smithi038.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-16T00:13:11.342 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-16T00:13:11.343 DEBUG:teuthology.orchestra.run.smithi038:> sudo rpm -qi grub2-tools 2024-09-16T00:13:11.381 INFO:teuthology.orchestra.run.smithi038.stdout:Name : grub2-tools 2024-09-16T00:13:11.381 INFO:teuthology.orchestra.run.smithi038.stdout:Epoch : 1 2024-09-16T00:13:11.382 INFO:teuthology.orchestra.run.smithi038.stdout:Version : 2.06 2024-09-16T00:13:11.382 INFO:teuthology.orchestra.run.smithi038.stdout:Release : 82.el9 2024-09-16T00:13:11.382 INFO:teuthology.orchestra.run.smithi038.stdout:Architecture: x86_64 2024-09-16T00:13:11.382 INFO:teuthology.orchestra.run.smithi038.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-16T00:13:11.382 INFO:teuthology.orchestra.run.smithi038.stdout:Group : Unspecified 2024-09-16T00:13:11.382 INFO:teuthology.orchestra.run.smithi038.stdout:Size : 8274670 2024-09-16T00:13:11.382 INFO:teuthology.orchestra.run.smithi038.stdout:License : GPLv3+ 2024-09-16T00:13:11.382 INFO:teuthology.orchestra.run.smithi038.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-16T00:13:11.382 INFO:teuthology.orchestra.run.smithi038.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-16T00:13:11.382 INFO:teuthology.orchestra.run.smithi038.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-16T00:13:11.382 INFO:teuthology.orchestra.run.smithi038.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-16T00:13:11.382 INFO:teuthology.orchestra.run.smithi038.stdout:Packager : builder@centos.org 2024-09-16T00:13:11.382 INFO:teuthology.orchestra.run.smithi038.stdout:Vendor : CentOS 2024-09-16T00:13:11.383 INFO:teuthology.orchestra.run.smithi038.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-16T00:13:11.383 INFO:teuthology.orchestra.run.smithi038.stdout:Summary : Support tools for GRUB. 2024-09-16T00:13:11.383 INFO:teuthology.orchestra.run.smithi038.stdout:Description : 2024-09-16T00:13:11.383 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:13:11.383 INFO:teuthology.orchestra.run.smithi038.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-16T00:13:11.383 INFO:teuthology.orchestra.run.smithi038.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-16T00:13:11.383 INFO:teuthology.orchestra.run.smithi038.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-16T00:13:11.383 INFO:teuthology.orchestra.run.smithi038.stdout:hardware devices. 2024-09-16T00:13:11.383 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:13:11.383 INFO:teuthology.orchestra.run.smithi038.stdout:This subpackage provides tools for support of all platforms. 2024-09-16T00:13:11.384 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-16T00:13:11.384 INFO:teuthology.task.kernel:Updating grub on smithi038 to boot 5.14.0-505.el9.x86_64 2024-09-16T00:13:11.384 DEBUG:teuthology.orchestra.run.smithi038:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-16T00:13:12.033 INFO:teuthology.orchestra.run.smithi038.stderr:Generating grub configuration file ... 2024-09-16T00:13:13.338 INFO:teuthology.orchestra.run.smithi038.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-16T00:13:13.363 INFO:teuthology.orchestra.run.smithi038.stderr:done 2024-09-16T00:13:13.365 DEBUG:teuthology.orchestra.run.smithi038:> mktemp 2024-09-16T00:13:13.381 INFO:teuthology.orchestra.run.smithi038.stdout:/tmp/tmp.c7cp7L0rWA 2024-09-16T00:13:13.381 DEBUG:teuthology.orchestra.run.smithi038:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.c7cp7L0rWA 2024-09-16T00:13:13.456 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 0666 /tmp/tmp.c7cp7L0rWA 2024-09-16T00:13:13.597 DEBUG:teuthology.orchestra.remote:smithi038:/tmp/tmp.c7cp7L0rWA is 6KB 2024-09-16T00:13:13.646 DEBUG:teuthology.orchestra.run.smithi038:> rm -fr /tmp/tmp.c7cp7L0rWA 2024-09-16T00:13:13.660 DEBUG:teuthology.orchestra.run.smithi038:> sudo /bin/ls /boot/loader/entries || true 2024-09-16T00:13:13.733 INFO:teuthology.orchestra.run.smithi038.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-16T00:13:13.733 INFO:teuthology.orchestra.run.smithi038.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-16T00:13:13.733 INFO:teuthology.orchestra.run.smithi038.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-16T00:13:13.733 INFO:teuthology.orchestra.run.smithi038.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-16T00:13:13.733 DEBUG:teuthology.orchestra.run.smithi038:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-16T00:13:13.886 DEBUG:teuthology.orchestra.run.smithi038:> sudo shutdown -r now 2024-09-16T00:13:43.888 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-16T00:13:43.889 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-09-16T00:13:43.890 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:14:02.301 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.38 2024-09-16T00:14:11.302 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-09-16T00:14:11.303 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:15:11.306 DEBUG:teuthology.orchestra.remote:timed out 2024-09-16T00:15:23.307 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-09-16T00:15:23.308 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:15:23.711 DEBUG:teuthology.orchestra.run.smithi038:> true 2024-09-16T00:15:24.071 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-09-16T00:15:24.071 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-505.el9.x86_64"... 2024-09-16T00:15:24.071 DEBUG:teuthology.orchestra.run.smithi038:> uname -r 2024-09-16T00:15:24.089 INFO:teuthology.orchestra.run.smithi038.stdout:5.14.0-505.el9.x86_64 2024-09-16T00:15:24.089 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-16T00:15:24.089 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-16T00:15:24.089 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-16T00:15:25.090 DEBUG:teuthology.parallel:result is None 2024-09-16T00:15:25.091 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-16T00:15:25.101 INFO:teuthology.task.internal:Creating test directory... 2024-09-16T00:15:25.101 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T00:15:25.103 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T00:15:25.106 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-16T00:15:25.125 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-16T00:15:25.135 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-16T00:15:25.146 INFO:teuthology.task.internal:Creating archive directory... 2024-09-16T00:15:25.147 DEBUG:teuthology.orchestra.run.smithi038:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T00:15:25.163 DEBUG:teuthology.orchestra.run.smithi073:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T00:15:25.165 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-16T00:15:25.203 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-16T00:15:25.214 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-16T00:15:25.214 DEBUG:teuthology.orchestra.run.smithi073:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T00:15:25.246 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:15:25.246 DEBUG:teuthology.orchestra.run.smithi122:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-16T00:15:25.263 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:15:25.263 DEBUG:teuthology.orchestra.run.smithi038:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T00:15:25.266 DEBUG:teuthology.orchestra.run.smithi073:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T00:15:25.290 DEBUG:teuthology.orchestra.run.smithi122:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-16T00:15:25.343 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T00:15:25.350 INFO:teuthology.orchestra.run.smithi073.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T00:15:25.359 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T00:15:25.365 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T00:15:25.371 INFO:teuthology.orchestra.run.smithi073.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T00:15:25.381 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-16T00:15:25.382 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-16T00:15:25.389 INFO:teuthology.task.internal:Configuring sudo... 2024-09-16T00:15:25.389 DEBUG:teuthology.orchestra.run.smithi038:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T00:15:25.392 DEBUG:teuthology.orchestra.run.smithi073:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T00:15:25.394 DEBUG:teuthology.orchestra.run.smithi122:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-16T00:15:25.427 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-16T00:15:25.438 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-16T00:15:25.438 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T00:15:25.463 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T00:15:25.465 DEBUG:teuthology.orchestra.run.smithi122:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-16T00:15:25.483 DEBUG:teuthology.orchestra.run.smithi038:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T00:15:25.566 DEBUG:teuthology.orchestra.run.smithi038:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T00:15:25.615 DEBUG:teuthology.orchestra.run.smithi038:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T00:15:25.702 DEBUG:teuthology.orchestra.run.smithi038:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T00:15:25.745 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-16T00:15:25.745 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T00:15:25.814 DEBUG:teuthology.orchestra.run.smithi073:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T00:15:25.862 DEBUG:teuthology.orchestra.run.smithi073:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T00:15:25.913 DEBUG:teuthology.orchestra.run.smithi073:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T00:15:25.996 DEBUG:teuthology.orchestra.run.smithi073:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T00:15:26.039 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-16T00:15:26.039 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T00:15:26.105 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T00:15:26.150 DEBUG:teuthology.orchestra.run.smithi122:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-16T00:15:26.203 DEBUG:teuthology.orchestra.run.smithi122:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T00:15:26.289 DEBUG:teuthology.orchestra.run.smithi122:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-16T00:15:26.331 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-16T00:15:26.331 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-16T00:15:26.397 DEBUG:teuthology.orchestra.run.smithi038:> sudo service rsyslog restart 2024-09-16T00:15:26.400 DEBUG:teuthology.orchestra.run.smithi073:> sudo service rsyslog restart 2024-09-16T00:15:26.403 DEBUG:teuthology.orchestra.run.smithi122:> sudo service rsyslog restart 2024-09-16T00:15:26.442 INFO:teuthology.orchestra.run.smithi073.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T00:15:26.450 INFO:teuthology.orchestra.run.smithi038.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T00:15:26.475 INFO:teuthology.orchestra.run.smithi122.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T00:15:26.830 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-16T00:15:26.838 INFO:teuthology.task.internal:Starting timer... 2024-09-16T00:15:26.838 INFO:teuthology.run_tasks:Running task pcp... 2024-09-16T00:15:26.860 INFO:teuthology.run_tasks:Running task selinux... 2024-09-16T00:15:26.869 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-09-16T00:15:26.870 DEBUG:teuthology.orchestra.run.smithi038:> sudo service auditd rotate 2024-09-16T00:15:26.931 INFO:teuthology.orchestra.run.smithi038.stdout:Rotating logs: 2024-09-16T00:15:26.933 DEBUG:teuthology.orchestra.run.smithi073:> sudo service auditd rotate 2024-09-16T00:15:26.985 INFO:teuthology.orchestra.run.smithi073.stdout:Rotating logs: 2024-09-16T00:15:26.987 DEBUG:teuthology.orchestra.run.smithi122:> sudo service auditd rotate 2024-09-16T00:15:27.045 INFO:teuthology.orchestra.run.smithi122.stdout:Rotating logs: 2024-09-16T00:15:27.047 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-16T00:15:27.047 DEBUG:teuthology.orchestra.run.smithi038:> /usr/sbin/getenforce 2024-09-16T00:15:27.080 INFO:teuthology.orchestra.run.smithi038.stdout:Permissive 2024-09-16T00:15:27.081 DEBUG:teuthology.orchestra.run.smithi073:> /usr/sbin/getenforce 2024-09-16T00:15:27.105 INFO:teuthology.orchestra.run.smithi073.stdout:Permissive 2024-09-16T00:15:27.105 DEBUG:teuthology.orchestra.run.smithi122:> /usr/sbin/getenforce 2024-09-16T00:15:27.131 INFO:teuthology.orchestra.run.smithi122.stdout:Permissive 2024-09-16T00:15:27.131 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi038.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi073.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi122.front.sepia.ceph.com': 'permissive'} 2024-09-16T00:15:27.132 DEBUG:teuthology.orchestra.run.smithi038:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T00:15:27.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:15:27.162 DEBUG:teuthology.orchestra.run.smithi073:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T00:15:27.190 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:15:27.191 DEBUG:teuthology.orchestra.run.smithi122:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T00:15:27.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:15:27.217 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-16T00:15:27.217 DEBUG:teuthology.orchestra.run.smithi038:> sudo /usr/sbin/setenforce permissive 2024-09-16T00:15:27.253 DEBUG:teuthology.orchestra.run.smithi073:> sudo /usr/sbin/setenforce permissive 2024-09-16T00:15:27.289 DEBUG:teuthology.orchestra.run.smithi122:> sudo /usr/sbin/setenforce permissive 2024-09-16T00:15:27.322 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-16T00:15:27.335 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-16T00:15:27.349 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-16T00:15:27.350 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi038.front.sepia.ceph.com,smithi073.front.sepia.ceph.com,smithi122.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-16T00:19:29.684 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi038.front.sepia.ceph.com'), Remote(name='ubuntu@smithi073.front.sepia.ceph.com'), Remote(name='ubuntu@smithi122.front.sepia.ceph.com')] 2024-09-16T00:19:29.688 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-09-16T00:19:29.689 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:19:29.772 DEBUG:teuthology.orchestra.run.smithi038:> true 2024-09-16T00:19:29.859 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi038.front.sepia.ceph.com' 2024-09-16T00:19:29.859 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-09-16T00:19:29.860 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:19:29.936 DEBUG:teuthology.orchestra.run.smithi073:> true 2024-09-16T00:19:30.013 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-09-16T00:19:30.014 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-16T00:19:30.014 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi122.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-16T00:19:30.090 DEBUG:teuthology.orchestra.run.smithi122:> true 2024-09-16T00:19:30.173 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi122.front.sepia.ceph.com' 2024-09-16T00:19:30.174 INFO:teuthology.run_tasks:Running task clock... 2024-09-16T00:19:30.184 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-16T00:19:30.184 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T00:19:30.185 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T00:19:30.187 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T00:19:30.188 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T00:19:30.190 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-16T00:19:30.190 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T00:19:30.219 INFO:teuthology.orchestra.run.smithi073.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T00:19:30.221 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T00:19:30.234 INFO:teuthology.orchestra.run.smithi073.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T00:19:30.238 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T00:19:30.244 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-16T00:19:30.259 INFO:teuthology.orchestra.run.smithi073.stderr:sudo: ntpd: command not found 2024-09-16T00:19:30.259 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-16T00:19:30.265 INFO:teuthology.orchestra.run.smithi038.stderr:sudo: ntpd: command not found 2024-09-16T00:19:30.270 INFO:teuthology.orchestra.run.smithi073.stdout:506 Cannot talk to daemon 2024-09-16T00:19:30.278 INFO:teuthology.orchestra.run.smithi038.stdout:506 Cannot talk to daemon 2024-09-16T00:19:30.284 INFO:teuthology.orchestra.run.smithi122.stderr:sudo: ntpd: command not found 2024-09-16T00:19:30.284 INFO:teuthology.orchestra.run.smithi073.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T00:19:30.294 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T00:19:30.295 INFO:teuthology.orchestra.run.smithi122.stdout:506 Cannot talk to daemon 2024-09-16T00:19:30.298 INFO:teuthology.orchestra.run.smithi073.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T00:19:30.309 INFO:teuthology.orchestra.run.smithi038.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T00:19:30.310 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-16T00:19:30.324 INFO:teuthology.orchestra.run.smithi122.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-16T00:19:30.355 INFO:teuthology.orchestra.run.smithi073.stderr:bash: line 1: ntpq: command not found 2024-09-16T00:19:30.358 INFO:teuthology.orchestra.run.smithi073.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T00:19:30.358 INFO:teuthology.orchestra.run.smithi073.stdout:=============================================================================== 2024-09-16T00:19:30.359 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:19:30.359 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:19:30.359 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:19:30.359 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:19:30.365 INFO:teuthology.orchestra.run.smithi038.stderr:bash: line 1: ntpq: command not found 2024-09-16T00:19:30.368 INFO:teuthology.orchestra.run.smithi038.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T00:19:30.368 INFO:teuthology.orchestra.run.smithi038.stdout:=============================================================================== 2024-09-16T00:19:30.368 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:19:30.368 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:19:30.369 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:19:30.369 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:19:30.385 INFO:teuthology.orchestra.run.smithi122.stderr:bash: line 1: ntpq: command not found 2024-09-16T00:19:30.389 INFO:teuthology.orchestra.run.smithi122.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T00:19:30.389 INFO:teuthology.orchestra.run.smithi122.stdout:=============================================================================== 2024-09-16T00:19:30.389 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:19:30.389 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:19:30.389 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:19:30.390 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:19:30.390 INFO:teuthology.run_tasks:Running task install... 2024-09-16T00:19:30.401 DEBUG:teuthology.task.install:project ceph 2024-09-16T00:19:30.401 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'}} 2024-09-16T00:19:30.401 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc'} 2024-09-16T00:19:30.401 INFO:teuthology.task.install:Using flavor: default 2024-09-16T00:19:30.407 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-16T00:19:30.407 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-16T00:19:30.407 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-16T00:19:30.408 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:19:30.409 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-16T00:19:30.410 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:19:30.411 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'tag': None, 'wait_for_package': False} 2024-09-16T00:19:30.411 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:19:30.556 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-16T00:19:30.556 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-16T00:19:30.562 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-16T00:19:30.563 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-16T00:19:30.572 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/ 2024-09-16T00:19:30.573 INFO:teuthology.task.install.rpm:Package version is 19.1.1-326.g8921c8ef 2024-09-16T00:19:30.665 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T00:19:30.665 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-16T00:19:30.665 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T00:19:30.671 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T00:19:30.671 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-16T00:19:30.671 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T00:19:30.679 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-16T00:19:30.680 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-16T00:19:30.680 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-16T00:19:30.692 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-16T00:19:30.693 DEBUG:teuthology.orchestra.run.smithi038:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T00:19:30.697 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-16T00:19:30.697 DEBUG:teuthology.orchestra.run.smithi122:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T00:19:30.705 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-16T00:19:30.705 DEBUG:teuthology.orchestra.run.smithi073:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/8921c8ef947807b717db041d85f0761b7914d5fc/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-16T00:19:30.760 DEBUG:teuthology.orchestra.run.smithi038:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T00:19:30.763 DEBUG:teuthology.orchestra.run.smithi122:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T00:19:30.770 DEBUG:teuthology.orchestra.run.smithi073:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-16T00:19:30.837 DEBUG:teuthology.orchestra.run.smithi038:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T00:19:30.838 DEBUG:teuthology.orchestra.run.smithi122:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T00:19:30.845 DEBUG:teuthology.orchestra.run.smithi073:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-16T00:19:30.902 INFO:teuthology.orchestra.run.smithi122.stdout:check_obsoletes = 1 2024-09-16T00:19:30.902 INFO:teuthology.orchestra.run.smithi038.stdout:check_obsoletes = 1 2024-09-16T00:19:30.903 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum clean all 2024-09-16T00:19:30.904 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum clean all 2024-09-16T00:19:30.911 INFO:teuthology.orchestra.run.smithi073.stdout:check_obsoletes = 1 2024-09-16T00:19:30.911 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum clean all 2024-09-16T00:19:31.127 INFO:teuthology.orchestra.run.smithi122.stdout:45 files removed 2024-09-16T00:19:31.135 INFO:teuthology.orchestra.run.smithi073.stdout:45 files removed 2024-09-16T00:19:31.144 INFO:teuthology.orchestra.run.smithi038.stdout:45 files removed 2024-09-16T00:19:31.154 DEBUG:teuthology.orchestra.run.smithi122:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-16T00:19:31.162 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-16T00:19:31.171 DEBUG:teuthology.orchestra.run.smithi038:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-16T00:19:31.691 INFO:teuthology.orchestra.run.smithi073.stdout:ceph packages for x86_64 326 kB/s | 83 kB 00:00 2024-09-16T00:19:31.700 INFO:teuthology.orchestra.run.smithi122.stdout:ceph packages for x86_64 310 kB/s | 83 kB 00:00 2024-09-16T00:19:31.750 INFO:teuthology.orchestra.run.smithi038.stdout:ceph packages for x86_64 301 kB/s | 83 kB 00:00 2024-09-16T00:19:31.958 INFO:teuthology.orchestra.run.smithi073.stdout:ceph noarch packages 52 kB/s | 12 kB 00:00 2024-09-16T00:19:31.976 INFO:teuthology.orchestra.run.smithi122.stdout:ceph noarch packages 51 kB/s | 12 kB 00:00 2024-09-16T00:19:32.042 INFO:teuthology.orchestra.run.smithi038.stdout:ceph noarch packages 48 kB/s | 12 kB 00:00 2024-09-16T00:19:32.217 INFO:teuthology.orchestra.run.smithi073.stdout:ceph source packages 8.4 kB/s | 1.9 kB 00:00 2024-09-16T00:19:32.243 INFO:teuthology.orchestra.run.smithi122.stdout:ceph source packages 8.2 kB/s | 1.9 kB 00:00 2024-09-16T00:19:32.326 INFO:teuthology.orchestra.run.smithi038.stdout:ceph source packages 7.7 kB/s | 1.9 kB 00:00 2024-09-16T00:19:32.667 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - BaseOS 20 MB/s | 8.2 MB 00:00 2024-09-16T00:19:32.810 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.2 MB 00:00 2024-09-16T00:19:32.977 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.2 MB 00:00 2024-09-16T00:19:34.711 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - AppStream 34 MB/s | 20 MB 00:00 2024-09-16T00:19:34.978 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - AppStream 20 MB/s | 20 MB 00:01 2024-09-16T00:19:36.078 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - AppStream 8.9 MB/s | 20 MB 00:02 2024-09-16T00:19:39.096 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - CRB 17 MB/s | 6.5 MB 00:00 2024-09-16T00:19:39.462 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-16T00:19:40.613 INFO:teuthology.orchestra.run.smithi038.stdout:CentOS Stream 9 - Extras packages 116 kB/s | 19 kB 00:00 2024-09-16T00:19:40.905 INFO:teuthology.orchestra.run.smithi122.stdout:CentOS Stream 9 - Extras packages 99 kB/s | 19 kB 00:00 2024-09-16T00:19:41.451 INFO:teuthology.orchestra.run.smithi038.stdout:Extra Packages for Enterprise Linux 32 MB/s | 23 MB 00:00 2024-09-16T00:19:41.834 INFO:teuthology.orchestra.run.smithi122.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-16T00:19:41.928 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - CRB 3.3 MB/s | 6.5 MB 00:01 2024-09-16T00:19:43.562 INFO:teuthology.orchestra.run.smithi073.stdout:CentOS Stream 9 - Extras packages 43 kB/s | 19 kB 00:00 2024-09-16T00:19:44.449 INFO:teuthology.orchestra.run.smithi073.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-16T00:19:47.140 INFO:teuthology.orchestra.run.smithi038.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-16T00:19:47.541 INFO:teuthology.orchestra.run.smithi122.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-16T00:19:49.687 INFO:teuthology.orchestra.run.smithi038.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T00:19:49.688 INFO:teuthology.orchestra.run.smithi038.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T00:19:49.703 INFO:teuthology.orchestra.run.smithi038.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-16T00:19:49.704 INFO:teuthology.orchestra.run.smithi038.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-16T00:19:49.705 INFO:teuthology.orchestra.run.smithi038.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-16T00:19:49.706 INFO:teuthology.orchestra.run.smithi038.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-16T00:19:49.708 INFO:teuthology.orchestra.run.smithi038.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-16T00:19:49.711 INFO:teuthology.orchestra.run.smithi038.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-16T00:19:49.712 INFO:teuthology.orchestra.run.smithi038.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-16T00:19:49.779 INFO:teuthology.orchestra.run.smithi038.stdout:Dependencies resolved. 2024-09-16T00:19:49.806 INFO:teuthology.orchestra.run.smithi038.stdout:====================================================================================== 2024-09-16T00:19:49.806 INFO:teuthology.orchestra.run.smithi038.stdout: Package Arch Version Repository Size 2024-09-16T00:19:49.806 INFO:teuthology.orchestra.run.smithi038.stdout:====================================================================================== 2024-09-16T00:19:49.806 INFO:teuthology.orchestra.run.smithi038.stdout:Installing: 2024-09-16T00:19:49.806 INFO:teuthology.orchestra.run.smithi038.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-16T00:19:49.806 INFO:teuthology.orchestra.run.smithi038.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-16T00:19:49.806 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-16T00:19:49.807 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-16T00:19:49.807 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-16T00:19:49.807 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-16T00:19:49.807 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-16T00:19:49.807 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-16T00:19:49.807 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-16T00:19:49.807 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-16T00:19:49.807 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-16T00:19:49.807 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-16T00:19:49.807 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-16T00:19:49.807 INFO:teuthology.orchestra.run.smithi038.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-16T00:19:49.807 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-16T00:19:49.807 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-16T00:19:49.808 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-16T00:19:49.808 INFO:teuthology.orchestra.run.smithi038.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-16T00:19:49.808 INFO:teuthology.orchestra.run.smithi038.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-16T00:19:49.808 INFO:teuthology.orchestra.run.smithi038.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-16T00:19:49.808 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-16T00:19:49.808 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-16T00:19:49.808 INFO:teuthology.orchestra.run.smithi038.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-16T00:19:49.808 INFO:teuthology.orchestra.run.smithi038.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-16T00:19:49.808 INFO:teuthology.orchestra.run.smithi038.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-16T00:19:49.808 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-16T00:19:49.808 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-16T00:19:49.808 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-16T00:19:49.809 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-16T00:19:49.809 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-16T00:19:49.809 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-16T00:19:49.809 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-16T00:19:49.809 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-16T00:19:49.809 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-16T00:19:49.809 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-16T00:19:49.809 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-16T00:19:49.809 INFO:teuthology.orchestra.run.smithi038.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-16T00:19:49.809 INFO:teuthology.orchestra.run.smithi038.stdout:Upgrading: 2024-09-16T00:19:49.809 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-16T00:19:49.809 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-16T00:19:49.809 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-16T00:19:49.810 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-16T00:19:49.810 INFO:teuthology.orchestra.run.smithi038.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-16T00:19:49.810 INFO:teuthology.orchestra.run.smithi038.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-16T00:19:49.810 INFO:teuthology.orchestra.run.smithi038.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-16T00:19:49.810 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-16T00:19:49.810 INFO:teuthology.orchestra.run.smithi038.stdout:Installing dependencies: 2024-09-16T00:19:49.810 INFO:teuthology.orchestra.run.smithi038.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-16T00:19:49.810 INFO:teuthology.orchestra.run.smithi038.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-16T00:19:49.810 INFO:teuthology.orchestra.run.smithi038.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T00:19:49.810 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-16T00:19:49.810 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-16T00:19:49.810 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-16T00:19:49.810 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-16T00:19:49.811 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-16T00:19:49.811 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-16T00:19:49.811 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-16T00:19:49.811 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-16T00:19:49.811 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T00:19:49.811 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T00:19:49.811 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T00:19:49.811 INFO:teuthology.orchestra.run.smithi038.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T00:19:49.811 INFO:teuthology.orchestra.run.smithi038.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T00:19:49.811 INFO:teuthology.orchestra.run.smithi038.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-16T00:19:49.811 INFO:teuthology.orchestra.run.smithi038.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T00:19:49.811 INFO:teuthology.orchestra.run.smithi038.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T00:19:49.811 INFO:teuthology.orchestra.run.smithi038.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-16T00:19:49.812 INFO:teuthology.orchestra.run.smithi038.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T00:19:49.812 INFO:teuthology.orchestra.run.smithi038.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T00:19:49.812 INFO:teuthology.orchestra.run.smithi038.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T00:19:49.812 INFO:teuthology.orchestra.run.smithi038.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T00:19:49.812 INFO:teuthology.orchestra.run.smithi038.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T00:19:49.812 INFO:teuthology.orchestra.run.smithi038.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-16T00:19:49.812 INFO:teuthology.orchestra.run.smithi038.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T00:19:49.812 INFO:teuthology.orchestra.run.smithi038.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-16T00:19:49.812 INFO:teuthology.orchestra.run.smithi038.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T00:19:49.812 INFO:teuthology.orchestra.run.smithi038.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-16T00:19:49.812 INFO:teuthology.orchestra.run.smithi038.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T00:19:49.812 INFO:teuthology.orchestra.run.smithi038.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-16T00:19:49.812 INFO:teuthology.orchestra.run.smithi038.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-16T00:19:49.813 INFO:teuthology.orchestra.run.smithi038.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-16T00:19:49.813 INFO:teuthology.orchestra.run.smithi038.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T00:19:49.813 INFO:teuthology.orchestra.run.smithi038.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T00:19:49.813 INFO:teuthology.orchestra.run.smithi038.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T00:19:49.813 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-16T00:19:49.813 INFO:teuthology.orchestra.run.smithi038.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-16T00:19:49.813 INFO:teuthology.orchestra.run.smithi038.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-16T00:19:49.813 INFO:teuthology.orchestra.run.smithi038.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T00:19:49.813 INFO:teuthology.orchestra.run.smithi038.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-16T00:19:49.813 INFO:teuthology.orchestra.run.smithi038.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T00:19:49.813 INFO:teuthology.orchestra.run.smithi038.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T00:19:49.813 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T00:19:49.814 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-16T00:19:49.814 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-16T00:19:49.814 INFO:teuthology.orchestra.run.smithi038.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T00:19:49.814 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T00:19:49.814 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T00:19:49.814 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T00:19:49.814 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-16T00:19:49.814 INFO:teuthology.orchestra.run.smithi038.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-16T00:19:49.814 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T00:19:49.814 INFO:teuthology.orchestra.run.smithi038.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T00:19:49.814 INFO:teuthology.orchestra.run.smithi038.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T00:19:49.815 INFO:teuthology.orchestra.run.smithi038.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-16T00:19:49.815 INFO:teuthology.orchestra.run.smithi038.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-16T00:19:49.815 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T00:19:49.815 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T00:19:49.815 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-16T00:19:49.815 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T00:19:49.815 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T00:19:49.815 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T00:19:49.815 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T00:19:49.815 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T00:19:49.815 INFO:teuthology.orchestra.run.smithi038.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T00:19:49.815 INFO:teuthology.orchestra.run.smithi038.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T00:19:49.815 INFO:teuthology.orchestra.run.smithi038.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T00:19:49.816 INFO:teuthology.orchestra.run.smithi038.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T00:19:49.816 INFO:teuthology.orchestra.run.smithi038.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T00:19:49.816 INFO:teuthology.orchestra.run.smithi038.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T00:19:49.816 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T00:19:49.816 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T00:19:49.816 INFO:teuthology.orchestra.run.smithi038.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T00:19:49.816 INFO:teuthology.orchestra.run.smithi038.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T00:19:49.816 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T00:19:49.816 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T00:19:49.816 INFO:teuthology.orchestra.run.smithi038.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T00:19:49.816 INFO:teuthology.orchestra.run.smithi038.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T00:19:49.816 INFO:teuthology.orchestra.run.smithi038.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-16T00:19:49.816 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T00:19:49.817 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T00:19:49.817 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T00:19:49.817 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T00:19:49.817 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-16T00:19:49.817 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T00:19:49.817 INFO:teuthology.orchestra.run.smithi038.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T00:19:49.817 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T00:19:49.817 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T00:19:49.817 INFO:teuthology.orchestra.run.smithi038.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T00:19:49.817 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T00:19:49.817 INFO:teuthology.orchestra.run.smithi038.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T00:19:49.817 INFO:teuthology.orchestra.run.smithi038.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T00:19:49.817 INFO:teuthology.orchestra.run.smithi038.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T00:19:49.818 INFO:teuthology.orchestra.run.smithi038.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T00:19:49.818 INFO:teuthology.orchestra.run.smithi038.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T00:19:49.818 INFO:teuthology.orchestra.run.smithi038.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-16T00:19:49.818 INFO:teuthology.orchestra.run.smithi038.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T00:19:49.818 INFO:teuthology.orchestra.run.smithi038.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T00:19:49.818 INFO:teuthology.orchestra.run.smithi038.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T00:19:49.818 INFO:teuthology.orchestra.run.smithi038.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T00:19:49.818 INFO:teuthology.orchestra.run.smithi038.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-16T00:19:49.818 INFO:teuthology.orchestra.run.smithi038.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-16T00:19:49.818 INFO:teuthology.orchestra.run.smithi038.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T00:19:49.818 INFO:teuthology.orchestra.run.smithi038.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T00:19:49.818 INFO:teuthology.orchestra.run.smithi038.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T00:19:49.818 INFO:teuthology.orchestra.run.smithi038.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-16T00:19:49.819 INFO:teuthology.orchestra.run.smithi038.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T00:19:49.819 INFO:teuthology.orchestra.run.smithi038.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-16T00:19:49.819 INFO:teuthology.orchestra.run.smithi038.stdout:Installing weak dependencies: 2024-09-16T00:19:49.819 INFO:teuthology.orchestra.run.smithi038.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-16T00:19:49.819 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:19:49.819 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction Summary 2024-09-16T00:19:49.819 INFO:teuthology.orchestra.run.smithi038.stdout:====================================================================================== 2024-09-16T00:19:49.819 INFO:teuthology.orchestra.run.smithi038.stdout:Install 148 Packages 2024-09-16T00:19:49.819 INFO:teuthology.orchestra.run.smithi038.stdout:Upgrade 8 Packages 2024-09-16T00:19:49.819 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:19:49.820 INFO:teuthology.orchestra.run.smithi038.stdout:Total download size: 226 M 2024-09-16T00:19:49.820 INFO:teuthology.orchestra.run.smithi038.stdout:Downloading Packages: 2024-09-16T00:19:49.826 INFO:teuthology.orchestra.run.smithi122.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T00:19:49.826 INFO:teuthology.orchestra.run.smithi122.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T00:19:49.838 INFO:teuthology.orchestra.run.smithi122.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-16T00:19:49.839 INFO:teuthology.orchestra.run.smithi122.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-16T00:19:49.840 INFO:teuthology.orchestra.run.smithi122.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-16T00:19:49.841 INFO:teuthology.orchestra.run.smithi122.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-16T00:19:49.842 INFO:teuthology.orchestra.run.smithi122.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-16T00:19:49.845 INFO:teuthology.orchestra.run.smithi122.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-16T00:19:49.845 INFO:teuthology.orchestra.run.smithi122.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-16T00:19:49.889 INFO:teuthology.orchestra.run.smithi073.stdout:lab-extras 11 kB/s | 1.7 kB 00:00 2024-09-16T00:19:49.908 INFO:teuthology.orchestra.run.smithi122.stdout:Dependencies resolved. 2024-09-16T00:19:49.930 INFO:teuthology.orchestra.run.smithi122.stdout:====================================================================================== 2024-09-16T00:19:49.930 INFO:teuthology.orchestra.run.smithi122.stdout: Package Arch Version Repository Size 2024-09-16T00:19:49.930 INFO:teuthology.orchestra.run.smithi122.stdout:====================================================================================== 2024-09-16T00:19:49.930 INFO:teuthology.orchestra.run.smithi122.stdout:Installing: 2024-09-16T00:19:49.930 INFO:teuthology.orchestra.run.smithi122.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-16T00:19:49.930 INFO:teuthology.orchestra.run.smithi122.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-16T00:19:49.930 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-16T00:19:49.930 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-16T00:19:49.930 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-16T00:19:49.930 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-16T00:19:49.931 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-16T00:19:49.931 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-16T00:19:49.931 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-16T00:19:49.931 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-16T00:19:49.931 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-16T00:19:49.931 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-16T00:19:49.931 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-16T00:19:49.931 INFO:teuthology.orchestra.run.smithi122.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-16T00:19:49.931 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-16T00:19:49.931 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-16T00:19:49.931 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-16T00:19:49.931 INFO:teuthology.orchestra.run.smithi122.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-16T00:19:49.931 INFO:teuthology.orchestra.run.smithi122.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-16T00:19:49.932 INFO:teuthology.orchestra.run.smithi122.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-16T00:19:49.932 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-16T00:19:49.932 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-16T00:19:49.932 INFO:teuthology.orchestra.run.smithi122.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-16T00:19:49.932 INFO:teuthology.orchestra.run.smithi122.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-16T00:19:49.932 INFO:teuthology.orchestra.run.smithi122.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-16T00:19:49.932 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-16T00:19:49.932 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-16T00:19:49.932 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-16T00:19:49.932 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-16T00:19:49.932 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-16T00:19:49.932 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-16T00:19:49.932 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-16T00:19:49.933 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-16T00:19:49.933 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-16T00:19:49.933 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-16T00:19:49.933 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-16T00:19:49.933 INFO:teuthology.orchestra.run.smithi122.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-16T00:19:49.933 INFO:teuthology.orchestra.run.smithi122.stdout:Upgrading: 2024-09-16T00:19:49.933 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-16T00:19:49.933 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-16T00:19:49.933 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-16T00:19:49.933 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-16T00:19:49.933 INFO:teuthology.orchestra.run.smithi122.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-16T00:19:49.933 INFO:teuthology.orchestra.run.smithi122.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-16T00:19:49.933 INFO:teuthology.orchestra.run.smithi122.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-16T00:19:49.933 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-16T00:19:49.934 INFO:teuthology.orchestra.run.smithi122.stdout:Installing dependencies: 2024-09-16T00:19:49.934 INFO:teuthology.orchestra.run.smithi122.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-16T00:19:49.934 INFO:teuthology.orchestra.run.smithi122.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-16T00:19:49.934 INFO:teuthology.orchestra.run.smithi122.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T00:19:49.934 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-16T00:19:49.934 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-16T00:19:49.934 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-16T00:19:49.934 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-16T00:19:49.934 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-16T00:19:49.934 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-16T00:19:49.934 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-16T00:19:49.934 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-16T00:19:49.934 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T00:19:49.934 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T00:19:49.935 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T00:19:49.935 INFO:teuthology.orchestra.run.smithi122.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T00:19:49.935 INFO:teuthology.orchestra.run.smithi122.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T00:19:49.935 INFO:teuthology.orchestra.run.smithi122.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-16T00:19:49.935 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T00:19:49.935 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T00:19:49.935 INFO:teuthology.orchestra.run.smithi122.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-16T00:19:49.935 INFO:teuthology.orchestra.run.smithi122.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T00:19:49.935 INFO:teuthology.orchestra.run.smithi122.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T00:19:49.935 INFO:teuthology.orchestra.run.smithi122.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T00:19:49.935 INFO:teuthology.orchestra.run.smithi122.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T00:19:49.935 INFO:teuthology.orchestra.run.smithi122.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T00:19:49.935 INFO:teuthology.orchestra.run.smithi122.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-16T00:19:49.936 INFO:teuthology.orchestra.run.smithi122.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T00:19:49.936 INFO:teuthology.orchestra.run.smithi122.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-16T00:19:49.936 INFO:teuthology.orchestra.run.smithi122.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T00:19:49.936 INFO:teuthology.orchestra.run.smithi122.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-16T00:19:49.936 INFO:teuthology.orchestra.run.smithi122.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T00:19:49.936 INFO:teuthology.orchestra.run.smithi122.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-16T00:19:49.936 INFO:teuthology.orchestra.run.smithi122.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-16T00:19:49.936 INFO:teuthology.orchestra.run.smithi122.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-16T00:19:49.936 INFO:teuthology.orchestra.run.smithi122.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T00:19:49.936 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T00:19:49.936 INFO:teuthology.orchestra.run.smithi122.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T00:19:49.936 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-16T00:19:49.936 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-16T00:19:49.936 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-16T00:19:49.937 INFO:teuthology.orchestra.run.smithi122.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T00:19:49.937 INFO:teuthology.orchestra.run.smithi122.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-16T00:19:49.937 INFO:teuthology.orchestra.run.smithi122.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T00:19:49.937 INFO:teuthology.orchestra.run.smithi122.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T00:19:49.937 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T00:19:49.937 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-16T00:19:49.937 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-16T00:19:49.937 INFO:teuthology.orchestra.run.smithi122.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T00:19:49.937 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T00:19:49.937 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T00:19:49.937 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T00:19:49.937 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-16T00:19:49.937 INFO:teuthology.orchestra.run.smithi122.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-16T00:19:49.938 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T00:19:49.938 INFO:teuthology.orchestra.run.smithi122.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T00:19:49.938 INFO:teuthology.orchestra.run.smithi122.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T00:19:49.938 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-16T00:19:49.938 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-16T00:19:49.938 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T00:19:49.938 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T00:19:49.938 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-16T00:19:49.938 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T00:19:49.938 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T00:19:49.938 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T00:19:49.938 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T00:19:49.938 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T00:19:49.938 INFO:teuthology.orchestra.run.smithi122.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T00:19:49.939 INFO:teuthology.orchestra.run.smithi122.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T00:19:49.939 INFO:teuthology.orchestra.run.smithi122.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T00:19:49.939 INFO:teuthology.orchestra.run.smithi122.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T00:19:49.939 INFO:teuthology.orchestra.run.smithi122.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T00:19:49.939 INFO:teuthology.orchestra.run.smithi122.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T00:19:49.939 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T00:19:49.939 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T00:19:49.939 INFO:teuthology.orchestra.run.smithi122.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T00:19:49.939 INFO:teuthology.orchestra.run.smithi122.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T00:19:49.939 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T00:19:49.939 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T00:19:49.939 INFO:teuthology.orchestra.run.smithi122.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T00:19:49.939 INFO:teuthology.orchestra.run.smithi122.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T00:19:49.939 INFO:teuthology.orchestra.run.smithi122.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-16T00:19:49.939 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T00:19:49.940 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T00:19:49.940 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T00:19:49.940 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T00:19:49.940 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-16T00:19:49.940 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T00:19:49.940 INFO:teuthology.orchestra.run.smithi122.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T00:19:49.940 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T00:19:49.940 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T00:19:49.940 INFO:teuthology.orchestra.run.smithi122.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T00:19:49.940 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T00:19:49.940 INFO:teuthology.orchestra.run.smithi122.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T00:19:49.940 INFO:teuthology.orchestra.run.smithi122.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T00:19:49.940 INFO:teuthology.orchestra.run.smithi122.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T00:19:49.941 INFO:teuthology.orchestra.run.smithi122.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T00:19:49.941 INFO:teuthology.orchestra.run.smithi122.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T00:19:49.941 INFO:teuthology.orchestra.run.smithi122.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-16T00:19:49.941 INFO:teuthology.orchestra.run.smithi122.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T00:19:49.941 INFO:teuthology.orchestra.run.smithi122.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T00:19:49.941 INFO:teuthology.orchestra.run.smithi122.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T00:19:49.941 INFO:teuthology.orchestra.run.smithi122.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T00:19:49.941 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-16T00:19:49.941 INFO:teuthology.orchestra.run.smithi122.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-16T00:19:49.941 INFO:teuthology.orchestra.run.smithi122.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T00:19:49.941 INFO:teuthology.orchestra.run.smithi122.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T00:19:49.941 INFO:teuthology.orchestra.run.smithi122.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T00:19:49.941 INFO:teuthology.orchestra.run.smithi122.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-16T00:19:49.941 INFO:teuthology.orchestra.run.smithi122.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T00:19:49.941 INFO:teuthology.orchestra.run.smithi122.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-16T00:19:49.942 INFO:teuthology.orchestra.run.smithi122.stdout:Installing weak dependencies: 2024-09-16T00:19:49.942 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-16T00:19:49.942 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-16T00:19:49.942 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction Summary 2024-09-16T00:19:49.942 INFO:teuthology.orchestra.run.smithi122.stdout:====================================================================================== 2024-09-16T00:19:49.942 INFO:teuthology.orchestra.run.smithi122.stdout:Install 148 Packages 2024-09-16T00:19:49.942 INFO:teuthology.orchestra.run.smithi122.stdout:Upgrade 8 Packages 2024-09-16T00:19:49.942 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-16T00:19:49.943 INFO:teuthology.orchestra.run.smithi122.stdout:Total download size: 226 M 2024-09-16T00:19:49.943 INFO:teuthology.orchestra.run.smithi122.stdout:Downloading Packages: 2024-09-16T00:19:50.517 INFO:teuthology.orchestra.run.smithi122.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 30 kB/s | 6.5 kB 00:00 2024-09-16T00:19:50.541 INFO:teuthology.orchestra.run.smithi038.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 27 kB/s | 6.5 kB 00:00 2024-09-16T00:19:51.034 INFO:teuthology.orchestra.run.smithi122.stdout:(2/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 1.7 MB/s | 896 kB 00:00 2024-09-16T00:19:51.151 INFO:teuthology.orchestra.run.smithi122.stdout:(3/156): ceph-immutable-object-cache-19.1.1-326 1.2 MB/s | 145 kB 00:00 2024-09-16T00:19:51.250 INFO:teuthology.orchestra.run.smithi038.stdout:(2/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 5.7 MB/s | 5.4 MB 00:00 2024-09-16T00:19:51.291 INFO:teuthology.orchestra.run.smithi038.stdout:(3/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 1.2 MB/s | 896 kB 00:00 2024-09-16T00:19:51.317 INFO:teuthology.orchestra.run.smithi038.stdout:(4/156): ceph-immutable-object-cache-19.1.1-326 2.1 MB/s | 145 kB 00:00 2024-09-16T00:19:51.501 INFO:teuthology.orchestra.run.smithi038.stdout:(5/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 8.3 MB/s | 1.5 MB 00:00 2024-09-16T00:19:51.651 INFO:teuthology.orchestra.run.smithi122.stdout:(4/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 4.8 MB/s | 2.4 MB 00:00 2024-09-16T00:19:51.926 INFO:teuthology.orchestra.run.smithi038.stdout:(6/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 11 MB/s | 4.7 MB 00:00 2024-09-16T00:19:51.926 INFO:teuthology.orchestra.run.smithi122.stdout:(5/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 3.3 MB/s | 5.4 MB 00:01 2024-09-16T00:19:51.969 INFO:teuthology.orchestra.run.smithi122.stdout:(6/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 4.8 MB/s | 1.5 MB 00:00 2024-09-16T00:19:52.124 INFO:teuthology.orchestra.run.smithi073.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T00:19:52.459 INFO:teuthology.orchestra.run.smithi073.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-16T00:19:52.459 INFO:teuthology.orchestra.run.smithi073.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-16T00:19:52.459 INFO:teuthology.orchestra.run.smithi073.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-16T00:19:52.459 INFO:teuthology.orchestra.run.smithi073.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-16T00:19:52.459 INFO:teuthology.orchestra.run.smithi073.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-16T00:19:52.459 INFO:teuthology.orchestra.run.smithi073.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-16T00:19:52.459 INFO:teuthology.orchestra.run.smithi073.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-16T00:19:52.459 INFO:teuthology.orchestra.run.smithi073.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-16T00:19:52.459 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2024-09-16T00:19:52.459 INFO:teuthology.orchestra.run.smithi073.stdout:====================================================================================== 2024-09-16T00:19:52.459 INFO:teuthology.orchestra.run.smithi073.stdout: Package Arch Version Repository Size 2024-09-16T00:19:52.460 INFO:teuthology.orchestra.run.smithi073.stdout:====================================================================================== 2024-09-16T00:19:52.460 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2024-09-16T00:19:52.460 INFO:teuthology.orchestra.run.smithi073.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-16T00:19:52.460 INFO:teuthology.orchestra.run.smithi073.stdout: ceph x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 6.5 k 2024-09-16T00:19:52.460 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-base x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.4 M 2024-09-16T00:19:52.460 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 896 k 2024-09-16T00:19:52.460 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-immutable-object-cache x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 145 k 2024-09-16T00:19:52.460 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 1.5 M 2024-09-16T00:19:52.460 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 146 k 2024-09-16T00:19:52.460 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-dashboard noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 3.6 M 2024-09-16T00:19:52.460 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-diskprediction-local noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 7.4 M 2024-09-16T00:19:52.460 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-rook noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 49 k 2024-09-16T00:19:52.460 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-radosgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 11 M 2024-09-16T00:19:52.460 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-test x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 49 M 2024-09-16T00:19:52.461 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-volume noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 285 k 2024-09-16T00:19:52.461 INFO:teuthology.orchestra.run.smithi073.stdout: cephadm noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 765 k 2024-09-16T00:19:52.461 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 228 k 2024-09-16T00:19:52.461 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-shell x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 27 k 2024-09-16T00:19:52.461 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-top noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 18 k 2024-09-16T00:19:52.461 INFO:teuthology.orchestra.run.smithi073.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-16T00:19:52.461 INFO:teuthology.orchestra.run.smithi073.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-16T00:19:52.461 INFO:teuthology.orchestra.run.smithi073.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-16T00:19:52.461 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 32 k 2024-09-16T00:19:52.461 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 751 k 2024-09-16T00:19:52.462 INFO:teuthology.orchestra.run.smithi073.stdout: librados-devel x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 127 k 2024-09-16T00:19:52.462 INFO:teuthology.orchestra.run.smithi073.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-16T00:19:52.462 INFO:teuthology.orchestra.run.smithi073.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-16T00:19:52.462 INFO:teuthology.orchestra.run.smithi073.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-16T00:19:52.462 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-16T00:19:52.462 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-16T00:19:52.462 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-16T00:19:52.462 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cephfs x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 161 k 2024-09-16T00:19:52.462 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rados x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 320 k 2024-09-16T00:19:52.462 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 300 k 2024-09-16T00:19:52.462 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rgw x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 99 k 2024-09-16T00:19:52.462 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-fuse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 85 k 2024-09-16T00:19:52.462 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-mirror x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.1 M 2024-09-16T00:19:52.463 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-nbd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 170 k 2024-09-16T00:19:52.463 INFO:teuthology.orchestra.run.smithi073.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-16T00:19:52.463 INFO:teuthology.orchestra.run.smithi073.stdout:Upgrading: 2024-09-16T00:19:52.463 INFO:teuthology.orchestra.run.smithi073.stdout: NetworkManager x86_64 1:1.49.90-1.el9 baseos 2.3 M 2024-09-16T00:19:52.463 INFO:teuthology.orchestra.run.smithi073.stdout: NetworkManager-libnm x86_64 1:1.49.90-1.el9 baseos 1.8 M 2024-09-16T00:19:52.463 INFO:teuthology.orchestra.run.smithi073.stdout: NetworkManager-team x86_64 1:1.49.90-1.el9 baseos 41 k 2024-09-16T00:19:52.463 INFO:teuthology.orchestra.run.smithi073.stdout: NetworkManager-tui x86_64 1:1.49.90-1.el9 baseos 249 k 2024-09-16T00:19:52.463 INFO:teuthology.orchestra.run.smithi073.stdout: librados2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.4 M 2024-09-16T00:19:52.463 INFO:teuthology.orchestra.run.smithi073.stdout: librbd1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 3.2 M 2024-09-16T00:19:52.463 INFO:teuthology.orchestra.run.smithi073.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-16T00:19:52.464 INFO:teuthology.orchestra.run.smithi073.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-16T00:19:52.464 INFO:teuthology.orchestra.run.smithi073.stdout:Installing dependencies: 2024-09-16T00:19:52.464 INFO:teuthology.orchestra.run.smithi073.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-16T00:19:52.464 INFO:teuthology.orchestra.run.smithi073.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-16T00:19:52.464 INFO:teuthology.orchestra.run.smithi073.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-16T00:19:52.464 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 21 M 2024-09-16T00:19:52.464 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-grafana-dashboards noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 32 k 2024-09-16T00:19:52.464 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mds x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 2.4 M 2024-09-16T00:19:52.464 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-modules-core noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 249 k 2024-09-16T00:19:52.464 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mon x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 4.7 M 2024-09-16T00:19:52.464 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-osd x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 17 M 2024-09-16T00:19:52.465 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-prometheus-alerts noarch 2:19.1.1-326.g8921c8ef.el9 ceph-noarch 16 k 2024-09-16T00:19:52.465 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-selinux x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 25 k 2024-09-16T00:19:52.465 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-16T00:19:52.465 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-16T00:19:52.465 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-16T00:19:52.465 INFO:teuthology.orchestra.run.smithi073.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-16T00:19:52.465 INFO:teuthology.orchestra.run.smithi073.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-16T00:19:52.465 INFO:teuthology.orchestra.run.smithi073.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-16T00:19:52.465 INFO:teuthology.orchestra.run.smithi073.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-16T00:19:52.465 INFO:teuthology.orchestra.run.smithi073.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-16T00:19:52.465 INFO:teuthology.orchestra.run.smithi073.stdout: libcephsqlite x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 163 k 2024-09-16T00:19:52.465 INFO:teuthology.orchestra.run.smithi073.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-16T00:19:52.465 INFO:teuthology.orchestra.run.smithi073.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-16T00:19:52.466 INFO:teuthology.orchestra.run.smithi073.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-16T00:19:52.466 INFO:teuthology.orchestra.run.smithi073.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-16T00:19:52.466 INFO:teuthology.orchestra.run.smithi073.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-16T00:19:52.466 INFO:teuthology.orchestra.run.smithi073.stdout: libradosstriper1 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 506 k 2024-09-16T00:19:52.466 INFO:teuthology.orchestra.run.smithi073.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-16T00:19:52.466 INFO:teuthology.orchestra.run.smithi073.stdout: librgw2 x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 5.3 M 2024-09-16T00:19:52.466 INFO:teuthology.orchestra.run.smithi073.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-16T00:19:52.466 INFO:teuthology.orchestra.run.smithi073.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-16T00:19:52.466 INFO:teuthology.orchestra.run.smithi073.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-16T00:19:52.466 INFO:teuthology.orchestra.run.smithi073.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-16T00:19:52.466 INFO:teuthology.orchestra.run.smithi073.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-16T00:19:52.466 INFO:teuthology.orchestra.run.smithi073.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-16T00:19:52.466 INFO:teuthology.orchestra.run.smithi073.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-16T00:19:52.466 INFO:teuthology.orchestra.run.smithi073.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-16T00:19:52.467 INFO:teuthology.orchestra.run.smithi073.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-16T00:19:52.467 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-16T00:19:52.467 INFO:teuthology.orchestra.run.smithi073.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-16T00:19:52.467 INFO:teuthology.orchestra.run.smithi073.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-16T00:19:52.467 INFO:teuthology.orchestra.run.smithi073.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-16T00:19:52.467 INFO:teuthology.orchestra.run.smithi073.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-16T00:19:52.467 INFO:teuthology.orchestra.run.smithi073.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-16T00:19:52.467 INFO:teuthology.orchestra.run.smithi073.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-16T00:19:52.467 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-16T00:19:52.467 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-argparse x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 45 k 2024-09-16T00:19:52.467 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-common x86_64 2:19.1.1-326.g8921c8ef.el9 ceph 136 k 2024-09-16T00:19:52.467 INFO:teuthology.orchestra.run.smithi073.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-16T00:19:52.467 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-16T00:19:52.468 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-16T00:19:52.468 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-16T00:19:52.468 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-16T00:19:52.468 INFO:teuthology.orchestra.run.smithi073.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-16T00:19:52.468 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-16T00:19:52.468 INFO:teuthology.orchestra.run.smithi073.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-16T00:19:52.468 INFO:teuthology.orchestra.run.smithi073.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-16T00:19:52.468 INFO:teuthology.orchestra.run.smithi073.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-16T00:19:52.468 INFO:teuthology.orchestra.run.smithi073.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-16T00:19:52.468 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-16T00:19:52.468 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-16T00:19:52.468 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-16T00:19:52.468 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-16T00:19:52.469 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-16T00:19:52.469 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-16T00:19:52.469 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-16T00:19:52.469 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-16T00:19:52.469 INFO:teuthology.orchestra.run.smithi073.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-16T00:19:52.469 INFO:teuthology.orchestra.run.smithi073.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-16T00:19:52.469 INFO:teuthology.orchestra.run.smithi073.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-16T00:19:52.469 INFO:teuthology.orchestra.run.smithi073.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-16T00:19:52.469 INFO:teuthology.orchestra.run.smithi073.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-16T00:19:52.469 INFO:teuthology.orchestra.run.smithi073.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-16T00:19:52.469 INFO:teuthology.orchestra.run.smithi073.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-16T00:19:52.469 INFO:teuthology.orchestra.run.smithi073.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-16T00:19:52.469 INFO:teuthology.orchestra.run.smithi073.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-16T00:19:52.470 INFO:teuthology.orchestra.run.smithi073.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-16T00:19:52.470 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-16T00:19:52.470 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-16T00:19:52.470 INFO:teuthology.orchestra.run.smithi073.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-16T00:19:52.470 INFO:teuthology.orchestra.run.smithi073.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-16T00:19:52.470 INFO:teuthology.orchestra.run.smithi073.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-16T00:19:52.470 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-16T00:19:52.470 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-16T00:19:52.470 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-16T00:19:52.470 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-16T00:19:52.470 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-16T00:19:52.470 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-16T00:19:52.471 INFO:teuthology.orchestra.run.smithi073.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-16T00:19:52.471 INFO:teuthology.orchestra.run.smithi073.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-16T00:19:52.471 INFO:teuthology.orchestra.run.smithi073.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-16T00:19:52.471 INFO:teuthology.orchestra.run.smithi073.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-16T00:19:52.471 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-16T00:19:52.471 INFO:teuthology.orchestra.run.smithi073.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-16T00:19:52.471 INFO:teuthology.orchestra.run.smithi073.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-16T00:19:52.471 INFO:teuthology.orchestra.run.smithi073.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-16T00:19:52.471 INFO:teuthology.orchestra.run.smithi073.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-16T00:19:52.471 INFO:teuthology.orchestra.run.smithi073.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-16T00:19:52.472 INFO:teuthology.orchestra.run.smithi073.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-16T00:19:52.472 INFO:teuthology.orchestra.run.smithi073.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-16T00:19:52.472 INFO:teuthology.orchestra.run.smithi073.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-16T00:19:52.472 INFO:teuthology.orchestra.run.smithi073.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-16T00:19:52.472 INFO:teuthology.orchestra.run.smithi073.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-16T00:19:52.472 INFO:teuthology.orchestra.run.smithi073.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-16T00:19:52.472 INFO:teuthology.orchestra.run.smithi073.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-16T00:19:52.472 INFO:teuthology.orchestra.run.smithi073.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-16T00:19:52.472 INFO:teuthology.orchestra.run.smithi073.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-16T00:19:52.473 INFO:teuthology.orchestra.run.smithi073.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-16T00:19:52.473 INFO:teuthology.orchestra.run.smithi073.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-16T00:19:52.473 INFO:teuthology.orchestra.run.smithi073.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-16T00:19:52.473 INFO:teuthology.orchestra.run.smithi073.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-16T00:19:52.473 INFO:teuthology.orchestra.run.smithi073.stdout:Installing weak dependencies: 2024-09-16T00:19:52.473 INFO:teuthology.orchestra.run.smithi073.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-16T00:19:52.473 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-16T00:19:52.473 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2024-09-16T00:19:52.473 INFO:teuthology.orchestra.run.smithi073.stdout:====================================================================================== 2024-09-16T00:19:52.473 INFO:teuthology.orchestra.run.smithi073.stdout:Install 148 Packages 2024-09-16T00:19:52.473 INFO:teuthology.orchestra.run.smithi073.stdout:Upgrade 8 Packages 2024-09-16T00:19:52.473 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-16T00:19:52.473 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 226 M 2024-09-16T00:19:52.473 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2024-09-16T00:19:52.501 INFO:teuthology.orchestra.run.smithi038.stdout:(7/156): ceph-common-19.1.1-326.g8921c8ef.el9.x 9.6 MB/s | 21 MB 00:02 2024-09-16T00:19:52.968 INFO:teuthology.orchestra.run.smithi038.stdout:(8/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 1.4 MB/s | 2.4 MB 00:01 2024-09-16T00:19:53.043 INFO:teuthology.orchestra.run.smithi038.stdout:(9/156): ceph-selinux-19.1.1-326.g8921c8ef.el9. 337 kB/s | 25 kB 00:00 2024-09-16T00:19:53.081 INFO:teuthology.orchestra.run.smithi073.stdout:(1/156): ceph-19.1.1-326.g8921c8ef.el9.x86_64.r 27 kB/s | 6.5 kB 00:00 2024-09-16T00:19:53.368 INFO:teuthology.orchestra.run.smithi122.stdout:(7/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 3.3 MB/s | 4.7 MB 00:01 2024-09-16T00:19:53.618 INFO:teuthology.orchestra.run.smithi122.stdout:(8/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86_ 10 MB/s | 17 MB 00:01 2024-09-16T00:19:53.623 INFO:teuthology.orchestra.run.smithi073.stdout:(2/156): ceph-fuse-19.1.1-326.g8921c8ef.el9.x86 1.6 MB/s | 896 kB 00:00 2024-09-16T00:19:53.669 INFO:teuthology.orchestra.run.smithi122.stdout:(9/156): ceph-selinux-19.1.1-326.g8921c8ef.el9. 504 kB/s | 25 kB 00:00 2024-09-16T00:19:53.724 INFO:teuthology.orchestra.run.smithi073.stdout:(3/156): ceph-immutable-object-cache-19.1.1-326 1.4 MB/s | 145 kB 00:00 2024-09-16T00:19:53.827 INFO:teuthology.orchestra.run.smithi038.stdout:(10/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9 8.0 MB/s | 11 MB 00:01 2024-09-16T00:19:54.002 INFO:teuthology.orchestra.run.smithi038.stdout:(11/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86 8.1 MB/s | 17 MB 00:02 2024-09-16T00:19:54.027 INFO:teuthology.orchestra.run.smithi038.stdout:(12/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 1.1 MB/s | 228 kB 00:00 2024-09-16T00:19:54.069 INFO:teuthology.orchestra.run.smithi038.stdout:(13/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 406 kB/s | 27 kB 00:00 2024-09-16T00:19:54.095 INFO:teuthology.orchestra.run.smithi038.stdout:(14/156): libcephfs-devel-19.1.1-326.g8921c8ef. 474 kB/s | 32 kB 00:00 2024-09-16T00:19:54.149 INFO:teuthology.orchestra.run.smithi073.stdout:(4/156): ceph-base-19.1.1-326.g8921c8ef.el9.x86 4.1 MB/s | 5.4 MB 00:01 2024-09-16T00:19:54.161 INFO:teuthology.orchestra.run.smithi038.stdout:(15/156): libcephsqlite-19.1.1-326.g8921c8ef.el 2.4 MB/s | 163 kB 00:00 2024-09-16T00:19:54.195 INFO:teuthology.orchestra.run.smithi038.stdout:(16/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 5.9 MB/s | 751 kB 00:00 2024-09-16T00:19:54.229 INFO:teuthology.orchestra.run.smithi038.stdout:(17/156): librados-devel-19.1.1-326.g8921c8ef.e 1.9 MB/s | 127 kB 00:00 2024-09-16T00:19:54.329 INFO:teuthology.orchestra.run.smithi038.stdout:(18/156): libradosstriper1-19.1.1-326.g8921c8ef 3.7 MB/s | 506 kB 00:00 2024-09-16T00:19:54.383 INFO:teuthology.orchestra.run.smithi073.stdout:(5/156): ceph-mgr-19.1.1-326.g8921c8ef.el9.x86_ 6.5 MB/s | 1.5 MB 00:00 2024-09-16T00:19:54.405 INFO:teuthology.orchestra.run.smithi038.stdout:(19/156): python3-ceph-argparse-19.1.1-326.g892 602 kB/s | 45 kB 00:00 2024-09-16T00:19:54.425 INFO:teuthology.orchestra.run.smithi073.stdout:(6/156): ceph-mds-19.1.1-326.g8921c8ef.el9.x86_ 3.4 MB/s | 2.4 MB 00:00 2024-09-16T00:19:54.480 INFO:teuthology.orchestra.run.smithi038.stdout:(20/156): python3-ceph-common-19.1.1-326.g8921c 1.8 MB/s | 136 kB 00:00 2024-09-16T00:19:54.563 INFO:teuthology.orchestra.run.smithi038.stdout:(21/156): python3-cephfs-19.1.1-326.g8921c8ef.e 1.9 MB/s | 161 kB 00:00 2024-09-16T00:19:54.664 INFO:teuthology.orchestra.run.smithi038.stdout:(22/156): python3-rados-19.1.1-326.g8921c8ef.el 3.1 MB/s | 320 kB 00:00 2024-09-16T00:19:54.764 INFO:teuthology.orchestra.run.smithi038.stdout:(23/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 2.9 MB/s | 300 kB 00:00 2024-09-16T00:19:54.856 INFO:teuthology.orchestra.run.smithi038.stdout:(24/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 1.1 MB/s | 99 kB 00:00 2024-09-16T00:19:54.948 INFO:teuthology.orchestra.run.smithi038.stdout:(25/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 928 kB/s | 85 kB 00:00 2024-09-16T00:19:55.033 INFO:teuthology.orchestra.run.smithi073.stdout:(7/156): ceph-mon-19.1.1-326.g8921c8ef.el9.x86_ 7.2 MB/s | 4.7 MB 00:00 2024-09-16T00:19:55.131 INFO:teuthology.orchestra.run.smithi038.stdout:(26/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 5.9 MB/s | 5.3 MB 00:00 2024-09-16T00:19:55.240 INFO:teuthology.orchestra.run.smithi038.stdout:(27/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 1.5 MB/s | 170 kB 00:00 2024-09-16T00:19:55.349 INFO:teuthology.orchestra.run.smithi038.stdout:(28/156): ceph-grafana-dashboards-19.1.1-326.g8 294 kB/s | 32 kB 00:00 2024-09-16T00:19:55.449 INFO:teuthology.orchestra.run.smithi038.stdout:(29/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 1.4 MB/s | 146 kB 00:00 2024-09-16T00:19:55.494 INFO:teuthology.orchestra.run.smithi122.stdout:(10/156): ceph-common-19.1.1-326.g8921c8ef.el9. 4.1 MB/s | 21 MB 00:05 2024-09-16T00:19:55.583 INFO:teuthology.orchestra.run.smithi038.stdout:(30/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 4.9 MB/s | 3.1 MB 00:00 2024-09-16T00:19:55.636 INFO:teuthology.orchestra.run.smithi122.stdout:(11/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 1.6 MB/s | 228 kB 00:00 2024-09-16T00:19:55.650 INFO:teuthology.orchestra.run.smithi073.stdout:(8/156): ceph-common-19.1.1-326.g8921c8ef.el9.x 7.5 MB/s | 21 MB 00:02 2024-09-16T00:19:55.736 INFO:teuthology.orchestra.run.smithi122.stdout:(12/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 270 kB/s | 27 kB 00:00 2024-09-16T00:19:55.750 INFO:teuthology.orchestra.run.smithi073.stdout:(9/156): ceph-selinux-19.1.1-326.g8921c8ef.el9. 252 kB/s | 25 kB 00:00 2024-09-16T00:19:55.828 INFO:teuthology.orchestra.run.smithi122.stdout:(13/156): libcephfs-devel-19.1.1-326.g8921c8ef. 347 kB/s | 32 kB 00:00 2024-09-16T00:19:56.092 INFO:teuthology.orchestra.run.smithi073.stdout:(10/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9 10 MB/s | 11 MB 00:01 2024-09-16T00:19:56.108 INFO:teuthology.orchestra.run.smithi038.stdout:(31/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 5.4 MB/s | 3.6 MB 00:00 2024-09-16T00:19:56.176 INFO:teuthology.orchestra.run.smithi073.stdout:(11/156): cephfs-mirror-19.1.1-326.g8921c8ef.el 2.7 MB/s | 228 kB 00:00 2024-09-16T00:19:56.200 INFO:teuthology.orchestra.run.smithi038.stdout:(32/156): ceph-mgr-modules-core-19.1.1-326.g892 2.7 MB/s | 249 kB 00:00 2024-09-16T00:19:56.268 INFO:teuthology.orchestra.run.smithi073.stdout:(12/156): cephfs-shell-19.1.1-326.g8921c8ef.el9 295 kB/s | 27 kB 00:00 2024-09-16T00:19:56.284 INFO:teuthology.orchestra.run.smithi038.stdout:(33/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 592 kB/s | 49 kB 00:00 2024-09-16T00:19:56.337 INFO:teuthology.orchestra.run.smithi122.stdout:(14/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 1.4 MB/s | 751 kB 00:00 2024-09-16T00:19:56.343 INFO:teuthology.orchestra.run.smithi073.stdout:(13/156): libcephfs-devel-19.1.1-326.g8921c8ef. 422 kB/s | 32 kB 00:00 2024-09-16T00:19:56.367 INFO:teuthology.orchestra.run.smithi038.stdout:(34/156): ceph-prometheus-alerts-19.1.1-326.g89 197 kB/s | 16 kB 00:00 2024-09-16T00:19:56.443 INFO:teuthology.orchestra.run.smithi073.stdout:(14/156): libcephfs2-19.1.1-326.g8921c8ef.el9.x 7.4 MB/s | 751 kB 00:00 2024-09-16T00:19:56.468 INFO:teuthology.orchestra.run.smithi038.stdout:(35/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 2.8 MB/s | 285 kB 00:00 2024-09-16T00:19:56.487 INFO:teuthology.orchestra.run.smithi122.stdout:(15/156): libcephsqlite-19.1.1-326.g8921c8ef.el 1.1 MB/s | 163 kB 00:00 2024-09-16T00:19:56.527 INFO:teuthology.orchestra.run.smithi073.stdout:(15/156): libcephsqlite-19.1.1-326.g8921c8ef.el 1.9 MB/s | 163 kB 00:00 2024-09-16T00:19:56.579 INFO:teuthology.orchestra.run.smithi122.stdout:(16/156): librados-devel-19.1.1-326.g8921c8ef.e 1.3 MB/s | 127 kB 00:00 2024-09-16T00:19:56.619 INFO:teuthology.orchestra.run.smithi073.stdout:(16/156): librados-devel-19.1.1-326.g8921c8ef.e 1.3 MB/s | 127 kB 00:00 2024-09-16T00:19:56.634 INFO:teuthology.orchestra.run.smithi038.stdout:(36/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 4.5 MB/s | 765 kB 00:00 2024-09-16T00:19:56.711 INFO:teuthology.orchestra.run.smithi073.stdout:(17/156): libradosstriper1-19.1.1-326.g8921c8ef 5.4 MB/s | 506 kB 00:00 2024-09-16T00:19:56.726 INFO:teuthology.orchestra.run.smithi038.stdout:(37/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 199 kB/s | 18 kB 00:00 2024-09-16T00:19:56.843 INFO:teuthology.orchestra.run.smithi038.stdout:(38/156): fuse-2.9.9-16.el9.x86_64.rpm 683 kB/s | 80 kB 00:00 2024-09-16T00:19:56.862 INFO:teuthology.orchestra.run.smithi122.stdout:(17/156): libradosstriper1-19.1.1-326.g8921c8ef 1.7 MB/s | 506 kB 00:00 2024-09-16T00:19:56.944 INFO:teuthology.orchestra.run.smithi038.stdout:(39/156): libgfortran-11.5.0-2.el9.x86_64.rpm 7.8 MB/s | 801 kB 00:00 2024-09-16T00:19:56.986 INFO:teuthology.orchestra.run.smithi038.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-16T00:19:57.028 INFO:teuthology.orchestra.run.smithi038.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.9 MB/s | 253 kB 00:00 2024-09-16T00:19:57.103 INFO:teuthology.orchestra.run.smithi038.stdout:(42/156): python3-cryptography-36.0.1-4.el9.x86 16 MB/s | 1.2 MB 00:00 2024-09-16T00:19:57.153 INFO:teuthology.orchestra.run.smithi038.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-16T00:19:57.194 INFO:teuthology.orchestra.run.smithi073.stdout:(18/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 11 MB/s | 5.3 MB 00:00 2024-09-16T00:19:57.203 INFO:teuthology.orchestra.run.smithi038.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 2.7 MB/s | 135 kB 00:00 2024-09-16T00:19:57.261 INFO:teuthology.orchestra.run.smithi073.stdout:(19/156): python3-ceph-argparse-19.1.1-326.g892 677 kB/s | 45 kB 00:00 2024-09-16T00:19:57.295 INFO:teuthology.orchestra.run.smithi038.stdout:(45/156): ceph-mgr-diskprediction-local-19.1.1- 4.3 MB/s | 7.4 MB 00:01 2024-09-16T00:19:57.321 INFO:teuthology.orchestra.run.smithi038.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-09-16T00:19:57.371 INFO:teuthology.orchestra.run.smithi038.stdout:(47/156): automake-1.16.2-8.el9.noarch.rpm 14 MB/s | 689 kB 00:00 2024-09-16T00:19:57.413 INFO:teuthology.orchestra.run.smithi122.stdout:(18/156): ceph-radosgw-19.1.1-326.g8921c8ef.el9 2.6 MB/s | 11 MB 00:04 2024-09-16T00:19:57.429 INFO:teuthology.orchestra.run.smithi038.stdout:(48/156): bison-3.7.4-5.el9.x86_64.rpm 16 MB/s | 943 kB 00:00 2024-09-16T00:19:57.445 INFO:teuthology.orchestra.run.smithi073.stdout:(20/156): ceph-osd-19.1.1-326.g8921c8ef.el9.x86 5.6 MB/s | 17 MB 00:03 2024-09-16T00:19:57.455 INFO:teuthology.orchestra.run.smithi038.stdout:(49/156): python3-urllib3-1.26.5-6.el9.noarch.r 1.3 MB/s | 215 kB 00:00 2024-09-16T00:19:57.471 INFO:teuthology.orchestra.run.smithi122.stdout:(19/156): python3-ceph-argparse-19.1.1-326.g892 777 kB/s | 45 kB 00:00 2024-09-16T00:19:57.480 INFO:teuthology.orchestra.run.smithi038.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-09-16T00:19:57.486 INFO:teuthology.orchestra.run.smithi073.stdout:(21/156): python3-ceph-common-19.1.1-326.g8921c 605 kB/s | 136 kB 00:00 2024-09-16T00:19:57.503 INFO:teuthology.orchestra.run.smithi073.stdout:(22/156): python3-cephfs-19.1.1-326.g8921c8ef.e 2.7 MB/s | 161 kB 00:00 2024-09-16T00:19:57.522 INFO:teuthology.orchestra.run.smithi038.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 325 kB/s | 22 kB 00:00 2024-09-16T00:19:57.556 INFO:teuthology.orchestra.run.smithi038.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 313 kB 00:00 2024-09-16T00:19:57.562 INFO:teuthology.orchestra.run.smithi073.stdout:(23/156): python3-rados-19.1.1-326.g8921c8ef.el 4.2 MB/s | 320 kB 00:00 2024-09-16T00:19:57.573 INFO:teuthology.orchestra.run.smithi038.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 652 kB/s | 33 kB 00:00 2024-09-16T00:19:57.580 INFO:teuthology.orchestra.run.smithi122.stdout:(20/156): python3-ceph-common-19.1.1-326.g8921c 1.2 MB/s | 136 kB 00:00 2024-09-16T00:19:57.587 INFO:teuthology.orchestra.run.smithi073.stdout:(24/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 3.5 MB/s | 300 kB 00:00 2024-09-16T00:19:57.606 INFO:teuthology.orchestra.run.smithi038.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 550 kB/s | 18 kB 00:00 2024-09-16T00:19:57.630 INFO:teuthology.orchestra.run.smithi073.stdout:(25/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 1.5 MB/s | 99 kB 00:00 2024-09-16T00:19:57.655 INFO:teuthology.orchestra.run.smithi073.stdout:(26/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 1.2 MB/s | 85 kB 00:00 2024-09-16T00:19:57.680 INFO:teuthology.orchestra.run.smithi122.stdout:(21/156): python3-cephfs-19.1.1-326.g8921c8ef.e 1.6 MB/s | 161 kB 00:00 2024-09-16T00:19:57.681 INFO:teuthology.orchestra.run.smithi038.stdout:(55/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 24 MB/s | 3.0 MB 00:00 2024-09-16T00:19:57.699 INFO:teuthology.orchestra.run.smithi038.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 121 kB/s | 11 kB 00:00 2024-09-16T00:19:57.732 INFO:teuthology.orchestra.run.smithi038.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-09-16T00:19:57.741 INFO:teuthology.orchestra.run.smithi073.stdout:(27/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 2.0 MB/s | 170 kB 00:00 2024-09-16T00:19:57.757 INFO:teuthology.orchestra.run.smithi038.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 777 kB/s | 45 kB 00:00 2024-09-16T00:19:57.791 INFO:teuthology.orchestra.run.smithi038.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2024-09-16T00:19:57.797 INFO:teuthology.orchestra.run.smithi073.stdout:(28/156): ceph-grafana-dashboards-19.1.1-326.g8 545 kB/s | 32 kB 00:00 2024-09-16T00:19:57.833 INFO:teuthology.orchestra.run.smithi038.stdout:(60/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 583 kB/s | 24 kB 00:00 2024-09-16T00:19:57.867 INFO:teuthology.orchestra.run.smithi038.stdout:(61/156): libtool-2.4.6-46.el9.x86_64.rpm 5.2 MB/s | 581 kB 00:00 2024-09-16T00:19:57.880 INFO:teuthology.orchestra.run.smithi073.stdout:(29/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 12 MB/s | 3.1 MB 00:00 2024-09-16T00:19:57.892 INFO:teuthology.orchestra.run.smithi038.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 812 kB/s | 47 kB 00:00 2024-09-16T00:19:57.906 INFO:teuthology.orchestra.run.smithi073.stdout:(30/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 1.3 MB/s | 146 kB 00:00 2024-09-16T00:19:57.918 INFO:teuthology.orchestra.run.smithi038.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-16T00:19:57.943 INFO:teuthology.orchestra.run.smithi038.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 3.7 MB/s | 188 kB 00:00 2024-09-16T00:19:57.968 INFO:teuthology.orchestra.run.smithi038.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 769 kB/s | 38 kB 00:00 2024-09-16T00:19:58.060 INFO:teuthology.orchestra.run.smithi038.stdout:(66/156): openblas-openmp-0.3.26-2.el9.x86_64.r 42 MB/s | 4.9 MB 00:00 2024-09-16T00:19:58.123 INFO:teuthology.orchestra.run.smithi073.stdout:(31/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 15 MB/s | 3.6 MB 00:00 2024-09-16T00:19:58.144 INFO:teuthology.orchestra.run.smithi038.stdout:(67/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 25 MB/s | 4.4 MB 00:00 2024-09-16T00:19:58.181 INFO:teuthology.orchestra.run.smithi073.stdout:(32/156): ceph-mgr-modules-core-19.1.1-326.g892 4.2 MB/s | 249 kB 00:00 2024-09-16T00:19:58.186 INFO:teuthology.orchestra.run.smithi038.stdout:(68/156): postgresql-13.16-1.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-16T00:19:58.189 INFO:teuthology.orchestra.run.smithi122.stdout:(22/156): ceph-test-19.1.1-326.g8921c8ef.el9.x8 11 MB/s | 49 MB 00:04 2024-09-16T00:19:58.219 INFO:teuthology.orchestra.run.smithi038.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 12 MB/s | 910 kB 00:00 2024-09-16T00:19:58.222 INFO:teuthology.orchestra.run.smithi122.stdout:(23/156): python3-rados-19.1.1-326.g8921c8ef.el 591 kB/s | 320 kB 00:00 2024-09-16T00:19:58.240 INFO:teuthology.orchestra.run.smithi073.stdout:(33/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 843 kB/s | 49 kB 00:00 2024-09-16T00:19:58.245 INFO:teuthology.orchestra.run.smithi038.stdout:(70/156): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-16T00:19:58.256 INFO:teuthology.orchestra.run.smithi122.stdout:(24/156): python3-rbd-19.1.1-326.g8921c8ef.el9. 4.4 MB/s | 300 kB 00:00 2024-09-16T00:19:58.299 INFO:teuthology.orchestra.run.smithi073.stdout:(34/156): ceph-prometheus-alerts-19.1.1-326.g89 281 kB/s | 16 kB 00:00 2024-09-16T00:19:58.303 INFO:teuthology.orchestra.run.smithi038.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 17 MB/s | 1.0 MB 00:00 2024-09-16T00:19:58.323 INFO:teuthology.orchestra.run.smithi122.stdout:(25/156): rbd-fuse-19.1.1-326.g8921c8ef.el9.x86 1.2 MB/s | 85 kB 00:00 2024-09-16T00:19:58.357 INFO:teuthology.orchestra.run.smithi122.stdout:(26/156): python3-rgw-19.1.1-326.g8921c8ef.el9. 744 kB/s | 99 kB 00:00 2024-09-16T00:19:58.366 INFO:teuthology.orchestra.run.smithi073.stdout:(35/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 4.2 MB/s | 285 kB 00:00 2024-09-16T00:19:58.387 INFO:teuthology.orchestra.run.smithi038.stdout:(72/156): postgresql-server-13.16-1.el9.x86_64. 35 MB/s | 5.8 MB 00:00 2024-09-16T00:19:58.412 INFO:teuthology.orchestra.run.smithi038.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 807 kB/s | 87 kB 00:00 2024-09-16T00:19:58.454 INFO:teuthology.orchestra.run.smithi038.stdout:(74/156): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-16T00:19:58.459 INFO:teuthology.orchestra.run.smithi073.stdout:(36/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 8.0 MB/s | 765 kB 00:00 2024-09-16T00:19:58.474 INFO:teuthology.orchestra.run.smithi122.stdout:(27/156): rbd-nbd-19.1.1-326.g8921c8ef.el9.x86_ 1.4 MB/s | 170 kB 00:00 2024-09-16T00:19:58.526 INFO:teuthology.orchestra.run.smithi073.stdout:(37/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 274 kB/s | 18 kB 00:00 2024-09-16T00:19:58.538 INFO:teuthology.orchestra.run.smithi038.stdout:(75/156): python3-babel-2.9.1-2.el9.noarch.rpm 40 MB/s | 6.0 MB 00:00 2024-09-16T00:19:58.557 INFO:teuthology.orchestra.run.smithi122.stdout:(28/156): ceph-grafana-dashboards-19.1.1-326.g8 382 kB/s | 32 kB 00:00 2024-09-16T00:19:58.563 INFO:teuthology.orchestra.run.smithi038.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 2.2 MB/s | 245 kB 00:00 2024-09-16T00:19:58.588 INFO:teuthology.orchestra.run.smithi038.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.9 MB/s | 248 kB 00:00 2024-09-16T00:19:58.610 INFO:teuthology.orchestra.run.smithi073.stdout:(38/156): fuse-2.9.9-16.el9.x86_64.rpm 958 kB/s | 80 kB 00:00 2024-09-16T00:19:58.613 INFO:teuthology.orchestra.run.smithi038.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 510 kB/s | 26 kB 00:00 2024-09-16T00:19:58.616 INFO:teuthology.orchestra.run.smithi122.stdout:(29/156): rbd-mirror-19.1.1-326.g8921c8ef.el9.x 11 MB/s | 3.1 MB 00:00 2024-09-16T00:19:58.639 INFO:teuthology.orchestra.run.smithi038.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 380 kB/s | 19 kB 00:00 2024-09-16T00:19:58.664 INFO:teuthology.orchestra.run.smithi038.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-09-16T00:19:58.675 INFO:teuthology.orchestra.run.smithi122.stdout:(30/156): ceph-mgr-cephadm-19.1.1-326.g8921c8ef 1.2 MB/s | 146 kB 00:00 2024-09-16T00:19:58.690 INFO:teuthology.orchestra.run.smithi038.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-09-16T00:19:58.702 INFO:teuthology.orchestra.run.smithi073.stdout:(39/156): libgfortran-11.5.0-2.el9.x86_64.rpm 8.5 MB/s | 801 kB 00:00 2024-09-16T00:19:58.740 INFO:teuthology.orchestra.run.smithi038.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.9 MB/s | 250 kB 00:00 2024-09-16T00:19:58.786 INFO:teuthology.orchestra.run.smithi073.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 2.2 MB/s | 191 kB 00:00 2024-09-16T00:19:58.807 INFO:teuthology.orchestra.run.smithi038.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 36 MB/s | 5.1 MB 00:00 2024-09-16T00:19:58.832 INFO:teuthology.orchestra.run.smithi038.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-09-16T00:19:58.857 INFO:teuthology.orchestra.run.smithi038.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-16T00:19:58.874 INFO:teuthology.orchestra.run.smithi038.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-16T00:19:58.886 INFO:teuthology.orchestra.run.smithi073.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 2.5 MB/s | 253 kB 00:00 2024-09-16T00:19:58.899 INFO:teuthology.orchestra.run.smithi038.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 6.3 MB/s | 269 kB 00:00 2024-09-16T00:19:58.925 INFO:teuthology.orchestra.run.smithi038.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-16T00:19:58.933 INFO:teuthology.orchestra.run.smithi122.stdout:(31/156): ceph-mgr-dashboard-19.1.1-326.g8921c8 11 MB/s | 3.6 MB 00:00 2024-09-16T00:19:58.959 INFO:teuthology.orchestra.run.smithi038.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.6 MB/s | 279 kB 00:00 2024-09-16T00:19:58.978 INFO:teuthology.orchestra.run.smithi073.stdout:(42/156): ceph-mgr-diskprediction-local-19.1.1- 6.9 MB/s | 7.4 MB 00:01 2024-09-16T00:19:58.984 INFO:teuthology.orchestra.run.smithi038.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 867 kB/s | 51 kB 00:00 2024-09-16T00:19:58.985 INFO:teuthology.orchestra.run.smithi122.stdout:(32/156): ceph-mgr-modules-core-19.1.1-326.g892 4.9 MB/s | 249 kB 00:00 2024-09-16T00:19:59.009 INFO:teuthology.orchestra.run.smithi038.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.0 MB/s | 54 kB 00:00 2024-09-16T00:19:59.042 INFO:teuthology.orchestra.run.smithi122.stdout:(33/156): ceph-mgr-rook-19.1.1-326.g8921c8ef.el 848 kB/s | 49 kB 00:00 2024-09-16T00:19:59.070 INFO:teuthology.orchestra.run.smithi073.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-09-16T00:19:59.120 INFO:teuthology.orchestra.run.smithi073.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-16T00:19:59.134 INFO:teuthology.orchestra.run.smithi122.stdout:(34/156): librgw2-19.1.1-326.g8921c8ef.el9.x86_ 2.4 MB/s | 5.3 MB 00:02 2024-09-16T00:19:59.151 INFO:teuthology.orchestra.run.smithi038.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 294 kB/s | 42 kB 00:00 2024-09-16T00:19:59.154 INFO:teuthology.orchestra.run.smithi073.stdout:(45/156): python3-cryptography-36.0.1-4.el9.x86 4.6 MB/s | 1.2 MB 00:00 2024-09-16T00:19:59.159 INFO:teuthology.orchestra.run.smithi122.stdout:(35/156): ceph-prometheus-alerts-19.1.1-326.g89 141 kB/s | 16 kB 00:00 2024-09-16T00:19:59.179 INFO:teuthology.orchestra.run.smithi073.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 2.1 MB/s | 125 kB 00:00 2024-09-16T00:19:59.193 INFO:teuthology.orchestra.run.smithi038.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.0 MB/s | 44 kB 00:00 2024-09-16T00:19:59.204 INFO:teuthology.orchestra.run.smithi073.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-16T00:19:59.235 INFO:teuthology.orchestra.run.smithi038.stdout:(94/156): qatlib-24.02.0-1.el9.x86_64.rpm 5.2 MB/s | 221 kB 00:00 2024-09-16T00:19:59.276 INFO:teuthology.orchestra.run.smithi122.stdout:(36/156): cephadm-19.1.1-326.g8921c8ef.el9.noar 6.4 MB/s | 765 kB 00:00 2024-09-16T00:19:59.277 INFO:teuthology.orchestra.run.smithi038.stdout:(95/156): qatlib-service-24.02.0-1.el9.x86_64.r 867 kB/s | 36 kB 00:00 2024-09-16T00:19:59.302 INFO:teuthology.orchestra.run.smithi122.stdout:(37/156): ceph-volume-19.1.1-326.g8921c8ef.el9. 1.7 MB/s | 285 kB 00:00 2024-09-16T00:19:59.321 INFO:teuthology.orchestra.run.smithi073.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 4.7 MB/s | 689 kB 00:00 2024-09-16T00:19:59.335 INFO:teuthology.orchestra.run.smithi122.stdout:(38/156): cephfs-top-19.1.1-326.g8921c8ef.el9.n 311 kB/s | 18 kB 00:00 2024-09-16T00:19:59.372 INFO:teuthology.orchestra.run.smithi073.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 5.5 MB/s | 943 kB 00:00 2024-09-16T00:19:59.394 INFO:teuthology.orchestra.run.smithi122.stdout:(39/156): fuse-2.9.9-16.el9.x86_64.rpm 869 kB/s | 80 kB 00:00 2024-09-16T00:19:59.397 INFO:teuthology.orchestra.run.smithi073.stdout:(50/156): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 107 kB 00:00 2024-09-16T00:19:59.414 INFO:teuthology.orchestra.run.smithi073.stdout:(51/156): elfutils-libelf-devel-0.191-4.el9.x86 520 kB/s | 22 kB 00:00 2024-09-16T00:19:59.439 INFO:teuthology.orchestra.run.smithi073.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 7.3 MB/s | 313 kB 00:00 2024-09-16T00:19:59.444 INFO:teuthology.orchestra.run.smithi038.stdout:(96/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 35 MB/s | 16 MB 00:00 2024-09-16T00:19:59.456 INFO:teuthology.orchestra.run.smithi073.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 780 kB/s | 33 kB 00:00 2024-09-16T00:19:59.469 INFO:teuthology.orchestra.run.smithi038.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 247 kB/s | 47 kB 00:00 2024-09-16T00:19:59.471 INFO:teuthology.orchestra.run.smithi122.stdout:(40/156): libquadmath-11.5.0-2.el9.x86_64.rpm 2.4 MB/s | 191 kB 00:00 2024-09-16T00:19:59.495 INFO:teuthology.orchestra.run.smithi038.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-16T00:19:59.520 INFO:teuthology.orchestra.run.smithi038.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-16T00:19:59.538 INFO:teuthology.orchestra.run.smithi122.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.7 MB/s | 253 kB 00:00 2024-09-16T00:19:59.545 INFO:teuthology.orchestra.run.smithi038.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-16T00:19:59.571 INFO:teuthology.orchestra.run.smithi038.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-16T00:19:59.571 INFO:teuthology.orchestra.run.smithi122.stdout:(42/156): libgfortran-11.5.0-2.el9.x86_64.rpm 3.3 MB/s | 801 kB 00:00 2024-09-16T00:19:59.596 INFO:teuthology.orchestra.run.smithi038.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 894 kB/s | 45 kB 00:00 2024-09-16T00:19:59.622 INFO:teuthology.orchestra.run.smithi122.stdout:(43/156): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-09-16T00:19:59.647 INFO:teuthology.orchestra.run.smithi122.stdout:(44/156): python3-ply-3.11-14.el9.noarch.rpm 1.4 MB/s | 106 kB 00:00 2024-09-16T00:19:59.689 INFO:teuthology.orchestra.run.smithi122.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 2.0 MB/s | 135 kB 00:00 2024-09-16T00:19:59.704 INFO:teuthology.orchestra.run.smithi038.stdout:(103/156): lua-devel-5.4.4-4.el9.x86_64.rpm 167 kB/s | 22 kB 00:00 2024-09-16T00:19:59.723 INFO:teuthology.orchestra.run.smithi122.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.6 MB/s | 125 kB 00:00 2024-09-16T00:19:59.765 INFO:teuthology.orchestra.run.smithi122.stdout:(47/156): python3-urllib3-1.26.5-6.el9.noarch.r 2.8 MB/s | 215 kB 00:00 2024-09-16T00:19:59.771 INFO:teuthology.orchestra.run.smithi038.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 8.0 MB/s | 548 kB 00:00 2024-09-16T00:19:59.805 INFO:teuthology.orchestra.run.smithi038.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 9.1 MB/s | 308 kB 00:00 2024-09-16T00:19:59.830 INFO:teuthology.orchestra.run.smithi038.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 785 kB/s | 19 kB 00:00 2024-09-16T00:19:59.890 INFO:teuthology.orchestra.run.smithi073.stdout:(54/156): ceph-test-19.1.1-326.g8921c8ef.el9.x8 12 MB/s | 49 MB 00:04 2024-09-16T00:19:59.907 INFO:teuthology.orchestra.run.smithi073.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 40 kB/s | 18 kB 00:00 2024-09-16T00:19:59.931 INFO:teuthology.orchestra.run.smithi038.stdout:(107/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.5 MB/s | 863 kB 00:00 2024-09-16T00:19:59.940 INFO:teuthology.orchestra.run.smithi073.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-09-16T00:19:59.965 INFO:teuthology.orchestra.run.smithi073.stdout:(57/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-09-16T00:20:00.006 INFO:teuthology.orchestra.run.smithi038.stdout:(108/156): libarrow-9.0.0-11.el9.x86_64.rpm 25 MB/s | 4.4 MB 00:00 2024-09-16T00:20:00.016 INFO:teuthology.orchestra.run.smithi073.stdout:(58/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.2 MB/s | 3.0 MB 00:00 2024-09-16T00:20:00.031 INFO:teuthology.orchestra.run.smithi038.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 2.1 MB/s | 49 kB 00:00 2024-09-16T00:20:00.041 INFO:teuthology.orchestra.run.smithi073.stdout:(59/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 450 kB/s | 45 kB 00:00 2024-09-16T00:20:00.056 INFO:teuthology.orchestra.run.smithi038.stdout:(110/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 207 kB/s | 26 kB 00:00 2024-09-16T00:20:00.081 INFO:teuthology.orchestra.run.smithi038.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.3 MB/s | 67 kB 00:00 2024-09-16T00:20:00.083 INFO:teuthology.orchestra.run.smithi073.stdout:(60/156): libtool-2.4.6-46.el9.x86_64.rpm 8.5 MB/s | 581 kB 00:00 2024-09-16T00:20:00.107 INFO:teuthology.orchestra.run.smithi038.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-09-16T00:20:00.117 INFO:teuthology.orchestra.run.smithi073.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 325 kB/s | 24 kB 00:00 2024-09-16T00:20:00.142 INFO:teuthology.orchestra.run.smithi073.stdout:(62/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.7 MB/s | 662 kB 00:00 2024-09-16T00:20:00.148 INFO:teuthology.orchestra.run.smithi038.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 12 MB/s | 839 kB 00:00 2024-09-16T00:20:00.167 INFO:teuthology.orchestra.run.smithi073.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 565 kB/s | 47 kB 00:00 2024-09-16T00:20:00.174 INFO:teuthology.orchestra.run.smithi038.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-16T00:20:00.192 INFO:teuthology.orchestra.run.smithi073.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-09-16T00:20:00.201 INFO:teuthology.orchestra.run.smithi038.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-16T00:20:00.218 INFO:teuthology.orchestra.run.smithi073.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 766 kB/s | 38 kB 00:00 2024-09-16T00:20:00.224 INFO:teuthology.orchestra.run.smithi038.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 567 kB/s | 14 kB 00:00 2024-09-16T00:20:00.243 INFO:teuthology.orchestra.run.smithi073.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.8 MB/s | 188 kB 00:00 2024-09-16T00:20:00.258 INFO:teuthology.orchestra.run.smithi038.stdout:(117/156): python3-asyncssh-2.13.2-5.el9.noarch 3.5 MB/s | 548 kB 00:00 2024-09-16T00:20:00.283 INFO:teuthology.orchestra.run.smithi038.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-16T00:20:00.317 INFO:teuthology.orchestra.run.smithi038.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-16T00:20:00.343 INFO:teuthology.orchestra.run.smithi038.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-16T00:20:00.368 INFO:teuthology.orchestra.run.smithi038.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-16T00:20:00.385 INFO:teuthology.orchestra.run.smithi073.stdout:(67/156): openblas-openmp-0.3.26-2.el9.x86_64.r 26 MB/s | 4.9 MB 00:00 2024-09-16T00:20:00.393 INFO:teuthology.orchestra.run.smithi038.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-16T00:20:00.410 INFO:teuthology.orchestra.run.smithi038.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 648 kB/s | 11 kB 00:00 2024-09-16T00:20:00.427 INFO:teuthology.orchestra.run.smithi038.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 1.0 MB/s | 18 kB 00:00 2024-09-16T00:20:00.444 INFO:teuthology.orchestra.run.smithi038.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 1.4 MB/s | 23 kB 00:00 2024-09-16T00:20:00.452 INFO:teuthology.orchestra.run.smithi073.stdout:(68/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-09-16T00:20:00.461 INFO:teuthology.orchestra.run.smithi038.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 1.1 MB/s | 19 kB 00:00 2024-09-16T00:20:00.485 INFO:teuthology.orchestra.run.smithi073.stdout:(69/156): postgresql-contrib-13.16-1.el9.x86_64 8.9 MB/s | 910 kB 00:00 2024-09-16T00:20:00.486 INFO:teuthology.orchestra.run.smithi038.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 790 kB/s | 20 kB 00:00 2024-09-16T00:20:00.490 INFO:teuthology.orchestra.run.smithi122.stdout:(48/156): ceph-mgr-diskprediction-local-19.1.1- 4.1 MB/s | 7.4 MB 00:01 2024-09-16T00:20:00.527 INFO:teuthology.orchestra.run.smithi073.stdout:(70/156): postgresql-13.16-1.el9.x86_64.rpm 5.6 MB/s | 1.6 MB 00:00 2024-09-16T00:20:00.545 INFO:teuthology.orchestra.run.smithi038.stdout:(128/156): python3-kubernetes-26.1.0-2.el9.noar 18 MB/s | 1.0 MB 00:00 2024-09-16T00:20:00.553 INFO:teuthology.orchestra.run.smithi073.stdout:(71/156): postgresql-private-libs-13.16-1.el9.x 1.3 MB/s | 137 kB 00:00 2024-09-16T00:20:00.570 INFO:teuthology.orchestra.run.smithi038.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-16T00:20:00.596 INFO:teuthology.orchestra.run.smithi073.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 2.1 MB/s | 87 kB 00:00 2024-09-16T00:20:00.613 INFO:teuthology.orchestra.run.smithi038.stdout:(130/156): python3-grpcio-1.46.7-10.el9.x86_64. 7.6 MB/s | 2.0 MB 00:00 2024-09-16T00:20:00.628 INFO:teuthology.orchestra.run.smithi073.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-09-16T00:20:00.637 INFO:teuthology.orchestra.run.smithi038.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.1 MB/s | 79 kB 00:00 2024-09-16T00:20:00.663 INFO:teuthology.orchestra.run.smithi038.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-16T00:20:00.688 INFO:teuthology.orchestra.run.smithi038.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 660 kB/s | 16 kB 00:00 2024-09-16T00:20:00.690 INFO:teuthology.orchestra.run.smithi122.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 712 kB/s | 689 kB 00:00 2024-09-16T00:20:00.712 INFO:teuthology.orchestra.run.smithi073.stdout:(74/156): postgresql-server-13.16-1.el9.x86_64. 26 MB/s | 5.8 MB 00:00 2024-09-16T00:20:00.713 INFO:teuthology.orchestra.run.smithi038.stdout:(134/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.6 MB/s | 90 kB 00:00 2024-09-16T00:20:00.737 INFO:teuthology.orchestra.run.smithi073.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 564 kB/s | 61 kB 00:00 2024-09-16T00:20:00.747 INFO:teuthology.orchestra.run.smithi038.stdout:(135/156): python3-pecan-1.4.2-3.el9.noarch.rpm 2.4 MB/s | 272 kB 00:00 2024-09-16T00:20:00.762 INFO:teuthology.orchestra.run.smithi073.stdout:(76/156): python3-devel-3.9.19-8.el9.x86_64.rpm 4.8 MB/s | 245 kB 00:00 2024-09-16T00:20:00.772 INFO:teuthology.orchestra.run.smithi038.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 483 kB/s | 28 kB 00:00 2024-09-16T00:20:00.788 INFO:teuthology.orchestra.run.smithi073.stdout:(77/156): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-16T00:20:00.797 INFO:teuthology.orchestra.run.smithi038.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 615 kB/s | 31 kB 00:00 2024-09-16T00:20:00.813 INFO:teuthology.orchestra.run.smithi073.stdout:(78/156): python3-jsonpatch-1.21-16.el9.noarch. 509 kB/s | 26 kB 00:00 2024-09-16T00:20:00.823 INFO:teuthology.orchestra.run.smithi038.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-16T00:20:00.838 INFO:teuthology.orchestra.run.smithi073.stdout:(79/156): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-09-16T00:20:00.857 INFO:teuthology.orchestra.run.smithi038.stdout:(139/156): python3-tempora-5.0.0-2.el9.noarch.r 1.0 MB/s | 36 kB 00:00 2024-09-16T00:20:00.863 INFO:teuthology.orchestra.run.smithi073.stdout:(80/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-16T00:20:00.882 INFO:teuthology.orchestra.run.smithi038.stdout:(140/156): python3-routes-2.5.1-5.el9.noarch.rp 1.7 MB/s | 188 kB 00:00 2024-09-16T00:20:00.888 INFO:teuthology.orchestra.run.smithi073.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-09-16T00:20:00.907 INFO:teuthology.orchestra.run.smithi038.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-09-16T00:20:00.933 INFO:teuthology.orchestra.run.smithi038.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 3.5 MB/s | 90 kB 00:00 2024-09-16T00:20:00.958 INFO:teuthology.orchestra.run.smithi038.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-16T00:20:00.981 INFO:teuthology.orchestra.run.smithi073.stdout:(82/156): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-09-16T00:20:00.983 INFO:teuthology.orchestra.run.smithi038.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 801 kB/s | 20 kB 00:00 2024-09-16T00:20:01.006 INFO:teuthology.orchestra.run.smithi073.stdout:(83/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-09-16T00:20:01.007 INFO:teuthology.orchestra.run.smithi122.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 759 kB/s | 943 kB 00:01 2024-09-16T00:20:01.025 INFO:teuthology.orchestra.run.smithi038.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 4.5 MB/s | 427 kB 00:00 2024-09-16T00:20:01.031 INFO:teuthology.orchestra.run.smithi073.stdout:(84/156): python3-oauthlib-3.1.1-5.el9.noarch.r 4.4 MB/s | 222 kB 00:00 2024-09-16T00:20:01.050 INFO:teuthology.orchestra.run.smithi038.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 2.8 MB/s | 191 kB 00:00 2024-09-16T00:20:01.056 INFO:teuthology.orchestra.run.smithi073.stdout:(85/156): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-16T00:20:01.082 INFO:teuthology.orchestra.run.smithi073.stdout:(86/156): python3-prettytable-0.7.2-27.el9.noar 838 kB/s | 42 kB 00:00 2024-09-16T00:20:01.107 INFO:teuthology.orchestra.run.smithi073.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-09-16T00:20:01.109 INFO:teuthology.orchestra.run.smithi038.stdout:(147/156): thrift-0.15.0-2.el9.x86_64.rpm 19 MB/s | 1.6 MB 00:00 2024-09-16T00:20:01.132 INFO:teuthology.orchestra.run.smithi073.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-16T00:20:01.157 INFO:teuthology.orchestra.run.smithi073.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-16T00:20:01.174 INFO:teuthology.orchestra.run.smithi122.stdout:(51/156): boost-program-options-1.75.0-8.el9.x8 156 kB/s | 107 kB 00:00 2024-09-16T00:20:01.183 INFO:teuthology.orchestra.run.smithi073.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-16T00:20:01.200 INFO:teuthology.orchestra.run.smithi073.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 1.2 MB/s | 54 kB 00:00 2024-09-16T00:20:01.241 INFO:teuthology.orchestra.run.smithi122.stdout:(52/156): elfutils-libelf-devel-0.191-4.el9.x86 39 kB/s | 22 kB 00:00 2024-09-16T00:20:01.275 INFO:teuthology.orchestra.run.smithi073.stdout:(92/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-16T00:20:01.292 INFO:teuthology.orchestra.run.smithi073.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 453 kB/s | 42 kB 00:00 2024-09-16T00:20:01.309 INFO:teuthology.orchestra.run.smithi073.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.3 MB/s | 44 kB 00:00 2024-09-16T00:20:01.334 INFO:teuthology.orchestra.run.smithi073.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 5.1 MB/s | 221 kB 00:00 2024-09-16T00:20:01.351 INFO:teuthology.orchestra.run.smithi073.stdout:(96/156): qatlib-service-24.02.0-1.el9.x86_64.r 858 kB/s | 36 kB 00:00 2024-09-16T00:20:01.368 INFO:teuthology.orchestra.run.smithi073.stdout:(97/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-09-16T00:20:01.393 INFO:teuthology.orchestra.run.smithi073.stdout:(98/156): socat-1.7.4.1-6.el9.x86_64.rpm 7.1 MB/s | 304 kB 00:00 2024-09-16T00:20:01.410 INFO:teuthology.orchestra.run.smithi073.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 1.3 MB/s | 58 kB 00:00 2024-09-16T00:20:01.437 INFO:teuthology.orchestra.run.smithi073.stdout:(100/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 2.2 MB/s | 96 kB 00:00 2024-09-16T00:20:01.461 INFO:teuthology.orchestra.run.smithi073.stdout:(101/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-16T00:20:01.486 INFO:teuthology.orchestra.run.smithi073.stdout:(102/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 892 kB/s | 45 kB 00:00 2024-09-16T00:20:01.733 INFO:teuthology.orchestra.run.smithi122.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 59 kB/s | 33 kB 00:00 2024-09-16T00:20:01.759 INFO:teuthology.orchestra.run.smithi038.stdout:(148/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 4.9 MB/s | 3.2 MB 00:00 2024-09-16T00:20:01.761 INFO:teuthology.orchestra.run.smithi073.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-09-16T00:20:01.816 INFO:teuthology.orchestra.run.smithi122.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 388 kB/s | 313 kB 00:00 2024-09-16T00:20:01.820 INFO:teuthology.orchestra.run.smithi073.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 9.2 MB/s | 548 kB 00:00 2024-09-16T00:20:01.845 INFO:teuthology.orchestra.run.smithi073.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 12 MB/s | 308 kB 00:00 2024-09-16T00:20:01.887 INFO:teuthology.orchestra.run.smithi073.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 688 kB/s | 19 kB 00:00 2024-09-16T00:20:01.943 INFO:teuthology.orchestra.run.smithi038.stdout:(149/156): NetworkManager-1.49.90-1.el9.x86_64. 13 MB/s | 2.3 MB 00:00 2024-09-16T00:20:02.018 INFO:teuthology.orchestra.run.smithi038.stdout:(150/156): NetworkManager-libnm-1.49.90-1.el9.x 24 MB/s | 1.8 MB 00:00 2024-09-16T00:20:02.071 INFO:teuthology.orchestra.run.smithi073.stdout:(107/156): libarrow-9.0.0-11.el9.x86_64.rpm 24 MB/s | 4.4 MB 00:00 2024-09-16T00:20:02.077 INFO:teuthology.orchestra.run.smithi038.stdout:(151/156): librados2-19.1.1-326.g8921c8ef.el9.x 3.3 MB/s | 3.4 MB 00:01 2024-09-16T00:20:02.096 INFO:teuthology.orchestra.run.smithi073.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-16T00:20:02.119 INFO:teuthology.orchestra.run.smithi038.stdout:(152/156): NetworkManager-team-1.49.90-1.el9.x8 406 kB/s | 41 kB 00:00 2024-09-16T00:20:02.121 INFO:teuthology.orchestra.run.smithi073.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-16T00:20:02.147 INFO:teuthology.orchestra.run.smithi073.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-16T00:20:02.172 INFO:teuthology.orchestra.run.smithi073.stdout:(111/156): lua-devel-5.4.4-4.el9.x86_64.rpm 31 kB/s | 22 kB 00:00 2024-09-16T00:20:02.186 INFO:teuthology.orchestra.run.smithi038.stdout:(153/156): openssl-3.2.2-6.el9.x86_64.rpm 20 MB/s | 1.4 MB 00:00 2024-09-16T00:20:02.198 INFO:teuthology.orchestra.run.smithi073.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-09-16T00:20:02.233 INFO:teuthology.orchestra.run.smithi122.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 36 kB/s | 18 kB 00:00 2024-09-16T00:20:02.239 INFO:teuthology.orchestra.run.smithi073.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-16T00:20:02.261 INFO:teuthology.orchestra.run.smithi038.stdout:(154/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 32 MB/s | 2.4 MB 00:00 2024-09-16T00:20:02.264 INFO:teuthology.orchestra.run.smithi073.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-16T00:20:02.286 INFO:teuthology.orchestra.run.smithi038.stdout:(155/156): NetworkManager-tui-1.49.90-1.el9.x86 1.2 MB/s | 249 kB 00:00 2024-09-16T00:20:02.290 INFO:teuthology.orchestra.run.smithi073.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 1.2 MB/s | 32 kB 00:00 2024-09-16T00:20:02.292 INFO:teuthology.orchestra.run.smithi122.stdout:(56/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 2.8 MB/s | 3.0 MB 00:01 2024-09-16T00:20:02.315 INFO:teuthology.orchestra.run.smithi073.stdout:(116/156): python3-certifi-2023.05.07-4.el9.noa 576 kB/s | 14 kB 00:00 2024-09-16T00:20:02.340 INFO:teuthology.orchestra.run.smithi073.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 6.7 MB/s | 172 kB 00:00 2024-09-16T00:20:02.350 INFO:teuthology.orchestra.run.smithi122.stdout:(57/156): libaio-devel-0.3.111-13.el9.x86_64.rp 21 kB/s | 11 kB 00:00 2024-09-16T00:20:02.374 INFO:teuthology.orchestra.run.smithi073.stdout:(118/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.1 MB/s | 839 kB 00:00 2024-09-16T00:20:02.399 INFO:teuthology.orchestra.run.smithi073.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-16T00:20:02.425 INFO:teuthology.orchestra.run.smithi073.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-09-16T00:20:02.450 INFO:teuthology.orchestra.run.smithi073.stdout:(121/156): python3-google-auth-2.34.0-1.el9.noa 4.2 MB/s | 220 kB 00:00 2024-09-16T00:20:02.467 INFO:teuthology.orchestra.run.smithi073.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 8.5 MB/s | 144 kB 00:00 2024-09-16T00:20:02.492 INFO:teuthology.orchestra.run.smithi073.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-09-16T00:20:02.517 INFO:teuthology.orchestra.run.smithi073.stdout:(124/156): python3-jaraco-classes-3.2.1-5.el9.n 708 kB/s | 18 kB 00:00 2024-09-16T00:20:02.543 INFO:teuthology.orchestra.run.smithi073.stdout:(125/156): python3-jaraco-collections-3.0.0-8.e 923 kB/s | 23 kB 00:00 2024-09-16T00:20:02.568 INFO:teuthology.orchestra.run.smithi073.stdout:(126/156): python3-jaraco-functools-3.5.0-2.el9 785 kB/s | 19 kB 00:00 2024-09-16T00:20:02.593 INFO:teuthology.orchestra.run.smithi073.stdout:(127/156): python3-jaraco-text-3.2.0-6.el9.noar 790 kB/s | 20 kB 00:00 2024-09-16T00:20:02.644 INFO:teuthology.orchestra.run.smithi073.stdout:(128/156): python3-grpcio-1.46.7-10.el9.x86_64. 9.3 MB/s | 2.0 MB 00:00 2024-09-16T00:20:02.669 INFO:teuthology.orchestra.run.smithi073.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-16T00:20:02.694 INFO:teuthology.orchestra.run.smithi073.stdout:(130/156): protobuf-compiler-3.14.0-13.el9.x86_ 715 kB/s | 863 kB 00:01 2024-09-16T00:20:02.719 INFO:teuthology.orchestra.run.smithi073.stdout:(131/156): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-09-16T00:20:02.761 INFO:teuthology.orchestra.run.smithi073.stdout:(132/156): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-09-16T00:20:02.787 INFO:teuthology.orchestra.run.smithi073.stdout:(133/156): python3-portend-3.1.0-2.el9.noarch.r 705 kB/s | 16 kB 00:00 2024-09-16T00:20:02.812 INFO:teuthology.orchestra.run.smithi073.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.1 MB/s | 272 kB 00:00 2024-09-16T00:20:02.837 INFO:teuthology.orchestra.run.smithi073.stdout:(135/156): python3-natsort-7.1.1-5.el9.noarch.r 404 kB/s | 58 kB 00:00 2024-09-16T00:20:02.863 INFO:teuthology.orchestra.run.smithi073.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 561 kB/s | 28 kB 00:00 2024-09-16T00:20:02.876 INFO:teuthology.orchestra.run.smithi122.stdout:(58/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 78 kB/s | 45 kB 00:00 2024-09-16T00:20:02.888 INFO:teuthology.orchestra.run.smithi073.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 613 kB/s | 31 kB 00:00 2024-09-16T00:20:02.905 INFO:teuthology.orchestra.run.smithi073.stdout:(138/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 765 kB/s | 90 kB 00:00 2024-09-16T00:20:02.930 INFO:teuthology.orchestra.run.smithi073.stdout:(139/156): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-09-16T00:20:02.955 INFO:teuthology.orchestra.run.smithi073.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 711 kB/s | 36 kB 00:00 2024-09-16T00:20:02.976 INFO:teuthology.orchestra.run.smithi122.stdout:(59/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 216 kB/s | 160 kB 00:00 2024-09-16T00:20:02.981 INFO:teuthology.orchestra.run.smithi073.stdout:(141/156): python3-routes-2.5.1-5.el9.noarch.rp 1.6 MB/s | 188 kB 00:00 2024-09-16T00:20:03.006 INFO:teuthology.orchestra.run.smithi073.stdout:(142/156): python3-typing-extensions-4.12.2-2.e 987 kB/s | 74 kB 00:00 2024-09-16T00:20:03.023 INFO:teuthology.orchestra.run.smithi073.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 5.3 MB/s | 90 kB 00:00 2024-09-16T00:20:03.048 INFO:teuthology.orchestra.run.smithi073.stdout:(144/156): python3-webob-1.8.8-2.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-09-16T00:20:03.065 INFO:teuthology.orchestra.run.smithi073.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 477 kB/s | 20 kB 00:00 2024-09-16T00:20:03.099 INFO:teuthology.orchestra.run.smithi073.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-16T00:20:03.124 INFO:teuthology.orchestra.run.smithi073.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.5 MB/s | 427 kB 00:00 2024-09-16T00:20:03.174 INFO:teuthology.orchestra.run.smithi073.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-16T00:20:03.251 INFO:teuthology.orchestra.run.smithi122.stdout:(60/156): librdkafka-1.6.1-102.el9.x86_64.rpm 736 kB/s | 662 kB 00:00 2024-09-16T00:20:03.341 INFO:teuthology.orchestra.run.smithi073.stdout:(149/156): NetworkManager-1.49.90-1.el9.x86_64. 14 MB/s | 2.3 MB 00:00 2024-09-16T00:20:03.408 INFO:teuthology.orchestra.run.smithi073.stdout:(150/156): NetworkManager-libnm-1.49.90-1.el9.x 27 MB/s | 1.8 MB 00:00 2024-09-16T00:20:03.450 INFO:teuthology.orchestra.run.smithi073.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 971 kB/s | 41 kB 00:00 2024-09-16T00:20:03.492 INFO:teuthology.orchestra.run.smithi073.stdout:(152/156): NetworkManager-tui-1.49.90-1.el9.x86 5.9 MB/s | 249 kB 00:00 2024-09-16T00:20:03.510 INFO:teuthology.orchestra.run.smithi122.stdout:(61/156): libuuid-devel-2.37.4-20.el9.x86_64.rp 46 kB/s | 24 kB 00:00 2024-09-16T00:20:03.560 INFO:teuthology.orchestra.run.smithi073.stdout:(153/156): openssl-3.2.2-6.el9.x86_64.rpm 20 MB/s | 1.4 MB 00:00 2024-09-16T00:20:03.643 INFO:teuthology.orchestra.run.smithi073.stdout:(154/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 29 MB/s | 2.4 MB 00:00 2024-09-16T00:20:03.760 INFO:teuthology.orchestra.run.smithi122.stdout:(62/156): libtool-2.4.6-46.el9.x86_64.rpm 657 kB/s | 581 kB 00:00 2024-09-16T00:20:03.844 INFO:teuthology.orchestra.run.smithi122.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 80 kB/s | 47 kB 00:00 2024-09-16T00:20:04.160 INFO:teuthology.orchestra.run.smithi073.stdout:(155/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 3.1 MB/s | 3.2 MB 00:01 2024-09-16T00:20:04.210 INFO:teuthology.orchestra.run.smithi073.stdout:(156/156): librados2-19.1.1-326.g8921c8ef.el9.x 3.1 MB/s | 3.4 MB 00:01 2024-09-16T00:20:04.216 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:20:04.217 INFO:teuthology.orchestra.run.smithi073.stdout:Total 19 MB/s | 226 MB 00:11 2024-09-16T00:20:04.286 INFO:teuthology.orchestra.run.smithi122.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 377 kB/s | 292 kB 00:00 2024-09-16T00:20:04.403 INFO:teuthology.orchestra.run.smithi122.stdout:(65/156): openblas-0.3.26-2.el9.x86_64.rpm 69 kB/s | 38 kB 00:00 2024-09-16T00:20:04.503 INFO:teuthology.orchestra.run.smithi122.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 254 kB/s | 188 kB 00:00 2024-09-16T00:20:05.388 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2024-09-16T00:20:05.540 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2024-09-16T00:20:05.540 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2024-09-16T00:20:05.578 INFO:teuthology.orchestra.run.smithi122.stdout:(67/156): postgresql-13.16-1.el9.x86_64.rpm 1.5 MB/s | 1.6 MB 00:01 2024-09-16T00:20:05.645 INFO:teuthology.orchestra.run.smithi122.stdout:(68/156): openblas-openmp-0.3.26-2.el9.x86_64.r 3.6 MB/s | 4.9 MB 00:01 2024-09-16T00:20:06.354 INFO:teuthology.orchestra.run.smithi122.stdout:(69/156): postgresql-private-libs-13.16-1.el9.x 194 kB/s | 137 kB 00:00 2024-09-16T00:20:06.471 INFO:teuthology.orchestra.run.smithi122.stdout:(70/156): openssl-devel-3.2.2-6.el9.x86_64.rpm 2.1 MB/s | 4.4 MB 00:02 2024-09-16T00:20:07.255 INFO:teuthology.orchestra.run.smithi122.stdout:(71/156): postgresql-contrib-13.16-1.el9.x86_64 543 kB/s | 910 kB 00:01 2024-09-16T00:20:07.345 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2024-09-16T00:20:07.346 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2024-09-16T00:20:07.886 INFO:teuthology.orchestra.run.smithi038.stdout:(156/156): ceph-test-19.1.1-326.g8921c8ef.el9.x 3.3 MB/s | 49 MB 00:14 2024-09-16T00:20:07.888 INFO:teuthology.orchestra.run.smithi122.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 138 kB/s | 87 kB 00:00 2024-09-16T00:20:07.892 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:20:07.893 INFO:teuthology.orchestra.run.smithi038.stdout:Total 13 MB/s | 226 MB 00:18 2024-09-16T00:20:08.113 INFO:teuthology.orchestra.run.smithi122.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 629 kB/s | 1.0 MB 00:01 2024-09-16T00:20:08.397 INFO:teuthology.orchestra.run.smithi122.stdout:(74/156): postgresql-server-13.16-1.el9.x86_64. 2.8 MB/s | 5.8 MB 00:02 2024-09-16T00:20:08.805 INFO:teuthology.orchestra.run.smithi122.stdout:(75/156): python3-colorama-0.4.6-3.el9.noarch.r 89 kB/s | 61 kB 00:00 2024-09-16T00:20:09.156 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction check 2024-09-16T00:20:09.306 INFO:teuthology.orchestra.run.smithi122.stdout:(76/156): python3-babel-2.9.1-2.el9.noarch.rpm 4.2 MB/s | 6.0 MB 00:01 2024-09-16T00:20:09.325 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction check succeeded. 2024-09-16T00:20:09.325 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction test 2024-09-16T00:20:09.331 INFO:teuthology.orchestra.run.smithi122.stdout:(77/156): python3-devel-3.9.19-8.el9.x86_64.rpm 264 kB/s | 245 kB 00:00 2024-09-16T00:20:09.723 INFO:teuthology.orchestra.run.smithi122.stdout:(78/156): python3-jinja2-2.11.3-6.el9.noarch.rp 271 kB/s | 248 kB 00:00 2024-09-16T00:20:09.840 INFO:teuthology.orchestra.run.smithi122.stdout:(79/156): python3-jsonpatch-1.21-16.el9.noarch. 48 kB/s | 26 kB 00:00 2024-09-16T00:20:09.865 INFO:teuthology.orchestra.run.smithi122.stdout:(80/156): python3-jsonpointer-2.0-4.el9.noarch. 36 kB/s | 19 kB 00:00 2024-09-16T00:20:10.440 INFO:teuthology.orchestra.run.smithi122.stdout:(81/156): python3-markupsafe-1.1.1-12.el9.x86_6 58 kB/s | 35 kB 00:00 2024-09-16T00:20:10.465 INFO:teuthology.orchestra.run.smithi122.stdout:(82/156): python3-mako-1.1.4-6.el9.noarch.rpm 232 kB/s | 172 kB 00:00 2024-09-16T00:20:11.166 INFO:teuthology.orchestra.run.smithi122.stdout:(83/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.9 MB/s | 5.1 MB 00:01 2024-09-16T00:20:11.316 INFO:teuthology.orchestra.run.smithi122.stdout:(84/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 286 kB/s | 250 kB 00:00 2024-09-16T00:20:11.349 INFO:teuthology.orchestra.run.smithi122.stdout:(85/156): python3-oauthlib-3.1.1-5.el9.noarch.r 251 kB/s | 222 kB 00:00 2024-09-16T00:20:11.710 INFO:teuthology.orchestra.run.smithi038.stdout:Transaction test succeeded. 2024-09-16T00:20:11.710 INFO:teuthology.orchestra.run.smithi038.stdout:Running transaction 2024-09-16T00:20:11.866 INFO:teuthology.orchestra.run.smithi122.stdout:(86/156): python3-packaging-20.9-5.el9.noarch.r 110 kB/s | 77 kB 00:00 2024-09-16T00:20:11.883 INFO:teuthology.orchestra.run.smithi122.stdout:(87/156): python3-prettytable-0.7.2-27.el9.noar 74 kB/s | 42 kB 00:00 2024-09-16T00:20:12.350 INFO:teuthology.orchestra.run.smithi122.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 269 kB/s | 269 kB 00:01 2024-09-16T00:20:12.600 INFO:teuthology.orchestra.run.smithi122.stdout:(89/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 216 kB/s | 159 kB 00:00 2024-09-16T00:20:12.892 INFO:teuthology.orchestra.run.smithi122.stdout:(90/156): python3-pyasn1-modules-0.4.8-6.el9.no 276 kB/s | 279 kB 00:01 2024-09-16T00:20:12.934 INFO:teuthology.orchestra.run.smithi122.stdout:(91/156): python3-pytz-2021.1-5.el9.noarch.rpm 87 kB/s | 51 kB 00:00 2024-09-16T00:20:13.209 INFO:teuthology.orchestra.run.smithi122.stdout:(92/156): python3-requests-oauthlib-1.3.0-12.el 88 kB/s | 54 kB 00:00 2024-09-16T00:20:13.501 INFO:teuthology.orchestra.run.smithi122.stdout:(93/156): python3-toml-0.10.2-6.el9.noarch.rpm 73 kB/s | 42 kB 00:00 2024-09-16T00:20:13.605 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2024-09-16T00:20:13.717 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-16T00:20:13.818 INFO:teuthology.orchestra.run.smithi122.stdout:(94/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 73 kB/s | 44 kB 00:00 2024-09-16T00:20:13.852 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-16T00:20:14.007 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-16T00:20:14.048 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-16T00:20:14.082 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-16T00:20:14.125 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-16T00:20:14.252 INFO:teuthology.orchestra.run.smithi122.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 294 kB/s | 221 kB 00:00 2024-09-16T00:20:14.435 INFO:teuthology.orchestra.run.smithi122.stdout:(96/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 11 MB/s | 16 MB 00:01 2024-09-16T00:20:14.461 INFO:teuthology.orchestra.run.smithi122.stdout:(97/156): qatlib-service-24.02.0-1.el9.x86_64.r 56 kB/s | 36 kB 00:00 2024-09-16T00:20:14.510 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-16T00:20:14.566 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:20:14.688 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:20:14.708 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:20:14.783 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:20:14.821 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T00:20:14.944 INFO:teuthology.orchestra.run.smithi122.stdout:(98/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 68 kB/s | 47 kB 00:00 2024-09-16T00:20:14.969 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-16T00:20:15.026 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-16T00:20:15.054 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:20:15.070 INFO:teuthology.orchestra.run.smithi122.stdout:(99/156): uuid-1.6.2-55.el9.x86_64.rpm 95 kB/s | 58 kB 00:00 2024-09-16T00:20:15.158 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:20:15.180 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T00:20:15.212 INFO:teuthology.orchestra.run.smithi122.stdout:(100/156): socat-1.7.4.1-6.el9.x86_64.rpm 392 kB/s | 304 kB 00:00 2024-09-16T00:20:15.226 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T00:20:15.248 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T00:20:15.268 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T00:20:15.466 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T00:20:15.603 INFO:teuthology.orchestra.run.smithi122.stdout:(101/156): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 145 kB/s | 96 kB 00:00 2024-09-16T00:20:15.606 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T00:20:15.670 INFO:teuthology.orchestra.run.smithi122.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 337 kB/s | 22 kB 00:00 2024-09-16T00:20:15.704 INFO:teuthology.orchestra.run.smithi122.stdout:(103/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 100 kB/s | 64 kB 00:00 2024-09-16T00:20:15.721 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-16T00:20:15.754 INFO:teuthology.orchestra.run.smithi122.stdout:(104/156): abseil-cpp-20211102.0-3.el9.x86_64.r 11 MB/s | 548 kB 00:00 2024-09-16T00:20:15.788 INFO:teuthology.orchestra.run.smithi122.stdout:(105/156): gperftools-libs-2.9.1-3.el9.x86_64.r 9.1 MB/s | 308 kB 00:00 2024-09-16T00:20:15.813 INFO:teuthology.orchestra.run.smithi122.stdout:(106/156): grpc-data-1.46.7-10.el9.noarch.rpm 780 kB/s | 19 kB 00:00 2024-09-16T00:20:15.820 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:20:15.838 INFO:teuthology.orchestra.run.smithi122.stdout:(107/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 71 kB/s | 45 kB 00:00 2024-09-16T00:20:15.889 INFO:teuthology.orchestra.run.smithi122.stdout:(108/156): protobuf-compiler-3.14.0-13.el9.x86_ 3.9 MB/s | 863 kB 00:00 2024-09-16T00:20:15.906 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:20:15.922 INFO:teuthology.orchestra.run.smithi122.stdout:(109/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 312 kB/s | 26 kB 00:00 2024-09-16T00:20:15.956 INFO:teuthology.orchestra.run.smithi122.stdout:(110/156): liboath-2.6.7-2.el9.x86_64.rpm 748 kB/s | 49 kB 00:00 2024-09-16T00:20:15.981 INFO:teuthology.orchestra.run.smithi122.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.2 MB/s | 67 kB 00:00 2024-09-16T00:20:16.006 INFO:teuthology.orchestra.run.smithi122.stdout:(112/156): luarocks-3.9.2-5.el9.noarch.rpm 3.0 MB/s | 151 kB 00:00 2024-09-16T00:20:16.065 INFO:teuthology.orchestra.run.smithi122.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 9.2 MB/s | 548 kB 00:00 2024-09-16T00:20:16.099 INFO:teuthology.orchestra.run.smithi122.stdout:(114/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 7.0 MB/s | 839 kB 00:00 2024-09-16T00:20:16.124 INFO:teuthology.orchestra.run.smithi122.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 741 kB/s | 43 kB 00:00 2024-09-16T00:20:16.130 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:20:16.149 INFO:teuthology.orchestra.run.smithi122.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 642 kB/s | 32 kB 00:00 2024-09-16T00:20:16.174 INFO:teuthology.orchestra.run.smithi122.stdout:(117/156): python3-certifi-2023.05.07-4.el9.noa 281 kB/s | 14 kB 00:00 2024-09-16T00:20:16.233 INFO:teuthology.orchestra.run.smithi122.stdout:(118/156): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-09-16T00:20:16.235 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-16T00:20:16.258 INFO:teuthology.orchestra.run.smithi122.stdout:(119/156): python3-cheroot-8.6.0-4.el9.noarch.r 1.5 MB/s | 172 kB 00:00 2024-09-16T00:20:16.284 INFO:teuthology.orchestra.run.smithi122.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.9 MB/s | 303 kB 00:00 2024-09-16T00:20:16.309 INFO:teuthology.orchestra.run.smithi122.stdout:(121/156): python3-cherrypy-18.6.1-2.el9.noarch 2.6 MB/s | 358 kB 00:00 2024-09-16T00:20:16.322 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-16T00:20:16.334 INFO:teuthology.orchestra.run.smithi122.stdout:(122/156): python3-google-auth-2.34.0-1.el9.noa 2.9 MB/s | 220 kB 00:00 2024-09-16T00:20:16.359 INFO:teuthology.orchestra.run.smithi122.stdout:(123/156): python3-jaraco-8.2.1-3.el9.noarch.rp 432 kB/s | 11 kB 00:00 2024-09-16T00:20:16.361 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-16T00:20:16.385 INFO:teuthology.orchestra.run.smithi122.stdout:(124/156): python3-grpcio-tools-1.46.7-10.el9.x 1.9 MB/s | 144 kB 00:00 2024-09-16T00:20:16.396 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-16T00:20:16.410 INFO:teuthology.orchestra.run.smithi122.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-09-16T00:20:16.435 INFO:teuthology.orchestra.run.smithi122.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 464 kB/s | 23 kB 00:00 2024-09-16T00:20:16.452 INFO:teuthology.orchestra.run.smithi122.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 465 kB/s | 19 kB 00:00 2024-09-16T00:20:16.460 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-16T00:20:16.477 INFO:teuthology.orchestra.run.smithi122.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 468 kB/s | 20 kB 00:00 2024-09-16T00:20:16.503 INFO:teuthology.orchestra.run.smithi122.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-16T00:20:16.504 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-16T00:20:16.528 INFO:teuthology.orchestra.run.smithi122.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-16T00:20:16.537 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-16T00:20:16.545 INFO:teuthology.orchestra.run.smithi122.stdout:(131/156): python3-natsort-7.1.1-5.el9.noarch.r 3.4 MB/s | 58 kB 00:00 2024-09-16T00:20:16.579 INFO:teuthology.orchestra.run.smithi122.stdout:(132/156): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-16T00:20:16.583 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-16T00:20:16.612 INFO:teuthology.orchestra.run.smithi122.stdout:(133/156): python3-kubernetes-26.1.0-2.el9.noar 6.4 MB/s | 1.0 MB 00:00 2024-09-16T00:20:16.616 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-16T00:20:16.629 INFO:teuthology.orchestra.run.smithi122.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 328 kB/s | 16 kB 00:00 2024-09-16T00:20:16.644 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-16T00:20:16.646 INFO:teuthology.orchestra.run.smithi122.stdout:(135/156): python3-pyperclip-1.8.2-2.el9.noarch 1.7 MB/s | 28 kB 00:00 2024-09-16T00:20:16.671 INFO:teuthology.orchestra.run.smithi122.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-09-16T00:20:16.697 INFO:teuthology.orchestra.run.smithi122.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 614 kB/s | 31 kB 00:00 2024-09-16T00:20:16.713 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-16T00:20:16.722 INFO:teuthology.orchestra.run.smithi122.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 2.3 MB/s | 59 kB 00:00 2024-09-16T00:20:16.747 INFO:teuthology.orchestra.run.smithi122.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-09-16T00:20:16.773 INFO:teuthology.orchestra.run.smithi122.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 713 kB/s | 36 kB 00:00 2024-09-16T00:20:16.774 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-16T00:20:16.814 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-16T00:20:16.840 INFO:teuthology.orchestra.run.smithi122.stdout:(141/156): python3-typing-extensions-4.12.2-2.e 803 kB/s | 74 kB 00:00 2024-09-16T00:20:16.849 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-16T00:20:16.925 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-16T00:20:16.960 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-16T00:20:17.003 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-16T00:20:17.032 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-16T00:20:17.065 INFO:teuthology.orchestra.run.smithi122.stdout:(142/156): python3-websocket-client-1.2.3-2.el9 400 kB/s | 90 kB 00:00 2024-09-16T00:20:17.090 INFO:teuthology.orchestra.run.smithi122.stdout:(143/156): python3-webob-1.8.8-2.el9.noarch.rpm 727 kB/s | 230 kB 00:00 2024-09-16T00:20:17.101 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-16T00:20:17.132 INFO:teuthology.orchestra.run.smithi122.stdout:(144/156): python3-grpcio-1.46.7-10.el9.x86_64. 2.4 MB/s | 2.0 MB 00:00 2024-09-16T00:20:17.149 INFO:teuthology.orchestra.run.smithi122.stdout:(145/156): python3-zc-lockfile-2.0-10.el9.noarc 342 kB/s | 20 kB 00:00 2024-09-16T00:20:17.155 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-16T00:20:17.174 INFO:teuthology.orchestra.run.smithi122.stdout:(146/156): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-16T00:20:17.200 INFO:teuthology.orchestra.run.smithi122.stdout:(147/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-16T00:20:17.237 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-16T00:20:17.250 INFO:teuthology.orchestra.run.smithi122.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-16T00:20:17.367 INFO:teuthology.orchestra.run.smithi122.stdout:(149/156): NetworkManager-1.49.90-1.el9.x86_64. 20 MB/s | 2.3 MB 00:00 2024-09-16T00:20:17.379 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-16T00:20:17.432 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-16T00:20:17.451 INFO:teuthology.orchestra.run.smithi122.stdout:(150/156): NetworkManager-libnm-1.49.90-1.el9.x 22 MB/s | 1.8 MB 00:00 2024-09-16T00:20:17.509 INFO:teuthology.orchestra.run.smithi122.stdout:(151/156): NetworkManager-team-1.49.90-1.el9.x8 702 kB/s | 41 kB 00:00 2024-09-16T00:20:17.560 INFO:teuthology.orchestra.run.smithi122.stdout:(152/156): librados2-19.1.1-326.g8921c8ef.el9.x 8.9 MB/s | 3.4 MB 00:00 2024-09-16T00:20:17.585 INFO:teuthology.orchestra.run.smithi122.stdout:(153/156): NetworkManager-tui-1.49.90-1.el9.x86 3.2 MB/s | 249 kB 00:00 2024-09-16T00:20:17.668 INFO:teuthology.orchestra.run.smithi122.stdout:(154/156): openssl-3.2.2-6.el9.x86_64.rpm 12 MB/s | 1.4 MB 00:00 2024-09-16T00:20:17.718 INFO:teuthology.orchestra.run.smithi122.stdout:(155/156): openssl-libs-3.2.2-6.el9.x86_64.rpm 18 MB/s | 2.4 MB 00:00 2024-09-16T00:20:17.852 INFO:teuthology.orchestra.run.smithi122.stdout:(156/156): librbd1-19.1.1-326.g8921c8ef.el9.x86 4.8 MB/s | 3.2 MB 00:00 2024-09-16T00:20:17.858 INFO:teuthology.orchestra.run.smithi122.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:20:17.858 INFO:teuthology.orchestra.run.smithi122.stdout:Total 8.1 MB/s | 226 MB 00:27 2024-09-16T00:20:17.957 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-16T00:20:18.046 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-16T00:20:18.074 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-16T00:20:18.117 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-16T00:20:18.154 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-16T00:20:18.196 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-16T00:20:18.306 INFO:teuthology.orchestra.run.smithi038.stdout: Preparing : 1/1 2024-09-16T00:20:18.349 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-16T00:20:18.400 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-16T00:20:18.440 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-16T00:20:18.454 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-16T00:20:18.489 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-16T00:20:18.536 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-16T00:20:18.558 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-16T00:20:18.569 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-16T00:20:18.600 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-16T00:20:18.604 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-16T00:20:18.625 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-16T00:20:18.652 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-16T00:20:18.682 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-16T00:20:18.697 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-16T00:20:18.739 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-16T00:20:18.781 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-16T00:20:18.924 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-16T00:20:18.974 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-16T00:20:19.016 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-16T00:20:19.079 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-16T00:20:19.089 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction check 2024-09-16T00:20:19.118 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-16T00:20:19.149 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-16T00:20:19.189 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-16T00:20:19.192 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:20:19.238 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction check succeeded. 2024-09-16T00:20:19.238 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction test 2024-09-16T00:20:19.326 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:20:19.360 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:20:19.374 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-16T00:20:19.398 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-16T00:20:19.453 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:20:19.476 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-16T00:20:19.489 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T00:20:19.627 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-16T00:20:19.668 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-16T00:20:19.672 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-16T00:20:19.700 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-16T00:20:19.721 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-16T00:20:19.734 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-16T00:20:19.748 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:20:19.774 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-16T00:20:19.806 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-16T00:20:19.835 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-16T00:20:19.836 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:20:19.865 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T00:20:19.869 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-16T00:20:19.903 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-16T00:20:19.915 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T00:20:19.941 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T00:20:19.944 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-16T00:20:19.967 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T00:20:19.979 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-16T00:20:20.073 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-16T00:20:20.126 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-16T00:20:20.153 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T00:20:20.157 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-16T00:20:20.260 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-16T00:20:20.308 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-16T00:20:20.334 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T00:20:20.464 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-16T00:20:20.590 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:20:20.676 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:20:20.895 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:20:21.006 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-16T00:20:21.038 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-16T00:20:21.096 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-16T00:20:21.117 INFO:teuthology.orchestra.run.smithi122.stdout:Transaction test succeeded. 2024-09-16T00:20:21.118 INFO:teuthology.orchestra.run.smithi122.stdout:Running transaction 2024-09-16T00:20:21.122 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-16T00:20:21.130 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-16T00:20:21.156 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-16T00:20:21.173 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-16T00:20:21.248 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-16T00:20:21.254 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-16T00:20:21.300 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-16T00:20:21.303 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-16T00:20:21.339 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-16T00:20:21.348 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-16T00:20:21.383 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-16T00:20:21.415 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-16T00:20:21.442 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-16T00:20:21.444 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-16T00:20:21.484 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-16T00:20:21.514 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-16T00:20:21.574 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-16T00:20:21.604 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-16T00:20:21.648 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-16T00:20:21.668 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-16T00:20:21.699 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-16T00:20:21.726 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-16T00:20:21.758 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-16T00:20:21.776 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-16T00:20:21.803 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-16T00:20:21.830 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-16T00:20:21.905 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-16T00:20:21.953 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-16T00:20:22.032 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-16T00:20:22.177 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-16T00:20:22.232 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-16T00:20:22.417 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-16T00:20:22.530 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-16T00:20:22.790 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-16T00:20:22.871 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-16T00:20:22.913 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-16T00:20:22.957 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-16T00:20:22.995 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-16T00:20:23.037 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-16T00:20:23.199 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-16T00:20:23.257 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-16T00:20:23.340 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-16T00:20:23.391 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-16T00:20:23.424 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-16T00:20:23.460 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-16T00:20:23.512 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-16T00:20:23.545 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-16T00:20:23.587 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-16T00:20:23.636 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-16T00:20:23.737 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-16T00:20:23.781 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-16T00:20:23.819 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-16T00:20:23.821 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-16T00:20:23.855 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-16T00:20:23.862 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-16T00:20:23.897 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-16T00:20:23.970 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-16T00:20:24.012 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-16T00:20:24.048 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-16T00:20:24.081 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-16T00:20:24.106 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-16T00:20:24.205 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-16T00:20:24.213 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-16T00:20:24.262 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T00:20:24.338 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-16T00:20:24.351 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T00:20:24.372 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-16T00:20:24.391 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-16T00:20:24.409 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-16T00:20:24.436 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-16T00:20:24.447 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-16T00:20:24.490 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-16T00:20:24.530 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-16T00:20:24.559 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-16T00:20:24.594 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-16T00:20:24.627 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-16T00:20:24.669 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-16T00:20:24.703 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-16T00:20:24.794 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-16T00:20:24.851 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-16T00:20:24.881 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-16T00:20:24.916 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-16T00:20:24.948 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T00:20:24.990 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-16T00:20:25.011 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T00:20:25.042 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-16T00:20:25.042 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-16T00:20:25.804 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-16T00:20:25.855 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-16T00:20:25.889 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-16T00:20:25.992 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-16T00:20:26.036 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-16T00:20:26.081 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-16T00:20:26.171 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-16T00:20:26.207 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-16T00:20:26.401 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-16T00:20:26.432 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-16T00:20:26.518 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-16T00:20:27.214 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-16T00:20:27.275 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:20:27.321 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:20:27.326 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-16T00:20:27.486 INFO:teuthology.orchestra.run.smithi122.stdout: Preparing : 1/1 2024-09-16T00:20:27.590 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/164 2024-09-16T00:20:27.628 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-09-16T00:20:27.681 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-09-16T00:20:27.715 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-09-16T00:20:27.741 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-09-16T00:20:27.783 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-09-16T00:20:27.862 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:20:27.900 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T00:20:27.984 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T00:20:27.984 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T00:20:27.984 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T00:20:27.984 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-16T00:20:28.018 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T00:20:28.157 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-09-16T00:20:28.208 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:20:28.330 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:20:28.351 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:20:28.433 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:20:28.464 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T00:20:28.613 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/164 2024-09-16T00:20:28.640 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-16T00:20:28.669 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : postgresql-13.16-1.el9.x86_64 12/164 2024-09-16T00:20:28.697 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:20:28.711 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-16T00:20:28.739 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-16T00:20:28.772 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-16T00:20:28.800 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:20:28.823 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T00:20:28.885 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 14/164 2024-09-16T00:20:28.908 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T00:20:28.932 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-09-16T00:20:29.099 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-16T00:20:29.109 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T00:20:29.142 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T00:20:29.235 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T00:20:29.253 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-09-16T00:20:29.264 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-16T00:20:29.301 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-16T00:20:29.389 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 17/164 2024-09-16T00:20:29.470 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:20:29.557 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:20:29.778 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: NetworkManager-1:1.49.90-1.el9.x86_64 18/164 2024-09-16T00:20:29.807 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-16T00:20:29.837 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T00:20:29.894 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : re2-1:20211101-20.el9.x86_64 19/164 2024-09-16T00:20:29.909 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T00:20:29.939 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-16T00:20:29.990 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-09-16T00:20:30.020 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 21/164 2024-09-16T00:20:30.055 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/164 2024-09-16T00:20:30.119 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/164 2024-09-16T00:20:30.181 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/164 2024-09-16T00:20:30.213 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/164 2024-09-16T00:20:30.256 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/164 2024-09-16T00:20:30.289 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/164 2024-09-16T00:20:30.320 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/164 2024-09-16T00:20:30.388 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/164 2024-09-16T00:20:30.442 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/164 2024-09-16T00:20:30.473 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/164 2024-09-16T00:20:30.508 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/164 2024-09-16T00:20:30.584 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/164 2024-09-16T00:20:30.627 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 34/164 2024-09-16T00:20:30.662 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 35/164 2024-09-16T00:20:30.691 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 36/164 2024-09-16T00:20:30.761 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/164 2024-09-16T00:20:30.806 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/164 2024-09-16T00:20:30.880 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/164 2024-09-16T00:20:31.012 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/164 2024-09-16T00:20:31.066 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/164 2024-09-16T00:20:31.594 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/164 2024-09-16T00:20:31.671 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/164 2024-09-16T00:20:31.708 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/164 2024-09-16T00:20:31.751 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/164 2024-09-16T00:20:31.789 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 46/164 2024-09-16T00:20:31.830 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/164 2024-09-16T00:20:31.977 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/164 2024-09-16T00:20:32.025 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/164 2024-09-16T00:20:32.080 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/164 2024-09-16T00:20:32.152 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 51/164 2024-09-16T00:20:32.184 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 52/164 2024-09-16T00:20:32.220 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 53/164 2024-09-16T00:20:32.268 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 54/164 2024-09-16T00:20:32.304 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 55/164 2024-09-16T00:20:32.320 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:20:32.348 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 56/164 2024-09-16T00:20:32.351 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:20:32.389 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 57/164 2024-09-16T00:20:32.526 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 58/164 2024-09-16T00:20:32.575 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 59/164 2024-09-16T00:20:32.616 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 60/164 2024-09-16T00:20:32.719 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-09-16T00:20:32.758 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 62/164 2024-09-16T00:20:32.797 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-09-16T00:20:32.824 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 64/164 2024-09-16T00:20:32.848 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-09-16T00:20:32.922 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-09-16T00:20:32.930 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:20:32.963 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T00:20:33.041 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T00:20:33.041 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T00:20:33.041 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T00:20:33.041 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:20:33.049 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-09-16T00:20:33.080 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-09-16T00:20:33.081 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T00:20:33.108 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-09-16T00:20:33.142 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-09-16T00:20:33.181 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-09-16T00:20:33.213 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-09-16T00:20:33.251 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-09-16T00:20:33.286 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-09-16T00:20:33.319 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-09-16T00:20:33.361 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-09-16T00:20:33.395 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-09-16T00:20:33.483 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-09-16T00:20:33.534 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-09-16T00:20:33.577 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-09-16T00:20:33.677 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-09-16T00:20:33.732 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 82/164 2024-09-16T00:20:34.454 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 83/164 2024-09-16T00:20:34.498 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 84/164 2024-09-16T00:20:34.532 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 85/164 2024-09-16T00:20:34.630 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 86/164 2024-09-16T00:20:34.687 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 87/164 2024-09-16T00:20:34.740 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 88/164 2024-09-16T00:20:34.828 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-09-16T00:20:34.868 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-09-16T00:20:35.052 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : openblas-0.3.26-2.el9.x86_64 91/164 2024-09-16T00:20:35.082 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 92/164 2024-09-16T00:20:35.160 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-09-16T00:20:35.802 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-09-16T00:20:35.914 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-09-16T00:20:37.146 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-09-16T00:20:37.236 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-09-16T00:20:37.264 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-09-16T00:20:37.305 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 99/164 2024-09-16T00:20:37.605 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-09-16T00:20:37.643 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T00:20:37.734 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 101/164 2024-09-16T00:20:37.765 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 102/164 2024-09-16T00:20:37.801 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-09-16T00:20:38.279 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-09-16T00:20:38.304 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T00:20:38.376 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 105/164 2024-09-16T00:20:38.406 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 106/164 2024-09-16T00:20:39.909 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T00:20:39.910 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /sys 2024-09-16T00:20:39.910 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /proc 2024-09-16T00:20:39.910 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /mnt 2024-09-16T00:20:39.910 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /var/tmp 2024-09-16T00:20:39.910 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /home 2024-09-16T00:20:39.911 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /root 2024-09-16T00:20:39.911 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /tmp 2024-09-16T00:20:39.911 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-16T00:20:40.080 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T00:20:40.095 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T00:20:40.096 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:20:40.096 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T00:20:40.096 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-16T00:20:40.662 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:20:40.702 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:20:41.247 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 107/164 2024-09-16T00:20:41.289 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T00:20:41.390 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 108/164 2024-09-16T00:20:41.390 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-16T00:20:41.390 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-16T00:20:41.390 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-16T00:20:41.424 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T00:20:41.558 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T00:20:41.584 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T00:20:41.584 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:20:41.584 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T00:20:41.585 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T00:20:41.585 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T00:20:41.585 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-16T00:20:41.808 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T00:20:41.832 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T00:20:41.832 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:20:41.832 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T00:20:41.832 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T00:20:41.832 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T00:20:41.833 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-16T00:20:42.259 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T00:20:42.282 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T00:20:42.282 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:20:42.282 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T00:20:42.282 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T00:20:42.283 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T00:20:42.283 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-16T00:20:42.408 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-16T00:20:42.463 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-16T00:20:42.518 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-16T00:20:42.701 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-16T00:20:42.752 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-16T00:20:42.829 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-16T00:20:42.918 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-16T00:20:43.057 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-16T00:20:43.118 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-16T00:20:43.150 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-16T00:20:43.178 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-16T00:20:43.640 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T00:20:43.668 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T00:20:43.700 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-16T00:20:43.762 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-16T00:20:43.791 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T00:20:44.834 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T00:20:44.868 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T00:20:45.572 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T00:20:45.572 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /sys 2024-09-16T00:20:45.572 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /proc 2024-09-16T00:20:45.572 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /mnt 2024-09-16T00:20:45.573 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /var/tmp 2024-09-16T00:20:45.573 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /home 2024-09-16T00:20:45.573 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /root 2024-09-16T00:20:45.573 INFO:teuthology.orchestra.run.smithi038.stdout:skipping the directory /tmp 2024-09-16T00:20:45.573 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:20:45.759 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T00:20:45.776 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T00:20:45.776 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:20:45.776 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T00:20:45.776 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:20:45.828 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T00:20:45.860 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T00:20:45.963 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T00:20:46.121 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-16T00:20:46.154 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T00:20:46.177 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T00:20:46.178 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:20:46.178 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T00:20:46.178 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T00:20:46.178 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T00:20:46.178 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-16T00:20:46.221 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T00:20:46.233 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T00:20:46.330 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-16T00:20:46.386 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-16T00:20:47.295 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T00:20:47.304 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-16T00:20:47.323 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T00:20:47.324 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:20:47.324 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T00:20:47.324 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T00:20:47.324 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T00:20:47.324 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:20:47.340 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T00:20:47.364 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T00:20:47.364 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:20:47.364 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T00:20:47.364 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T00:20:47.364 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T00:20:47.364 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-16T00:20:47.408 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T00:20:47.430 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T00:20:47.431 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:20:47.431 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T00:20:47.431 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-16T00:20:47.484 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T00:20:47.507 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T00:20:47.507 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:20:47.507 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-16T00:20:47.507 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T00:20:47.507 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T00:20:47.507 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-16T00:20:47.554 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T00:20:47.579 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T00:20:47.579 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:20:47.579 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T00:20:47.580 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T00:20:47.580 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T00:20:47.580 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:20:47.809 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T00:20:47.832 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T00:20:47.832 INFO:teuthology.orchestra.run.smithi073.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:20:47.832 INFO:teuthology.orchestra.run.smithi073.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T00:20:47.833 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T00:20:47.833 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T00:20:47.833 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-16T00:20:48.038 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T00:20:48.063 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T00:20:48.063 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:20:48.063 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T00:20:48.063 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T00:20:48.063 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T00:20:48.063 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:20:48.169 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-16T00:20:48.244 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-16T00:20:48.307 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-16T00:20:48.503 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-16T00:20:48.557 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-16T00:20:48.637 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-16T00:20:48.726 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-16T00:20:48.869 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-16T00:20:48.915 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-16T00:20:48.946 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-16T00:20:48.982 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-16T00:20:49.404 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T00:20:49.431 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T00:20:49.463 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-16T00:20:49.531 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-16T00:20:49.556 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T00:20:50.653 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T00:20:50.680 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T00:20:51.693 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T00:20:51.722 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T00:20:51.831 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T00:20:52.025 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-16T00:20:52.049 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T00:20:52.074 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T00:20:52.075 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:20:52.075 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T00:20:52.075 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T00:20:52.075 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T00:20:52.075 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:20:52.125 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T00:20:52.137 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T00:20:52.238 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-16T00:20:52.297 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-16T00:20:52.362 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-16T00:20:52.403 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-16T00:20:52.444 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-16T00:20:52.477 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-16T00:20:52.504 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-16T00:20:52.682 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-16T00:20:52.734 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-16T00:20:52.765 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-16T00:20:52.794 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-16T00:20:52.840 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-16T00:20:53.197 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 109/164 2024-09-16T00:20:53.197 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /sys 2024-09-16T00:20:53.197 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /proc 2024-09-16T00:20:53.198 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /mnt 2024-09-16T00:20:53.198 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /var/tmp 2024-09-16T00:20:53.198 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /home 2024-09-16T00:20:53.198 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /root 2024-09-16T00:20:53.198 INFO:teuthology.orchestra.run.smithi122.stdout:skipping the directory /tmp 2024-09-16T00:20:53.198 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-16T00:20:53.275 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-16T00:20:53.310 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T00:20:53.335 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T00:20:53.335 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:20:53.335 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T00:20:53.335 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T00:20:53.335 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T00:20:53.335 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:20:53.368 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T00:20:53.378 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T00:20:53.384 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 110/164 2024-09-16T00:20:53.385 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:20:53.385 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-16T00:20:53.385 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-16T00:20:53.402 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T00:20:53.402 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:20:53.402 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T00:20:53.402 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:20:53.443 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:20:53.454 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T00:20:53.478 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T00:20:53.478 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:20:53.478 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-16T00:20:53.478 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T00:20:53.479 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T00:20:53.479 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:20:53.497 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:20:53.519 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:20:53.796 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T00:20:53.821 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T00:20:53.821 INFO:teuthology.orchestra.run.smithi038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:20:53.821 INFO:teuthology.orchestra.run.smithi038.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T00:20:53.821 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T00:20:53.821 INFO:teuthology.orchestra.run.smithi038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T00:20:53.821 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:20:54.344 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-16T00:20:54.426 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-16T00:20:54.467 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-16T00:20:54.548 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-16T00:20:54.588 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-16T00:20:54.613 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-16T00:20:54.639 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-16T00:20:54.639 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T00:20:54.715 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T00:20:54.715 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T00:20:54.799 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T00:20:54.824 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-16T00:20:54.829 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:20:54.829 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:20:54.873 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T00:20:54.900 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 111/164 2024-09-16T00:20:54.900 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:20:54.900 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-16T00:20:54.900 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T00:20:54.900 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-16T00:20:54.900 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-16T00:20:55.009 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:20:55.009 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T00:20:55.093 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T00:20:55.123 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T00:20:55.147 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 112/164 2024-09-16T00:20:55.148 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:20:55.148 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-16T00:20:55.148 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T00:20:55.148 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-16T00:20:55.148 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-16T00:20:55.156 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:20:55.591 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T00:20:55.615 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 113/164 2024-09-16T00:20:55.615 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:20:55.615 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-16T00:20:55.615 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T00:20:55.615 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-16T00:20:55.615 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-16T00:20:55.720 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-09-16T00:20:55.769 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-09-16T00:20:55.834 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-09-16T00:20:56.022 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-09-16T00:20:56.069 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-09-16T00:20:56.144 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-09-16T00:20:56.234 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-09-16T00:20:56.373 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-09-16T00:20:56.425 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-09-16T00:20:56.457 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-09-16T00:20:56.484 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : fuse-2.9.9-16.el9.x86_64 124/164 2024-09-16T00:20:56.939 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T00:20:56.966 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 125/164 2024-09-16T00:20:56.989 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 126/164 2024-09-16T00:20:57.044 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 127/164 2024-09-16T00:20:57.073 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T00:20:58.119 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 128/164 2024-09-16T00:20:58.158 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T00:20:58.666 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-16T00:20:58.708 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-16T00:20:58.749 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-16T00:20:58.780 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-16T00:20:58.808 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-16T00:20:59.008 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-16T00:20:59.046 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-16T00:20:59.077 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-16T00:20:59.106 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-16T00:20:59.118 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 129/164 2024-09-16T00:20:59.142 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T00:20:59.144 INFO:teuthology.orchestra.run.smithi038.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-16T00:20:59.245 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 130/164 2024-09-16T00:20:59.404 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 131/164 2024-09-16T00:20:59.428 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T00:20:59.452 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 132/164 2024-09-16T00:20:59.452 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:20:59.452 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-16T00:20:59.452 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T00:20:59.453 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-16T00:20:59.453 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-16T00:20:59.503 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T00:20:59.515 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 133/164 2024-09-16T00:20:59.604 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 134/164 2024-09-16T00:20:59.660 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 135/164 2024-09-16T00:20:59.781 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:20:59.817 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:20:59.838 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:21:00.578 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libtool-2.4.6-46.el9.x86_64 136/164 2024-09-16T00:21:00.614 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T00:21:00.638 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 137/164 2024-09-16T00:21:00.638 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:21:00.638 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-16T00:21:00.638 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T00:21:00.638 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-16T00:21:00.638 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-16T00:21:00.681 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T00:21:00.705 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.1.1-326.g8921c8 138/164 2024-09-16T00:21:00.705 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:21:00.705 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-16T00:21:00.705 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-16T00:21:00.742 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-16T00:21:00.757 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T00:21:00.781 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 139/164 2024-09-16T00:21:00.781 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:21:00.781 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-16T00:21:00.781 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T00:21:00.781 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-16T00:21:00.781 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-16T00:21:00.837 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-16T00:21:00.888 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:21:00.888 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-16T00:21:00.888 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-16T00:21:00.888 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-16T00:21:00.888 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-16T00:21:00.888 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-16T00:21:00.888 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-16T00:21:00.888 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-16T00:21:00.888 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:21:00.888 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:21:00.888 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T00:21:00.889 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-16T00:21:00.889 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-16T00:21:00.889 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:21:00.889 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-16T00:21:00.889 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-16T00:21:00.889 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-16T00:21:00.889 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-16T00:21:00.889 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-16T00:21:00.889 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-16T00:21:00.889 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-16T00:21:00.889 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-16T00:21:00.889 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-16T00:21:00.889 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-16T00:21:00.890 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-16T00:21:00.890 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-16T00:21:00.890 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-16T00:21:00.890 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-16T00:21:00.890 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-16T00:21:00.902 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-16T00:21:00.903 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-16T00:21:00.903 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-16T00:21:00.903 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-16T00:21:00.903 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-16T00:21:00.903 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-16T00:21:00.903 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-16T00:21:00.903 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-16T00:21:00.903 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-16T00:21:00.903 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-16T00:21:00.903 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-16T00:21:00.903 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-16T00:21:00.903 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-16T00:21:00.904 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-16T00:21:00.904 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-16T00:21:00.904 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-16T00:21:00.904 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-16T00:21:00.904 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-16T00:21:00.904 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-16T00:21:00.904 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-16T00:21:00.904 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-16T00:21:00.904 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-16T00:21:00.904 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-16T00:21:00.904 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-16T00:21:00.904 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-16T00:21:00.904 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-16T00:21:00.905 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-16T00:21:00.905 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-16T00:21:00.905 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-16T00:21:00.905 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-16T00:21:00.905 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-16T00:21:00.905 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-16T00:21:00.905 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-16T00:21:00.905 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-16T00:21:00.905 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-16T00:21:00.905 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-16T00:21:00.905 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-16T00:21:00.905 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-16T00:21:00.905 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-16T00:21:00.906 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-16T00:21:00.906 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-16T00:21:00.906 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-16T00:21:00.906 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-16T00:21:00.906 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-16T00:21:00.906 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-16T00:21:00.906 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-16T00:21:00.906 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-16T00:21:00.906 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-16T00:21:00.906 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-16T00:21:00.906 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-16T00:21:00.907 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-16T00:21:00.907 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-16T00:21:00.907 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-16T00:21:00.907 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-16T00:21:00.907 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-16T00:21:00.907 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-16T00:21:00.907 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-16T00:21:00.907 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-16T00:21:00.907 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-16T00:21:00.907 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-16T00:21:00.907 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-16T00:21:00.907 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-16T00:21:00.907 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-16T00:21:00.908 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-16T00:21:00.908 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-16T00:21:00.908 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-16T00:21:00.908 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-16T00:21:00.908 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-16T00:21:00.908 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-16T00:21:00.908 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-16T00:21:00.908 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-16T00:21:00.908 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-16T00:21:00.908 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-16T00:21:00.908 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-16T00:21:00.908 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-16T00:21:00.908 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-16T00:21:00.908 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-16T00:21:00.909 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-16T00:21:00.909 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-16T00:21:00.909 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-16T00:21:00.909 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-16T00:21:00.909 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-16T00:21:00.909 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-16T00:21:00.909 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-16T00:21:00.909 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-16T00:21:00.909 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-16T00:21:00.909 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-16T00:21:00.909 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-16T00:21:00.909 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-16T00:21:00.910 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-16T00:21:00.910 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-16T00:21:00.910 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-16T00:21:00.910 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-16T00:21:00.910 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-16T00:21:00.910 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-16T00:21:00.910 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-16T00:21:00.910 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-16T00:21:00.910 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-16T00:21:00.910 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-16T00:21:00.910 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-16T00:21:00.910 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-16T00:21:00.910 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-16T00:21:00.910 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-16T00:21:00.911 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-16T00:21:00.911 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-16T00:21:00.911 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-16T00:21:00.911 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-16T00:21:00.911 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-16T00:21:00.911 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-16T00:21:00.911 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-16T00:21:00.911 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-16T00:21:00.911 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-16T00:21:00.911 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-16T00:21:00.911 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-16T00:21:00.911 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-16T00:21:00.912 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-16T00:21:00.912 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-16T00:21:00.912 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-16T00:21:00.912 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-16T00:21:00.912 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-16T00:21:00.912 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-16T00:21:00.912 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-16T00:21:00.912 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-16T00:21:00.912 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-16T00:21:00.912 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-16T00:21:00.912 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-16T00:21:00.912 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-16T00:21:00.912 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-16T00:21:00.913 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-16T00:21:00.913 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-16T00:21:00.913 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-16T00:21:00.913 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-16T00:21:00.913 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-16T00:21:00.913 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-16T00:21:00.913 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-16T00:21:00.913 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-16T00:21:00.982 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-16T00:21:01.024 INFO:teuthology.orchestra.run.smithi038.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-16T00:21:01.049 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-16T00:21:01.075 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-16T00:21:01.076 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T00:21:01.083 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T00:21:01.106 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 140/164 2024-09-16T00:21:01.107 INFO:teuthology.orchestra.run.smithi122.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-16T00:21:01.107 INFO:teuthology.orchestra.run.smithi122.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-16T00:21:01.107 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T00:21:01.107 INFO:teuthology.orchestra.run.smithi122.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-16T00:21:01.107 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-16T00:21:01.168 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T00:21:01.168 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T00:21:01.243 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T00:21:01.268 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-16T00:21:01.274 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:21:01.274 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:21:01.462 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:21:01.462 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T00:21:01.546 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T00:21:01.613 INFO:teuthology.orchestra.run.smithi038.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:21:03.734 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:21:03.734 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-16T00:21:03.734 INFO:teuthology.orchestra.run.smithi073.stdout:Upgraded: 2024-09-16T00:21:03.734 INFO:teuthology.orchestra.run.smithi073.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-16T00:21:03.734 INFO:teuthology.orchestra.run.smithi073.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-16T00:21:03.734 INFO:teuthology.orchestra.run.smithi073.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-16T00:21:03.735 INFO:teuthology.orchestra.run.smithi073.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-16T00:21:03.735 INFO:teuthology.orchestra.run.smithi073.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:03.735 INFO:teuthology.orchestra.run.smithi073.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:03.735 INFO:teuthology.orchestra.run.smithi073.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-16T00:21:03.735 INFO:teuthology.orchestra.run.smithi073.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-16T00:21:03.735 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2024-09-16T00:21:03.735 INFO:teuthology.orchestra.run.smithi073.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-16T00:21:03.735 INFO:teuthology.orchestra.run.smithi073.stdout: automake-1.16.2-8.el9.noarch 2024-09-16T00:21:03.735 INFO:teuthology.orchestra.run.smithi073.stdout: bison-3.7.4-5.el9.x86_64 2024-09-16T00:21:03.735 INFO:teuthology.orchestra.run.smithi073.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T00:21:03.735 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:03.735 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:03.735 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:03.736 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:03.736 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:21:03.736 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:03.736 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:03.736 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:03.736 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:21:03.736 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:21:03.736 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:21:03.736 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:21:03.736 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:21:03.736 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:03.736 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:03.737 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:21:03.737 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:03.737 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:03.737 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:03.737 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:21:03.737 INFO:teuthology.orchestra.run.smithi073.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:21:03.737 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:03.737 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:03.737 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:21:03.737 INFO:teuthology.orchestra.run.smithi073.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-16T00:21:03.737 INFO:teuthology.orchestra.run.smithi073.stdout: flex-2.6.4-9.el9.x86_64 2024-09-16T00:21:03.737 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T00:21:03.737 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T00:21:03.738 INFO:teuthology.orchestra.run.smithi073.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T00:21:03.738 INFO:teuthology.orchestra.run.smithi073.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-16T00:21:03.738 INFO:teuthology.orchestra.run.smithi073.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T00:21:03.738 INFO:teuthology.orchestra.run.smithi073.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-16T00:21:03.738 INFO:teuthology.orchestra.run.smithi073.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-16T00:21:03.738 INFO:teuthology.orchestra.run.smithi073.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T00:21:03.738 INFO:teuthology.orchestra.run.smithi073.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T00:21:03.738 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:03.738 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:03.738 INFO:teuthology.orchestra.run.smithi073.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:03.738 INFO:teuthology.orchestra.run.smithi073.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T00:21:03.738 INFO:teuthology.orchestra.run.smithi073.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T00:21:03.738 INFO:teuthology.orchestra.run.smithi073.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T00:21:03.738 INFO:teuthology.orchestra.run.smithi073.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T00:21:03.739 INFO:teuthology.orchestra.run.smithi073.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T00:21:03.739 INFO:teuthology.orchestra.run.smithi073.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:03.739 INFO:teuthology.orchestra.run.smithi073.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:03.739 INFO:teuthology.orchestra.run.smithi073.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T00:21:03.739 INFO:teuthology.orchestra.run.smithi073.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:03.739 INFO:teuthology.orchestra.run.smithi073.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-16T00:21:03.739 INFO:teuthology.orchestra.run.smithi073.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T00:21:03.739 INFO:teuthology.orchestra.run.smithi073.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-16T00:21:03.739 INFO:teuthology.orchestra.run.smithi073.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-16T00:21:03.739 INFO:teuthology.orchestra.run.smithi073.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T00:21:03.739 INFO:teuthology.orchestra.run.smithi073.stdout: lua-5.4.4-4.el9.x86_64 2024-09-16T00:21:03.739 INFO:teuthology.orchestra.run.smithi073.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-16T00:21:03.739 INFO:teuthology.orchestra.run.smithi073.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-16T00:21:03.739 INFO:teuthology.orchestra.run.smithi073.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T00:21:03.740 INFO:teuthology.orchestra.run.smithi073.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T00:21:03.740 INFO:teuthology.orchestra.run.smithi073.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-16T00:21:03.740 INFO:teuthology.orchestra.run.smithi073.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T00:21:03.740 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-13.16-1.el9.x86_64 2024-09-16T00:21:03.740 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-16T00:21:03.740 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-16T00:21:03.740 INFO:teuthology.orchestra.run.smithi073.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-16T00:21:03.740 INFO:teuthology.orchestra.run.smithi073.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-16T00:21:03.740 INFO:teuthology.orchestra.run.smithi073.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-16T00:21:03.740 INFO:teuthology.orchestra.run.smithi073.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T00:21:03.740 INFO:teuthology.orchestra.run.smithi073.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-16T00:21:03.740 INFO:teuthology.orchestra.run.smithi073.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T00:21:03.740 INFO:teuthology.orchestra.run.smithi073.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T00:21:03.741 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T00:21:03.741 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:03.741 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:03.741 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:03.741 INFO:teuthology.orchestra.run.smithi073.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T00:21:03.741 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T00:21:03.741 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T00:21:03.741 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T00:21:03.741 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-16T00:21:03.741 INFO:teuthology.orchestra.run.smithi073.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-16T00:21:03.741 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T00:21:03.741 INFO:teuthology.orchestra.run.smithi073.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T00:21:03.741 INFO:teuthology.orchestra.run.smithi073.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T00:21:03.742 INFO:teuthology.orchestra.run.smithi073.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-16T00:21:03.742 INFO:teuthology.orchestra.run.smithi073.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-16T00:21:03.742 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T00:21:03.742 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T00:21:03.742 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T00:21:03.742 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T00:21:03.742 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T00:21:03.742 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T00:21:03.742 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T00:21:03.742 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T00:21:03.742 INFO:teuthology.orchestra.run.smithi073.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T00:21:03.742 INFO:teuthology.orchestra.run.smithi073.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T00:21:03.742 INFO:teuthology.orchestra.run.smithi073.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T00:21:03.742 INFO:teuthology.orchestra.run.smithi073.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T00:21:03.743 INFO:teuthology.orchestra.run.smithi073.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T00:21:03.743 INFO:teuthology.orchestra.run.smithi073.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T00:21:03.743 INFO:teuthology.orchestra.run.smithi073.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T00:21:03.743 INFO:teuthology.orchestra.run.smithi073.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T00:21:03.743 INFO:teuthology.orchestra.run.smithi073.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T00:21:03.743 INFO:teuthology.orchestra.run.smithi073.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T00:21:03.743 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T00:21:03.743 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T00:21:03.743 INFO:teuthology.orchestra.run.smithi073.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T00:21:03.743 INFO:teuthology.orchestra.run.smithi073.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T00:21:03.743 INFO:teuthology.orchestra.run.smithi073.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-16T00:21:03.743 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T00:21:03.743 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T00:21:03.743 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T00:21:03.743 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T00:21:03.744 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-16T00:21:03.744 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T00:21:03.744 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:03.744 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:03.744 INFO:teuthology.orchestra.run.smithi073.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T00:21:03.744 INFO:teuthology.orchestra.run.smithi073.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T00:21:03.744 INFO:teuthology.orchestra.run.smithi073.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T00:21:03.744 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:03.744 INFO:teuthology.orchestra.run.smithi073.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T00:21:03.744 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T00:21:03.744 INFO:teuthology.orchestra.run.smithi073.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T00:21:03.744 INFO:teuthology.orchestra.run.smithi073.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T00:21:03.744 INFO:teuthology.orchestra.run.smithi073.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T00:21:03.744 INFO:teuthology.orchestra.run.smithi073.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T00:21:03.745 INFO:teuthology.orchestra.run.smithi073.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T00:21:03.745 INFO:teuthology.orchestra.run.smithi073.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-16T00:21:03.745 INFO:teuthology.orchestra.run.smithi073.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T00:21:03.745 INFO:teuthology.orchestra.run.smithi073.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T00:21:03.745 INFO:teuthology.orchestra.run.smithi073.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T00:21:03.745 INFO:teuthology.orchestra.run.smithi073.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T00:21:03.745 INFO:teuthology.orchestra.run.smithi073.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-16T00:21:03.745 INFO:teuthology.orchestra.run.smithi073.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-16T00:21:03.745 INFO:teuthology.orchestra.run.smithi073.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-16T00:21:03.745 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:03.745 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:03.745 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:03.745 INFO:teuthology.orchestra.run.smithi073.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T00:21:03.745 INFO:teuthology.orchestra.run.smithi073.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T00:21:03.746 INFO:teuthology.orchestra.run.smithi073.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T00:21:03.746 INFO:teuthology.orchestra.run.smithi073.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-16T00:21:03.746 INFO:teuthology.orchestra.run.smithi073.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-16T00:21:03.746 INFO:teuthology.orchestra.run.smithi073.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T00:21:03.746 INFO:teuthology.orchestra.run.smithi073.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-16T00:21:03.746 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-16T00:21:03.746 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2024-09-16T00:21:04.189 DEBUG:teuthology.parallel:result is None 2024-09-16T00:21:05.653 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 141/164 2024-09-16T00:21:05.694 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 142/164 2024-09-16T00:21:05.735 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 143/164 2024-09-16T00:21:05.767 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 144/164 2024-09-16T00:21:05.803 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 145/164 2024-09-16T00:21:05.986 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 146/164 2024-09-16T00:21:06.033 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 147/164 2024-09-16T00:21:06.063 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 148/164 2024-09-16T00:21:06.093 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : NetworkManager-team-1:1.49.90-1.el9.x86_64 149/164 2024-09-16T00:21:06.139 INFO:teuthology.orchestra.run.smithi122.stdout: Upgrading : NetworkManager-tui-1:1.49.90-1.el9.x86_64 150/164 2024-09-16T00:21:06.688 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:21:06.720 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:21:06.741 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 151/164 2024-09-16T00:21:07.533 INFO:teuthology.orchestra.run.smithi038.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:21:07.533 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-16T00:21:07.533 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-16T00:21:07.534 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-16T00:21:07.534 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-16T00:21:07.534 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-16T00:21:07.534 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-16T00:21:07.534 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-16T00:21:07.534 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:21:07.534 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:21:07.534 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T00:21:07.534 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-16T00:21:07.534 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-16T00:21:07.534 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:21:07.534 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-16T00:21:07.535 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-16T00:21:07.535 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-16T00:21:07.535 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-16T00:21:07.535 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-16T00:21:07.535 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-16T00:21:07.535 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-16T00:21:07.535 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-16T00:21:07.535 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-16T00:21:07.535 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-16T00:21:07.535 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-16T00:21:07.535 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-16T00:21:07.535 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-16T00:21:07.535 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-16T00:21:07.536 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-16T00:21:07.536 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-16T00:21:07.536 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-16T00:21:07.536 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-16T00:21:07.536 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-16T00:21:07.536 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-16T00:21:07.536 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-16T00:21:07.536 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-16T00:21:07.536 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-16T00:21:07.536 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-16T00:21:07.536 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-16T00:21:07.536 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-16T00:21:07.536 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-16T00:21:07.537 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-16T00:21:07.537 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-16T00:21:07.537 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-16T00:21:07.537 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-16T00:21:07.537 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-16T00:21:07.537 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-16T00:21:07.537 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-16T00:21:07.537 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-16T00:21:07.537 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-16T00:21:07.537 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-16T00:21:07.537 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-16T00:21:07.537 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-16T00:21:07.537 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-16T00:21:07.538 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-16T00:21:07.538 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-16T00:21:07.538 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-16T00:21:07.538 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-16T00:21:07.538 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-16T00:21:07.538 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-16T00:21:07.538 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-16T00:21:07.538 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-16T00:21:07.538 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-16T00:21:07.538 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-16T00:21:07.538 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-16T00:21:07.538 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-16T00:21:07.538 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-16T00:21:07.538 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-16T00:21:07.539 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-16T00:21:07.539 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-16T00:21:07.539 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-16T00:21:07.539 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-16T00:21:07.539 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-16T00:21:07.539 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-16T00:21:07.539 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-16T00:21:07.539 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-16T00:21:07.539 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-16T00:21:07.539 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-16T00:21:07.539 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-16T00:21:07.539 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-16T00:21:07.540 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-16T00:21:07.540 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-16T00:21:07.540 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-16T00:21:07.540 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-16T00:21:07.540 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-16T00:21:07.540 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-16T00:21:07.540 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-16T00:21:07.540 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-16T00:21:07.540 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-16T00:21:07.540 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-16T00:21:07.540 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-16T00:21:07.540 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-16T00:21:07.540 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-16T00:21:07.540 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-16T00:21:07.541 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-16T00:21:07.541 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-16T00:21:07.541 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-16T00:21:07.541 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-16T00:21:07.541 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-16T00:21:07.541 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-16T00:21:07.541 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-16T00:21:07.541 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-16T00:21:07.541 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-16T00:21:07.541 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-16T00:21:07.541 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-16T00:21:07.541 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-16T00:21:07.541 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-16T00:21:07.541 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-16T00:21:07.542 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-16T00:21:07.542 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-16T00:21:07.542 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-16T00:21:07.542 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-16T00:21:07.542 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-16T00:21:07.542 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-16T00:21:07.542 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-16T00:21:07.542 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-16T00:21:07.542 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-16T00:21:07.542 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-16T00:21:07.542 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-16T00:21:07.542 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-16T00:21:07.543 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-16T00:21:07.543 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-16T00:21:07.543 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-16T00:21:07.543 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-16T00:21:07.543 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-16T00:21:07.543 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-16T00:21:07.543 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-16T00:21:07.543 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-16T00:21:07.543 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-16T00:21:07.543 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-16T00:21:07.543 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-16T00:21:07.543 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-16T00:21:07.543 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-16T00:21:07.543 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-16T00:21:07.544 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-16T00:21:07.544 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-16T00:21:07.544 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-16T00:21:07.544 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-16T00:21:07.544 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-16T00:21:07.544 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-16T00:21:07.544 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-16T00:21:07.544 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-16T00:21:07.544 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-16T00:21:07.544 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-16T00:21:07.544 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-16T00:21:07.544 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-16T00:21:07.544 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-16T00:21:07.545 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-16T00:21:07.545 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-16T00:21:07.545 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-16T00:21:07.545 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-16T00:21:07.545 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-16T00:21:07.545 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-16T00:21:07.545 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-16T00:21:07.545 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-16T00:21:07.545 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-16T00:21:07.545 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-16T00:21:07.545 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-16T00:21:07.545 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-16T00:21:07.545 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-16T00:21:07.545 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-16T00:21:07.545 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-16T00:21:07.546 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-16T00:21:07.546 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-16T00:21:07.587 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 152/164 2024-09-16T00:21:07.675 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 153/164 2024-09-16T00:21:07.716 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-09-16T00:21:07.798 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-09-16T00:21:07.828 INFO:teuthology.orchestra.run.smithi122.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-09-16T00:21:07.854 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 157/164 2024-09-16T00:21:07.879 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 158/164 2024-09-16T00:21:07.880 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T00:21:07.981 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/164 2024-09-16T00:21:07.981 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T00:21:08.064 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-09-16T00:21:08.089 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 161/164 2024-09-16T00:21:08.095 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:21:08.095 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:21:08.275 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 162/164 2024-09-16T00:21:08.275 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T00:21:08.367 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 163/164 2024-09-16T00:21:08.432 INFO:teuthology.orchestra.run.smithi122.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:21:10.254 INFO:teuthology.orchestra.run.smithi038.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:21:10.254 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:21:10.254 INFO:teuthology.orchestra.run.smithi038.stdout:Upgraded: 2024-09-16T00:21:10.255 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-16T00:21:10.255 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-16T00:21:10.255 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-16T00:21:10.255 INFO:teuthology.orchestra.run.smithi038.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-16T00:21:10.255 INFO:teuthology.orchestra.run.smithi038.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:10.256 INFO:teuthology.orchestra.run.smithi038.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:10.256 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-16T00:21:10.256 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-16T00:21:10.256 INFO:teuthology.orchestra.run.smithi038.stdout:Installed: 2024-09-16T00:21:10.257 INFO:teuthology.orchestra.run.smithi038.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-16T00:21:10.257 INFO:teuthology.orchestra.run.smithi038.stdout: automake-1.16.2-8.el9.noarch 2024-09-16T00:21:10.257 INFO:teuthology.orchestra.run.smithi038.stdout: bison-3.7.4-5.el9.x86_64 2024-09-16T00:21:10.257 INFO:teuthology.orchestra.run.smithi038.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T00:21:10.257 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:10.257 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:10.258 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:10.258 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:10.258 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:21:10.258 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:10.258 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:10.259 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:10.259 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:21:10.259 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:21:10.259 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:21:10.259 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:21:10.259 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:21:10.260 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:10.260 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:10.260 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:21:10.260 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:10.260 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:10.261 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:10.261 INFO:teuthology.orchestra.run.smithi038.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:21:10.261 INFO:teuthology.orchestra.run.smithi038.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:21:10.261 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:10.261 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:10.262 INFO:teuthology.orchestra.run.smithi038.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:21:10.262 INFO:teuthology.orchestra.run.smithi038.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-16T00:21:10.262 INFO:teuthology.orchestra.run.smithi038.stdout: flex-2.6.4-9.el9.x86_64 2024-09-16T00:21:10.262 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T00:21:10.262 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T00:21:10.262 INFO:teuthology.orchestra.run.smithi038.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T00:21:10.263 INFO:teuthology.orchestra.run.smithi038.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-16T00:21:10.263 INFO:teuthology.orchestra.run.smithi038.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T00:21:10.263 INFO:teuthology.orchestra.run.smithi038.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-16T00:21:10.263 INFO:teuthology.orchestra.run.smithi038.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-16T00:21:10.263 INFO:teuthology.orchestra.run.smithi038.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T00:21:10.263 INFO:teuthology.orchestra.run.smithi038.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T00:21:10.263 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:10.263 INFO:teuthology.orchestra.run.smithi038.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:10.263 INFO:teuthology.orchestra.run.smithi038.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:10.263 INFO:teuthology.orchestra.run.smithi038.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T00:21:10.264 INFO:teuthology.orchestra.run.smithi038.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T00:21:10.264 INFO:teuthology.orchestra.run.smithi038.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T00:21:10.264 INFO:teuthology.orchestra.run.smithi038.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T00:21:10.264 INFO:teuthology.orchestra.run.smithi038.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T00:21:10.264 INFO:teuthology.orchestra.run.smithi038.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:10.264 INFO:teuthology.orchestra.run.smithi038.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:10.264 INFO:teuthology.orchestra.run.smithi038.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T00:21:10.264 INFO:teuthology.orchestra.run.smithi038.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:10.264 INFO:teuthology.orchestra.run.smithi038.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-16T00:21:10.264 INFO:teuthology.orchestra.run.smithi038.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T00:21:10.264 INFO:teuthology.orchestra.run.smithi038.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-16T00:21:10.264 INFO:teuthology.orchestra.run.smithi038.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-16T00:21:10.264 INFO:teuthology.orchestra.run.smithi038.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T00:21:10.265 INFO:teuthology.orchestra.run.smithi038.stdout: lua-5.4.4-4.el9.x86_64 2024-09-16T00:21:10.265 INFO:teuthology.orchestra.run.smithi038.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-16T00:21:10.265 INFO:teuthology.orchestra.run.smithi038.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-16T00:21:10.265 INFO:teuthology.orchestra.run.smithi038.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T00:21:10.265 INFO:teuthology.orchestra.run.smithi038.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T00:21:10.265 INFO:teuthology.orchestra.run.smithi038.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-16T00:21:10.265 INFO:teuthology.orchestra.run.smithi038.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T00:21:10.265 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-13.16-1.el9.x86_64 2024-09-16T00:21:10.265 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-16T00:21:10.265 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-16T00:21:10.266 INFO:teuthology.orchestra.run.smithi038.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-16T00:21:10.266 INFO:teuthology.orchestra.run.smithi038.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-16T00:21:10.266 INFO:teuthology.orchestra.run.smithi038.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-16T00:21:10.266 INFO:teuthology.orchestra.run.smithi038.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T00:21:10.266 INFO:teuthology.orchestra.run.smithi038.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-16T00:21:10.266 INFO:teuthology.orchestra.run.smithi038.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T00:21:10.266 INFO:teuthology.orchestra.run.smithi038.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T00:21:10.266 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T00:21:10.266 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:10.266 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:10.266 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:10.267 INFO:teuthology.orchestra.run.smithi038.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T00:21:10.267 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T00:21:10.267 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T00:21:10.267 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T00:21:10.267 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-16T00:21:10.267 INFO:teuthology.orchestra.run.smithi038.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-16T00:21:10.267 INFO:teuthology.orchestra.run.smithi038.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T00:21:10.267 INFO:teuthology.orchestra.run.smithi038.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T00:21:10.267 INFO:teuthology.orchestra.run.smithi038.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T00:21:10.267 INFO:teuthology.orchestra.run.smithi038.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-16T00:21:10.268 INFO:teuthology.orchestra.run.smithi038.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-16T00:21:10.268 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T00:21:10.268 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T00:21:10.268 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T00:21:10.268 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T00:21:10.268 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T00:21:10.268 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T00:21:10.268 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T00:21:10.268 INFO:teuthology.orchestra.run.smithi038.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T00:21:10.269 INFO:teuthology.orchestra.run.smithi038.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T00:21:10.269 INFO:teuthology.orchestra.run.smithi038.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T00:21:10.269 INFO:teuthology.orchestra.run.smithi038.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T00:21:10.269 INFO:teuthology.orchestra.run.smithi038.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T00:21:10.269 INFO:teuthology.orchestra.run.smithi038.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T00:21:10.269 INFO:teuthology.orchestra.run.smithi038.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T00:21:10.269 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T00:21:10.269 INFO:teuthology.orchestra.run.smithi038.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T00:21:10.269 INFO:teuthology.orchestra.run.smithi038.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T00:21:10.269 INFO:teuthology.orchestra.run.smithi038.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T00:21:10.269 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T00:21:10.269 INFO:teuthology.orchestra.run.smithi038.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T00:21:10.269 INFO:teuthology.orchestra.run.smithi038.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T00:21:10.270 INFO:teuthology.orchestra.run.smithi038.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T00:21:10.270 INFO:teuthology.orchestra.run.smithi038.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-16T00:21:10.270 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T00:21:10.270 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T00:21:10.270 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T00:21:10.270 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T00:21:10.270 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-16T00:21:10.270 INFO:teuthology.orchestra.run.smithi038.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T00:21:10.270 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:10.270 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:10.270 INFO:teuthology.orchestra.run.smithi038.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T00:21:10.270 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T00:21:10.270 INFO:teuthology.orchestra.run.smithi038.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T00:21:10.271 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:10.271 INFO:teuthology.orchestra.run.smithi038.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T00:21:10.271 INFO:teuthology.orchestra.run.smithi038.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T00:21:10.271 INFO:teuthology.orchestra.run.smithi038.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T00:21:10.271 INFO:teuthology.orchestra.run.smithi038.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T00:21:10.271 INFO:teuthology.orchestra.run.smithi038.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T00:21:10.271 INFO:teuthology.orchestra.run.smithi038.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T00:21:10.271 INFO:teuthology.orchestra.run.smithi038.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T00:21:10.271 INFO:teuthology.orchestra.run.smithi038.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-16T00:21:10.271 INFO:teuthology.orchestra.run.smithi038.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T00:21:10.272 INFO:teuthology.orchestra.run.smithi038.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T00:21:10.272 INFO:teuthology.orchestra.run.smithi038.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T00:21:10.272 INFO:teuthology.orchestra.run.smithi038.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T00:21:10.272 INFO:teuthology.orchestra.run.smithi038.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-16T00:21:10.272 INFO:teuthology.orchestra.run.smithi038.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-16T00:21:10.272 INFO:teuthology.orchestra.run.smithi038.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-16T00:21:10.272 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:10.272 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:10.272 INFO:teuthology.orchestra.run.smithi038.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:10.272 INFO:teuthology.orchestra.run.smithi038.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T00:21:10.272 INFO:teuthology.orchestra.run.smithi038.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T00:21:10.272 INFO:teuthology.orchestra.run.smithi038.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T00:21:10.272 INFO:teuthology.orchestra.run.smithi038.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-16T00:21:10.272 INFO:teuthology.orchestra.run.smithi038.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-16T00:21:10.273 INFO:teuthology.orchestra.run.smithi038.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T00:21:10.273 INFO:teuthology.orchestra.run.smithi038.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-16T00:21:10.273 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:21:10.273 INFO:teuthology.orchestra.run.smithi038.stdout:Complete! 2024-09-16T00:21:10.795 DEBUG:teuthology.parallel:result is None 2024-09-16T00:21:14.304 INFO:teuthology.orchestra.run.smithi122.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:21:14.304 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 1/164 2024-09-16T00:21:14.304 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2/164 2024-09-16T00:21:14.304 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 3/164 2024-09-16T00:21:14.305 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 4/164 2024-09-16T00:21:14.305 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-immutable-object-cache-2:19.1.1-326.g8921c8 5/164 2024-09-16T00:21:14.305 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 6/164 2024-09-16T00:21:14.305 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 7/164 2024-09-16T00:21:14.305 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 8/164 2024-09-16T00:21:14.305 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 9/164 2024-09-16T00:21:14.305 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 10/164 2024-09-16T00:21:14.305 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 11/164 2024-09-16T00:21:14.305 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 12/164 2024-09-16T00:21:14.305 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 13/164 2024-09-16T00:21:14.305 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 14/164 2024-09-16T00:21:14.305 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_6 15/164 2024-09-16T00:21:14.306 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 16/164 2024-09-16T00:21:14.306 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 17/164 2024-09-16T00:21:14.306 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 18/164 2024-09-16T00:21:14.306 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_ 19/164 2024-09-16T00:21:14.306 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 20/164 2024-09-16T00:21:14.306 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9 21/164 2024-09-16T00:21:14.306 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x 22/164 2024-09-16T00:21:14.306 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 23/164 2024-09-16T00:21:14.306 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 24/164 2024-09-16T00:21:14.306 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 25/164 2024-09-16T00:21:14.306 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 26/164 2024-09-16T00:21:14.306 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 27/164 2024-09-16T00:21:14.306 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 28/164 2024-09-16T00:21:14.307 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 29/164 2024-09-16T00:21:14.307 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.e 30/164 2024-09-16T00:21:14.307 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noar 31/164 2024-09-16T00:21:14.318 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.no 32/164 2024-09-16T00:21:14.319 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.1.1-326.g8921 33/164 2024-09-16T00:21:14.319 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9 34/164 2024-09-16T00:21:14.319 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 35/164 2024-09-16T00:21:14.319 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el 36/164 2024-09-16T00:21:14.319 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 37/164 2024-09-16T00:21:14.319 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 38/164 2024-09-16T00:21:14.319 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 39/164 2024-09-16T00:21:14.319 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/164 2024-09-16T00:21:14.319 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/164 2024-09-16T00:21:14.319 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/164 2024-09-16T00:21:14.319 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-09-16T00:21:14.319 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-09-16T00:21:14.319 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-09-16T00:21:14.320 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-09-16T00:21:14.320 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-09-16T00:21:14.320 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/164 2024-09-16T00:21:14.320 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-09-16T00:21:14.320 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-09-16T00:21:14.320 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-09-16T00:21:14.320 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/164 2024-09-16T00:21:14.320 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-09-16T00:21:14.320 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-09-16T00:21:14.320 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-09-16T00:21:14.320 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-09-16T00:21:14.320 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-09-16T00:21:14.320 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-09-16T00:21:14.321 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-09-16T00:21:14.321 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-09-16T00:21:14.321 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 61/164 2024-09-16T00:21:14.321 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 62/164 2024-09-16T00:21:14.321 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-09-16T00:21:14.321 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-09-16T00:21:14.321 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-09-16T00:21:14.321 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 66/164 2024-09-16T00:21:14.321 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 67/164 2024-09-16T00:21:14.321 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 68/164 2024-09-16T00:21:14.321 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : postgresql-13.16-1.el9.x86_64 69/164 2024-09-16T00:21:14.321 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 70/164 2024-09-16T00:21:14.322 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 71/164 2024-09-16T00:21:14.322 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 72/164 2024-09-16T00:21:14.322 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-09-16T00:21:14.322 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-09-16T00:21:14.322 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-09-16T00:21:14.322 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 76/164 2024-09-16T00:21:14.322 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 77/164 2024-09-16T00:21:14.322 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 78/164 2024-09-16T00:21:14.322 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 79/164 2024-09-16T00:21:14.322 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 80/164 2024-09-16T00:21:14.322 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 81/164 2024-09-16T00:21:14.322 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 82/164 2024-09-16T00:21:14.322 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 83/164 2024-09-16T00:21:14.323 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 84/164 2024-09-16T00:21:14.323 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 85/164 2024-09-16T00:21:14.323 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/164 2024-09-16T00:21:14.323 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 87/164 2024-09-16T00:21:14.323 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 88/164 2024-09-16T00:21:14.323 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 89/164 2024-09-16T00:21:14.323 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 90/164 2024-09-16T00:21:14.323 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 91/164 2024-09-16T00:21:14.323 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 92/164 2024-09-16T00:21:14.323 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 93/164 2024-09-16T00:21:14.323 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 94/164 2024-09-16T00:21:14.323 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 95/164 2024-09-16T00:21:14.323 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 96/164 2024-09-16T00:21:14.324 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 97/164 2024-09-16T00:21:14.324 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 98/164 2024-09-16T00:21:14.324 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 99/164 2024-09-16T00:21:14.324 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 100/164 2024-09-16T00:21:14.324 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 101/164 2024-09-16T00:21:14.324 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 102/164 2024-09-16T00:21:14.324 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 103/164 2024-09-16T00:21:14.324 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 104/164 2024-09-16T00:21:14.324 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 105/164 2024-09-16T00:21:14.324 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 106/164 2024-09-16T00:21:14.324 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 107/164 2024-09-16T00:21:14.324 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 108/164 2024-09-16T00:21:14.324 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 109/164 2024-09-16T00:21:14.325 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 110/164 2024-09-16T00:21:14.325 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 111/164 2024-09-16T00:21:14.325 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 112/164 2024-09-16T00:21:14.325 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 113/164 2024-09-16T00:21:14.325 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 114/164 2024-09-16T00:21:14.325 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 115/164 2024-09-16T00:21:14.325 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 116/164 2024-09-16T00:21:14.325 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 117/164 2024-09-16T00:21:14.325 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 118/164 2024-09-16T00:21:14.325 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 119/164 2024-09-16T00:21:14.325 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 120/164 2024-09-16T00:21:14.325 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 121/164 2024-09-16T00:21:14.325 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 122/164 2024-09-16T00:21:14.326 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-09-16T00:21:14.326 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-09-16T00:21:14.326 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-09-16T00:21:14.326 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-09-16T00:21:14.326 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-09-16T00:21:14.326 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-09-16T00:21:14.326 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-09-16T00:21:14.327 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-09-16T00:21:14.327 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-09-16T00:21:14.327 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-09-16T00:21:14.327 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-09-16T00:21:14.327 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-09-16T00:21:14.327 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-09-16T00:21:14.327 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-09-16T00:21:14.327 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-09-16T00:21:14.327 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-09-16T00:21:14.327 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-09-16T00:21:14.327 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-09-16T00:21:14.327 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-09-16T00:21:14.327 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 142/164 2024-09-16T00:21:14.328 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 143/164 2024-09-16T00:21:14.328 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-09-16T00:21:14.328 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-09-16T00:21:14.328 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-09-16T00:21:14.328 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : re2-1:20211101-20.el9.x86_64 147/164 2024-09-16T00:21:14.328 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-09-16T00:21:14.328 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 149/164 2024-09-16T00:21:14.328 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-09-16T00:21:14.328 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 151/164 2024-09-16T00:21:14.328 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-09-16T00:21:14.328 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-1:1.49.90-1.el9.x86_64 153/164 2024-09-16T00:21:14.329 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 154/164 2024-09-16T00:21:14.329 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-libnm-1:1.49.90-1.el9.x86_64 155/164 2024-09-16T00:21:14.329 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 156/164 2024-09-16T00:21:14.329 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-team-1:1.49.90-1.el9.x86_64 157/164 2024-09-16T00:21:14.329 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 158/164 2024-09-16T00:21:14.329 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-tui-1:1.49.90-1.el9.x86_64 159/164 2024-09-16T00:21:14.329 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 160/164 2024-09-16T00:21:14.329 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 161/164 2024-09-16T00:21:14.330 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 162/164 2024-09-16T00:21:14.330 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 163/164 2024-09-16T00:21:17.086 INFO:teuthology.orchestra.run.smithi122.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 164/164 2024-09-16T00:21:17.086 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-16T00:21:17.086 INFO:teuthology.orchestra.run.smithi122.stdout:Upgraded: 2024-09-16T00:21:17.086 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-1:1.49.90-1.el9.x86_64 2024-09-16T00:21:17.086 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-libnm-1:1.49.90-1.el9.x86_64 2024-09-16T00:21:17.086 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-team-1:1.49.90-1.el9.x86_64 2024-09-16T00:21:17.086 INFO:teuthology.orchestra.run.smithi122.stdout: NetworkManager-tui-1:1.49.90-1.el9.x86_64 2024-09-16T00:21:17.086 INFO:teuthology.orchestra.run.smithi122.stdout: librados2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:17.086 INFO:teuthology.orchestra.run.smithi122.stdout: librbd1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:17.087 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-16T00:21:17.087 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-16T00:21:17.087 INFO:teuthology.orchestra.run.smithi122.stdout:Installed: 2024-09-16T00:21:17.087 INFO:teuthology.orchestra.run.smithi122.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-16T00:21:17.087 INFO:teuthology.orchestra.run.smithi122.stdout: automake-1.16.2-8.el9.noarch 2024-09-16T00:21:17.087 INFO:teuthology.orchestra.run.smithi122.stdout: bison-3.7.4-5.el9.x86_64 2024-09-16T00:21:17.087 INFO:teuthology.orchestra.run.smithi122.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-16T00:21:17.087 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:17.087 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-base-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:17.087 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:17.087 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:17.087 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-grafana-dashboards-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:21:17.087 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-immutable-object-cache-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:17.088 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mds-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:17.088 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:17.088 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:21:17.088 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-dashboard-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:21:17.088 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-diskprediction-local-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:21:17.088 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-modules-core-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:21:17.088 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mgr-rook-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:21:17.088 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-mon-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:17.088 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-osd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:17.088 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-prometheus-alerts-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:21:17.088 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-radosgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:17.088 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-selinux-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:17.088 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-test-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:17.089 INFO:teuthology.orchestra.run.smithi122.stdout: ceph-volume-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:21:17.089 INFO:teuthology.orchestra.run.smithi122.stdout: cephadm-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:21:17.089 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:17.089 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-shell-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:17.089 INFO:teuthology.orchestra.run.smithi122.stdout: cephfs-top-2:19.1.1-326.g8921c8ef.el9.noarch 2024-09-16T00:21:17.089 INFO:teuthology.orchestra.run.smithi122.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-16T00:21:17.089 INFO:teuthology.orchestra.run.smithi122.stdout: flex-2.6.4-9.el9.x86_64 2024-09-16T00:21:17.089 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-16T00:21:17.089 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-16T00:21:17.089 INFO:teuthology.orchestra.run.smithi122.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-16T00:21:17.089 INFO:teuthology.orchestra.run.smithi122.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-16T00:21:17.090 INFO:teuthology.orchestra.run.smithi122.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-16T00:21:17.090 INFO:teuthology.orchestra.run.smithi122.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-16T00:21:17.090 INFO:teuthology.orchestra.run.smithi122.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-16T00:21:17.090 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-16T00:21:17.090 INFO:teuthology.orchestra.run.smithi122.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-16T00:21:17.090 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:17.090 INFO:teuthology.orchestra.run.smithi122.stdout: libcephfs2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:17.090 INFO:teuthology.orchestra.run.smithi122.stdout: libcephsqlite-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:17.090 INFO:teuthology.orchestra.run.smithi122.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-16T00:21:17.090 INFO:teuthology.orchestra.run.smithi122.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-16T00:21:17.090 INFO:teuthology.orchestra.run.smithi122.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-16T00:21:17.091 INFO:teuthology.orchestra.run.smithi122.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-16T00:21:17.091 INFO:teuthology.orchestra.run.smithi122.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-16T00:21:17.091 INFO:teuthology.orchestra.run.smithi122.stdout: librados-devel-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:17.091 INFO:teuthology.orchestra.run.smithi122.stdout: libradosstriper1-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:17.091 INFO:teuthology.orchestra.run.smithi122.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-16T00:21:17.091 INFO:teuthology.orchestra.run.smithi122.stdout: librgw2-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:17.091 INFO:teuthology.orchestra.run.smithi122.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-16T00:21:17.091 INFO:teuthology.orchestra.run.smithi122.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-16T00:21:17.091 INFO:teuthology.orchestra.run.smithi122.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-16T00:21:17.091 INFO:teuthology.orchestra.run.smithi122.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-16T00:21:17.091 INFO:teuthology.orchestra.run.smithi122.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-16T00:21:17.091 INFO:teuthology.orchestra.run.smithi122.stdout: lua-5.4.4-4.el9.x86_64 2024-09-16T00:21:17.091 INFO:teuthology.orchestra.run.smithi122.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-16T00:21:17.091 INFO:teuthology.orchestra.run.smithi122.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-16T00:21:17.092 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-16T00:21:17.092 INFO:teuthology.orchestra.run.smithi122.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-16T00:21:17.092 INFO:teuthology.orchestra.run.smithi122.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-16T00:21:17.092 INFO:teuthology.orchestra.run.smithi122.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-16T00:21:17.092 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-13.16-1.el9.x86_64 2024-09-16T00:21:17.092 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-16T00:21:17.092 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-16T00:21:17.092 INFO:teuthology.orchestra.run.smithi122.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-16T00:21:17.092 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-16T00:21:17.092 INFO:teuthology.orchestra.run.smithi122.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-16T00:21:17.092 INFO:teuthology.orchestra.run.smithi122.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-16T00:21:17.092 INFO:teuthology.orchestra.run.smithi122.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-16T00:21:17.092 INFO:teuthology.orchestra.run.smithi122.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-16T00:21:17.093 INFO:teuthology.orchestra.run.smithi122.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-16T00:21:17.093 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-16T00:21:17.093 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-argparse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:17.093 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ceph-common-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:17.093 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cephfs-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:17.093 INFO:teuthology.orchestra.run.smithi122.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-16T00:21:17.093 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-16T00:21:17.093 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-16T00:21:17.093 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-16T00:21:17.093 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-16T00:21:17.093 INFO:teuthology.orchestra.run.smithi122.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-16T00:21:17.093 INFO:teuthology.orchestra.run.smithi122.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-16T00:21:17.093 INFO:teuthology.orchestra.run.smithi122.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-16T00:21:17.094 INFO:teuthology.orchestra.run.smithi122.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-16T00:21:17.094 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-16T00:21:17.094 INFO:teuthology.orchestra.run.smithi122.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-16T00:21:17.094 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-16T00:21:17.094 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-16T00:21:17.094 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-16T00:21:17.094 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-16T00:21:17.094 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-16T00:21:17.094 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-16T00:21:17.094 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-16T00:21:17.094 INFO:teuthology.orchestra.run.smithi122.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-16T00:21:17.094 INFO:teuthology.orchestra.run.smithi122.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-16T00:21:17.094 INFO:teuthology.orchestra.run.smithi122.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-16T00:21:17.094 INFO:teuthology.orchestra.run.smithi122.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-16T00:21:17.095 INFO:teuthology.orchestra.run.smithi122.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-16T00:21:17.095 INFO:teuthology.orchestra.run.smithi122.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-16T00:21:17.095 INFO:teuthology.orchestra.run.smithi122.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-16T00:21:17.095 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-16T00:21:17.095 INFO:teuthology.orchestra.run.smithi122.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-16T00:21:17.095 INFO:teuthology.orchestra.run.smithi122.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-16T00:21:17.095 INFO:teuthology.orchestra.run.smithi122.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-16T00:21:17.095 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-16T00:21:17.095 INFO:teuthology.orchestra.run.smithi122.stdout: python3-ply-3.11-14.el9.noarch 2024-09-16T00:21:17.095 INFO:teuthology.orchestra.run.smithi122.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-16T00:21:17.095 INFO:teuthology.orchestra.run.smithi122.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-16T00:21:17.096 INFO:teuthology.orchestra.run.smithi122.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-16T00:21:17.096 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-16T00:21:17.096 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-16T00:21:17.096 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-16T00:21:17.096 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-16T00:21:17.096 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-16T00:21:17.096 INFO:teuthology.orchestra.run.smithi122.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-16T00:21:17.096 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rados-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:17.096 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:17.097 INFO:teuthology.orchestra.run.smithi122.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-16T00:21:17.097 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-16T00:21:17.097 INFO:teuthology.orchestra.run.smithi122.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-16T00:21:17.097 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rgw-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:17.097 INFO:teuthology.orchestra.run.smithi122.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-16T00:21:17.097 INFO:teuthology.orchestra.run.smithi122.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-16T00:21:17.097 INFO:teuthology.orchestra.run.smithi122.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-16T00:21:17.097 INFO:teuthology.orchestra.run.smithi122.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-16T00:21:17.097 INFO:teuthology.orchestra.run.smithi122.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-16T00:21:17.097 INFO:teuthology.orchestra.run.smithi122.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-16T00:21:17.097 INFO:teuthology.orchestra.run.smithi122.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-16T00:21:17.097 INFO:teuthology.orchestra.run.smithi122.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-16T00:21:17.097 INFO:teuthology.orchestra.run.smithi122.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-16T00:21:17.097 INFO:teuthology.orchestra.run.smithi122.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-16T00:21:17.098 INFO:teuthology.orchestra.run.smithi122.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-16T00:21:17.098 INFO:teuthology.orchestra.run.smithi122.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-16T00:21:17.098 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-16T00:21:17.098 INFO:teuthology.orchestra.run.smithi122.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-16T00:21:17.098 INFO:teuthology.orchestra.run.smithi122.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-16T00:21:17.098 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-fuse-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:17.098 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-mirror-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:17.098 INFO:teuthology.orchestra.run.smithi122.stdout: rbd-nbd-2:19.1.1-326.g8921c8ef.el9.x86_64 2024-09-16T00:21:17.098 INFO:teuthology.orchestra.run.smithi122.stdout: re2-1:20211101-20.el9.x86_64 2024-09-16T00:21:17.098 INFO:teuthology.orchestra.run.smithi122.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-16T00:21:17.098 INFO:teuthology.orchestra.run.smithi122.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-16T00:21:17.098 INFO:teuthology.orchestra.run.smithi122.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-16T00:21:17.098 INFO:teuthology.orchestra.run.smithi122.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-16T00:21:17.098 INFO:teuthology.orchestra.run.smithi122.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-16T00:21:17.099 INFO:teuthology.orchestra.run.smithi122.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-16T00:21:17.099 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-16T00:21:17.099 INFO:teuthology.orchestra.run.smithi122.stdout:Complete! 2024-09-16T00:21:17.533 DEBUG:teuthology.parallel:result is None 2024-09-16T00:21:17.533 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:21:17.675 DEBUG:teuthology.orchestra.run.smithi038:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T00:21:17.703 INFO:teuthology.orchestra.run.smithi038.stdout:19.1.1-326.g8921c8ef.el9 2024-09-16T00:21:17.704 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-16T00:21:17.704 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-16T00:21:17.707 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:21:17.849 DEBUG:teuthology.orchestra.run.smithi073:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T00:21:17.876 INFO:teuthology.orchestra.run.smithi073.stdout:19.1.1-326.g8921c8ef.el9 2024-09-16T00:21:17.876 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-16T00:21:17.876 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-16T00:21:17.878 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:21:18.014 DEBUG:teuthology.orchestra.run.smithi122:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-16T00:21:18.036 INFO:teuthology.orchestra.run.smithi122.stdout:19.1.1-326.g8921c8ef.el9 2024-09-16T00:21:18.037 INFO:teuthology.packaging:The installed version of ceph is 19.1.1-326.g8921c8ef.el9 2024-09-16T00:21:18.037 INFO:teuthology.task.install:The correct ceph version 19.1.1-326.g8921c8ef is installed. 2024-09-16T00:21:18.038 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-16T00:21:18.039 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-16T00:21:18.039 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T00:21:18.072 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-16T00:21:18.072 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T00:21:18.105 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-16T00:21:18.105 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-16T00:21:18.132 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-16T00:21:18.133 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-16T00:21:18.133 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T00:21:18.162 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T00:21:18.232 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-16T00:21:18.232 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T00:21:18.261 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T00:21:18.331 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-16T00:21:18.331 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/daemon-helper 2024-09-16T00:21:18.356 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-16T00:21:18.421 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-16T00:21:18.421 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-16T00:21:18.421 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T00:21:18.454 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T00:21:18.522 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-16T00:21:18.522 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T00:21:18.563 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T00:21:18.632 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-16T00:21:18.632 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-16T00:21:18.657 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-16T00:21:18.722 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-16T00:21:18.722 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-16T00:21:18.722 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T00:21:18.748 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T00:21:18.814 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-16T00:21:18.814 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T00:21:18.840 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T00:21:18.905 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-16T00:21:18.905 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/usr/bin/stdin-killer 2024-09-16T00:21:18.930 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-16T00:21:18.995 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-16T00:21:19.166 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120, 'mon warn on pool no app': False}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'subvols': {'create': 2}} 2024-09-16T00:21:19.166 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:21:19.167 INFO:tasks.cephadm:Cluster fsid is 983b2072-73c1-11ef-bceb-c7b262605968 2024-09-16T00:21:19.167 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-16T00:21:19.167 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.38', 'mon.b': '172.21.15.73', 'mon.c': '172.21.15.122'} 2024-09-16T00:21:19.167 INFO:tasks.cephadm:First mon is mon.a on smithi038 2024-09-16T00:21:19.167 INFO:tasks.cephadm:First mgr is x 2024-09-16T00:21:19.167 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-16T00:21:19.167 DEBUG:teuthology.orchestra.run.smithi038:> sudo hostname $(hostname -s) 2024-09-16T00:21:19.195 DEBUG:teuthology.orchestra.run.smithi073:> sudo hostname $(hostname -s) 2024-09-16T00:21:19.222 DEBUG:teuthology.orchestra.run.smithi122:> sudo hostname $(hostname -s) 2024-09-16T00:21:19.247 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-16T00:21:19.248 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:21:19.384 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '8921c8ef947807b717db041d85f0761b7914d5fc', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/56081/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.1.1-326-g8921c8ef', 'node_name': '172.21.5.31+adami01', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.1.1-326.g8921c8ef'}, 'url': 'https://4.chacra.ceph.com/r/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-15 16:06:42.385367', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2024-09-16T00:21:19.530 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref squid, sha1 8921c8ef947807b717db041d85f0761b7914d5fc from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:21:19.531 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm 2024-09-16T00:21:19.531 INFO:tasks.cephadm:Downloading cephadm from url: https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm 2024-09-16T00:21:19.532 DEBUG:teuthology.orchestra.run.smithi038:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-16T00:21:19.890 INFO:teuthology.orchestra.run.smithi038.stdout:-rw-r--r--. 1 ubuntu ubuntu 784481 Sep 16 00:21 /home/ubuntu/cephtest/cephadm 2024-09-16T00:21:19.890 DEBUG:teuthology.orchestra.run.smithi073:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-16T00:21:20.269 INFO:teuthology.orchestra.run.smithi073.stdout:-rw-r--r--. 1 ubuntu ubuntu 784481 Sep 16 00:21 /home/ubuntu/cephtest/cephadm 2024-09-16T00:21:20.269 DEBUG:teuthology.orchestra.run.smithi122:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/squid/8921c8ef947807b717db041d85f0761b7914d5fc/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-16T00:21:20.663 INFO:teuthology.orchestra.run.smithi122.stdout:-rw-r--r--. 1 ubuntu ubuntu 784481 Sep 16 00:21 /home/ubuntu/cephtest/cephadm 2024-09-16T00:21:20.663 DEBUG:teuthology.orchestra.run.smithi038:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-16T00:21:20.685 DEBUG:teuthology.orchestra.run.smithi073:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-16T00:21:20.703 DEBUG:teuthology.orchestra.run.smithi122:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-16T00:21:20.946 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc on all hosts... 2024-09-16T00:21:20.947 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc pull 2024-09-16T00:21:20.949 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc pull 2024-09-16T00:21:20.951 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc pull 2024-09-16T00:21:22.570 INFO:teuthology.orchestra.run.smithi038.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-16T00:21:22.641 INFO:teuthology.orchestra.run.smithi073.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-16T00:21:22.657 INFO:teuthology.orchestra.run.smithi122.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-16T00:22:01.009 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-16T00:22:01.010 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-16T00:22:01.010 INFO:teuthology.orchestra.run.smithi038.stdout: "image_id": "f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee", 2024-09-16T00:22:01.010 INFO:teuthology.orchestra.run.smithi038.stdout: "repo_digests": [ 2024-09-16T00:22:01.010 INFO:teuthology.orchestra.run.smithi038.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80" 2024-09-16T00:22:01.010 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-09-16T00:22:01.010 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-16T00:22:16.566 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-16T00:22:16.566 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-16T00:22:16.566 INFO:teuthology.orchestra.run.smithi122.stdout: "image_id": "f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee", 2024-09-16T00:22:16.566 INFO:teuthology.orchestra.run.smithi122.stdout: "repo_digests": [ 2024-09-16T00:22:16.566 INFO:teuthology.orchestra.run.smithi122.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80" 2024-09-16T00:22:16.566 INFO:teuthology.orchestra.run.smithi122.stdout: ] 2024-09-16T00:22:16.566 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-16T00:22:18.529 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-16T00:22:18.529 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-16T00:22:18.529 INFO:teuthology.orchestra.run.smithi073.stdout: "image_id": "f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee", 2024-09-16T00:22:18.529 INFO:teuthology.orchestra.run.smithi073.stdout: "repo_digests": [ 2024-09-16T00:22:18.529 INFO:teuthology.orchestra.run.smithi073.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80" 2024-09-16T00:22:18.530 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-09-16T00:22:18.530 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-16T00:22:18.554 DEBUG:teuthology.orchestra.run.smithi038:> sudo mkdir -p /etc/ceph 2024-09-16T00:22:18.588 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /etc/ceph 2024-09-16T00:22:18.615 DEBUG:teuthology.orchestra.run.smithi122:> sudo mkdir -p /etc/ceph 2024-09-16T00:22:18.648 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 777 /etc/ceph 2024-09-16T00:22:18.678 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 777 /etc/ceph 2024-09-16T00:22:18.703 DEBUG:teuthology.orchestra.run.smithi122:> sudo chmod 777 /etc/ceph 2024-09-16T00:22:18.731 INFO:tasks.cephadm:Writing seed config... 2024-09-16T00:22:18.732 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-09-16T00:22:18.732 INFO:tasks.cephadm: override: [client] debug client = 20 2024-09-16T00:22:18.732 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-16T00:22:18.732 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-09-16T00:22:18.732 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-09-16T00:22:18.733 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-16T00:22:18.733 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-09-16T00:22:18.733 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-09-16T00:22:18.733 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-09-16T00:22:18.733 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-09-16T00:22:18.733 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-09-16T00:22:18.733 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-09-16T00:22:18.733 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-09-16T00:22:18.733 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-09-16T00:22:18.733 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-09-16T00:22:18.733 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-09-16T00:22:18.733 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-09-16T00:22:18.733 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-09-16T00:22:18.733 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-09-16T00:22:18.734 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-09-16T00:22:18.734 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-09-16T00:22:18.734 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-09-16T00:22:18.734 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-16T00:22:18.734 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-09-16T00:22:18.734 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-16T00:22:18.734 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-16T00:22:18.734 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-16T00:22:18.734 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-16T00:22:18.734 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-16T00:22:18.734 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-16T00:22:18.734 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-09-16T00:22:18.734 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-16T00:22:18.735 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-16T00:22:18.735 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-16T00:22:18.735 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-09-16T00:22:18.735 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-09-16T00:22:18.735 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-16T00:22:18.735 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-16T00:22:18.735 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-09-16T00:22:18.735 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-16T00:22:18.735 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-16T00:22:18.735 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-16T00:22:18.735 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-16T00:22:18.735 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-16T00:22:18.735 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-09-16T00:22:18.735 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-16T00:22:18.736 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-16T00:22:18.736 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-16T00:22:18.737 DEBUG:teuthology.orchestra.run.smithi038:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-16T00:22:18.755 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 983b2072-73c1-11ef-bceb-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-09-16T00:22:18.756 DEBUG:teuthology.orchestra.run.smithi038:mon.a> sudo journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@mon.a.service 2024-09-16T00:22:18.799 DEBUG:teuthology.orchestra.run.smithi038:mgr.x> sudo journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@mgr.x.service 2024-09-16T00:22:18.842 INFO:tasks.cephadm:Bootstrapping... 2024-09-16T00:22:18.842 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc -v bootstrap --fsid 983b2072-73c1-11ef-bceb-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.38 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:22:19.058 INFO:teuthology.orchestra.run.smithi038.stdout:-------------------------------------------------------------------------------- 2024-09-16T00:22:19.058 INFO:teuthology.orchestra.run.smithi038.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc', '-v', 'bootstrap', '--fsid', '983b2072-73c1-11ef-bceb-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.38', '--skip-admin-label'] 2024-09-16T00:22:19.059 INFO:teuthology.orchestra.run.smithi038.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-16T00:22:19.059 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying podman|docker is present... 2024-09-16T00:22:19.083 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stdout 5.2.2 2024-09-16T00:22:19.084 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying lvm2 is present... 2024-09-16T00:22:19.084 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying time synchronization is in place... 2024-09-16T00:22:19.092 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-16T00:22:19.092 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-16T00:22:19.098 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-16T00:22:19.098 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout inactive 2024-09-16T00:22:19.105 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout enabled 2024-09-16T00:22:19.112 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout active 2024-09-16T00:22:19.112 INFO:teuthology.orchestra.run.smithi038.stdout:Unit chronyd.service is enabled and running 2024-09-16T00:22:19.112 INFO:teuthology.orchestra.run.smithi038.stdout:Repeating the final host check... 2024-09-16T00:22:19.135 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stdout 5.2.2 2024-09-16T00:22:19.135 INFO:teuthology.orchestra.run.smithi038.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-16T00:22:19.136 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl is present 2024-09-16T00:22:19.136 INFO:teuthology.orchestra.run.smithi038.stdout:lvcreate is present 2024-09-16T00:22:19.142 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-16T00:22:19.142 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-16T00:22:19.149 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-16T00:22:19.149 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout inactive 2024-09-16T00:22:19.155 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout enabled 2024-09-16T00:22:19.162 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout active 2024-09-16T00:22:19.162 INFO:teuthology.orchestra.run.smithi038.stdout:Unit chronyd.service is enabled and running 2024-09-16T00:22:19.163 INFO:teuthology.orchestra.run.smithi038.stdout:Host looks OK 2024-09-16T00:22:19.163 INFO:teuthology.orchestra.run.smithi038.stdout:Cluster fsid: 983b2072-73c1-11ef-bceb-c7b262605968 2024-09-16T00:22:19.163 INFO:teuthology.orchestra.run.smithi038.stdout:Acquiring lock 140214518572032 on /run/cephadm/983b2072-73c1-11ef-bceb-c7b262605968.lock 2024-09-16T00:22:19.163 INFO:teuthology.orchestra.run.smithi038.stdout:Lock 140214518572032 acquired on /run/cephadm/983b2072-73c1-11ef-bceb-c7b262605968.lock 2024-09-16T00:22:19.163 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying IP 172.21.15.38 port 3300 ... 2024-09-16T00:22:19.163 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying IP 172.21.15.38 port 6789 ... 2024-09-16T00:22:19.164 INFO:teuthology.orchestra.run.smithi038.stdout:Base mon IP(s) is [172.21.15.38:3300, 172.21.15.38:6789], mon addrv is [v2:172.21.15.38:3300,v1:172.21.15.38:6789] 2024-09-16T00:22:19.167 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.38 metric 100 2024-09-16T00:22:19.167 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.38 metric 100 2024-09-16T00:22:19.170 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-16T00:22:19.170 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-16T00:22:19.170 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1794sec hoplimit 64 pref medium 2024-09-16T00:22:19.172 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-16T00:22:19.172 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-16T00:22:19.173 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-16T00:22:19.173 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-16T00:22:19.173 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:15fc/64 scope link 2024-09-16T00:22:19.173 INFO:teuthology.orchestra.run.smithi038.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-16T00:22:19.173 INFO:teuthology.orchestra.run.smithi038.stdout:Mon IP `172.21.15.38` is in CIDR network `172.21.0.0/20` 2024-09-16T00:22:19.173 INFO:teuthology.orchestra.run.smithi038.stdout:Mon IP `172.21.15.38` is in CIDR network `172.21.0.0/20` 2024-09-16T00:22:19.174 INFO:teuthology.orchestra.run.smithi038.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-16T00:22:19.174 INFO:teuthology.orchestra.run.smithi038.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-16T00:22:19.175 INFO:teuthology.orchestra.run.smithi038.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-16T00:22:19.778 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stdout f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee 2024-09-16T00:22:19.778 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc... 2024-09-16T00:22:19.778 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stderr Getting image source signatures 2024-09-16T00:22:19.778 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stderr Copying blob sha256:c0574da0c3174f3dc49daa274908f97d06456bd6a3cfab87e94022c1369a83a6 2024-09-16T00:22:19.778 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-16T00:22:19.778 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stderr Copying config sha256:f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee 2024-09-16T00:22:19.778 INFO:teuthology.orchestra.run.smithi038.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-16T00:22:20.715 INFO:teuthology.orchestra.run.smithi038.stdout:ceph: stdout ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable) 2024-09-16T00:22:20.715 INFO:teuthology.orchestra.run.smithi038.stdout:Ceph version: ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable) 2024-09-16T00:22:20.715 INFO:teuthology.orchestra.run.smithi038.stdout:Extracting ceph user uid/gid from container image... 2024-09-16T00:22:21.805 INFO:teuthology.orchestra.run.smithi038.stdout:stat: stdout 167 167 2024-09-16T00:22:21.805 INFO:teuthology.orchestra.run.smithi038.stdout:Creating initial keys... 2024-09-16T00:22:28.784 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph-authtool: stdout AQBAeudmaoHwFRAAU6MUyds/HIR41V7AVNqRrQ== 2024-09-16T00:22:30.373 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph-authtool: stdout AQBFeudm2JA+KRAAkE1rA/N6Mm+cB9mfGcSmJA== 2024-09-16T00:22:31.321 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph-authtool: stdout AQBGeudmFw0mKhAAOdumWG/BG+KFdYTpODXtuw== 2024-09-16T00:22:31.322 INFO:teuthology.orchestra.run.smithi038.stdout:Creating initial monmap... 2024-09-16T00:22:32.253 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-16T00:22:32.253 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-16T00:22:32.253 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 983b2072-73c1-11ef-bceb-c7b262605968 2024-09-16T00:22:32.254 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-16T00:22:32.254 INFO:teuthology.orchestra.run.smithi038.stdout:monmaptool for a [v2:172.21.15.38:3300,v1:172.21.15.38:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-16T00:22:32.254 INFO:teuthology.orchestra.run.smithi038.stdout:setting min_mon_release = quincy 2024-09-16T00:22:32.254 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/monmaptool: set fsid to 983b2072-73c1-11ef-bceb-c7b262605968 2024-09-16T00:22:32.254 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-16T00:22:32.254 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:22:32.254 INFO:teuthology.orchestra.run.smithi038.stdout:Creating mon... 2024-09-16T00:22:33.351 INFO:teuthology.orchestra.run.smithi038.stdout:create mon.a on 2024-09-16T00:22:33.575 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-16T00:22:33.732 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-16T00:22:33.914 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-983b2072-73c1-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-983b2072-73c1-11ef-bceb-c7b262605968.target. 2024-09-16T00:22:33.915 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-983b2072-73c1-11ef-bceb-c7b262605968.target → /etc/systemd/system/ceph-983b2072-73c1-11ef-bceb-c7b262605968.target. 2024-09-16T00:22:34.242 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-983b2072-73c1-11ef-bceb-c7b262605968@mon.a 2024-09-16T00:22:34.243 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Failed to reset failed state of unit ceph-983b2072-73c1-11ef-bceb-c7b262605968@mon.a.service: Unit ceph-983b2072-73c1-11ef-bceb-c7b262605968@mon.a.service not loaded. 2024-09-16T00:22:34.410 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-983b2072-73c1-11ef-bceb-c7b262605968.target.wants/ceph-983b2072-73c1-11ef-bceb-c7b262605968@mon.a.service → /etc/systemd/system/ceph-983b2072-73c1-11ef-bceb-c7b262605968@.service. 2024-09-16T00:22:34.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:34 smithi038 podman[30559]: 2024-09-16 00:22:34.553742303 +0000 UTC m=+0.016209707 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:22:34.933 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T00:22:34.933 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout disabled 2024-09-16T00:22:34.940 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T00:22:34.941 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout inactive 2024-09-16T00:22:34.941 INFO:teuthology.orchestra.run.smithi038.stdout:firewalld.service is not enabled 2024-09-16T00:22:34.941 INFO:teuthology.orchestra.run.smithi038.stdout:Not possible to enable service . firewalld.service is not available 2024-09-16T00:22:34.941 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for mon to start... 2024-09-16T00:22:34.941 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for mon... 2024-09-16T00:22:35.064 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:34 smithi038 podman[30559]: 2024-09-16 00:22:34.845012583 +0000 UTC m=+0.307479977 container init 282e3793608133cfcf985ae5a3105aec77f125b4a23d8d39b1d5dc3dd22a163b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mon-a, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, ceph=True, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-16T00:22:35.064 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:34 smithi038 podman[30559]: 2024-09-16 00:22:34.848235058 +0000 UTC m=+0.310702453 container start 282e3793608133cfcf985ae5a3105aec77f125b4a23d8d39b1d5dc3dd22a163b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:22:35.064 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:34 smithi038 ceph-mon[30578]: mkfs 983b2072-73c1-11ef-bceb-c7b262605968 2024-09-16T00:22:35.064 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:34 smithi038 ceph-mon[30578]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-16T00:22:35.064 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:34 smithi038 bash[30559]: 282e3793608133cfcf985ae5a3105aec77f125b4a23d8d39b1d5dc3dd22a163b 2024-09-16T00:22:35.064 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:34 smithi038 systemd[1]: Started Ceph mon.a for 983b2072-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:22:36.026 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout cluster: 2024-09-16T00:22:36.026 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout id: 983b2072-73c1-11ef-bceb-c7b262605968 2024-09-16T00:22:36.026 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-16T00:22:36.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-09-16T00:22:36.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout services: 2024-09-16T00:22:36.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.536654s) 2024-09-16T00:22:36.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-16T00:22:36.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-16T00:22:36.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-09-16T00:22:36.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout data: 2024-09-16T00:22:36.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-16T00:22:36.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-16T00:22:36.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-16T00:22:36.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout pgs: 2024-09-16T00:22:36.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-09-16T00:22:36.027 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.359+0000 7fd3b459f640 1 Processor -- start 2024-09-16T00:22:36.028 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.359+0000 7fd3b459f640 1 -- start start 2024-09-16T00:22:36.028 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.360+0000 7fd3b459f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3ac108050 0x7fd3ac10a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:36.028 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.360+0000 7fd3b459f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac07be50 con 0x7fd3ac108050 2024-09-16T00:22:36.028 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.360+0000 7fd3b233b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3ac108050 0x7fd3ac10a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:36.028 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.360+0000 7fd3b233b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3ac108050 0x7fd3ac10a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48188/0 (socket says 172.21.15.38:48188) 2024-09-16T00:22:36.028 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.360+0000 7fd3b233b640 1 -- 172.21.15.38:0/1261456221 learned_addr learned my addr 172.21.15.38:0/1261456221 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:22:36.028 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.360+0000 7fd3b233b640 1 -- 172.21.15.38:0/1261456221 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3ac10a980 con 0x7fd3ac108050 2024-09-16T00:22:36.028 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.362+0000 7fd3b233b640 1 --2- 172.21.15.38:0/1261456221 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3ac108050 0x7fd3ac10a440 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4009820 tx=0x7fd3a402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7063ef6fa0950ea server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:36.028 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.362+0000 7fd3b1339640 1 -- 172.21.15.38:0/1261456221 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3a4037070 con 0x7fd3ac108050 2024-09-16T00:22:36.028 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.362+0000 7fd3b1339640 1 -- 172.21.15.38:0/1261456221 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3a4032440 con 0x7fd3ac108050 2024-09-16T00:22:36.028 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.362+0000 7fd3b1339640 1 -- 172.21.15.38:0/1261456221 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3a4032740 con 0x7fd3ac108050 2024-09-16T00:22:36.028 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.363+0000 7fd3b459f640 1 -- 172.21.15.38:0/1261456221 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3ac108050 msgr2=0x7fd3ac10a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:36.029 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.363+0000 7fd3b459f640 1 --2- 172.21.15.38:0/1261456221 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3ac108050 0x7fd3ac10a440 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd3a4009820 tx=0x7fd3a402a270 comp rx=0 tx=0).stop 2024-09-16T00:22:36.029 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.363+0000 7fd3b459f640 1 -- 172.21.15.38:0/1261456221 shutdown_connections 2024-09-16T00:22:36.029 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.363+0000 7fd3b459f640 1 --2- 172.21.15.38:0/1261456221 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3ac108050 0x7fd3ac10a440 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:36.029 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.363+0000 7fd3b459f640 1 -- 172.21.15.38:0/1261456221 >> 172.21.15.38:0/1261456221 conn(0x7fd3ac101c30 msgr2=0x7fd3ac1040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:36.029 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.363+0000 7fd3b459f640 1 -- 172.21.15.38:0/1261456221 shutdown_connections 2024-09-16T00:22:36.029 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.363+0000 7fd3b459f640 1 -- 172.21.15.38:0/1261456221 wait complete. 2024-09-16T00:22:36.029 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.364+0000 7fd3b459f640 1 Processor -- start 2024-09-16T00:22:36.029 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.364+0000 7fd3b459f640 1 -- start start 2024-09-16T00:22:36.029 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.364+0000 7fd3b459f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3ac108050 0x7fd3ac1078d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:36.029 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.364+0000 7fd3b459f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd3ac10ae60 con 0x7fd3ac108050 2024-09-16T00:22:36.029 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.364+0000 7fd3b233b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3ac108050 0x7fd3ac1078d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:36.029 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.364+0000 7fd3b233b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3ac108050 0x7fd3ac1078d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:48202/0 (socket says 172.21.15.38:48202) 2024-09-16T00:22:36.030 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.364+0000 7fd3b233b640 1 -- 172.21.15.38:0/2541889795 learned_addr learned my addr 172.21.15.38:0/2541889795 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:22:36.030 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.364+0000 7fd3b233b640 1 -- 172.21.15.38:0/2541889795 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3ac107e10 con 0x7fd3ac108050 2024-09-16T00:22:36.030 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.364+0000 7fd3b233b640 1 --2- 172.21.15.38:0/2541889795 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3ac108050 0x7fd3ac1078d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd3a40090b0 tx=0x7fd3a40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:36.030 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.365+0000 7fd3ab7fe640 1 -- 172.21.15.38:0/2541889795 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3a4041070 con 0x7fd3ac108050 2024-09-16T00:22:36.030 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.365+0000 7fd3ab7fe640 1 -- 172.21.15.38:0/2541889795 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd3a403c7a0 con 0x7fd3ac108050 2024-09-16T00:22:36.030 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.365+0000 7fd3ab7fe640 1 -- 172.21.15.38:0/2541889795 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd3a4037040 con 0x7fd3ac108050 2024-09-16T00:22:36.030 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.365+0000 7fd3b459f640 1 -- 172.21.15.38:0/2541889795 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3ac106000 con 0x7fd3ac108050 2024-09-16T00:22:36.030 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.365+0000 7fd3b459f640 1 -- 172.21.15.38:0/2541889795 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3ac106540 con 0x7fd3ac108050 2024-09-16T00:22:36.030 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.365+0000 7fd3ab7fe640 1 -- 172.21.15.38:0/2541889795 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd3a4044070 con 0x7fd3ac108050 2024-09-16T00:22:36.030 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.365+0000 7fd3ab7fe640 1 -- 172.21.15.38:0/2541889795 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd3a403d3f0 con 0x7fd3ac108050 2024-09-16T00:22:36.030 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.366+0000 7fd3b459f640 1 -- 172.21.15.38:0/2541889795 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3ac1084b0 con 0x7fd3ac108050 2024-09-16T00:22:36.030 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.368+0000 7fd3ab7fe640 1 -- 172.21.15.38:0/2541889795 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fd3a4046450 con 0x7fd3ac108050 2024-09-16T00:22:36.031 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.429+0000 7fd3b459f640 1 -- 172.21.15.38:0/2541889795 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fd3ac1947a0 con 0x7fd3ac108050 2024-09-16T00:22:36.031 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.430+0000 7fd3ab7fe640 1 -- 172.21.15.38:0/2541889795 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fd3a4046630 con 0x7fd3ac108050 2024-09-16T00:22:36.031 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.430+0000 7fd3b459f640 1 -- 172.21.15.38:0/2541889795 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3ac108050 msgr2=0x7fd3ac1078d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:36.031 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.430+0000 7fd3b459f640 1 --2- 172.21.15.38:0/2541889795 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3ac108050 0x7fd3ac1078d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd3a40090b0 tx=0x7fd3a40046c0 comp rx=0 tx=0).stop 2024-09-16T00:22:36.031 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.430+0000 7fd3b459f640 1 -- 172.21.15.38:0/2541889795 shutdown_connections 2024-09-16T00:22:36.031 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.430+0000 7fd3b459f640 1 --2- 172.21.15.38:0/2541889795 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3ac108050 0x7fd3ac1078d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:36.031 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.430+0000 7fd3b459f640 1 -- 172.21.15.38:0/2541889795 >> 172.21.15.38:0/2541889795 conn(0x7fd3ac101c30 msgr2=0x7fd3ac102d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:36.031 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.430+0000 7fd3b459f640 1 -- 172.21.15.38:0/2541889795 shutdown_connections 2024-09-16T00:22:36.031 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:35.431+0000 7fd3b459f640 1 -- 172.21.15.38:0/2541889795 wait complete. 2024-09-16T00:22:36.031 INFO:teuthology.orchestra.run.smithi038.stdout:mon is available 2024-09-16T00:22:36.031 INFO:teuthology.orchestra.run.smithi038.stdout:Assimilating anything we can from ceph.conf... 2024-09-16T00:22:36.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:35 smithi038 ceph-mon[30578]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-16T00:22:36.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:35 smithi038 ceph-mon[30578]: monmap epoch 1 2024-09-16T00:22:36.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:35 smithi038 ceph-mon[30578]: fsid 983b2072-73c1-11ef-bceb-c7b262605968 2024-09-16T00:22:36.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:35 smithi038 ceph-mon[30578]: last_changed 2024-09-16T00:22:31.653387+0000 2024-09-16T00:22:36.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:35 smithi038 ceph-mon[30578]: created 2024-09-16T00:22:31.653387+0000 2024-09-16T00:22:36.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:35 smithi038 ceph-mon[30578]: min_mon_release 19 (squid) 2024-09-16T00:22:36.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:35 smithi038 ceph-mon[30578]: election_strategy: 1 2024-09-16T00:22:36.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:35 smithi038 ceph-mon[30578]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-09-16T00:22:36.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:35 smithi038 ceph-mon[30578]: fsmap 2024-09-16T00:22:36.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:35 smithi038 ceph-mon[30578]: osdmap e1: 0 total, 0 up, 0 in 2024-09-16T00:22:36.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:35 smithi038 ceph-mon[30578]: mgrmap e1: no daemons active 2024-09-16T00:22:36.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:35 smithi038 ceph-mon[30578]: from='client.? 172.21.15.38:0/2541889795' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-16T00:22:37.067 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-09-16T00:22:37.067 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout [global] 2024-09-16T00:22:37.067 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout fsid = 983b2072-73c1-11ef-bceb-c7b262605968 2024-09-16T00:22:37.067 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.38:3300,v1:172.21.15.38:6789] 2024-09-16T00:22:37.067 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-16T00:22:37.067 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-16T00:22:37.068 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-16T00:22:37.068 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-16T00:22:37.068 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-09-16T00:22:37.068 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-16T00:22:37.068 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-16T00:22:37.068 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-09-16T00:22:37.068 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout [osd] 2024-09-16T00:22:37.068 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-16T00:22:37.068 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-16T00:22:37.068 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-16T00:22:37.068 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-16T00:22:37.068 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-16T00:22:37.069 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.425+0000 7f11134ea640 1 Processor -- start 2024-09-16T00:22:37.069 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.425+0000 7f11134ea640 1 -- start start 2024-09-16T00:22:37.069 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.425+0000 7f11134ea640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f110c107620 0x7f110c107a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:37.069 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.425+0000 7f11134ea640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f110c107ff0 con 0x7f110c107620 2024-09-16T00:22:37.069 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.425+0000 7f1111286640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f110c107620 0x7f110c107a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:37.069 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.425+0000 7f1111286640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f110c107620 0x7f110c107a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51022/0 (socket says 172.21.15.38:51022) 2024-09-16T00:22:37.069 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.425+0000 7f1111286640 1 -- 172.21.15.38:0/2605350683 learned_addr learned my addr 172.21.15.38:0/2605350683 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:22:37.069 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.425+0000 7f1111286640 1 -- 172.21.15.38:0/2605350683 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f110c108820 con 0x7f110c107620 2024-09-16T00:22:37.069 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.426+0000 7f1111286640 1 --2- 172.21.15.38:0/2605350683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f110c107620 0x7f110c107a20 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f1108009820 tx=0x7f110802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5efbf60863bcdffc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:37.069 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.426+0000 7f1103fff640 1 -- 172.21.15.38:0/2605350683 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1108037070 con 0x7f110c107620 2024-09-16T00:22:37.069 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.426+0000 7f1103fff640 1 -- 172.21.15.38:0/2605350683 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1108032440 con 0x7f110c107620 2024-09-16T00:22:37.069 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.426+0000 7f1103fff640 1 -- 172.21.15.38:0/2605350683 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1108032740 con 0x7f110c107620 2024-09-16T00:22:37.069 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.427+0000 7f11134ea640 1 -- 172.21.15.38:0/2605350683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f110c107620 msgr2=0x7f110c107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:37.070 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.427+0000 7f11134ea640 1 --2- 172.21.15.38:0/2605350683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f110c107620 0x7f110c107a20 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f1108009820 tx=0x7f110802a270 comp rx=0 tx=0).stop 2024-09-16T00:22:37.070 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.427+0000 7f11134ea640 1 -- 172.21.15.38:0/2605350683 shutdown_connections 2024-09-16T00:22:37.070 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.427+0000 7f11134ea640 1 --2- 172.21.15.38:0/2605350683 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f110c107620 0x7f110c107a20 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:37.070 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.427+0000 7f11134ea640 1 -- 172.21.15.38:0/2605350683 >> 172.21.15.38:0/2605350683 conn(0x7f110c102ea0 msgr2=0x7f110c105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:37.070 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.427+0000 7f11134ea640 1 -- 172.21.15.38:0/2605350683 shutdown_connections 2024-09-16T00:22:37.070 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.427+0000 7f11134ea640 1 -- 172.21.15.38:0/2605350683 wait complete. 2024-09-16T00:22:37.070 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.428+0000 7f11134ea640 1 Processor -- start 2024-09-16T00:22:37.070 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.428+0000 7f11134ea640 1 -- start start 2024-09-16T00:22:37.070 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.428+0000 7f11134ea640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f110c107620 0x7f110c19cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:37.070 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.428+0000 7f11134ea640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f110c108dc0 con 0x7f110c107620 2024-09-16T00:22:37.070 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.428+0000 7f1111286640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f110c107620 0x7f110c19cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:37.070 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.428+0000 7f1111286640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f110c107620 0x7f110c19cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51026/0 (socket says 172.21.15.38:51026) 2024-09-16T00:22:37.070 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.428+0000 7f1111286640 1 -- 172.21.15.38:0/587372253 learned_addr learned my addr 172.21.15.38:0/587372253 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:22:37.070 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.428+0000 7f1111286640 1 -- 172.21.15.38:0/587372253 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f110c19d4e0 con 0x7f110c107620 2024-09-16T00:22:37.070 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.428+0000 7f1111286640 1 --2- 172.21.15.38:0/587372253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f110c107620 0x7f110c19cfa0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f1108009820 tx=0x7f11080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:37.071 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.429+0000 7f11027fc640 1 -- 172.21.15.38:0/587372253 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1108041070 con 0x7f110c107620 2024-09-16T00:22:37.071 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.429+0000 7f11027fc640 1 -- 172.21.15.38:0/587372253 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f110803c7a0 con 0x7f110c107620 2024-09-16T00:22:37.071 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.429+0000 7f11134ea640 1 -- 172.21.15.38:0/587372253 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f110c19d770 con 0x7f110c107620 2024-09-16T00:22:37.071 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.429+0000 7f11027fc640 1 -- 172.21.15.38:0/587372253 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1108037040 con 0x7f110c107620 2024-09-16T00:22:37.071 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.429+0000 7f11134ea640 1 -- 172.21.15.38:0/587372253 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f110c1a0460 con 0x7f110c107620 2024-09-16T00:22:37.071 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.429+0000 7f11027fc640 1 -- 172.21.15.38:0/587372253 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f1108044070 con 0x7f110c107620 2024-09-16T00:22:37.071 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.429+0000 7f11027fc640 1 -- 172.21.15.38:0/587372253 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f1108046dd0 con 0x7f110c107620 2024-09-16T00:22:37.071 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.430+0000 7f11134ea640 1 -- 172.21.15.38:0/587372253 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f110c109410 con 0x7f110c107620 2024-09-16T00:22:37.071 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.432+0000 7f11027fc640 1 -- 172.21.15.38:0/587372253 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f1108046450 con 0x7f110c107620 2024-09-16T00:22:37.071 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.491+0000 7f11134ea640 1 -- 172.21.15.38:0/587372253 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f110c1a0630 con 0x7f110c107620 2024-09-16T00:22:37.071 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.495+0000 7f11027fc640 1 -- 172.21.15.38:0/587372253 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+429 (secure 0 0 0) 0x7f1108046630 con 0x7f110c107620 2024-09-16T00:22:37.071 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.495+0000 7f11027fc640 1 -- 172.21.15.38:0/587372253 <== mon.0 v2:172.21.15.38:3300/0 8 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f1108055020 con 0x7f110c107620 2024-09-16T00:22:37.071 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.496+0000 7f11134ea640 1 -- 172.21.15.38:0/587372253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f110c107620 msgr2=0x7f110c19cfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:37.071 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.496+0000 7f11134ea640 1 --2- 172.21.15.38:0/587372253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f110c107620 0x7f110c19cfa0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f1108009820 tx=0x7f11080046c0 comp rx=0 tx=0).stop 2024-09-16T00:22:37.072 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.496+0000 7f11134ea640 1 -- 172.21.15.38:0/587372253 shutdown_connections 2024-09-16T00:22:37.072 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.496+0000 7f11134ea640 1 --2- 172.21.15.38:0/587372253 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f110c107620 0x7f110c19cfa0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:37.072 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.496+0000 7f11134ea640 1 -- 172.21.15.38:0/587372253 >> 172.21.15.38:0/587372253 conn(0x7f110c102ea0 msgr2=0x7f110c07c960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:37.072 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.496+0000 7f11134ea640 1 -- 172.21.15.38:0/587372253 shutdown_connections 2024-09-16T00:22:37.072 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:36.496+0000 7f11134ea640 1 -- 172.21.15.38:0/587372253 wait complete. 2024-09-16T00:22:37.072 INFO:teuthology.orchestra.run.smithi038.stdout:Generating new minimal ceph.conf... 2024-09-16T00:22:37.175 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:36 smithi038 ceph-mon[30578]: from='client.? 172.21.15.38:0/587372253' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-16T00:22:37.175 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:36 smithi038 ceph-mon[30578]: from='client.? 172.21.15.38:0/587372253' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-16T00:22:38.088 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.465+0000 7f8caaf7a640 1 Processor -- start 2024-09-16T00:22:38.088 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.465+0000 7f8caaf7a640 1 -- start start 2024-09-16T00:22:38.089 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.465+0000 7f8caaf7a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ca41080d0 0x7f8ca410a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:38.089 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.465+0000 7f8caaf7a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca407c5f0 con 0x7f8ca41080d0 2024-09-16T00:22:38.089 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.465+0000 7f8ca8d16640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ca41080d0 0x7f8ca410a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:38.089 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.465+0000 7f8ca8d16640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ca41080d0 0x7f8ca410a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51040/0 (socket says 172.21.15.38:51040) 2024-09-16T00:22:38.089 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.465+0000 7f8ca8d16640 1 -- 172.21.15.38:0/782105099 learned_addr learned my addr 172.21.15.38:0/782105099 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:22:38.089 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.465+0000 7f8ca8d16640 1 -- 172.21.15.38:0/782105099 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca410aa00 con 0x7f8ca41080d0 2024-09-16T00:22:38.089 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.466+0000 7f8ca8d16640 1 --2- 172.21.15.38:0/782105099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ca41080d0 0x7f8ca410a4c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8c98009820 tx=0x7f8c9802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35927c9d721d5847 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:38.089 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.466+0000 7f8ca37fe640 1 -- 172.21.15.38:0/782105099 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c98037070 con 0x7f8ca41080d0 2024-09-16T00:22:38.089 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.466+0000 7f8ca37fe640 1 -- 172.21.15.38:0/782105099 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8c98032900 con 0x7f8ca41080d0 2024-09-16T00:22:38.089 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.466+0000 7f8ca37fe640 1 -- 172.21.15.38:0/782105099 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c98030850 con 0x7f8ca41080d0 2024-09-16T00:22:38.089 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.467+0000 7f8caaf7a640 1 -- 172.21.15.38:0/782105099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ca41080d0 msgr2=0x7f8ca410a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:38.089 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.467+0000 7f8caaf7a640 1 --2- 172.21.15.38:0/782105099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ca41080d0 0x7f8ca410a4c0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f8c98009820 tx=0x7f8c9802a270 comp rx=0 tx=0).stop 2024-09-16T00:22:38.090 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.467+0000 7f8caaf7a640 1 -- 172.21.15.38:0/782105099 shutdown_connections 2024-09-16T00:22:38.090 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.467+0000 7f8caaf7a640 1 --2- 172.21.15.38:0/782105099 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ca41080d0 0x7f8ca410a4c0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:38.090 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.467+0000 7f8caaf7a640 1 -- 172.21.15.38:0/782105099 >> 172.21.15.38:0/782105099 conn(0x7f8ca4101c70 msgr2=0x7f8ca41040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:38.090 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.467+0000 7f8caaf7a640 1 -- 172.21.15.38:0/782105099 shutdown_connections 2024-09-16T00:22:38.090 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.467+0000 7f8caaf7a640 1 -- 172.21.15.38:0/782105099 wait complete. 2024-09-16T00:22:38.090 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.468+0000 7f8caaf7a640 1 Processor -- start 2024-09-16T00:22:38.090 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.468+0000 7f8caaf7a640 1 -- start start 2024-09-16T00:22:38.090 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.468+0000 7f8caaf7a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ca41080d0 0x7f8ca4107950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:38.090 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.468+0000 7f8caaf7a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8ca410b980 con 0x7f8ca41080d0 2024-09-16T00:22:38.090 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.468+0000 7f8ca8d16640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ca41080d0 0x7f8ca4107950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:38.090 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.468+0000 7f8ca8d16640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ca41080d0 0x7f8ca4107950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51052/0 (socket says 172.21.15.38:51052) 2024-09-16T00:22:38.090 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.468+0000 7f8ca8d16640 1 -- 172.21.15.38:0/3065390911 learned_addr learned my addr 172.21.15.38:0/3065390911 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:22:38.091 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.468+0000 7f8ca8d16640 1 -- 172.21.15.38:0/3065390911 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ca4107e90 con 0x7f8ca41080d0 2024-09-16T00:22:38.091 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.469+0000 7f8ca8d16640 1 --2- 172.21.15.38:0/3065390911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ca41080d0 0x7f8ca4107950 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8c980097f0 tx=0x7f8c98030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:38.091 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.469+0000 7f8ca1ffb640 1 -- 172.21.15.38:0/3065390911 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c98037040 con 0x7f8ca41080d0 2024-09-16T00:22:38.091 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.469+0000 7f8ca1ffb640 1 -- 172.21.15.38:0/3065390911 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8c98040070 con 0x7f8ca41080d0 2024-09-16T00:22:38.091 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.469+0000 7f8caaf7a640 1 -- 172.21.15.38:0/3065390911 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ca4106080 con 0x7f8ca41080d0 2024-09-16T00:22:38.091 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.469+0000 7f8caaf7a640 1 -- 172.21.15.38:0/3065390911 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ca41064a0 con 0x7f8ca41080d0 2024-09-16T00:22:38.091 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.470+0000 7f8ca1ffb640 1 -- 172.21.15.38:0/3065390911 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8c9803a390 con 0x7f8ca41080d0 2024-09-16T00:22:38.091 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.470+0000 7f8ca1ffb640 1 -- 172.21.15.38:0/3065390911 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f8c9803a660 con 0x7f8ca41080d0 2024-09-16T00:22:38.091 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.470+0000 7f8ca1ffb640 1 -- 172.21.15.38:0/3065390911 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8c980444e0 con 0x7f8ca41080d0 2024-09-16T00:22:38.091 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.471+0000 7f8caaf7a640 1 -- 172.21.15.38:0/3065390911 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c880052c0 con 0x7f8ca41080d0 2024-09-16T00:22:38.091 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.473+0000 7f8ca1ffb640 1 -- 172.21.15.38:0/3065390911 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f8c9804a020 con 0x7f8ca41080d0 2024-09-16T00:22:38.091 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.533+0000 7f8caaf7a640 1 -- 172.21.15.38:0/3065390911 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f8c88005880 con 0x7f8ca41080d0 2024-09-16T00:22:38.091 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.533+0000 7f8ca1ffb640 1 -- 172.21.15.38:0/3065390911 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f8c9803b430 con 0x7f8ca41080d0 2024-09-16T00:22:38.091 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.534+0000 7f8caaf7a640 1 -- 172.21.15.38:0/3065390911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ca41080d0 msgr2=0x7f8ca4107950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:38.092 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.534+0000 7f8caaf7a640 1 --2- 172.21.15.38:0/3065390911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ca41080d0 0x7f8ca4107950 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f8c980097f0 tx=0x7f8c98030d80 comp rx=0 tx=0).stop 2024-09-16T00:22:38.092 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.534+0000 7f8caaf7a640 1 -- 172.21.15.38:0/3065390911 shutdown_connections 2024-09-16T00:22:38.092 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.534+0000 7f8caaf7a640 1 --2- 172.21.15.38:0/3065390911 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8ca41080d0 0x7f8ca4107950 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:38.092 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.534+0000 7f8caaf7a640 1 -- 172.21.15.38:0/3065390911 >> 172.21.15.38:0/3065390911 conn(0x7f8ca4101c70 msgr2=0x7f8ca41027d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:38.092 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.534+0000 7f8caaf7a640 1 -- 172.21.15.38:0/3065390911 shutdown_connections 2024-09-16T00:22:38.092 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:37.534+0000 7f8caaf7a640 1 -- 172.21.15.38:0/3065390911 wait complete. 2024-09-16T00:22:38.092 INFO:teuthology.orchestra.run.smithi038.stdout:Restarting the monitor... 2024-09-16T00:22:38.214 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:37 smithi038 ceph-mon[30578]: from='client.? 172.21.15.38:0/3065390911' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:22:38.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:38 smithi038 systemd[1]: Stopping Ceph mon.a for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:22:38.660 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:38 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mon-a[30574]: 2024-09-16T00:22:38.360+0000 7facfa696640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:22:38.660 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:38 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mon-a[30574]: 2024-09-16T00:22:38.360+0000 7facfa696640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-16T00:22:38.660 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:38 smithi038 podman[30821]: 2024-09-16 00:22:38.391508469 +0000 UTC m=+0.138474746 container died 282e3793608133cfcf985ae5a3105aec77f125b4a23d8d39b1d5dc3dd22a163b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mon-a, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-16T00:22:38.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:38 smithi038 podman[30821]: 2024-09-16 00:22:38.759788299 +0000 UTC m=+0.506754566 container cleanup 282e3793608133cfcf985ae5a3105aec77f125b4a23d8d39b1d5dc3dd22a163b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mon-a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-8921c8e) 2024-09-16T00:22:38.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:38 smithi038 bash[30821]: ceph-983b2072-73c1-11ef-bceb-c7b262605968-mon-a 2024-09-16T00:22:39.390 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 podman[30835]: 2024-09-16 00:22:39.05252909 +0000 UTC m=+0.659419499 container remove 282e3793608133cfcf985ae5a3105aec77f125b4a23d8d39b1d5dc3dd22a163b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mon-a, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-16T00:22:39.390 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 systemd[1]: ceph-983b2072-73c1-11ef-bceb-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-16T00:22:39.390 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 systemd[1]: Stopped Ceph mon.a for 983b2072-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:22:39.390 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 systemd[1]: Starting Ceph mon.a for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:22:39.647 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 podman[30926]: 2024-09-16 00:22:39.293467221 +0000 UTC m=+0.016142782 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc 2024-09-16T00:22:39.647 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 podman[30926]: 2024-09-16 00:22:39.411965302 +0000 UTC m=+0.134640889 container create e8fd9f4da33c2aeedbf006f12065ead99db393676b9dd914abc8c6920451f9fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mon-a, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:22:39.647 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 podman[30926]: 2024-09-16 00:22:39.62256239 +0000 UTC m=+0.345237947 container init e8fd9f4da33c2aeedbf006f12065ead99db393676b9dd914abc8c6920451f9fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mon-a, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:22:39.647 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 podman[30926]: 2024-09-16 00:22:39.625666256 +0000 UTC m=+0.348341810 container start e8fd9f4da33c2aeedbf006f12065ead99db393676b9dd914abc8c6920451f9fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:22:39.710 INFO:teuthology.orchestra.run.smithi038.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-16T00:22:39.958 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: set uid:gid to 167:167 (ceph:ceph) 2024-09-16T00:22:39.958 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable), process ceph-mon, pid 2 2024-09-16T00:22:39.958 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: pidfile_write: ignore empty --pid-file 2024-09-16T00:22:39.958 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: load: jerasure load: lrc 2024-09-16T00:22:39.958 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: RocksDB version: 7.9.2 2024-09-16T00:22:39.959 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Git sha 0 2024-09-16T00:22:39.959 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Compile date 2024-09-15 15:04:35 2024-09-16T00:22:39.959 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: DB SUMMARY 2024-09-16T00:22:39.959 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: DB Session ID: IDCEG8EQVZFMENBK1A1S 2024-09-16T00:22:39.959 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: CURRENT file: CURRENT 2024-09-16T00:22:39.959 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: IDENTITY file: IDENTITY 2024-09-16T00:22:39.959 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-16T00:22:39.959 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-16T00:22:39.959 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105374 ; 2024-09-16T00:22:39.959 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.error_if_exists: 0 2024-09-16T00:22:39.959 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.create_if_missing: 0 2024-09-16T00:22:39.960 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.paranoid_checks: 1 2024-09-16T00:22:39.960 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-16T00:22:39.960 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-16T00:22:39.960 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-16T00:22:39.960 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.env: 0x564e5aeac940 2024-09-16T00:22:39.960 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.fs: PosixFileSystem 2024-09-16T00:22:39.960 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.info_log: 0x564e5ba596e0 2024-09-16T00:22:39.960 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.max_file_opening_threads: 16 2024-09-16T00:22:39.960 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.statistics: (nil) 2024-09-16T00:22:39.960 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.use_fsync: 0 2024-09-16T00:22:39.960 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.max_log_file_size: 0 2024-09-16T00:22:39.960 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-16T00:22:39.961 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-16T00:22:39.961 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.keep_log_file_num: 1000 2024-09-16T00:22:39.961 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.recycle_log_file_num: 0 2024-09-16T00:22:39.961 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.allow_fallocate: 1 2024-09-16T00:22:39.961 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.allow_mmap_reads: 0 2024-09-16T00:22:39.961 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.allow_mmap_writes: 0 2024-09-16T00:22:39.961 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.use_direct_reads: 0 2024-09-16T00:22:39.961 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-16T00:22:39.961 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.create_missing_column_families: 0 2024-09-16T00:22:39.961 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.db_log_dir: 2024-09-16T00:22:39.961 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.wal_dir: 2024-09-16T00:22:39.962 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-16T00:22:39.962 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-16T00:22:39.962 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-16T00:22:39.962 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-16T00:22:39.962 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-16T00:22:39.962 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-16T00:22:39.962 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.advise_random_on_open: 1 2024-09-16T00:22:39.962 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.db_write_buffer_size: 0 2024-09-16T00:22:39.962 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.write_buffer_manager: 0x564e5ba68dc0 2024-09-16T00:22:39.962 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-16T00:22:39.962 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-16T00:22:39.962 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-16T00:22:39.963 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.rate_limiter: (nil) 2024-09-16T00:22:39.963 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-16T00:22:39.963 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.wal_recovery_mode: 2 2024-09-16T00:22:39.963 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.enable_thread_tracking: 0 2024-09-16T00:22:39.963 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.enable_pipelined_write: 0 2024-09-16T00:22:39.963 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.unordered_write: 0 2024-09-16T00:22:39.963 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-16T00:22:39.963 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-16T00:22:39.963 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-16T00:22:39.963 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-16T00:22:39.963 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.row_cache: None 2024-09-16T00:22:39.964 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.wal_filter: None 2024-09-16T00:22:39.964 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-16T00:22:39.964 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.allow_ingest_behind: 0 2024-09-16T00:22:39.964 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.two_write_queues: 0 2024-09-16T00:22:39.964 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.manual_wal_flush: 0 2024-09-16T00:22:39.964 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.wal_compression: 0 2024-09-16T00:22:39.964 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.atomic_flush: 0 2024-09-16T00:22:39.964 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-16T00:22:39.964 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-16T00:22:39.964 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-16T00:22:39.964 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.log_readahead_size: 0 2024-09-16T00:22:39.964 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-16T00:22:39.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.best_efforts_recovery: 0 2024-09-16T00:22:39.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-16T00:22:39.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-16T00:22:39.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.allow_data_in_errors: 0 2024-09-16T00:22:39.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.db_host_id: __hostname__ 2024-09-16T00:22:39.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-16T00:22:39.966 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.max_background_jobs: 2 2024-09-16T00:22:39.966 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.max_background_compactions: -1 2024-09-16T00:22:39.966 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.max_subcompactions: 1 2024-09-16T00:22:39.966 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-16T00:22:39.966 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-16T00:22:39.966 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-16T00:22:39.967 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.max_total_wal_size: 0 2024-09-16T00:22:39.967 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-16T00:22:39.967 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-16T00:22:39.967 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-16T00:22:39.967 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-16T00:22:39.967 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.max_open_files: -1 2024-09-16T00:22:39.967 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.bytes_per_sync: 0 2024-09-16T00:22:39.967 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-16T00:22:39.967 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-16T00:22:39.967 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.compaction_readahead_size: 0 2024-09-16T00:22:39.967 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.max_background_flushes: -1 2024-09-16T00:22:39.967 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Compression algorithms supported: 2024-09-16T00:22:39.967 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: kZSTD supported: 0 2024-09-16T00:22:39.967 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: kXpressCompression supported: 0 2024-09-16T00:22:39.967 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: kBZip2Compression supported: 0 2024-09-16T00:22:39.968 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-16T00:22:39.968 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: kLZ4Compression supported: 1 2024-09-16T00:22:39.968 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: kZlibCompression supported: 1 2024-09-16T00:22:39.968 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: kLZ4HCCompression supported: 1 2024-09-16T00:22:39.968 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: kSnappyCompression supported: 1 2024-09-16T00:22:39.968 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-16T00:22:39.968 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-16T00:22:39.968 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-16T00:22:39.968 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-16T00:22:39.968 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-16T00:22:39.968 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.merge_operator: 2024-09-16T00:22:39.968 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.compaction_filter: None 2024-09-16T00:22:39.968 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.compaction_filter_factory: None 2024-09-16T00:22:39.968 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.sst_partitioner_factory: None 2024-09-16T00:22:39.969 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-16T00:22:39.969 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-16T00:22:39.969 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x564e5ba59320) 2024-09-16T00:22:39.969 INFO:journalctl@ceph.mon.a.smithi038.stdout: cache_index_and_filter_blocks: 1 2024-09-16T00:22:39.969 INFO:journalctl@ceph.mon.a.smithi038.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-16T00:22:39.969 INFO:journalctl@ceph.mon.a.smithi038.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-16T00:22:39.969 INFO:journalctl@ceph.mon.a.smithi038.stdout: pin_top_level_index_and_filter: 1 2024-09-16T00:22:39.969 INFO:journalctl@ceph.mon.a.smithi038.stdout: index_type: 0 2024-09-16T00:22:39.969 INFO:journalctl@ceph.mon.a.smithi038.stdout: data_block_index_type: 0 2024-09-16T00:22:39.969 INFO:journalctl@ceph.mon.a.smithi038.stdout: index_shortening: 1 2024-09-16T00:22:39.969 INFO:journalctl@ceph.mon.a.smithi038.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-16T00:22:39.969 INFO:journalctl@ceph.mon.a.smithi038.stdout: checksum: 4 2024-09-16T00:22:39.970 INFO:journalctl@ceph.mon.a.smithi038.stdout: no_block_cache: 0 2024-09-16T00:22:39.970 INFO:journalctl@ceph.mon.a.smithi038.stdout: block_cache: 0x564e5ba4d1f0 2024-09-16T00:22:39.970 INFO:journalctl@ceph.mon.a.smithi038.stdout: block_cache_name: BinnedLRUCache 2024-09-16T00:22:39.970 INFO:journalctl@ceph.mon.a.smithi038.stdout: block_cache_options: 2024-09-16T00:22:39.970 INFO:journalctl@ceph.mon.a.smithi038.stdout: capacity : 536870912 2024-09-16T00:22:39.970 INFO:journalctl@ceph.mon.a.smithi038.stdout: num_shard_bits : 4 2024-09-16T00:22:39.970 INFO:journalctl@ceph.mon.a.smithi038.stdout: strict_capacity_limit : 0 2024-09-16T00:22:39.970 INFO:journalctl@ceph.mon.a.smithi038.stdout: high_pri_pool_ratio: 0.000 2024-09-16T00:22:39.970 INFO:journalctl@ceph.mon.a.smithi038.stdout: block_cache_compressed: (nil) 2024-09-16T00:22:39.970 INFO:journalctl@ceph.mon.a.smithi038.stdout: persistent_cache: (nil) 2024-09-16T00:22:39.970 INFO:journalctl@ceph.mon.a.smithi038.stdout: block_size: 4096 2024-09-16T00:22:39.970 INFO:journalctl@ceph.mon.a.smithi038.stdout: block_size_deviation: 10 2024-09-16T00:22:39.970 INFO:journalctl@ceph.mon.a.smithi038.stdout: block_restart_interval: 16 2024-09-16T00:22:39.970 INFO:journalctl@ceph.mon.a.smithi038.stdout: index_block_restart_interval: 1 2024-09-16T00:22:39.971 INFO:journalctl@ceph.mon.a.smithi038.stdout: metadata_block_size: 4096 2024-09-16T00:22:39.971 INFO:journalctl@ceph.mon.a.smithi038.stdout: partition_filters: 0 2024-09-16T00:22:39.971 INFO:journalctl@ceph.mon.a.smithi038.stdout: use_delta_encoding: 1 2024-09-16T00:22:39.971 INFO:journalctl@ceph.mon.a.smithi038.stdout: filter_policy: bloomfilter 2024-09-16T00:22:39.971 INFO:journalctl@ceph.mon.a.smithi038.stdout: whole_key_filtering: 1 2024-09-16T00:22:39.971 INFO:journalctl@ceph.mon.a.smithi038.stdout: verify_compression: 0 2024-09-16T00:22:39.971 INFO:journalctl@ceph.mon.a.smithi038.stdout: read_amp_bytes_per_bit: 0 2024-09-16T00:22:39.971 INFO:journalctl@ceph.mon.a.smithi038.stdout: format_version: 5 2024-09-16T00:22:39.971 INFO:journalctl@ceph.mon.a.smithi038.stdout: enable_index_compression: 1 2024-09-16T00:22:39.971 INFO:journalctl@ceph.mon.a.smithi038.stdout: block_align: 0 2024-09-16T00:22:39.971 INFO:journalctl@ceph.mon.a.smithi038.stdout: max_auto_readahead_size: 262144 2024-09-16T00:22:39.971 INFO:journalctl@ceph.mon.a.smithi038.stdout: prepopulate_block_cache: 0 2024-09-16T00:22:39.971 INFO:journalctl@ceph.mon.a.smithi038.stdout: initial_auto_readahead_size: 8192 2024-09-16T00:22:39.971 INFO:journalctl@ceph.mon.a.smithi038.stdout: num_file_reads_for_auto_readahead: 2 2024-09-16T00:22:39.972 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.write_buffer_size: 33554432 2024-09-16T00:22:39.972 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.max_write_buffer_number: 2 2024-09-16T00:22:39.972 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.compression: NoCompression 2024-09-16T00:22:39.972 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.bottommost_compression: Disabled 2024-09-16T00:22:39.972 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.prefix_extractor: nullptr 2024-09-16T00:22:39.972 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-16T00:22:39.972 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.num_levels: 7 2024-09-16T00:22:39.972 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-16T00:22:39.972 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-16T00:22:39.972 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-16T00:22:39.972 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-16T00:22:39.972 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-16T00:22:39.972 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-16T00:22:39.972 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-16T00:22:39.973 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-16T00:22:39.973 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-16T00:22:39.973 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-16T00:22:39.973 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-16T00:22:39.973 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-16T00:22:39.973 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-16T00:22:39.973 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.compression_opts.level: 32767 2024-09-16T00:22:39.973 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.compression_opts.strategy: 0 2024-09-16T00:22:39.973 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-16T00:22:39.973 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-16T00:22:39.973 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-16T00:22:39.973 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-16T00:22:39.974 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.compression_opts.enabled: false 2024-09-16T00:22:39.974 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-16T00:22:39.974 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-16T00:22:39.974 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-16T00:22:39.974 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-16T00:22:39.974 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.target_file_size_base: 67108864 2024-09-16T00:22:39.974 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-16T00:22:39.974 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-16T00:22:39.974 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-16T00:22:39.974 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-16T00:22:39.974 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-16T00:22:39.974 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-16T00:22:39.974 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-16T00:22:39.975 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-16T00:22:39.975 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-16T00:22:39.975 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-16T00:22:39.975 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-16T00:22:39.975 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-16T00:22:39.975 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-16T00:22:39.975 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-16T00:22:39.975 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.arena_block_size: 1048576 2024-09-16T00:22:39.975 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-16T00:22:39.975 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-16T00:22:39.975 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.disable_auto_compactions: 0 2024-09-16T00:22:39.975 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-16T00:22:39.975 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-16T00:22:39.976 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-16T00:22:39.976 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-16T00:22:39.976 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-16T00:22:39.976 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-16T00:22:39.976 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-16T00:22:39.976 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-16T00:22:39.976 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-16T00:22:39.976 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-16T00:22:39.976 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-16T00:22:39.976 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.inplace_update_support: 0 2024-09-16T00:22:39.976 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-16T00:22:39.976 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-16T00:22:39.976 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-16T00:22:39.977 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-16T00:22:39.977 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.bloom_locality: 0 2024-09-16T00:22:39.977 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.max_successive_merges: 0 2024-09-16T00:22:39.977 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-16T00:22:39.977 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.paranoid_file_checks: 0 2024-09-16T00:22:39.977 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.force_consistency_checks: 1 2024-09-16T00:22:39.977 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.report_bg_io_stats: 0 2024-09-16T00:22:39.977 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.ttl: 2592000 2024-09-16T00:22:39.977 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-16T00:22:39.977 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-16T00:22:39.977 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-16T00:22:39.977 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.enable_blob_files: false 2024-09-16T00:22:39.978 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.min_blob_size: 0 2024-09-16T00:22:39.978 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.blob_file_size: 268435456 2024-09-16T00:22:39.978 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-16T00:22:39.978 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-16T00:22:39.978 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-16T00:22:39.978 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-16T00:22:39.978 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-16T00:22:39.978 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.blob_file_starting_level: 0 2024-09-16T00:22:39.978 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-16T00:22:39.978 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-16T00:22:39.978 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-16T00:22:39.978 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 6106ab4c-a44a-45f4-8093-7055068b9fa4 2024-09-16T00:22:39.978 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726446159652924, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-16T00:22:39.979 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-16T00:22:39.979 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726446159653990, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100422, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 328, "table_properties": {"data_size": 98316, "index_size": 298, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14322, "raw_average_key_size": 49, "raw_value_size": 90567, "raw_average_value_size": 315, "num_data_blocks": 12, "num_entries": 287, "num_filter_entries": 287, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726446159, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "6106ab4c-a44a-45f4-8093-7055068b9fa4", "db_session_id": "IDCEG8EQVZFMENBK1A1S", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-16T00:22:39.979 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726446159654086, "job": 1, "event": "recovery_finished"} 2024-09-16T00:22:39.979 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-16T00:22:39.979 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-16T00:22:39.979 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x564e5ba7ae00 2024-09-16T00:22:39.979 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: DB pointer 0x564e5bb76000 2024-09-16T00:22:39.979 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-16T00:22:39.979 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-16T00:22:39.979 INFO:journalctl@ceph.mon.a.smithi038.stdout: ** DB Stats ** 2024-09-16T00:22:39.979 INFO:journalctl@ceph.mon.a.smithi038.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T00:22:39.980 INFO:journalctl@ceph.mon.a.smithi038.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-16T00:22:39.980 INFO:journalctl@ceph.mon.a.smithi038.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T00:22:39.980 INFO:journalctl@ceph.mon.a.smithi038.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T00:22:39.980 INFO:journalctl@ceph.mon.a.smithi038.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-16T00:22:39.980 INFO:journalctl@ceph.mon.a.smithi038.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T00:22:39.980 INFO:journalctl@ceph.mon.a.smithi038.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T00:22:39.980 INFO:journalctl@ceph.mon.a.smithi038.stdout: 2024-09-16T00:22:39.980 INFO:journalctl@ceph.mon.a.smithi038.stdout: ** Compaction Stats [default] ** 2024-09-16T00:22:39.980 INFO:journalctl@ceph.mon.a.smithi038.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T00:22:39.980 INFO:journalctl@ceph.mon.a.smithi038.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-16T00:22:39.980 INFO:journalctl@ceph.mon.a.smithi038.stdout: L0 2/0 99.93 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T00:22:39.980 INFO:journalctl@ceph.mon.a.smithi038.stdout: Sum 2/0 99.93 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T00:22:39.981 INFO:journalctl@ceph.mon.a.smithi038.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 144.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T00:22:39.981 INFO:journalctl@ceph.mon.a.smithi038.stdout: 2024-09-16T00:22:39.981 INFO:journalctl@ceph.mon.a.smithi038.stdout: ** Compaction Stats [default] ** 2024-09-16T00:22:39.981 INFO:journalctl@ceph.mon.a.smithi038.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T00:22:39.981 INFO:journalctl@ceph.mon.a.smithi038.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T00:22:39.981 INFO:journalctl@ceph.mon.a.smithi038.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 144.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-16T00:22:39.981 INFO:journalctl@ceph.mon.a.smithi038.stdout: 2024-09-16T00:22:39.981 INFO:journalctl@ceph.mon.a.smithi038.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-16T00:22:39.982 INFO:journalctl@ceph.mon.a.smithi038.stdout: 2024-09-16T00:22:39.982 INFO:journalctl@ceph.mon.a.smithi038.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T00:22:39.982 INFO:journalctl@ceph.mon.a.smithi038.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-16T00:22:39.982 INFO:journalctl@ceph.mon.a.smithi038.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T00:22:39.982 INFO:journalctl@ceph.mon.a.smithi038.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T00:22:39.982 INFO:journalctl@ceph.mon.a.smithi038.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T00:22:39.982 INFO:journalctl@ceph.mon.a.smithi038.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T00:22:39.982 INFO:journalctl@ceph.mon.a.smithi038.stdout: Cumulative compaction: 0.00 GB write, 16.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T00:22:39.982 INFO:journalctl@ceph.mon.a.smithi038.stdout: Interval compaction: 0.00 GB write, 16.64 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T00:22:39.982 INFO:journalctl@ceph.mon.a.smithi038.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T00:22:39.983 INFO:journalctl@ceph.mon.a.smithi038.stdout: Block cache BinnedLRUCache@0x564e5ba4d1f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-16T00:22:39.983 INFO:journalctl@ceph.mon.a.smithi038.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-09-16T00:22:39.983 INFO:journalctl@ceph.mon.a.smithi038.stdout: 2024-09-16T00:22:39.983 INFO:journalctl@ceph.mon.a.smithi038.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T00:22:39.983 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: starting mon.a rank 0 at public addrs [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] at bind addrs [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 983b2072-73c1-11ef-bceb-c7b262605968 2024-09-16T00:22:39.983 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: mon.a@-1(???) e1 preinit fsid 983b2072-73c1-11ef-bceb-c7b262605968 2024-09-16T00:22:39.983 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: mon.a@-1(???).mds e1 new map 2024-09-16T00:22:39.983 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: mon.a@-1(???).mds e1 print_map 2024-09-16T00:22:39.983 INFO:journalctl@ceph.mon.a.smithi038.stdout: e1 2024-09-16T00:22:39.983 INFO:journalctl@ceph.mon.a.smithi038.stdout: btime 2024-09-16T00:22:34:896635+0000 2024-09-16T00:22:39.983 INFO:journalctl@ceph.mon.a.smithi038.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:22:39.983 INFO:journalctl@ceph.mon.a.smithi038.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-16T00:22:39.983 INFO:journalctl@ceph.mon.a.smithi038.stdout: legacy client fscid: -1 2024-09-16T00:22:39.983 INFO:journalctl@ceph.mon.a.smithi038.stdout: 2024-09-16T00:22:39.984 INFO:journalctl@ceph.mon.a.smithi038.stdout: No filesystems configured 2024-09-16T00:22:39.984 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-16T00:22:39.984 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T00:22:39.984 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T00:22:39.984 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T00:22:39.984 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-16T00:22:39.984 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 bash[30926]: e8fd9f4da33c2aeedbf006f12065ead99db393676b9dd914abc8c6920451f9fc 2024-09-16T00:22:39.984 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 systemd[1]: Started Ceph mon.a for 983b2072-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:22:39.984 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-16T00:22:39.984 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: monmap epoch 1 2024-09-16T00:22:39.984 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: fsid 983b2072-73c1-11ef-bceb-c7b262605968 2024-09-16T00:22:39.984 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: last_changed 2024-09-16T00:22:31.653387+0000 2024-09-16T00:22:39.984 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: created 2024-09-16T00:22:31.653387+0000 2024-09-16T00:22:39.984 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: min_mon_release 19 (squid) 2024-09-16T00:22:39.985 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: election_strategy: 1 2024-09-16T00:22:39.985 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-09-16T00:22:39.985 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: fsmap 2024-09-16T00:22:39.985 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: osdmap e1: 0 total, 0 up, 0 in 2024-09-16T00:22:39.985 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:39 smithi038 ceph-mon[30945]: mgrmap e1: no daemons active 2024-09-16T00:22:40.740 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.159+0000 7f59e6981640 1 Processor -- start 2024-09-16T00:22:40.747 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.159+0000 7f59e6981640 1 -- start start 2024-09-16T00:22:40.748 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.159+0000 7f59e6981640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59e0108160 0x7f59e010a550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:40.748 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.159+0000 7f59e6981640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f59e007c650 con 0x7f59e0108160 2024-09-16T00:22:40.748 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.160+0000 7f59dffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59e0108160 0x7f59e010a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:40.748 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.160+0000 7f59dffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59e0108160 0x7f59e010a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51058/0 (socket says 172.21.15.38:51058) 2024-09-16T00:22:40.748 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.160+0000 7f59dffff640 1 -- 172.21.15.38:0/781204627 learned_addr learned my addr 172.21.15.38:0/781204627 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:22:40.748 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.160+0000 7f59dffff640 1 -- 172.21.15.38:0/781204627 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59e010aa90 con 0x7f59e0108160 2024-09-16T00:22:40.749 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.161+0000 7f59dffff640 1 --2- 172.21.15.38:0/781204627 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59e0108160 0x7f59e010a550 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f59d0009820 tx=0x7f59d002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d24cc102113963 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:40.749 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.162+0000 7f59deffd640 1 -- 172.21.15.38:0/781204627 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59d0037070 con 0x7f59e0108160 2024-09-16T00:22:40.749 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.162+0000 7f59deffd640 1 -- 172.21.15.38:0/781204627 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f59d0032900 con 0x7f59e0108160 2024-09-16T00:22:40.749 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.162+0000 7f59deffd640 1 -- 172.21.15.38:0/781204627 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59d0030850 con 0x7f59e0108160 2024-09-16T00:22:40.749 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.162+0000 7f59e6981640 1 -- 172.21.15.38:0/781204627 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59e0108160 msgr2=0x7f59e010a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:40.749 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.162+0000 7f59e6981640 1 --2- 172.21.15.38:0/781204627 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59e0108160 0x7f59e010a550 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f59d0009820 tx=0x7f59d002a270 comp rx=0 tx=0).stop 2024-09-16T00:22:40.750 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.162+0000 7f59e6981640 1 -- 172.21.15.38:0/781204627 shutdown_connections 2024-09-16T00:22:40.750 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.162+0000 7f59e6981640 1 --2- 172.21.15.38:0/781204627 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59e0108160 0x7f59e010a550 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:40.750 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.162+0000 7f59e6981640 1 -- 172.21.15.38:0/781204627 >> 172.21.15.38:0/781204627 conn(0x7f59e0101cc0 msgr2=0x7f59e01040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:40.750 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.163+0000 7f59e6981640 1 -- 172.21.15.38:0/781204627 shutdown_connections 2024-09-16T00:22:40.750 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.163+0000 7f59e6981640 1 -- 172.21.15.38:0/781204627 wait complete. 2024-09-16T00:22:40.750 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.163+0000 7f59e6981640 1 Processor -- start 2024-09-16T00:22:40.751 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.163+0000 7f59e6981640 1 -- start start 2024-09-16T00:22:40.751 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.164+0000 7f59e6981640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59e0108160 0x7f59e019b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:40.751 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.164+0000 7f59e6981640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f59e010ba10 con 0x7f59e0108160 2024-09-16T00:22:40.751 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.164+0000 7f59dffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59e0108160 0x7f59e019b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:40.751 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.164+0000 7f59dffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59e0108160 0x7f59e019b5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51068/0 (socket says 172.21.15.38:51068) 2024-09-16T00:22:40.751 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.164+0000 7f59dffff640 1 -- 172.21.15.38:0/2949300591 learned_addr learned my addr 172.21.15.38:0/2949300591 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:22:40.752 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.164+0000 7f59dffff640 1 -- 172.21.15.38:0/2949300591 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59e019bb20 con 0x7f59e0108160 2024-09-16T00:22:40.752 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.164+0000 7f59dffff640 1 --2- 172.21.15.38:0/2949300591 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59e0108160 0x7f59e019b5e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f59d00097f0 tx=0x7f59d0030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:40.752 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.164+0000 7f59dd7fa640 1 -- 172.21.15.38:0/2949300591 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59d0037040 con 0x7f59e0108160 2024-09-16T00:22:40.752 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.164+0000 7f59dd7fa640 1 -- 172.21.15.38:0/2949300591 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f59d0040070 con 0x7f59e0108160 2024-09-16T00:22:40.752 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.164+0000 7f59e6981640 1 -- 172.21.15.38:0/2949300591 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59e019bdb0 con 0x7f59e0108160 2024-09-16T00:22:40.752 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.164+0000 7f59dd7fa640 1 -- 172.21.15.38:0/2949300591 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f59d003a390 con 0x7f59e0108160 2024-09-16T00:22:40.753 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.164+0000 7f59e6981640 1 -- 172.21.15.38:0/2949300591 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59e019eaa0 con 0x7f59e0108160 2024-09-16T00:22:40.753 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.165+0000 7f59dd7fa640 1 -- 172.21.15.38:0/2949300591 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f59d003a5e0 con 0x7f59e0108160 2024-09-16T00:22:40.753 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.165+0000 7f59dd7fa640 1 -- 172.21.15.38:0/2949300591 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f59d0045950 con 0x7f59e0108160 2024-09-16T00:22:40.753 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.165+0000 7f59e6981640 1 -- 172.21.15.38:0/2949300591 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59e01085c0 con 0x7f59e0108160 2024-09-16T00:22:40.753 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.168+0000 7f59dd7fa640 1 -- 172.21.15.38:0/2949300591 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f59d0045460 con 0x7f59e0108160 2024-09-16T00:22:40.754 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.227+0000 7f59e6981640 1 -- 172.21.15.38:0/2949300591 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f59e019ec70 con 0x7f59e0108160 2024-09-16T00:22:40.754 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.230+0000 7f59dd7fa640 1 -- 172.21.15.38:0/2949300591 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f59d003b430 con 0x7f59e0108160 2024-09-16T00:22:40.754 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.230+0000 7f59e6981640 1 -- 172.21.15.38:0/2949300591 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59e0108160 msgr2=0x7f59e019b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:40.754 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.230+0000 7f59e6981640 1 --2- 172.21.15.38:0/2949300591 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59e0108160 0x7f59e019b5e0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f59d00097f0 tx=0x7f59d0030d80 comp rx=0 tx=0).stop 2024-09-16T00:22:40.754 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.230+0000 7f59e6981640 1 -- 172.21.15.38:0/2949300591 shutdown_connections 2024-09-16T00:22:40.754 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.230+0000 7f59e6981640 1 --2- 172.21.15.38:0/2949300591 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f59e0108160 0x7f59e019b5e0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:40.755 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.230+0000 7f59e6981640 1 -- 172.21.15.38:0/2949300591 >> 172.21.15.38:0/2949300591 conn(0x7f59e0101cc0 msgr2=0x7f59e0102800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:40.755 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.231+0000 7f59e6981640 1 -- 172.21.15.38:0/2949300591 shutdown_connections 2024-09-16T00:22:40.755 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:40.231+0000 7f59e6981640 1 -- 172.21.15.38:0/2949300591 wait complete. 2024-09-16T00:22:40.755 INFO:teuthology.orchestra.run.smithi038.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-16T00:22:40.762 INFO:teuthology.orchestra.run.smithi038.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:22:40.762 INFO:teuthology.orchestra.run.smithi038.stdout:Creating mgr... 2024-09-16T00:22:40.763 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-16T00:22:40.763 INFO:teuthology.orchestra.run.smithi038.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-16T00:22:41.037 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-983b2072-73c1-11ef-bceb-c7b262605968@mgr.x 2024-09-16T00:22:41.037 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Failed to reset failed state of unit ceph-983b2072-73c1-11ef-bceb-c7b262605968@mgr.x.service: Unit ceph-983b2072-73c1-11ef-bceb-c7b262605968@mgr.x.service not loaded. 2024-09-16T00:22:41.194 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-983b2072-73c1-11ef-bceb-c7b262605968.target.wants/ceph-983b2072-73c1-11ef-bceb-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-983b2072-73c1-11ef-bceb-c7b262605968@.service. 2024-09-16T00:22:41.430 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:41 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/2949300591' entity='client.admin' 2024-09-16T00:22:41.715 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:41 smithi038 podman[31157]: 2024-09-16 00:22:41.653330091 +0000 UTC m=+0.335836346 container start d898ac20a9d134ac46b1a5105f46db363aafc064eee96ca783d3eaf498a9fac2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x, RELEASE=squid-8921c8e, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-16T00:22:41.758 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T00:22:41.758 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout disabled 2024-09-16T00:22:41.765 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T00:22:41.765 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout inactive 2024-09-16T00:22:41.766 INFO:teuthology.orchestra.run.smithi038.stdout:firewalld.service is not enabled 2024-09-16T00:22:41.766 INFO:teuthology.orchestra.run.smithi038.stdout:Not possible to enable service . firewalld.service is not available 2024-09-16T00:22:41.775 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T00:22:41.775 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout disabled 2024-09-16T00:22:41.782 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T00:22:41.782 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout inactive 2024-09-16T00:22:41.782 INFO:teuthology.orchestra.run.smithi038.stdout:firewalld.service is not enabled 2024-09-16T00:22:41.782 INFO:teuthology.orchestra.run.smithi038.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-16T00:22:41.783 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for mgr to start... 2024-09-16T00:22:41.783 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for mgr... 2024-09-16T00:22:41.996 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:41 smithi038 bash[31157]: d898ac20a9d134ac46b1a5105f46db363aafc064eee96ca783d3eaf498a9fac2 2024-09-16T00:22:41.997 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:41 smithi038 systemd[1]: Started Ceph mgr.x for 983b2072-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:22:41.997 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:41 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:41.778+0000 7f2c9b74f040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T00:22:41.997 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:41 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:41.839+0000 7f2c9b74f040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T00:22:41.997 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:41 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:41.996+0000 7f2c9b74f040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T00:22:42.273 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:42 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:42.068+0000 7f2c9b74f040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T00:22:42.273 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:42 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:42.228+0000 7f2c9b74f040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:22:42.573 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:42 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/1873366123' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T00:22:42.573 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:42 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:42.383+0000 7f2c9b74f040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T00:22:42.874 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:42 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:42.716+0000 7f2c9b74f040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T00:22:42.874 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:42 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:42.781+0000 7f2c9b74f040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T00:22:42.886 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-09-16T00:22:42.886 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-09-16T00:22:42.886 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "fsid": "983b2072-73c1-11ef-bceb-c7b262605968", 2024-09-16T00:22:42.886 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T00:22:42.886 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T00:22:42.886 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T00:22:42.886 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T00:22:42.886 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:22:42.886 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T00:22:42.886 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T00:22:42.886 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 0 2024-09-16T00:22:42.887 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:22:42.887 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T00:22:42.887 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "a" 2024-09-16T00:22:42.887 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:22:42.887 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-16T00:22:42.887 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T00:22:42.887 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:22:42.887 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-16T00:22:42.887 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T00:22:42.887 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:22:42.887 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T00:22:42.887 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:22:42.887 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T00:22:42.888 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T00:22:42.888 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T00:22:42.888 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T00:22:42.888 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T00:22:42.888 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T00:22:42.888 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:22:42.888 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T00:22:42.888 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T00:22:42.888 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T00:22:42.888 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T00:22:42.888 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T00:22:42.888 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T00:22:42.888 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T00:22:42.889 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T00:22:42.889 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T00:22:42.889 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:22:42.889 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T00:22:42.889 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:22:42.889 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "btime": "2024-09-16T00:22:34:896635+0000", 2024-09-16T00:22:42.889 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T00:22:42.889 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T00:22:42.889 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:22:42.889 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T00:22:42.889 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-16T00:22:42.889 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T00:22:42.889 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T00:22:42.890 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T00:22:42.890 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T00:22:42.890 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T00:22:42.890 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:22:42.890 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T00:22:42.890 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:22:42.890 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T00:22:42.890 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:22:42.890 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T00:22:34.897652+0000", 2024-09-16T00:22:42.890 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T00:22:42.890 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:22:42.890 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T00:22:42.890 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-09-16T00:22:42.891 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.200+0000 7f4808e3f640 1 Processor -- start 2024-09-16T00:22:42.891 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.200+0000 7f4808e3f640 1 -- start start 2024-09-16T00:22:42.891 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.200+0000 7f4808e3f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4804108050 0x7f480410a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:42.891 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.200+0000 7f4808e3f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f480407be50 con 0x7f4804108050 2024-09-16T00:22:42.891 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.201+0000 7f480259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4804108050 0x7f480410a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:42.891 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.201+0000 7f480259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4804108050 0x7f480410a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51092/0 (socket says 172.21.15.38:51092) 2024-09-16T00:22:42.891 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.201+0000 7f480259c640 1 -- 172.21.15.38:0/3578338321 learned_addr learned my addr 172.21.15.38:0/3578338321 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:22:42.891 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.201+0000 7f480259c640 1 -- 172.21.15.38:0/3578338321 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f480410a980 con 0x7f4804108050 2024-09-16T00:22:42.891 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.202+0000 7f480259c640 1 --2- 172.21.15.38:0/3578338321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4804108050 0x7f480410a440 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f47f4009820 tx=0x7f47f402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff3ab336d0b0cad6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:42.891 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.202+0000 7f480159a640 1 -- 172.21.15.38:0/3578338321 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47f4037070 con 0x7f4804108050 2024-09-16T00:22:42.891 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.202+0000 7f480159a640 1 -- 172.21.15.38:0/3578338321 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f47f4032900 con 0x7f4804108050 2024-09-16T00:22:42.891 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.202+0000 7f480159a640 1 -- 172.21.15.38:0/3578338321 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47f4030850 con 0x7f4804108050 2024-09-16T00:22:42.891 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.203+0000 7f4808e3f640 1 -- 172.21.15.38:0/3578338321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4804108050 msgr2=0x7f480410a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:42.892 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.203+0000 7f4808e3f640 1 --2- 172.21.15.38:0/3578338321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4804108050 0x7f480410a440 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f47f4009820 tx=0x7f47f402a270 comp rx=0 tx=0).stop 2024-09-16T00:22:42.892 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.203+0000 7f4808e3f640 1 -- 172.21.15.38:0/3578338321 shutdown_connections 2024-09-16T00:22:42.892 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.203+0000 7f4808e3f640 1 --2- 172.21.15.38:0/3578338321 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4804108050 0x7f480410a440 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:42.892 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.203+0000 7f4808e3f640 1 -- 172.21.15.38:0/3578338321 >> 172.21.15.38:0/3578338321 conn(0x7f4804101c30 msgr2=0x7f48041040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:42.892 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.203+0000 7f4808e3f640 1 -- 172.21.15.38:0/3578338321 shutdown_connections 2024-09-16T00:22:42.892 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.203+0000 7f4808e3f640 1 -- 172.21.15.38:0/3578338321 wait complete. 2024-09-16T00:22:42.892 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.204+0000 7f4808e3f640 1 Processor -- start 2024-09-16T00:22:42.892 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.204+0000 7f4808e3f640 1 -- start start 2024-09-16T00:22:42.892 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.204+0000 7f4808e3f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4804108050 0x7f48041078d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:42.892 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.204+0000 7f4808e3f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f480410b9c0 con 0x7f4804108050 2024-09-16T00:22:42.892 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.204+0000 7f480259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4804108050 0x7f48041078d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:42.892 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.204+0000 7f480259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4804108050 0x7f48041078d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51108/0 (socket says 172.21.15.38:51108) 2024-09-16T00:22:42.893 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.204+0000 7f480259c640 1 -- 172.21.15.38:0/1873366123 learned_addr learned my addr 172.21.15.38:0/1873366123 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:22:42.893 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.204+0000 7f480259c640 1 -- 172.21.15.38:0/1873366123 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4804107e10 con 0x7f4804108050 2024-09-16T00:22:42.893 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.205+0000 7f480259c640 1 --2- 172.21.15.38:0/1873366123 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4804108050 0x7f48041078d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f47f4032ce0 tx=0x7f47f4032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:42.893 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.205+0000 7f47f37fe640 1 -- 172.21.15.38:0/1873366123 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47f4042070 con 0x7f4804108050 2024-09-16T00:22:42.893 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.205+0000 7f47f37fe640 1 -- 172.21.15.38:0/1873366123 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f47f40308c0 con 0x7f4804108050 2024-09-16T00:22:42.893 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.205+0000 7f4808e3f640 1 -- 172.21.15.38:0/1873366123 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4804106000 con 0x7f4804108050 2024-09-16T00:22:42.893 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.205+0000 7f47f37fe640 1 -- 172.21.15.38:0/1873366123 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47f4037040 con 0x7f4804108050 2024-09-16T00:22:42.893 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.205+0000 7f4808e3f640 1 -- 172.21.15.38:0/1873366123 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4804106540 con 0x7f4804108050 2024-09-16T00:22:42.893 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.205+0000 7f47f37fe640 1 -- 172.21.15.38:0/1873366123 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f47f4045070 con 0x7f4804108050 2024-09-16T00:22:42.893 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.205+0000 7f47f37fe640 1 -- 172.21.15.38:0/1873366123 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f47f4047bc0 con 0x7f4804108050 2024-09-16T00:22:42.893 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.206+0000 7f4808e3f640 1 -- 172.21.15.38:0/1873366123 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48041084b0 con 0x7f4804108050 2024-09-16T00:22:42.893 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.210+0000 7f47f37fe640 1 -- 172.21.15.38:0/1873366123 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7f47f403c7e0 con 0x7f4804108050 2024-09-16T00:22:42.894 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.272+0000 7f4808e3f640 1 -- 172.21.15.38:0/1873366123 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f4804106880 con 0x7f4804108050 2024-09-16T00:22:42.894 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.273+0000 7f47f37fe640 1 -- 172.21.15.38:0/1873366123 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7f47f403c9c0 con 0x7f4804108050 2024-09-16T00:22:42.894 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.274+0000 7f4808e3f640 1 -- 172.21.15.38:0/1873366123 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4804108050 msgr2=0x7f48041078d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:42.894 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.274+0000 7f4808e3f640 1 --2- 172.21.15.38:0/1873366123 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4804108050 0x7f48041078d0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f47f4032ce0 tx=0x7f47f4032d10 comp rx=0 tx=0).stop 2024-09-16T00:22:42.894 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.274+0000 7f4808e3f640 1 -- 172.21.15.38:0/1873366123 shutdown_connections 2024-09-16T00:22:42.894 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.274+0000 7f4808e3f640 1 --2- 172.21.15.38:0/1873366123 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4804108050 0x7f48041078d0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:42.894 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.274+0000 7f4808e3f640 1 -- 172.21.15.38:0/1873366123 >> 172.21.15.38:0/1873366123 conn(0x7f4804101c30 msgr2=0x7f48041027d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:42.894 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.274+0000 7f4808e3f640 1 -- 172.21.15.38:0/1873366123 shutdown_connections 2024-09-16T00:22:42.894 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:42.274+0000 7f4808e3f640 1 -- 172.21.15.38:0/1873366123 wait complete. 2024-09-16T00:22:42.894 INFO:teuthology.orchestra.run.smithi038.stdout:mgr not available, waiting (1/15)... 2024-09-16T00:22:43.215 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:42 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:42.975+0000 7f2c9b74f040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T00:22:44.215 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:43 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:43.903+0000 7f2c9b74f040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T00:22:44.215 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:43 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:43.974+0000 7f2c9b74f040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T00:22:45.002 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:44 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:44.729+0000 7f2c9b74f040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T00:22:45.003 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:44 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:44.849+0000 7f2c9b74f040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T00:22:45.299 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:45 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:45.002+0000 7f2c9b74f040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:22:45.299 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:45 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:45.066+0000 7f2c9b74f040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T00:22:45.299 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:45 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:45.225+0000 7f2c9b74f040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T00:22:45.300 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:45 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:45.298+0000 7f2c9b74f040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T00:22:45.553 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:45 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/1769316337' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T00:22:45.553 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:45 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:45.383+0000 7f2c9b74f040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T00:22:45.553 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:45 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:45.552+0000 7f2c9b74f040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T00:22:45.844 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:45 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:45.620+0000 7f2c9b74f040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T00:22:45.844 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:45 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:45.693+0000 7f2c9b74f040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T00:22:45.844 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:45 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:45.780+0000 7f2c9b74f040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T00:22:45.844 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:45 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:45.843+0000 7f2c9b74f040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T00:22:46.076 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-09-16T00:22:46.077 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-09-16T00:22:46.077 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "fsid": "983b2072-73c1-11ef-bceb-c7b262605968", 2024-09-16T00:22:46.077 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T00:22:46.077 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T00:22:46.077 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T00:22:46.077 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T00:22:46.077 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:22:46.077 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T00:22:46.077 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T00:22:46.077 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 0 2024-09-16T00:22:46.078 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:22:46.078 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T00:22:46.078 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "a" 2024-09-16T00:22:46.078 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:22:46.078 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-16T00:22:46.078 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T00:22:46.078 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:22:46.078 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-16T00:22:46.078 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T00:22:46.078 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:22:46.078 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T00:22:46.078 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:22:46.079 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T00:22:46.079 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T00:22:46.079 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T00:22:46.079 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T00:22:46.079 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T00:22:46.079 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T00:22:46.079 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:22:46.079 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T00:22:46.079 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T00:22:46.079 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T00:22:46.079 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T00:22:46.079 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T00:22:46.079 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T00:22:46.080 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T00:22:46.080 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T00:22:46.080 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T00:22:46.080 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:22:46.080 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T00:22:46.080 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:22:46.080 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "btime": "2024-09-16T00:22:34:896635+0000", 2024-09-16T00:22:46.080 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T00:22:46.080 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T00:22:46.080 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:22:46.080 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T00:22:46.080 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-16T00:22:46.080 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T00:22:46.080 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T00:22:46.081 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T00:22:46.081 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T00:22:46.081 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T00:22:46.081 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:22:46.081 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T00:22:46.081 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:22:46.081 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T00:22:46.081 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:22:46.081 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T00:22:34.897652+0000", 2024-09-16T00:22:46.081 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T00:22:46.081 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:22:46.081 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T00:22:46.081 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-09-16T00:22:46.081 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.306+0000 7fc75264f640 1 Processor -- start 2024-09-16T00:22:46.082 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.306+0000 7fc75264f640 1 -- start start 2024-09-16T00:22:46.082 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.306+0000 7fc75264f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc74c107620 0x7fc74c107a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:46.082 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.306+0000 7fc75264f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc74c107ff0 con 0x7fc74c107620 2024-09-16T00:22:46.082 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.306+0000 7fc74bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc74c107620 0x7fc74c107a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:46.082 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.306+0000 7fc74bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc74c107620 0x7fc74c107a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51116/0 (socket says 172.21.15.38:51116) 2024-09-16T00:22:46.082 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.306+0000 7fc74bfff640 1 -- 172.21.15.38:0/2402361254 learned_addr learned my addr 172.21.15.38:0/2402361254 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:22:46.082 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.307+0000 7fc74bfff640 1 -- 172.21.15.38:0/2402361254 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc74c108820 con 0x7fc74c107620 2024-09-16T00:22:46.082 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.307+0000 7fc74bfff640 1 --2- 172.21.15.38:0/2402361254 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc74c107620 0x7fc74c107a20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc73c009820 tx=0x7fc73c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b0a710128d9d2eb server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:46.082 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.308+0000 7fc74affd640 1 -- 172.21.15.38:0/2402361254 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc73c037070 con 0x7fc74c107620 2024-09-16T00:22:46.082 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.308+0000 7fc74affd640 1 -- 172.21.15.38:0/2402361254 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc73c032900 con 0x7fc74c107620 2024-09-16T00:22:46.082 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.308+0000 7fc74affd640 1 -- 172.21.15.38:0/2402361254 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc73c030850 con 0x7fc74c107620 2024-09-16T00:22:46.082 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.308+0000 7fc75264f640 1 -- 172.21.15.38:0/2402361254 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc74c107620 msgr2=0x7fc74c107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:46.082 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.308+0000 7fc75264f640 1 --2- 172.21.15.38:0/2402361254 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc74c107620 0x7fc74c107a20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc73c009820 tx=0x7fc73c02a270 comp rx=0 tx=0).stop 2024-09-16T00:22:46.082 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.308+0000 7fc75264f640 1 -- 172.21.15.38:0/2402361254 shutdown_connections 2024-09-16T00:22:46.083 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.308+0000 7fc75264f640 1 --2- 172.21.15.38:0/2402361254 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc74c107620 0x7fc74c107a20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:46.083 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.308+0000 7fc75264f640 1 -- 172.21.15.38:0/2402361254 >> 172.21.15.38:0/2402361254 conn(0x7fc74c102ea0 msgr2=0x7fc74c105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:46.083 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.308+0000 7fc75264f640 1 -- 172.21.15.38:0/2402361254 shutdown_connections 2024-09-16T00:22:46.083 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.308+0000 7fc75264f640 1 -- 172.21.15.38:0/2402361254 wait complete. 2024-09-16T00:22:46.083 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.309+0000 7fc75264f640 1 Processor -- start 2024-09-16T00:22:46.083 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.309+0000 7fc75264f640 1 -- start start 2024-09-16T00:22:46.083 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.309+0000 7fc75264f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc74c107620 0x7fc74c19d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:46.083 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.309+0000 7fc75264f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc74c10bbd0 con 0x7fc74c107620 2024-09-16T00:22:46.083 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.309+0000 7fc74bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc74c107620 0x7fc74c19d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:46.083 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.309+0000 7fc74bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc74c107620 0x7fc74c19d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51128/0 (socket says 172.21.15.38:51128) 2024-09-16T00:22:46.083 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.309+0000 7fc74bfff640 1 -- 172.21.15.38:0/1769316337 learned_addr learned my addr 172.21.15.38:0/1769316337 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:22:46.083 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.310+0000 7fc74bfff640 1 -- 172.21.15.38:0/1769316337 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc74c19dd80 con 0x7fc74c107620 2024-09-16T00:22:46.083 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.310+0000 7fc74bfff640 1 --2- 172.21.15.38:0/1769316337 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc74c107620 0x7fc74c19d840 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc73c0097f0 tx=0x7fc73c030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:46.084 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.310+0000 7fc7497fa640 1 -- 172.21.15.38:0/1769316337 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc73c037040 con 0x7fc74c107620 2024-09-16T00:22:46.084 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.310+0000 7fc7497fa640 1 -- 172.21.15.38:0/1769316337 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc73c040070 con 0x7fc74c107620 2024-09-16T00:22:46.084 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.310+0000 7fc75264f640 1 -- 172.21.15.38:0/1769316337 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc74c19e010 con 0x7fc74c107620 2024-09-16T00:22:46.084 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.310+0000 7fc7497fa640 1 -- 172.21.15.38:0/1769316337 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc73c03a390 con 0x7fc74c107620 2024-09-16T00:22:46.084 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.310+0000 7fc75264f640 1 -- 172.21.15.38:0/1769316337 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc74c1a0d00 con 0x7fc74c107620 2024-09-16T00:22:46.084 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.310+0000 7fc7497fa640 1 -- 172.21.15.38:0/1769316337 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc73c03a5e0 con 0x7fc74c107620 2024-09-16T00:22:46.084 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.310+0000 7fc7497fa640 1 -- 172.21.15.38:0/1769316337 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc73c045950 con 0x7fc74c107620 2024-09-16T00:22:46.084 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.311+0000 7fc75264f640 1 -- 172.21.15.38:0/1769316337 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc74c107aa0 con 0x7fc74c107620 2024-09-16T00:22:46.084 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.313+0000 7fc7497fa640 1 -- 172.21.15.38:0/1769316337 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+74432 (secure 0 0 0) 0x7fc73c045460 con 0x7fc74c107620 2024-09-16T00:22:46.084 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.401+0000 7fc75264f640 1 -- 172.21.15.38:0/1769316337 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7fc74c1a0ed0 con 0x7fc74c107620 2024-09-16T00:22:46.084 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.401+0000 7fc7497fa640 1 -- 172.21.15.38:0/1769316337 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1291 (secure 0 0 0) 0x7fc73c03b430 con 0x7fc74c107620 2024-09-16T00:22:46.084 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.403+0000 7fc75264f640 1 -- 172.21.15.38:0/1769316337 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc74c107620 msgr2=0x7fc74c19d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:46.084 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.403+0000 7fc75264f640 1 --2- 172.21.15.38:0/1769316337 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc74c107620 0x7fc74c19d840 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fc73c0097f0 tx=0x7fc73c030d80 comp rx=0 tx=0).stop 2024-09-16T00:22:46.085 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.403+0000 7fc75264f640 1 -- 172.21.15.38:0/1769316337 shutdown_connections 2024-09-16T00:22:46.085 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.403+0000 7fc75264f640 1 --2- 172.21.15.38:0/1769316337 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc74c107620 0x7fc74c19d840 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:46.085 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.403+0000 7fc75264f640 1 -- 172.21.15.38:0/1769316337 >> 172.21.15.38:0/1769316337 conn(0x7fc74c102ea0 msgr2=0x7fc74c103950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:46.085 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.403+0000 7fc75264f640 1 -- 172.21.15.38:0/1769316337 shutdown_connections 2024-09-16T00:22:46.085 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:45.403+0000 7fc75264f640 1 -- 172.21.15.38:0/1769316337 wait complete. 2024-09-16T00:22:46.085 INFO:teuthology.orchestra.run.smithi038.stdout:mgr not available, waiting (2/15)... 2024-09-16T00:22:46.214 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:45 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:45.905+0000 7f2c9b74f040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T00:22:47.215 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:46 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:46.734+0000 7f2c9b74f040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T00:22:47.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:46 smithi038 ceph-mon[30945]: Activating manager daemon x 2024-09-16T00:22:47.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:46 smithi038 ceph-mon[30945]: mgrmap e2: x(active, starting, since 0.00785527s) 2024-09-16T00:22:47.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:46 smithi038 ceph-mon[30945]: from='mgr.14100 172.21.15.38:0/1866938801' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T00:22:47.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:46 smithi038 ceph-mon[30945]: from='mgr.14100 172.21.15.38:0/1866938801' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T00:22:47.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:46 smithi038 ceph-mon[30945]: from='mgr.14100 172.21.15.38:0/1866938801' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T00:22:47.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:46 smithi038 ceph-mon[30945]: from='mgr.14100 172.21.15.38:0/1866938801' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:22:47.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:46 smithi038 ceph-mon[30945]: from='mgr.14100 172.21.15.38:0/1866938801' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T00:22:47.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:46 smithi038 ceph-mon[30945]: Manager daemon x is now available 2024-09-16T00:22:47.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:46 smithi038 ceph-mon[30945]: from='mgr.14100 172.21.15.38:0/1866938801' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-16T00:22:47.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:46 smithi038 ceph-mon[30945]: from='mgr.14100 172.21.15.38:0/1866938801' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-16T00:22:47.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:46 smithi038 ceph-mon[30945]: from='mgr.14100 172.21.15.38:0/1866938801' entity='mgr.x' 2024-09-16T00:22:47.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:46 smithi038 ceph-mon[30945]: from='mgr.14100 172.21.15.38:0/1866938801' entity='mgr.x' 2024-09-16T00:22:47.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:46 smithi038 ceph-mon[30945]: from='mgr.14100 172.21.15.38:0/1866938801' entity='mgr.x' 2024-09-16T00:22:48.760 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:48 smithi038 ceph-mon[30945]: mgrmap e3: x(active, since 1.01218s) 2024-09-16T00:22:49.341 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-09-16T00:22:49.341 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-09-16T00:22:49.341 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "fsid": "983b2072-73c1-11ef-bceb-c7b262605968", 2024-09-16T00:22:49.341 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "health": { 2024-09-16T00:22:49.341 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-16T00:22:49.342 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-16T00:22:49.342 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-16T00:22:49.342 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:22:49.342 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-16T00:22:49.342 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-16T00:22:49.342 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 0 2024-09-16T00:22:49.342 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:22:49.342 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-16T00:22:49.342 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "a" 2024-09-16T00:22:49.342 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:22:49.342 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-16T00:22:49.342 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-16T00:22:49.342 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:22:49.342 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-16T00:22:49.343 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-16T00:22:49.343 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:22:49.343 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-16T00:22:49.343 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:22:49.343 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-16T00:22:49.343 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-16T00:22:49.343 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-16T00:22:49.351 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-16T00:22:49.351 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-16T00:22:49.351 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-16T00:22:49.351 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:22:49.351 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-16T00:22:49.351 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-16T00:22:49.351 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-16T00:22:49.351 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-16T00:22:49.352 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-16T00:22:49.352 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-16T00:22:49.352 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-16T00:22:49.352 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-16T00:22:49.352 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-16T00:22:49.352 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:22:49.352 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-16T00:22:49.352 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:22:49.352 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "btime": "2024-09-16T00:22:34:896635+0000", 2024-09-16T00:22:49.352 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-16T00:22:49.352 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-16T00:22:49.352 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:22:49.352 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-16T00:22:49.353 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T00:22:49.353 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-16T00:22:49.353 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-16T00:22:49.353 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-16T00:22:49.353 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-16T00:22:49.353 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "restful" 2024-09-16T00:22:49.353 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ], 2024-09-16T00:22:49.353 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T00:22:49.353 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:22:49.353 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-16T00:22:49.353 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-16T00:22:49.353 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "modified": "2024-09-16T00:22:34.897652+0000", 2024-09-16T00:22:49.353 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-16T00:22:49.353 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout }, 2024-09-16T00:22:49.354 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-16T00:22:49.354 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-09-16T00:22:49.354 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.504+0000 7f556fd54640 1 Processor -- start 2024-09-16T00:22:49.354 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.505+0000 7f556fd54640 1 -- start start 2024-09-16T00:22:49.354 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.505+0000 7f556fd54640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55681080d0 0x7f556810a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:49.354 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.505+0000 7f556fd54640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f556807bdd0 con 0x7f55681080d0 2024-09-16T00:22:49.354 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.505+0000 7f556daf0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55681080d0 0x7f556810a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:49.354 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.505+0000 7f556daf0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55681080d0 0x7f556810a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41530/0 (socket says 172.21.15.38:41530) 2024-09-16T00:22:49.354 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.505+0000 7f556daf0640 1 -- 172.21.15.38:0/245982666 learned_addr learned my addr 172.21.15.38:0/245982666 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:22:49.354 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.505+0000 7f556daf0640 1 -- 172.21.15.38:0/245982666 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f556810aa00 con 0x7f55681080d0 2024-09-16T00:22:49.354 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.506+0000 7f556daf0640 1 --2- 172.21.15.38:0/245982666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55681080d0 0x7f556810a4c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5558009820 tx=0x7f555802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ecd9a477d638d31 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:49.354 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.506+0000 7f556caee640 1 -- 172.21.15.38:0/245982666 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5558037070 con 0x7f55681080d0 2024-09-16T00:22:49.354 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.506+0000 7f556caee640 1 -- 172.21.15.38:0/245982666 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5558032900 con 0x7f55681080d0 2024-09-16T00:22:49.355 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.506+0000 7f556caee640 1 -- 172.21.15.38:0/245982666 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5558030850 con 0x7f55681080d0 2024-09-16T00:22:49.355 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.507+0000 7f556fd54640 1 -- 172.21.15.38:0/245982666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55681080d0 msgr2=0x7f556810a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:49.355 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.507+0000 7f556fd54640 1 --2- 172.21.15.38:0/245982666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55681080d0 0x7f556810a4c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5558009820 tx=0x7f555802a270 comp rx=0 tx=0).stop 2024-09-16T00:22:49.355 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.507+0000 7f556fd54640 1 -- 172.21.15.38:0/245982666 shutdown_connections 2024-09-16T00:22:49.355 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.507+0000 7f556fd54640 1 --2- 172.21.15.38:0/245982666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55681080d0 0x7f556810a4c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.355 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.507+0000 7f556fd54640 1 -- 172.21.15.38:0/245982666 >> 172.21.15.38:0/245982666 conn(0x7f5568101c30 msgr2=0x7f55681040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:49.355 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.507+0000 7f556fd54640 1 -- 172.21.15.38:0/245982666 shutdown_connections 2024-09-16T00:22:49.355 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.507+0000 7f556fd54640 1 -- 172.21.15.38:0/245982666 wait complete. 2024-09-16T00:22:49.355 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.508+0000 7f556fd54640 1 Processor -- start 2024-09-16T00:22:49.355 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.508+0000 7f556fd54640 1 -- start start 2024-09-16T00:22:49.355 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.508+0000 7f556fd54640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55681080d0 0x7f5568107950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:49.355 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.508+0000 7f556fd54640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f556810b980 con 0x7f55681080d0 2024-09-16T00:22:49.355 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.508+0000 7f556daf0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55681080d0 0x7f5568107950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:49.355 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.508+0000 7f556daf0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55681080d0 0x7f5568107950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41538/0 (socket says 172.21.15.38:41538) 2024-09-16T00:22:49.356 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.508+0000 7f556daf0640 1 -- 172.21.15.38:0/1896943365 learned_addr learned my addr 172.21.15.38:0/1896943365 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:22:49.356 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.509+0000 7f556daf0640 1 -- 172.21.15.38:0/1896943365 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5568107e90 con 0x7f55681080d0 2024-09-16T00:22:49.356 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.509+0000 7f556daf0640 1 --2- 172.21.15.38:0/1896943365 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55681080d0 0x7f5568107950 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5558032ce0 tx=0x7f5558032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:49.356 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.509+0000 7f5566ffd640 1 -- 172.21.15.38:0/1896943365 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5558042070 con 0x7f55681080d0 2024-09-16T00:22:49.356 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.509+0000 7f5566ffd640 1 -- 172.21.15.38:0/1896943365 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f55580308c0 con 0x7f55681080d0 2024-09-16T00:22:49.356 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.509+0000 7f556fd54640 1 -- 172.21.15.38:0/1896943365 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5568106080 con 0x7f55681080d0 2024-09-16T00:22:49.356 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.509+0000 7f5566ffd640 1 -- 172.21.15.38:0/1896943365 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5558037040 con 0x7f55681080d0 2024-09-16T00:22:49.356 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.509+0000 7f556fd54640 1 -- 172.21.15.38:0/1896943365 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55681065c0 con 0x7f55681080d0 2024-09-16T00:22:49.357 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.510+0000 7f5566ffd640 1 -- 172.21.15.38:0/1896943365 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 3) ==== 46344+0+0 (secure 0 0 0) 0x7f5558045070 con 0x7f55681080d0 2024-09-16T00:22:49.357 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.510+0000 7f556fd54640 1 -- 172.21.15.38:0/1896943365 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5568108530 con 0x7f55681080d0 2024-09-16T00:22:49.357 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.510+0000 7f5566ffd640 1 --2- 172.21.15.38:0/1896943365 >> [v2:172.21.15.38:6800/583006373,v1:172.21.15.38:6801/583006373] conn(0x7f55500390a0 0x7f555003b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:49.357 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.510+0000 7f5566ffd640 1 -- 172.21.15.38:0/1896943365 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f555806c010 con 0x7f55681080d0 2024-09-16T00:22:49.357 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.510+0000 7f556d2ef640 1 --2- 172.21.15.38:0/1896943365 >> [v2:172.21.15.38:6800/583006373,v1:172.21.15.38:6801/583006373] conn(0x7f55500390a0 0x7f555003b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:49.357 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.511+0000 7f556d2ef640 1 --2- 172.21.15.38:0/1896943365 >> [v2:172.21.15.38:6800/583006373,v1:172.21.15.38:6801/583006373] conn(0x7f55500390a0 0x7f555003b560 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f555c0098c0 tx=0x7f555c006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:49.357 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.515+0000 7f5566ffd640 1 -- 172.21.15.38:0/1896943365 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f555803c780 con 0x7f55681080d0 2024-09-16T00:22:49.357 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.760+0000 7f556fd54640 1 -- 172.21.15.38:0/1896943365 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f5568106e00 con 0x7f55681080d0 2024-09-16T00:22:49.357 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.761+0000 7f5566ffd640 1 -- 172.21.15.38:0/1896943365 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f555803cca0 con 0x7f55681080d0 2024-09-16T00:22:49.357 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.762+0000 7f5566ffd640 1 -- 172.21.15.38:0/1896943365 <== mon.0 v2:172.21.15.38:3300/0 8 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1290 (secure 0 0 0) 0x7f5558004430 con 0x7f55681080d0 2024-09-16T00:22:49.357 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.764+0000 7f556fd54640 1 -- 172.21.15.38:0/1896943365 >> [v2:172.21.15.38:6800/583006373,v1:172.21.15.38:6801/583006373] conn(0x7f55500390a0 msgr2=0x7f555003b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:49.358 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.764+0000 7f556fd54640 1 --2- 172.21.15.38:0/1896943365 >> [v2:172.21.15.38:6800/583006373,v1:172.21.15.38:6801/583006373] conn(0x7f55500390a0 0x7f555003b560 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f555c0098c0 tx=0x7f555c006db0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.358 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.764+0000 7f556fd54640 1 -- 172.21.15.38:0/1896943365 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55681080d0 msgr2=0x7f5568107950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:49.358 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.764+0000 7f556fd54640 1 --2- 172.21.15.38:0/1896943365 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55681080d0 0x7f5568107950 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f5558032ce0 tx=0x7f5558032d10 comp rx=0 tx=0).stop 2024-09-16T00:22:49.358 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.764+0000 7f556fd54640 1 -- 172.21.15.38:0/1896943365 shutdown_connections 2024-09-16T00:22:49.358 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.764+0000 7f556fd54640 1 --2- 172.21.15.38:0/1896943365 >> [v2:172.21.15.38:6800/583006373,v1:172.21.15.38:6801/583006373] conn(0x7f55500390a0 0x7f555003b560 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.358 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.764+0000 7f556fd54640 1 --2- 172.21.15.38:0/1896943365 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f55681080d0 0x7f5568107950 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:49.358 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.764+0000 7f556fd54640 1 -- 172.21.15.38:0/1896943365 >> 172.21.15.38:0/1896943365 conn(0x7f5568101c30 msgr2=0x7f55681027e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:49.358 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.765+0000 7f556fd54640 1 -- 172.21.15.38:0/1896943365 shutdown_connections 2024-09-16T00:22:49.358 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:48.765+0000 7f556fd54640 1 -- 172.21.15.38:0/1896943365 wait complete. 2024-09-16T00:22:49.358 INFO:teuthology.orchestra.run.smithi038.stdout:mgr is available 2024-09-16T00:22:49.929 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:49 smithi038 ceph-mon[30945]: mgrmap e4: x(active, since 2s) 2024-09-16T00:22:49.930 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:49 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/1896943365' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-16T00:22:50.502 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-09-16T00:22:50.502 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout [global] 2024-09-16T00:22:50.502 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout fsid = 983b2072-73c1-11ef-bceb-c7b262605968 2024-09-16T00:22:50.502 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.38:3300,v1:172.21.15.38:6789] 2024-09-16T00:22:50.502 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-16T00:22:50.502 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-16T00:22:50.502 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-16T00:22:50.502 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-16T00:22:50.502 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-09-16T00:22:50.503 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-16T00:22:50.503 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-16T00:22:50.503 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 2024-09-16T00:22:50.503 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout [osd] 2024-09-16T00:22:50.503 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-16T00:22:50.503 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-16T00:22:50.503 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-16T00:22:50.503 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-16T00:22:50.503 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-16T00:22:50.503 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.742+0000 7f5dbdff5640 1 Processor -- start 2024-09-16T00:22:50.503 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.742+0000 7f5dbdff5640 1 -- start start 2024-09-16T00:22:50.503 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.742+0000 7f5dbdff5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5db8107620 0x7f5db8107a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:50.503 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.742+0000 7f5dbdff5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5db8107ff0 con 0x7f5db8107620 2024-09-16T00:22:50.504 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.742+0000 7f5db77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5db8107620 0x7f5db8107a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:50.504 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.742+0000 7f5db77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5db8107620 0x7f5db8107a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41544/0 (socket says 172.21.15.38:41544) 2024-09-16T00:22:50.504 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.742+0000 7f5db77fe640 1 -- 172.21.15.38:0/231457279 learned_addr learned my addr 172.21.15.38:0/231457279 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:22:50.504 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.743+0000 7f5db77fe640 1 -- 172.21.15.38:0/231457279 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db8108820 con 0x7f5db8107620 2024-09-16T00:22:50.504 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.743+0000 7f5db77fe640 1 --2- 172.21.15.38:0/231457279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5db8107620 0x7f5db8107a20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5da8009820 tx=0x7f5da802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28f1b715868f9a03 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:50.504 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.744+0000 7f5db67fc640 1 -- 172.21.15.38:0/231457279 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5da8037070 con 0x7f5db8107620 2024-09-16T00:22:50.504 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.744+0000 7f5db67fc640 1 -- 172.21.15.38:0/231457279 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5da8032900 con 0x7f5db8107620 2024-09-16T00:22:50.504 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.744+0000 7f5db67fc640 1 -- 172.21.15.38:0/231457279 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5da8030850 con 0x7f5db8107620 2024-09-16T00:22:50.504 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.744+0000 7f5dbdff5640 1 -- 172.21.15.38:0/231457279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5db8107620 msgr2=0x7f5db8107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:50.504 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.744+0000 7f5dbdff5640 1 --2- 172.21.15.38:0/231457279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5db8107620 0x7f5db8107a20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5da8009820 tx=0x7f5da802a270 comp rx=0 tx=0).stop 2024-09-16T00:22:50.504 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.744+0000 7f5dbdff5640 1 -- 172.21.15.38:0/231457279 shutdown_connections 2024-09-16T00:22:50.504 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.744+0000 7f5dbdff5640 1 --2- 172.21.15.38:0/231457279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5db8107620 0x7f5db8107a20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.504 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.744+0000 7f5dbdff5640 1 -- 172.21.15.38:0/231457279 >> 172.21.15.38:0/231457279 conn(0x7f5db8102ea0 msgr2=0x7f5db8105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:50.505 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.744+0000 7f5dbdff5640 1 -- 172.21.15.38:0/231457279 shutdown_connections 2024-09-16T00:22:50.505 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.744+0000 7f5dbdff5640 1 -- 172.21.15.38:0/231457279 wait complete. 2024-09-16T00:22:50.505 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.745+0000 7f5dbdff5640 1 Processor -- start 2024-09-16T00:22:50.505 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.745+0000 7f5dbdff5640 1 -- start start 2024-09-16T00:22:50.505 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.745+0000 7f5dbdff5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5db8107620 0x7f5db819b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:50.505 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.745+0000 7f5dbdff5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5db810bbd0 con 0x7f5db8107620 2024-09-16T00:22:50.505 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.746+0000 7f5db77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5db8107620 0x7f5db819b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:50.505 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.746+0000 7f5db77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5db8107620 0x7f5db819b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41552/0 (socket says 172.21.15.38:41552) 2024-09-16T00:22:50.505 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.746+0000 7f5db77fe640 1 -- 172.21.15.38:0/632876724 learned_addr learned my addr 172.21.15.38:0/632876724 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:22:50.505 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.746+0000 7f5db77fe640 1 -- 172.21.15.38:0/632876724 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5db819bb50 con 0x7f5db8107620 2024-09-16T00:22:50.505 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.746+0000 7f5db77fe640 1 --2- 172.21.15.38:0/632876724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5db8107620 0x7f5db819b610 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5da80097f0 tx=0x7f5da8030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:50.505 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.746+0000 7f5db4ff9640 1 -- 172.21.15.38:0/632876724 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5da8037040 con 0x7f5db8107620 2024-09-16T00:22:50.506 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.746+0000 7f5db4ff9640 1 -- 172.21.15.38:0/632876724 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f5da8040070 con 0x7f5db8107620 2024-09-16T00:22:50.506 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.746+0000 7f5dbdff5640 1 -- 172.21.15.38:0/632876724 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5db819bde0 con 0x7f5db8107620 2024-09-16T00:22:50.506 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.746+0000 7f5db4ff9640 1 -- 172.21.15.38:0/632876724 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5da803a390 con 0x7f5db8107620 2024-09-16T00:22:50.506 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.746+0000 7f5dbdff5640 1 -- 172.21.15.38:0/632876724 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5db819ead0 con 0x7f5db8107620 2024-09-16T00:22:50.506 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.747+0000 7f5db4ff9640 1 -- 172.21.15.38:0/632876724 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f5da803a5e0 con 0x7f5db8107620 2024-09-16T00:22:50.506 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.747+0000 7f5db4ff9640 1 --2- 172.21.15.38:0/632876724 >> [v2:172.21.15.38:6800/583006373,v1:172.21.15.38:6801/583006373] conn(0x7f5da00391b0 0x7f5da003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:50.506 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.747+0000 7f5dbdff5640 1 -- 172.21.15.38:0/632876724 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5db8107aa0 con 0x7f5db8107620 2024-09-16T00:22:50.506 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.747+0000 7f5db4ff9640 1 -- 172.21.15.38:0/632876724 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5da806c380 con 0x7f5db8107620 2024-09-16T00:22:50.506 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.747+0000 7f5db6ffd640 1 --2- 172.21.15.38:0/632876724 >> [v2:172.21.15.38:6800/583006373,v1:172.21.15.38:6801/583006373] conn(0x7f5da00391b0 0x7f5da003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:50.506 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.748+0000 7f5db6ffd640 1 --2- 172.21.15.38:0/632876724 >> [v2:172.21.15.38:6800/583006373,v1:172.21.15.38:6801/583006373] conn(0x7f5da00391b0 0x7f5da003b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5dac0098c0 tx=0x7f5dac006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:50.506 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.752+0000 7f5db4ff9640 1 -- 172.21.15.38:0/632876724 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5da806c0e0 con 0x7f5db8107620 2024-09-16T00:22:50.506 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.929+0000 7f5dbdff5640 1 -- 172.21.15.38:0/632876724 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f5db819c260 con 0x7f5db8107620 2024-09-16T00:22:50.506 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.930+0000 7f5db4ff9640 1 -- 172.21.15.38:0/632876724 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+429 (secure 0 0 0) 0x7f5da803a8d0 con 0x7f5db8107620 2024-09-16T00:22:50.507 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.933+0000 7f5dbdff5640 1 -- 172.21.15.38:0/632876724 >> [v2:172.21.15.38:6800/583006373,v1:172.21.15.38:6801/583006373] conn(0x7f5da00391b0 msgr2=0x7f5da003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:50.507 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.933+0000 7f5dbdff5640 1 --2- 172.21.15.38:0/632876724 >> [v2:172.21.15.38:6800/583006373,v1:172.21.15.38:6801/583006373] conn(0x7f5da00391b0 0x7f5da003b670 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5dac0098c0 tx=0x7f5dac006db0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.507 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.933+0000 7f5dbdff5640 1 -- 172.21.15.38:0/632876724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5db8107620 msgr2=0x7f5db819b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:50.507 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.933+0000 7f5dbdff5640 1 --2- 172.21.15.38:0/632876724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5db8107620 0x7f5db819b610 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5da80097f0 tx=0x7f5da8030d80 comp rx=0 tx=0).stop 2024-09-16T00:22:50.507 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.933+0000 7f5dbdff5640 1 -- 172.21.15.38:0/632876724 shutdown_connections 2024-09-16T00:22:50.507 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.933+0000 7f5dbdff5640 1 --2- 172.21.15.38:0/632876724 >> [v2:172.21.15.38:6800/583006373,v1:172.21.15.38:6801/583006373] conn(0x7f5da00391b0 0x7f5da003b670 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.507 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.933+0000 7f5dbdff5640 1 --2- 172.21.15.38:0/632876724 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5db8107620 0x7f5db819b610 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:50.507 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.933+0000 7f5dbdff5640 1 -- 172.21.15.38:0/632876724 >> 172.21.15.38:0/632876724 conn(0x7f5db8102ea0 msgr2=0x7f5db8103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:50.507 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.933+0000 7f5dbdff5640 1 -- 172.21.15.38:0/632876724 shutdown_connections 2024-09-16T00:22:50.507 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:49.934+0000 7f5dbdff5640 1 -- 172.21.15.38:0/632876724 wait complete. 2024-09-16T00:22:50.507 INFO:teuthology.orchestra.run.smithi038.stdout:Enabling cephadm module... 2024-09-16T00:22:51.062 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:50 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/632876724' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-16T00:22:52.107 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:51 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/399819412' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-16T00:22:52.107 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:51 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: ignoring --setuser ceph since I am not root 2024-09-16T00:22:52.107 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:51 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: ignoring --setgroup ceph since I am not root 2024-09-16T00:22:52.108 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:51 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:51.941+0000 7fc98e861040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T00:22:52.108 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:52 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:52.001+0000 7fc98e861040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T00:22:52.381 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:52 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:52.156+0000 7fc98e861040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T00:22:52.381 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:52 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:52.226+0000 7fc98e861040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T00:22:52.381 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:52 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:52.380+0000 7fc98e861040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:22:52.441 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.153+0000 7f871485a640 1 Processor -- start 2024-09-16T00:22:52.441 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.153+0000 7f871485a640 1 -- start start 2024-09-16T00:22:52.442 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.154+0000 7f871485a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87101064e0 0x7f87101068e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:52.442 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.154+0000 7f871485a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8710106eb0 con 0x7f87101064e0 2024-09-16T00:22:52.442 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.154+0000 7f870e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87101064e0 0x7f87101068e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:52.442 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.154+0000 7f870e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87101064e0 0x7f87101068e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41556/0 (socket says 172.21.15.38:41556) 2024-09-16T00:22:52.442 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.154+0000 7f870e59c640 1 -- 172.21.15.38:0/473478438 learned_addr learned my addr 172.21.15.38:0/473478438 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:22:52.442 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.154+0000 7f870e59c640 1 -- 172.21.15.38:0/473478438 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8710107690 con 0x7f87101064e0 2024-09-16T00:22:52.442 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.155+0000 7f870e59c640 1 --2- 172.21.15.38:0/473478438 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87101064e0 0x7f87101068e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8700009e80 tx=0x7f870002a7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61644e06faedcf2b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:52.442 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.155+0000 7f870d59a640 1 -- 172.21.15.38:0/473478438 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8700036070 con 0x7f87101064e0 2024-09-16T00:22:52.442 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.155+0000 7f870d59a640 1 -- 172.21.15.38:0/473478438 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f87000318c0 con 0x7f87101064e0 2024-09-16T00:22:52.442 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.155+0000 7f870d59a640 1 -- 172.21.15.38:0/473478438 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8700007ab0 con 0x7f87101064e0 2024-09-16T00:22:52.442 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.156+0000 7f871485a640 1 -- 172.21.15.38:0/473478438 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87101064e0 msgr2=0x7f87101068e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:52.442 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.156+0000 7f871485a640 1 --2- 172.21.15.38:0/473478438 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87101064e0 0x7f87101068e0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8700009e80 tx=0x7f870002a7a0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.443 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.156+0000 7f871485a640 1 -- 172.21.15.38:0/473478438 shutdown_connections 2024-09-16T00:22:52.443 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.156+0000 7f871485a640 1 --2- 172.21.15.38:0/473478438 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87101064e0 0x7f87101068e0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.443 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.156+0000 7f871485a640 1 -- 172.21.15.38:0/473478438 >> 172.21.15.38:0/473478438 conn(0x7f8710101c30 msgr2=0x7f87101040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:52.443 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.156+0000 7f871485a640 1 -- 172.21.15.38:0/473478438 shutdown_connections 2024-09-16T00:22:52.443 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.156+0000 7f871485a640 1 -- 172.21.15.38:0/473478438 wait complete. 2024-09-16T00:22:52.443 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.157+0000 7f871485a640 1 Processor -- start 2024-09-16T00:22:52.443 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.157+0000 7f871485a640 1 -- start start 2024-09-16T00:22:52.443 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.157+0000 7f871485a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87101064e0 0x7f871019b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:52.443 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.157+0000 7f871485a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f871010aa40 con 0x7f87101064e0 2024-09-16T00:22:52.443 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.157+0000 7f870e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87101064e0 0x7f871019b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:52.443 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.157+0000 7f870e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87101064e0 0x7f871019b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41564/0 (socket says 172.21.15.38:41564) 2024-09-16T00:22:52.443 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.157+0000 7f870e59c640 1 -- 172.21.15.38:0/399819412 learned_addr learned my addr 172.21.15.38:0/399819412 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:22:52.443 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.157+0000 7f870e59c640 1 -- 172.21.15.38:0/399819412 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f871019bb60 con 0x7f87101064e0 2024-09-16T00:22:52.444 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.158+0000 7f870e59c640 1 --2- 172.21.15.38:0/399819412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87101064e0 0x7f871019b620 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f870002acc0 tx=0x7f8700031ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:52.444 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.158+0000 7f86ff7fe640 1 -- 172.21.15.38:0/399819412 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8700036050 con 0x7f87101064e0 2024-09-16T00:22:52.444 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.158+0000 7f86ff7fe640 1 -- 172.21.15.38:0/399819412 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f8700037040 con 0x7f87101064e0 2024-09-16T00:22:52.444 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.158+0000 7f871485a640 1 -- 172.21.15.38:0/399819412 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f871019bdf0 con 0x7f87101064e0 2024-09-16T00:22:52.444 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.158+0000 7f86ff7fe640 1 -- 172.21.15.38:0/399819412 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f870003d8c0 con 0x7f87101064e0 2024-09-16T00:22:52.444 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.158+0000 7f871485a640 1 -- 172.21.15.38:0/399819412 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f871019eae0 con 0x7f87101064e0 2024-09-16T00:22:52.444 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.159+0000 7f86ff7fe640 1 -- 172.21.15.38:0/399819412 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 4) ==== 46450+0+0 (secure 0 0 0) 0x7f8700044020 con 0x7f87101064e0 2024-09-16T00:22:52.444 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.159+0000 7f871485a640 1 -- 172.21.15.38:0/399819412 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f871010abc0 con 0x7f87101064e0 2024-09-16T00:22:52.444 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.159+0000 7f86ff7fe640 1 --2- 172.21.15.38:0/399819412 >> [v2:172.21.15.38:6800/583006373,v1:172.21.15.38:6801/583006373] conn(0x7f86f80391b0 0x7f86f803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:52.444 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.159+0000 7f86ff7fe640 1 -- 172.21.15.38:0/399819412 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f8700042020 con 0x7f87101064e0 2024-09-16T00:22:52.444 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.159+0000 7f870dd9b640 1 --2- 172.21.15.38:0/399819412 >> [v2:172.21.15.38:6800/583006373,v1:172.21.15.38:6801/583006373] conn(0x7f86f80391b0 0x7f86f803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:52.444 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.160+0000 7f870dd9b640 1 --2- 172.21.15.38:0/399819412 >> [v2:172.21.15.38:6800/583006373,v1:172.21.15.38:6801/583006373] conn(0x7f86f80391b0 0x7f86f803b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f87040098c0 tx=0x7f8704006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:52.444 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.164+0000 7f86ff7fe640 1 -- 172.21.15.38:0/399819412 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f870004b020 con 0x7f87101064e0 2024-09-16T00:22:52.445 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.388+0000 7f871485a640 1 -- 172.21.15.38:0/399819412 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f871002da30 con 0x7f87101064e0 2024-09-16T00:22:52.445 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.778+0000 7f86ff7fe640 1 -- 172.21.15.38:0/399819412 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f87000371f0 con 0x7f87101064e0 2024-09-16T00:22:52.445 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.780+0000 7f86ff7fe640 1 -- 172.21.15.38:0/399819412 <== mon.0 v2:172.21.15.38:3300/0 8 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f8700046790 con 0x7f87101064e0 2024-09-16T00:22:52.445 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.781+0000 7f871485a640 1 -- 172.21.15.38:0/399819412 >> [v2:172.21.15.38:6800/583006373,v1:172.21.15.38:6801/583006373] conn(0x7f86f80391b0 msgr2=0x7f86f803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:52.445 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.781+0000 7f871485a640 1 --2- 172.21.15.38:0/399819412 >> [v2:172.21.15.38:6800/583006373,v1:172.21.15.38:6801/583006373] conn(0x7f86f80391b0 0x7f86f803b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f87040098c0 tx=0x7f8704006db0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.445 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.781+0000 7f871485a640 1 -- 172.21.15.38:0/399819412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87101064e0 msgr2=0x7f871019b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:52.445 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.781+0000 7f871485a640 1 --2- 172.21.15.38:0/399819412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87101064e0 0x7f871019b620 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f870002acc0 tx=0x7f8700031ca0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.445 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.781+0000 7f871485a640 1 -- 172.21.15.38:0/399819412 shutdown_connections 2024-09-16T00:22:52.445 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.781+0000 7f871485a640 1 --2- 172.21.15.38:0/399819412 >> [v2:172.21.15.38:6800/583006373,v1:172.21.15.38:6801/583006373] conn(0x7f86f80391b0 0x7f86f803b670 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.446 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.781+0000 7f871485a640 1 --2- 172.21.15.38:0/399819412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87101064e0 0x7f871019b620 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:52.446 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.781+0000 7f871485a640 1 -- 172.21.15.38:0/399819412 >> 172.21.15.38:0/399819412 conn(0x7f8710101c30 msgr2=0x7f8710102bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:52.446 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.781+0000 7f871485a640 1 -- 172.21.15.38:0/399819412 shutdown_connections 2024-09-16T00:22:52.446 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:51.781+0000 7f871485a640 1 -- 172.21.15.38:0/399819412 wait complete. 2024-09-16T00:22:52.715 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:52 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:52.531+0000 7fc98e861040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T00:22:53.104 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:52 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/399819412' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-16T00:22:53.104 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:52 smithi038 ceph-mon[30945]: mgrmap e5: x(active, since 5s) 2024-09-16T00:22:53.105 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:52 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:52.865+0000 7fc98e861040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T00:22:53.105 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:52 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:52.931+0000 7fc98e861040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T00:22:53.422 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:53 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:53.126+0000 7fc98e861040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T00:22:53.730 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-09-16T00:22:53.731 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-16T00:22:53.731 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T00:22:53.731 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-16T00:22:53.731 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-16T00:22:53.731 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-09-16T00:22:53.731 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:52.897+0000 7f2c04fab640 1 Processor -- start 2024-09-16T00:22:53.731 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:52.897+0000 7f2c04fab640 1 -- start start 2024-09-16T00:22:53.731 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:52.897+0000 7f2c04fab640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c00108090 0x7f2c0010a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:53.731 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:52.897+0000 7f2c04fab640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0007be10 con 0x7f2c00108090 2024-09-16T00:22:53.731 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:52.897+0000 7f2bfe59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c00108090 0x7f2c0010a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:53.731 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:52.897+0000 7f2bfe59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c00108090 0x7f2c0010a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41590/0 (socket says 172.21.15.38:41590) 2024-09-16T00:22:53.731 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:52.897+0000 7f2bfe59c640 1 -- 172.21.15.38:0/3383436734 learned_addr learned my addr 172.21.15.38:0/3383436734 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:22:53.732 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:52.898+0000 7f2bfe59c640 1 -- 172.21.15.38:0/3383436734 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c0010a9c0 con 0x7f2c00108090 2024-09-16T00:22:53.732 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:52.898+0000 7f2bfe59c640 1 --2- 172.21.15.38:0/3383436734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c00108090 0x7f2c0010a480 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0009820 tx=0x7f2bf002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b69c412e3e886b72 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:53.732 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:52.899+0000 7f2bfdd9b640 1 -- 172.21.15.38:0/3383436734 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2bf0037070 con 0x7f2c00108090 2024-09-16T00:22:53.732 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:52.899+0000 7f2bfdd9b640 1 -- 172.21.15.38:0/3383436734 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2bf0032900 con 0x7f2c00108090 2024-09-16T00:22:53.732 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:52.899+0000 7f2bfdd9b640 1 -- 172.21.15.38:0/3383436734 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2bf0030850 con 0x7f2c00108090 2024-09-16T00:22:53.732 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:52.899+0000 7f2c04fab640 1 -- 172.21.15.38:0/3383436734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c00108090 msgr2=0x7f2c0010a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:53.732 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:52.899+0000 7f2c04fab640 1 --2- 172.21.15.38:0/3383436734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c00108090 0x7f2c0010a480 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2bf0009820 tx=0x7f2bf002a270 comp rx=0 tx=0).stop 2024-09-16T00:22:53.732 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:52.899+0000 7f2c04fab640 1 -- 172.21.15.38:0/3383436734 shutdown_connections 2024-09-16T00:22:53.732 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:52.899+0000 7f2c04fab640 1 --2- 172.21.15.38:0/3383436734 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c00108090 0x7f2c0010a480 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:53.732 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:52.899+0000 7f2c04fab640 1 -- 172.21.15.38:0/3383436734 >> 172.21.15.38:0/3383436734 conn(0x7f2c00101c30 msgr2=0x7f2c001040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:53.732 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:52.900+0000 7f2c04fab640 1 -- 172.21.15.38:0/3383436734 shutdown_connections 2024-09-16T00:22:53.732 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:52.900+0000 7f2c04fab640 1 -- 172.21.15.38:0/3383436734 wait complete. 2024-09-16T00:22:53.732 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:52.900+0000 7f2c04fab640 1 Processor -- start 2024-09-16T00:22:53.733 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:52.900+0000 7f2c04fab640 1 -- start start 2024-09-16T00:22:53.733 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:52.901+0000 7f2c04fab640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c00108090 0x7f2c0019d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:53.733 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:52.901+0000 7f2c04fab640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2c0010b9a0 con 0x7f2c00108090 2024-09-16T00:22:53.733 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:52.901+0000 7f2bfe59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c00108090 0x7f2c0019d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:53.733 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:52.901+0000 7f2bfe59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c00108090 0x7f2c0019d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41604/0 (socket says 172.21.15.38:41604) 2024-09-16T00:22:53.733 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:52.901+0000 7f2bfe59c640 1 -- 172.21.15.38:0/400508077 learned_addr learned my addr 172.21.15.38:0/400508077 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:22:53.733 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:52.901+0000 7f2bfe59c640 1 -- 172.21.15.38:0/400508077 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c0019dcf0 con 0x7f2c00108090 2024-09-16T00:22:53.733 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:52.901+0000 7f2bfe59c640 1 --2- 172.21.15.38:0/400508077 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c00108090 0x7f2c0019d7b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2bf00097f0 tx=0x7f2bf0030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:53.733 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:52.901+0000 7f2bf77fe640 1 -- 172.21.15.38:0/400508077 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2bf0037040 con 0x7f2c00108090 2024-09-16T00:22:53.733 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:52.901+0000 7f2bf77fe640 1 -- 172.21.15.38:0/400508077 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2bf0040070 con 0x7f2c00108090 2024-09-16T00:22:53.733 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:52.901+0000 7f2c04fab640 1 -- 172.21.15.38:0/400508077 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c0019df80 con 0x7f2c00108090 2024-09-16T00:22:53.733 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:52.901+0000 7f2bf77fe640 1 -- 172.21.15.38:0/400508077 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2bf003a390 con 0x7f2c00108090 2024-09-16T00:22:53.733 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:52.901+0000 7f2c04fab640 1 -- 172.21.15.38:0/400508077 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c001a0c70 con 0x7f2c00108090 2024-09-16T00:22:53.733 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:52.902+0000 7f2bf77fe640 1 -- 172.21.15.38:0/400508077 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7f2bf003a5e0 con 0x7f2c00108090 2024-09-16T00:22:53.734 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:52.902+0000 7f2c04fab640 1 -- 172.21.15.38:0/400508077 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c001084f0 con 0x7f2c00108090 2024-09-16T00:22:53.734 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:52.903+0000 7f2bf77fe640 1 --2- 172.21.15.38:0/400508077 >> [v2:172.21.15.38:6800/583006373,v1:172.21.15.38:6801/583006373] conn(0x7f2bec039200 0x7f2bec03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:53.734 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:52.903+0000 7f2bf77fe640 1 -- 172.21.15.38:0/400508077 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f2bf006c440 con 0x7f2c00108090 2024-09-16T00:22:53.734 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:52.903+0000 7f2bf7fff640 1 -- 172.21.15.38:0/400508077 >> [v2:172.21.15.38:6800/583006373,v1:172.21.15.38:6801/583006373] conn(0x7f2bec039200 msgr2=0x7f2bec03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/583006373 2024-09-16T00:22:53.734 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:52.903+0000 7f2bf7fff640 1 --2- 172.21.15.38:0/400508077 >> [v2:172.21.15.38:6800/583006373,v1:172.21.15.38:6801/583006373] conn(0x7f2bec039200 0x7f2bec03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T00:22:53.734 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:52.907+0000 7f2bf77fe640 1 -- 172.21.15.38:0/400508077 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2bf006c1a0 con 0x7f2c00108090 2024-09-16T00:22:53.734 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:53.104+0000 7f2bf7fff640 1 -- 172.21.15.38:0/400508077 >> [v2:172.21.15.38:6800/583006373,v1:172.21.15.38:6801/583006373] conn(0x7f2bec039200 msgr2=0x7f2bec03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/583006373 2024-09-16T00:22:53.734 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:53.104+0000 7f2bf7fff640 1 --2- 172.21.15.38:0/400508077 >> [v2:172.21.15.38:6800/583006373,v1:172.21.15.38:6801/583006373] conn(0x7f2bec039200 0x7f2bec03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T00:22:53.734 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:53.129+0000 7f2c04fab640 1 -- 172.21.15.38:0/400508077 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f2c001a1360 con 0x7f2c00108090 2024-09-16T00:22:53.734 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:53.129+0000 7f2bf77fe640 1 -- 172.21.15.38:0/400508077 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7f2bf0043760 con 0x7f2c00108090 2024-09-16T00:22:53.734 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:53.131+0000 7f2c04fab640 1 -- 172.21.15.38:0/400508077 >> [v2:172.21.15.38:6800/583006373,v1:172.21.15.38:6801/583006373] conn(0x7f2bec039200 msgr2=0x7f2bec03b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:22:53.734 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:53.131+0000 7f2c04fab640 1 --2- 172.21.15.38:0/400508077 >> [v2:172.21.15.38:6800/583006373,v1:172.21.15.38:6801/583006373] conn(0x7f2bec039200 0x7f2bec03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:53.734 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:53.131+0000 7f2c04fab640 1 -- 172.21.15.38:0/400508077 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c00108090 msgr2=0x7f2c0019d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:53.735 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:53.131+0000 7f2c04fab640 1 --2- 172.21.15.38:0/400508077 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c00108090 0x7f2c0019d7b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f2bf00097f0 tx=0x7f2bf0030d80 comp rx=0 tx=0).stop 2024-09-16T00:22:53.735 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:53.131+0000 7f2c04fab640 1 -- 172.21.15.38:0/400508077 shutdown_connections 2024-09-16T00:22:53.735 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:53.131+0000 7f2c04fab640 1 --2- 172.21.15.38:0/400508077 >> [v2:172.21.15.38:6800/583006373,v1:172.21.15.38:6801/583006373] conn(0x7f2bec039200 0x7f2bec03b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:53.735 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:53.131+0000 7f2c04fab640 1 --2- 172.21.15.38:0/400508077 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2c00108090 0x7f2c0019d7b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:53.735 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:53.131+0000 7f2c04fab640 1 -- 172.21.15.38:0/400508077 >> 172.21.15.38:0/400508077 conn(0x7f2c00101c30 msgr2=0x7f2c001027a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:53.735 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:53.132+0000 7f2c04fab640 1 -- 172.21.15.38:0/400508077 shutdown_connections 2024-09-16T00:22:53.735 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:53.132+0000 7f2c04fab640 1 -- 172.21.15.38:0/400508077 wait complete. 2024-09-16T00:22:53.735 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for the mgr to restart... 2024-09-16T00:22:53.735 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for mgr epoch 5... 2024-09-16T00:22:53.975 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:53 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/400508077' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-16T00:22:54.354 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:54 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:54.061+0000 7fc98e861040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T00:22:54.355 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:54 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:54.133+0000 7fc98e861040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T00:22:55.151 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:54 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:54.880+0000 7fc98e861040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T00:22:55.151 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:55 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:55.001+0000 7fc98e861040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T00:22:55.441 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:55 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:55.150+0000 7fc98e861040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:22:55.441 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:55 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:55.214+0000 7fc98e861040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T00:22:55.441 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:55 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:55.370+0000 7fc98e861040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T00:22:55.441 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:55 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:55.440+0000 7fc98e861040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T00:22:55.715 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:55 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:55.503+0000 7fc98e861040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T00:22:55.715 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:55 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:55.669+0000 7fc98e861040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T00:22:56.016 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:55 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:55.737+0000 7fc98e861040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T00:22:56.016 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:55 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:55.806+0000 7fc98e861040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T00:22:56.017 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:55 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:55.892+0000 7fc98e861040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T00:22:56.017 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:55 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:55.954+0000 7fc98e861040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T00:22:56.017 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:56 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:56.016+0000 7fc98e861040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T00:22:57.215 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:22:56 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:22:56.842+0000 7fc98e861040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T00:22:57.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:56 smithi038 ceph-mon[30945]: Active manager daemon x restarted 2024-09-16T00:22:57.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:56 smithi038 ceph-mon[30945]: Activating manager daemon x 2024-09-16T00:22:57.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:56 smithi038 ceph-mon[30945]: osdmap e2: 0 total, 0 up, 0 in 2024-09-16T00:22:57.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:56 smithi038 ceph-mon[30945]: mgrmap e6: x(active, starting, since 0.00633339s) 2024-09-16T00:22:57.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:56 smithi038 ceph-mon[30945]: from='mgr.14118 172.21.15.38:0/3698772563' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:22:57.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:56 smithi038 ceph-mon[30945]: from='mgr.14118 172.21.15.38:0/3698772563' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T00:22:57.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:56 smithi038 ceph-mon[30945]: from='mgr.14118 172.21.15.38:0/3698772563' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T00:22:57.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:56 smithi038 ceph-mon[30945]: from='mgr.14118 172.21.15.38:0/3698772563' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T00:22:57.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:56 smithi038 ceph-mon[30945]: from='mgr.14118 172.21.15.38:0/3698772563' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T00:22:57.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:56 smithi038 ceph-mon[30945]: Manager daemon x is now available 2024-09-16T00:22:57.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:56 smithi038 ceph-mon[30945]: from='mgr.14118 172.21.15.38:0/3698772563' entity='mgr.x' 2024-09-16T00:22:57.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:56 smithi038 ceph-mon[30945]: from='mgr.14118 172.21.15.38:0/3698772563' entity='mgr.x' 2024-09-16T00:22:57.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:56 smithi038 ceph-mon[30945]: from='mgr.14118 172.21.15.38:0/3698772563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:22:57.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:56 smithi038 ceph-mon[30945]: from='mgr.14118 172.21.15.38:0/3698772563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:22:57.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:56 smithi038 ceph-mon[30945]: from='mgr.14118 172.21.15.38:0/3698772563' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-16T00:22:57.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:56 smithi038 ceph-mon[30945]: from='mgr.14118 172.21.15.38:0/3698772563' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-16T00:22:58.153 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:57 smithi038 ceph-mon[30945]: Found migration_current of "None". Setting to last migration. 2024-09-16T00:22:58.154 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:57 smithi038 ceph-mon[30945]: mgrmap e7: x(active, since 1.00857s) 2024-09-16T00:22:58.421 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-09-16T00:22:58.421 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-16T00:22:58.421 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-16T00:22:58.421 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-09-16T00:22:58.421 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.147+0000 7fd4b0f80640 1 Processor -- start 2024-09-16T00:22:58.421 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.147+0000 7fd4b0f80640 1 -- start start 2024-09-16T00:22:58.421 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.147+0000 7fd4b0f80640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4ac108130 0x7fd4ac10a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:58.421 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.147+0000 7fd4b0f80640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ac07bdc0 con 0x7fd4ac108130 2024-09-16T00:22:58.422 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.148+0000 7fd4aa59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4ac108130 0x7fd4ac10a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:58.422 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.148+0000 7fd4aa59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4ac108130 0x7fd4ac10a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41618/0 (socket says 172.21.15.38:41618) 2024-09-16T00:22:58.422 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.148+0000 7fd4aa59c640 1 -- 172.21.15.38:0/3249611607 learned_addr learned my addr 172.21.15.38:0/3249611607 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:22:58.422 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.148+0000 7fd4aa59c640 1 -- 172.21.15.38:0/3249611607 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4ac10aa60 con 0x7fd4ac108130 2024-09-16T00:22:58.422 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.148+0000 7fd4aa59c640 1 --2- 172.21.15.38:0/3249611607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4ac108130 0x7fd4ac10a520 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd49c009820 tx=0x7fd49c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b638c339bbf12e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:58.422 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.149+0000 7fd4a9d9b640 1 -- 172.21.15.38:0/3249611607 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd49c037070 con 0x7fd4ac108130 2024-09-16T00:22:58.422 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.149+0000 7fd4a9d9b640 1 -- 172.21.15.38:0/3249611607 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd49c032900 con 0x7fd4ac108130 2024-09-16T00:22:58.422 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.149+0000 7fd4a9d9b640 1 -- 172.21.15.38:0/3249611607 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd49c030850 con 0x7fd4ac108130 2024-09-16T00:22:58.422 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.149+0000 7fd4b0f80640 1 -- 172.21.15.38:0/3249611607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4ac108130 msgr2=0x7fd4ac10a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:58.422 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.149+0000 7fd4b0f80640 1 --2- 172.21.15.38:0/3249611607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4ac108130 0x7fd4ac10a520 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd49c009820 tx=0x7fd49c02a270 comp rx=0 tx=0).stop 2024-09-16T00:22:58.422 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.149+0000 7fd4b0f80640 1 -- 172.21.15.38:0/3249611607 shutdown_connections 2024-09-16T00:22:58.422 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.149+0000 7fd4b0f80640 1 --2- 172.21.15.38:0/3249611607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4ac108130 0x7fd4ac10a520 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.423 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.149+0000 7fd4b0f80640 1 -- 172.21.15.38:0/3249611607 >> 172.21.15.38:0/3249611607 conn(0x7fd4ac101c30 msgr2=0x7fd4ac1040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:58.423 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.149+0000 7fd4b0f80640 1 -- 172.21.15.38:0/3249611607 shutdown_connections 2024-09-16T00:22:58.423 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.149+0000 7fd4b0f80640 1 -- 172.21.15.38:0/3249611607 wait complete. 2024-09-16T00:22:58.423 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.150+0000 7fd4b0f80640 1 Processor -- start 2024-09-16T00:22:58.423 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.150+0000 7fd4b0f80640 1 -- start start 2024-09-16T00:22:58.423 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.150+0000 7fd4b0f80640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4ac108130 0x7fd4ac19b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:58.423 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.150+0000 7fd4b0f80640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd4ac10b9e0 con 0x7fd4ac108130 2024-09-16T00:22:58.423 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.151+0000 7fd4aa59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4ac108130 0x7fd4ac19b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:58.423 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.151+0000 7fd4aa59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4ac108130 0x7fd4ac19b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41626/0 (socket says 172.21.15.38:41626) 2024-09-16T00:22:58.423 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.151+0000 7fd4aa59c640 1 -- 172.21.15.38:0/57375269 learned_addr learned my addr 172.21.15.38:0/57375269 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:22:58.423 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.151+0000 7fd4aa59c640 1 -- 172.21.15.38:0/57375269 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4ac19bb70 con 0x7fd4ac108130 2024-09-16T00:22:58.423 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.151+0000 7fd4aa59c640 1 --2- 172.21.15.38:0/57375269 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4ac108130 0x7fd4ac19b630 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd49c031000 tx=0x7fd49c0326e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:58.423 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.151+0000 7fd4a3fff640 1 -- 172.21.15.38:0/57375269 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd49c042070 con 0x7fd4ac108130 2024-09-16T00:22:58.423 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.151+0000 7fd4b0f80640 1 -- 172.21.15.38:0/57375269 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4ac19be00 con 0x7fd4ac108130 2024-09-16T00:22:58.424 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.151+0000 7fd4a3fff640 1 -- 172.21.15.38:0/57375269 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fd49c032be0 con 0x7fd4ac108130 2024-09-16T00:22:58.424 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.151+0000 7fd4a3fff640 1 -- 172.21.15.38:0/57375269 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd49c037040 con 0x7fd4ac108130 2024-09-16T00:22:58.424 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.151+0000 7fd4b0f80640 1 -- 172.21.15.38:0/57375269 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4ac19eaf0 con 0x7fd4ac108130 2024-09-16T00:22:58.424 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.152+0000 7fd4a3fff640 1 -- 172.21.15.38:0/57375269 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 5) ==== 46461+0+0 (secure 0 0 0) 0x7fd49c045070 con 0x7fd4ac108130 2024-09-16T00:22:58.424 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.153+0000 7fd4a3fff640 1 --2- 172.21.15.38:0/57375269 >> [v2:172.21.15.38:6800/583006373,v1:172.21.15.38:6801/583006373] conn(0x7fd4980391b0 0x7fd49803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:58.424 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.153+0000 7fd4a3fff640 1 -- 172.21.15.38:0/57375269 --> [v2:172.21.15.38:6800/583006373,v1:172.21.15.38:6801/583006373] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd49803bd80 con 0x7fd4980391b0 2024-09-16T00:22:58.424 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.153+0000 7fd4a3fff640 1 -- 172.21.15.38:0/57375269 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd49c06c1f0 con 0x7fd4ac108130 2024-09-16T00:22:58.424 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.153+0000 7fd4a1d9b640 1 -- 172.21.15.38:0/57375269 >> [v2:172.21.15.38:6800/583006373,v1:172.21.15.38:6801/583006373] conn(0x7fd4980391b0 msgr2=0x7fd49803b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/583006373 2024-09-16T00:22:58.424 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.153+0000 7fd4a1d9b640 1 --2- 172.21.15.38:0/57375269 >> [v2:172.21.15.38:6800/583006373,v1:172.21.15.38:6801/583006373] conn(0x7fd4980391b0 0x7fd49803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T00:22:58.424 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.354+0000 7fd4a1d9b640 1 -- 172.21.15.38:0/57375269 >> [v2:172.21.15.38:6800/583006373,v1:172.21.15.38:6801/583006373] conn(0x7fd4980391b0 msgr2=0x7fd49803b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/583006373 2024-09-16T00:22:58.424 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.354+0000 7fd4a1d9b640 1 --2- 172.21.15.38:0/57375269 >> [v2:172.21.15.38:6800/583006373,v1:172.21.15.38:6801/583006373] conn(0x7fd4980391b0 0x7fd49803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T00:22:58.424 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.755+0000 7fd4a1d9b640 1 -- 172.21.15.38:0/57375269 >> [v2:172.21.15.38:6800/583006373,v1:172.21.15.38:6801/583006373] conn(0x7fd4980391b0 msgr2=0x7fd49803b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/583006373 2024-09-16T00:22:58.425 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:54.755+0000 7fd4a1d9b640 1 --2- 172.21.15.38:0/57375269 >> [v2:172.21.15.38:6800/583006373,v1:172.21.15.38:6801/583006373] conn(0x7fd4980391b0 0x7fd49803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T00:22:58.425 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:55.556+0000 7fd4a1d9b640 1 -- 172.21.15.38:0/57375269 >> [v2:172.21.15.38:6800/583006373,v1:172.21.15.38:6801/583006373] conn(0x7fd4980391b0 msgr2=0x7fd49803b670 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/583006373 2024-09-16T00:22:58.425 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:55.556+0000 7fd4a1d9b640 1 --2- 172.21.15.38:0/57375269 >> [v2:172.21.15.38:6800/583006373,v1:172.21.15.38:6801/583006373] conn(0x7fd4980391b0 0x7fd49803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T00:22:58.425 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:56.848+0000 7fd4a3fff640 1 -- 172.21.15.38:0/57375269 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mgrmap(e 6) ==== 46228+0+0 (secure 0 0 0) 0x7fd49c06ad70 con 0x7fd4ac108130 2024-09-16T00:22:58.425 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:56.848+0000 7fd4a3fff640 1 -- 172.21.15.38:0/57375269 >> [v2:172.21.15.38:6800/583006373,v1:172.21.15.38:6801/583006373] conn(0x7fd4980391b0 msgr2=0x7fd49803b670 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:22:58.425 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:56.848+0000 7fd4a3fff640 1 --2- 172.21.15.38:0/57375269 >> [v2:172.21.15.38:6800/583006373,v1:172.21.15.38:6801/583006373] conn(0x7fd4980391b0 0x7fd49803b670 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.425 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:57.849+0000 7fd4a3fff640 1 -- 172.21.15.38:0/57375269 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7fd49c02ed80 con 0x7fd4ac108130 2024-09-16T00:22:58.425 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:57.849+0000 7fd4a3fff640 1 --2- 172.21.15.38:0/57375269 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7fd49803cae0 0x7fd49803eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:58.425 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:57.849+0000 7fd4a3fff640 1 -- 172.21.15.38:0/57375269 --> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fd49c001d30 con 0x7fd49803cae0 2024-09-16T00:22:58.425 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:57.849+0000 7fd4a1d9b640 1 --2- 172.21.15.38:0/57375269 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7fd49803cae0 0x7fd49803eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:58.425 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:57.850+0000 7fd4a1d9b640 1 --2- 172.21.15.38:0/57375269 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7fd49803cae0 0x7fd49803eed0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd494003980 tx=0x7fd494006e10 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:58.425 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:57.851+0000 7fd4a3fff640 1 -- 172.21.15.38:0/57375269 <== mgr.14118 v2:172.21.15.38:6800/1779324916 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7fd49c001d30 con 0x7fd49803cae0 2024-09-16T00:22:58.425 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:57.858+0000 7fd4b0f80640 1 -- 172.21.15.38:0/57375269 --> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7fd4ac108590 con 0x7fd49803cae0 2024-09-16T00:22:58.426 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:57.858+0000 7fd4a3fff640 1 -- 172.21.15.38:0/57375269 <== mgr.14118 v2:172.21.15.38:6800/1779324916 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7fd4ac108590 con 0x7fd49803cae0 2024-09-16T00:22:58.426 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:57.858+0000 7fd4b0f80640 1 -- 172.21.15.38:0/57375269 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7fd49803cae0 msgr2=0x7fd49803eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:58.426 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:57.858+0000 7fd4b0f80640 1 --2- 172.21.15.38:0/57375269 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7fd49803cae0 0x7fd49803eed0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd494003980 tx=0x7fd494006e10 comp rx=0 tx=0).stop 2024-09-16T00:22:58.426 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:57.858+0000 7fd4b0f80640 1 -- 172.21.15.38:0/57375269 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4ac108130 msgr2=0x7fd4ac19b630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:58.426 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:57.858+0000 7fd4b0f80640 1 --2- 172.21.15.38:0/57375269 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4ac108130 0x7fd4ac19b630 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fd49c031000 tx=0x7fd49c0326e0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.426 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:57.858+0000 7fd4b0f80640 1 -- 172.21.15.38:0/57375269 shutdown_connections 2024-09-16T00:22:58.426 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:57.858+0000 7fd4b0f80640 1 --2- 172.21.15.38:0/57375269 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7fd49803cae0 0x7fd49803eed0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.426 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:57.858+0000 7fd4b0f80640 1 --2- 172.21.15.38:0/57375269 >> [v2:172.21.15.38:6800/583006373,v1:172.21.15.38:6801/583006373] conn(0x7fd4980391b0 0x7fd49803b670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.426 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:57.859+0000 7fd4b0f80640 1 --2- 172.21.15.38:0/57375269 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd4ac108130 0x7fd4ac19b630 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:58.426 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:57.859+0000 7fd4b0f80640 1 -- 172.21.15.38:0/57375269 >> 172.21.15.38:0/57375269 conn(0x7fd4ac101c30 msgr2=0x7fd4ac1028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:58.426 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:57.859+0000 7fd4b0f80640 1 -- 172.21.15.38:0/57375269 shutdown_connections 2024-09-16T00:22:58.426 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:57.859+0000 7fd4b0f80640 1 -- 172.21.15.38:0/57375269 wait complete. 2024-09-16T00:22:58.426 INFO:teuthology.orchestra.run.smithi038.stdout:mgr epoch 5 is available 2024-09-16T00:22:58.426 INFO:teuthology.orchestra.run.smithi038.stdout:Setting orchestrator backend to cephadm... 2024-09-16T00:22:59.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:59 smithi038 ceph-mon[30945]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-16T00:22:59.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:59 smithi038 ceph-mon[30945]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-16T00:22:59.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:59 smithi038 ceph-mon[30945]: from='mgr.14118 172.21.15.38:0/3698772563' entity='mgr.x' 2024-09-16T00:22:59.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:59 smithi038 ceph-mon[30945]: from='mgr.14118 172.21.15.38:0/3698772563' entity='mgr.x' 2024-09-16T00:22:59.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:59 smithi038 ceph-mon[30945]: from='mgr.14118 172.21.15.38:0/3698772563' entity='mgr.x' 2024-09-16T00:22:59.644 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:22:59 smithi038 ceph-mon[30945]: from='mgr.14118 172.21.15.38:0/3698772563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:22:59.646 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:58.819+0000 7f2f8a722640 1 Processor -- start 2024-09-16T00:22:59.652 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:58.819+0000 7f2f8a722640 1 -- start start 2024-09-16T00:22:59.652 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:58.819+0000 7f2f8a722640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f841080b0 0x7f2f8410a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:59.652 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:58.819+0000 7f2f8a722640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8407bdf0 con 0x7f2f841080b0 2024-09-16T00:22:59.652 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:58.819+0000 7f2f83fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f841080b0 0x7f2f8410a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:59.652 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:58.819+0000 7f2f83fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f841080b0 0x7f2f8410a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34334/0 (socket says 172.21.15.38:34334) 2024-09-16T00:22:59.652 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:58.819+0000 7f2f83fff640 1 -- 172.21.15.38:0/2952946537 learned_addr learned my addr 172.21.15.38:0/2952946537 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:22:59.652 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:58.820+0000 7f2f83fff640 1 -- 172.21.15.38:0/2952946537 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f8410a9e0 con 0x7f2f841080b0 2024-09-16T00:22:59.652 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:58.820+0000 7f2f83fff640 1 --2- 172.21.15.38:0/2952946537 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f841080b0 0x7f2f8410a4a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2f74009820 tx=0x7f2f7402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c060d4e32aa0f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:59.652 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:58.820+0000 7f2f82ffd640 1 -- 172.21.15.38:0/2952946537 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f74037070 con 0x7f2f841080b0 2024-09-16T00:22:59.653 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:58.820+0000 7f2f82ffd640 1 -- 172.21.15.38:0/2952946537 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2f74032900 con 0x7f2f841080b0 2024-09-16T00:22:59.653 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:58.820+0000 7f2f82ffd640 1 -- 172.21.15.38:0/2952946537 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f74030850 con 0x7f2f841080b0 2024-09-16T00:22:59.653 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:58.821+0000 7f2f8a722640 1 -- 172.21.15.38:0/2952946537 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f841080b0 msgr2=0x7f2f8410a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:59.653 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:58.821+0000 7f2f8a722640 1 --2- 172.21.15.38:0/2952946537 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f841080b0 0x7f2f8410a4a0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f2f74009820 tx=0x7f2f7402a270 comp rx=0 tx=0).stop 2024-09-16T00:22:59.653 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:58.821+0000 7f2f8a722640 1 -- 172.21.15.38:0/2952946537 shutdown_connections 2024-09-16T00:22:59.653 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:58.821+0000 7f2f8a722640 1 --2- 172.21.15.38:0/2952946537 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f841080b0 0x7f2f8410a4a0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:59.653 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:58.821+0000 7f2f8a722640 1 -- 172.21.15.38:0/2952946537 >> 172.21.15.38:0/2952946537 conn(0x7f2f84101c30 msgr2=0x7f2f841040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:59.653 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:58.821+0000 7f2f8a722640 1 -- 172.21.15.38:0/2952946537 shutdown_connections 2024-09-16T00:22:59.653 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:58.821+0000 7f2f8a722640 1 -- 172.21.15.38:0/2952946537 wait complete. 2024-09-16T00:22:59.653 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:58.822+0000 7f2f8a722640 1 Processor -- start 2024-09-16T00:22:59.653 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:58.822+0000 7f2f8a722640 1 -- start start 2024-09-16T00:22:59.653 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:58.822+0000 7f2f8a722640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f841080b0 0x7f2f84107930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:59.654 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:58.822+0000 7f2f8a722640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2f8410b9c0 con 0x7f2f841080b0 2024-09-16T00:22:59.654 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:58.822+0000 7f2f83fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f841080b0 0x7f2f84107930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:59.654 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:58.822+0000 7f2f83fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f841080b0 0x7f2f84107930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34350/0 (socket says 172.21.15.38:34350) 2024-09-16T00:22:59.654 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:58.822+0000 7f2f83fff640 1 -- 172.21.15.38:0/3040084311 learned_addr learned my addr 172.21.15.38:0/3040084311 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:22:59.654 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:58.822+0000 7f2f83fff640 1 -- 172.21.15.38:0/3040084311 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f84107e70 con 0x7f2f841080b0 2024-09-16T00:22:59.654 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:58.823+0000 7f2f83fff640 1 --2- 172.21.15.38:0/3040084311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f841080b0 0x7f2f84107930 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2f740097f0 tx=0x7f2f74030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:59.654 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:58.823+0000 7f2f817fa640 1 -- 172.21.15.38:0/3040084311 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f74037040 con 0x7f2f841080b0 2024-09-16T00:22:59.654 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:58.823+0000 7f2f8a722640 1 -- 172.21.15.38:0/3040084311 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f84106060 con 0x7f2f841080b0 2024-09-16T00:22:59.654 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:58.823+0000 7f2f817fa640 1 -- 172.21.15.38:0/3040084311 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f2f74040070 con 0x7f2f841080b0 2024-09-16T00:22:59.654 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:58.823+0000 7f2f817fa640 1 -- 172.21.15.38:0/3040084311 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f7403a390 con 0x7f2f841080b0 2024-09-16T00:22:59.654 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:58.823+0000 7f2f8a722640 1 -- 172.21.15.38:0/3040084311 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f84106540 con 0x7f2f841080b0 2024-09-16T00:22:59.654 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:58.824+0000 7f2f817fa640 1 -- 172.21.15.38:0/3040084311 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 7) ==== 46355+0+0 (secure 0 0 0) 0x7f2f7403a630 con 0x7f2f841080b0 2024-09-16T00:22:59.654 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:58.824+0000 7f2f817fa640 1 --2- 172.21.15.38:0/3040084311 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f2f700390a0 0x7f2f7003b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:22:59.655 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:58.824+0000 7f2f817fa640 1 -- 172.21.15.38:0/3040084311 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2f7406c600 con 0x7f2f841080b0 2024-09-16T00:22:59.655 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:58.824+0000 7f2f8a722640 1 -- 172.21.15.38:0/3040084311 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f84108510 con 0x7f2f841080b0 2024-09-16T00:22:59.655 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:58.824+0000 7f2f837fe640 1 --2- 172.21.15.38:0/3040084311 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f2f700390a0 0x7f2f7003b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:22:59.655 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:58.825+0000 7f2f837fe640 1 --2- 172.21.15.38:0/3040084311 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f2f700390a0 0x7f2f7003b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c0098c0 tx=0x7f2f6c006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:22:59.655 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:58.833+0000 7f2f817fa640 1 -- 172.21.15.38:0/3040084311 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2f7404a200 con 0x7f2f841080b0 2024-09-16T00:22:59.655 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:59.021+0000 7f2f8a722640 1 -- 172.21.15.38:0/3040084311 --> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f2f84106eb0 con 0x7f2f700390a0 2024-09-16T00:22:59.655 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:59.029+0000 7f2f817fa640 1 -- 172.21.15.38:0/3040084311 <== mgr.14118 v2:172.21.15.38:6800/1779324916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2f84106eb0 con 0x7f2f700390a0 2024-09-16T00:22:59.655 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:59.031+0000 7f2f8a722640 1 -- 172.21.15.38:0/3040084311 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f2f700390a0 msgr2=0x7f2f7003b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:59.655 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:59.031+0000 7f2f8a722640 1 --2- 172.21.15.38:0/3040084311 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f2f700390a0 0x7f2f7003b560 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c0098c0 tx=0x7f2f6c006db0 comp rx=0 tx=0).stop 2024-09-16T00:22:59.655 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:59.031+0000 7f2f8a722640 1 -- 172.21.15.38:0/3040084311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f841080b0 msgr2=0x7f2f84107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:22:59.655 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:59.031+0000 7f2f8a722640 1 --2- 172.21.15.38:0/3040084311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f841080b0 0x7f2f84107930 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2f740097f0 tx=0x7f2f74030d80 comp rx=0 tx=0).stop 2024-09-16T00:22:59.655 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:59.031+0000 7f2f8a722640 1 -- 172.21.15.38:0/3040084311 shutdown_connections 2024-09-16T00:22:59.656 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:59.031+0000 7f2f8a722640 1 --2- 172.21.15.38:0/3040084311 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f2f700390a0 0x7f2f7003b560 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:59.656 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:59.031+0000 7f2f8a722640 1 --2- 172.21.15.38:0/3040084311 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2f841080b0 0x7f2f84107930 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:22:59.656 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:59.031+0000 7f2f8a722640 1 -- 172.21.15.38:0/3040084311 >> 172.21.15.38:0/3040084311 conn(0x7f2f84101c30 msgr2=0x7f2f841027e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:22:59.656 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:59.031+0000 7f2f8a722640 1 -- 172.21.15.38:0/3040084311 shutdown_connections 2024-09-16T00:22:59.656 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:22:59.031+0000 7f2f8a722640 1 -- 172.21.15.38:0/3040084311 wait complete. 2024-09-16T00:23:00.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:00 smithi038 ceph-mon[30945]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:23:00.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:00 smithi038 ceph-mon[30945]: mgrmap e8: x(active, since 2s) 2024-09-16T00:23:00.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:00 smithi038 ceph-mon[30945]: from='mgr.14118 172.21.15.38:0/3698772563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:23:00.465 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:00 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:23:00.209+0000 7fc8fb386640 -1 log_channel(cephadm) log [ERR] : [16/Sep/2024:00:23:00] ENGINE Error in HTTPServer.serve 2024-09-16T00:23:00.465 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:00 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: Traceback (most recent call last): 2024-09-16T00:23:00.465 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:00 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T00:23:00.466 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:00 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: self._connections.run(self.expiration_interval) 2024-09-16T00:23:00.466 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:00 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T00:23:00.466 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:00 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: self._run(expiration_interval) 2024-09-16T00:23:00.466 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:00 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T00:23:00.466 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:00 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T00:23:00.466 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:00 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T00:23:00.466 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:00 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T00:23:00.466 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:00 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T00:23:00.466 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:00 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: s = self.context.wrap_socket( 2024-09-16T00:23:00.466 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:00 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T00:23:00.466 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:00 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: return self.sslsocket_class._create( 2024-09-16T00:23:00.466 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:00 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T00:23:00.466 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:00 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: self.do_handshake() 2024-09-16T00:23:00.467 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:00 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T00:23:00.467 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:00 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: self._sslobj.do_handshake() 2024-09-16T00:23:00.467 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:00 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T00:23:00.467 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:00 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:23:00.884 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-16T00:23:00.885 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.078+0000 7f9714f2c640 1 Processor -- start 2024-09-16T00:23:00.885 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.078+0000 7f9714f2c640 1 -- start start 2024-09-16T00:23:00.885 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.079+0000 7f9714f2c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97101080b0 0x7f971010a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:00.885 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.079+0000 7f9714f2c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f971007be40 con 0x7f97101080b0 2024-09-16T00:23:00.885 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.079+0000 7f970e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97101080b0 0x7f971010a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:00.885 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.079+0000 7f970e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97101080b0 0x7f971010a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34358/0 (socket says 172.21.15.38:34358) 2024-09-16T00:23:00.885 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.079+0000 7f970e59c640 1 -- 172.21.15.38:0/2387925058 learned_addr learned my addr 172.21.15.38:0/2387925058 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:23:00.885 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.079+0000 7f970e59c640 1 -- 172.21.15.38:0/2387925058 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f971010a9e0 con 0x7f97101080b0 2024-09-16T00:23:00.885 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.080+0000 7f970e59c640 1 --2- 172.21.15.38:0/2387925058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97101080b0 0x7f971010a4a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9700009820 tx=0x7f970002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f742358e5a9143b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:00.885 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.080+0000 7f970d59a640 1 -- 172.21.15.38:0/2387925058 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9700037070 con 0x7f97101080b0 2024-09-16T00:23:00.885 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.080+0000 7f970d59a640 1 -- 172.21.15.38:0/2387925058 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9700032900 con 0x7f97101080b0 2024-09-16T00:23:00.885 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.080+0000 7f970d59a640 1 -- 172.21.15.38:0/2387925058 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9700030850 con 0x7f97101080b0 2024-09-16T00:23:00.885 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.081+0000 7f9714f2c640 1 -- 172.21.15.38:0/2387925058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97101080b0 msgr2=0x7f971010a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:00.886 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.081+0000 7f9714f2c640 1 --2- 172.21.15.38:0/2387925058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97101080b0 0x7f971010a4a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9700009820 tx=0x7f970002a270 comp rx=0 tx=0).stop 2024-09-16T00:23:00.886 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.081+0000 7f9714f2c640 1 -- 172.21.15.38:0/2387925058 shutdown_connections 2024-09-16T00:23:00.886 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.081+0000 7f9714f2c640 1 --2- 172.21.15.38:0/2387925058 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97101080b0 0x7f971010a4a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:00.886 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.081+0000 7f9714f2c640 1 -- 172.21.15.38:0/2387925058 >> 172.21.15.38:0/2387925058 conn(0x7f9710101c30 msgr2=0x7f97101040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:00.886 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.081+0000 7f9714f2c640 1 -- 172.21.15.38:0/2387925058 shutdown_connections 2024-09-16T00:23:00.886 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.081+0000 7f9714f2c640 1 -- 172.21.15.38:0/2387925058 wait complete. 2024-09-16T00:23:00.886 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.082+0000 7f9714f2c640 1 Processor -- start 2024-09-16T00:23:00.886 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.082+0000 7f9714f2c640 1 -- start start 2024-09-16T00:23:00.886 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.082+0000 7f9714f2c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97101080b0 0x7f9710107930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:00.886 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.082+0000 7f9714f2c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f971010ba20 con 0x7f97101080b0 2024-09-16T00:23:00.886 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.083+0000 7f970e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97101080b0 0x7f9710107930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:00.886 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.083+0000 7f970e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97101080b0 0x7f9710107930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34360/0 (socket says 172.21.15.38:34360) 2024-09-16T00:23:00.886 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.083+0000 7f970e59c640 1 -- 172.21.15.38:0/3378533407 learned_addr learned my addr 172.21.15.38:0/3378533407 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:23:00.887 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.083+0000 7f970e59c640 1 -- 172.21.15.38:0/3378533407 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9710107e70 con 0x7f97101080b0 2024-09-16T00:23:00.887 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.083+0000 7f970e59c640 1 --2- 172.21.15.38:0/3378533407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97101080b0 0x7f9710107930 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f97000097f0 tx=0x7f9700030ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:00.887 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.083+0000 7f96ff7fe640 1 -- 172.21.15.38:0/3378533407 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9700042070 con 0x7f97101080b0 2024-09-16T00:23:00.887 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.083+0000 7f9714f2c640 1 -- 172.21.15.38:0/3378533407 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9710106060 con 0x7f97101080b0 2024-09-16T00:23:00.887 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.083+0000 7f96ff7fe640 1 -- 172.21.15.38:0/3378533407 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f9700032ca0 con 0x7f97101080b0 2024-09-16T00:23:00.887 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.083+0000 7f9714f2c640 1 -- 172.21.15.38:0/3378533407 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97101065a0 con 0x7f97101080b0 2024-09-16T00:23:00.887 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.083+0000 7f96ff7fe640 1 -- 172.21.15.38:0/3378533407 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9700037040 con 0x7f97101080b0 2024-09-16T00:23:00.887 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.085+0000 7f96ff7fe640 1 -- 172.21.15.38:0/3378533407 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f9700045070 con 0x7f97101080b0 2024-09-16T00:23:00.887 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.085+0000 7f9714f2c640 1 -- 172.21.15.38:0/3378533407 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9710108510 con 0x7f97101080b0 2024-09-16T00:23:00.887 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.085+0000 7f96ff7fe640 1 --2- 172.21.15.38:0/3378533407 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f96f80391b0 0x7f96f803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:00.887 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.085+0000 7f96ff7fe640 1 -- 172.21.15.38:0/3378533407 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f970006c2c0 con 0x7f97101080b0 2024-09-16T00:23:00.887 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.085+0000 7f970dd9b640 1 --2- 172.21.15.38:0/3378533407 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f96f80391b0 0x7f96f803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:00.887 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.086+0000 7f970dd9b640 1 --2- 172.21.15.38:0/3378533407 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f96f80391b0 0x7f96f803b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f97040098c0 tx=0x7f9704006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:00.888 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.090+0000 7f96ff7fe640 1 -- 172.21.15.38:0/3378533407 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9700045360 con 0x7f97101080b0 2024-09-16T00:23:00.888 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.272+0000 7f9714f2c640 1 -- 172.21.15.38:0/3378533407 --> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f9710106e00 con 0x7f96f80391b0 2024-09-16T00:23:00.888 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.272+0000 7f96ff7fe640 1 -- 172.21.15.38:0/3378533407 <== mgr.14118 v2:172.21.15.38:6800/1779324916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f9710106e00 con 0x7f96f80391b0 2024-09-16T00:23:00.888 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.274+0000 7f9714f2c640 1 -- 172.21.15.38:0/3378533407 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f96f80391b0 msgr2=0x7f96f803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:00.888 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.274+0000 7f9714f2c640 1 --2- 172.21.15.38:0/3378533407 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f96f80391b0 0x7f96f803b670 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f97040098c0 tx=0x7f9704006db0 comp rx=0 tx=0).stop 2024-09-16T00:23:00.888 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.274+0000 7f9714f2c640 1 -- 172.21.15.38:0/3378533407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97101080b0 msgr2=0x7f9710107930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:00.888 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.274+0000 7f9714f2c640 1 --2- 172.21.15.38:0/3378533407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97101080b0 0x7f9710107930 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f97000097f0 tx=0x7f9700030ec0 comp rx=0 tx=0).stop 2024-09-16T00:23:00.888 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.274+0000 7f9714f2c640 1 -- 172.21.15.38:0/3378533407 shutdown_connections 2024-09-16T00:23:00.888 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.274+0000 7f9714f2c640 1 --2- 172.21.15.38:0/3378533407 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f96f80391b0 0x7f96f803b670 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:00.888 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.275+0000 7f9714f2c640 1 --2- 172.21.15.38:0/3378533407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f97101080b0 0x7f9710107930 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:00.888 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.275+0000 7f9714f2c640 1 -- 172.21.15.38:0/3378533407 >> 172.21.15.38:0/3378533407 conn(0x7f9710101c30 msgr2=0x7f9710102800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:00.888 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.275+0000 7f9714f2c640 1 -- 172.21.15.38:0/3378533407 shutdown_connections 2024-09-16T00:23:00.888 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:00.275+0000 7f9714f2c640 1 -- 172.21.15.38:0/3378533407 wait complete. 2024-09-16T00:23:00.889 INFO:teuthology.orchestra.run.smithi038.stdout:Generating ssh key... 2024-09-16T00:23:01.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:01 smithi038 ceph-mon[30945]: [16/Sep/2024:00:23:00] ENGINE Bus STARTING 2024-09-16T00:23:01.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:01 smithi038 ceph-mon[30945]: [16/Sep/2024:00:23:00] ENGINE Serving on https://172.21.15.38:7150 2024-09-16T00:23:01.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:01 smithi038 ceph-mon[30945]: [16/Sep/2024:00:23:00] ENGINE Error in HTTPServer.serve 2024-09-16T00:23:01.715 INFO:journalctl@ceph.mon.a.smithi038.stdout: Traceback (most recent call last): 2024-09-16T00:23:01.715 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T00:23:01.715 INFO:journalctl@ceph.mon.a.smithi038.stdout: self._connections.run(self.expiration_interval) 2024-09-16T00:23:01.715 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T00:23:01.715 INFO:journalctl@ceph.mon.a.smithi038.stdout: self._run(expiration_interval) 2024-09-16T00:23:01.715 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T00:23:01.715 INFO:journalctl@ceph.mon.a.smithi038.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T00:23:01.715 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T00:23:01.716 INFO:journalctl@ceph.mon.a.smithi038.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T00:23:01.716 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T00:23:01.716 INFO:journalctl@ceph.mon.a.smithi038.stdout: s = self.context.wrap_socket( 2024-09-16T00:23:01.716 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T00:23:01.716 INFO:journalctl@ceph.mon.a.smithi038.stdout: return self.sslsocket_class._create( 2024-09-16T00:23:01.716 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T00:23:01.716 INFO:journalctl@ceph.mon.a.smithi038.stdout: self.do_handshake() 2024-09-16T00:23:01.716 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T00:23:01.716 INFO:journalctl@ceph.mon.a.smithi038.stdout: self._sslobj.do_handshake() 2024-09-16T00:23:01.716 INFO:journalctl@ceph.mon.a.smithi038.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T00:23:01.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:01 smithi038 ceph-mon[30945]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:23:01.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:01 smithi038 ceph-mon[30945]: [16/Sep/2024:00:23:00] ENGINE Serving on http://172.21.15.38:8765 2024-09-16T00:23:01.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:01 smithi038 ceph-mon[30945]: [16/Sep/2024:00:23:00] ENGINE Bus STARTED 2024-09-16T00:23:02.125 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:01 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: Generating public/private rsa key pair. 2024-09-16T00:23:02.125 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:01 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: Your identification has been saved in /tmp/tmp9m1j55be/key 2024-09-16T00:23:02.125 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:01 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: Your public key has been saved in /tmp/tmp9m1j55be/key.pub 2024-09-16T00:23:02.126 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:01 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: The key fingerprint is: 2024-09-16T00:23:02.126 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:01 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: SHA256:qys+LIAY62ytWGhLem9r4T7LrbYA4F2UtAefJnSP5DQ ceph-983b2072-73c1-11ef-bceb-c7b262605968 2024-09-16T00:23:02.126 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:01 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: The key's randomart image is: 2024-09-16T00:23:02.126 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:01 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: +---[RSA 3072]----+ 2024-09-16T00:23:02.126 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:01 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: | .=.E | 2024-09-16T00:23:02.126 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:01 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: | o.O = | 2024-09-16T00:23:02.126 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:01 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: |. + B . | 2024-09-16T00:23:02.126 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:01 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: |+ . . + | 2024-09-16T00:23:02.126 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:01 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: |++ . S | 2024-09-16T00:23:02.126 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:01 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: |=o . . | 2024-09-16T00:23:02.126 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:01 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: |+++o . . | 2024-09-16T00:23:02.126 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:01 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: |==++X. . | 2024-09-16T00:23:02.126 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:01 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: |++.OXO+. | 2024-09-16T00:23:02.126 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:01 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: +----[SHA256]-----+ 2024-09-16T00:23:02.419 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.272+0000 7f38f4471640 1 Processor -- start 2024-09-16T00:23:02.419 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.272+0000 7f38f4471640 1 -- start start 2024-09-16T00:23:02.419 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.273+0000 7f38f4471640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38ec108050 0x7f38ec10a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:02.419 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.273+0000 7f38f4471640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec07be50 con 0x7f38ec108050 2024-09-16T00:23:02.419 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.273+0000 7f38f220d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38ec108050 0x7f38ec10a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:02.419 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.273+0000 7f38f220d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38ec108050 0x7f38ec10a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34372/0 (socket says 172.21.15.38:34372) 2024-09-16T00:23:02.419 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.273+0000 7f38f220d640 1 -- 172.21.15.38:0/1175783578 learned_addr learned my addr 172.21.15.38:0/1175783578 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:23:02.419 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.273+0000 7f38f220d640 1 -- 172.21.15.38:0/1175783578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38ec10a980 con 0x7f38ec108050 2024-09-16T00:23:02.420 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.274+0000 7f38f220d640 1 --2- 172.21.15.38:0/1175783578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38ec108050 0x7f38ec10a440 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f38e8009820 tx=0x7f38e802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a32b80c66d7e53a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:02.420 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.274+0000 7f38f120b640 1 -- 172.21.15.38:0/1175783578 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38e8037070 con 0x7f38ec108050 2024-09-16T00:23:02.420 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.274+0000 7f38f120b640 1 -- 172.21.15.38:0/1175783578 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f38e8032900 con 0x7f38ec108050 2024-09-16T00:23:02.420 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.274+0000 7f38f120b640 1 -- 172.21.15.38:0/1175783578 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38e8030850 con 0x7f38ec108050 2024-09-16T00:23:02.420 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.275+0000 7f38f4471640 1 -- 172.21.15.38:0/1175783578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38ec108050 msgr2=0x7f38ec10a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:02.420 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.275+0000 7f38f4471640 1 --2- 172.21.15.38:0/1175783578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38ec108050 0x7f38ec10a440 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f38e8009820 tx=0x7f38e802a270 comp rx=0 tx=0).stop 2024-09-16T00:23:02.420 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.275+0000 7f38f4471640 1 -- 172.21.15.38:0/1175783578 shutdown_connections 2024-09-16T00:23:02.420 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.275+0000 7f38f4471640 1 --2- 172.21.15.38:0/1175783578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38ec108050 0x7f38ec10a440 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:02.420 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.275+0000 7f38f4471640 1 -- 172.21.15.38:0/1175783578 >> 172.21.15.38:0/1175783578 conn(0x7f38ec101c30 msgr2=0x7f38ec1040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:02.420 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.275+0000 7f38f4471640 1 -- 172.21.15.38:0/1175783578 shutdown_connections 2024-09-16T00:23:02.420 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.275+0000 7f38f4471640 1 -- 172.21.15.38:0/1175783578 wait complete. 2024-09-16T00:23:02.420 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.276+0000 7f38f4471640 1 Processor -- start 2024-09-16T00:23:02.421 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.276+0000 7f38f4471640 1 -- start start 2024-09-16T00:23:02.421 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.276+0000 7f38f4471640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38ec108050 0x7f38ec19b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:02.421 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.276+0000 7f38f4471640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f38ec10b9c0 con 0x7f38ec108050 2024-09-16T00:23:02.421 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.276+0000 7f38f220d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38ec108050 0x7f38ec19b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:02.421 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.276+0000 7f38f220d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38ec108050 0x7f38ec19b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34386/0 (socket says 172.21.15.38:34386) 2024-09-16T00:23:02.421 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.276+0000 7f38f220d640 1 -- 172.21.15.38:0/2189366056 learned_addr learned my addr 172.21.15.38:0/2189366056 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:23:02.421 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.276+0000 7f38f220d640 1 -- 172.21.15.38:0/2189366056 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38ec19bad0 con 0x7f38ec108050 2024-09-16T00:23:02.421 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.276+0000 7f38f220d640 1 --2- 172.21.15.38:0/2189366056 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38ec108050 0x7f38ec19b590 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f38e8032ce0 tx=0x7f38e8032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:02.421 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.276+0000 7f38e37fe640 1 -- 172.21.15.38:0/2189366056 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38e8042070 con 0x7f38ec108050 2024-09-16T00:23:02.421 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.277+0000 7f38e37fe640 1 -- 172.21.15.38:0/2189366056 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f38e80308c0 con 0x7f38ec108050 2024-09-16T00:23:02.421 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.277+0000 7f38f4471640 1 -- 172.21.15.38:0/2189366056 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38ec19bd60 con 0x7f38ec108050 2024-09-16T00:23:02.421 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.277+0000 7f38e37fe640 1 -- 172.21.15.38:0/2189366056 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f38e8037040 con 0x7f38ec108050 2024-09-16T00:23:02.421 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.277+0000 7f38f4471640 1 -- 172.21.15.38:0/2189366056 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38ec19ea50 con 0x7f38ec108050 2024-09-16T00:23:02.422 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.277+0000 7f38e37fe640 1 -- 172.21.15.38:0/2189366056 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f38e8045070 con 0x7f38ec108050 2024-09-16T00:23:02.422 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.278+0000 7f38e37fe640 1 --2- 172.21.15.38:0/2189366056 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f38d80391b0 0x7f38d803b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:02.422 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.278+0000 7f38f4471640 1 -- 172.21.15.38:0/2189366056 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38ec1084b0 con 0x7f38ec108050 2024-09-16T00:23:02.422 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.278+0000 7f38e37fe640 1 -- 172.21.15.38:0/2189366056 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f38e806c330 con 0x7f38ec108050 2024-09-16T00:23:02.422 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.278+0000 7f38f1a0c640 1 --2- 172.21.15.38:0/2189366056 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f38d80391b0 0x7f38d803b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:02.422 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.278+0000 7f38f1a0c640 1 --2- 172.21.15.38:0/2189366056 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f38d80391b0 0x7f38d803b670 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f38dc0098c0 tx=0x7f38dc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:02.422 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.283+0000 7f38e37fe640 1 -- 172.21.15.38:0/2189366056 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f38e804b170 con 0x7f38ec108050 2024-09-16T00:23:02.422 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.461+0000 7f38f4471640 1 -- 172.21.15.38:0/2189366056 --> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f38ec102ee0 con 0x7f38d80391b0 2024-09-16T00:23:02.422 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.767+0000 7f38e37fe640 1 -- 172.21.15.38:0/2189366056 <== mgr.14118 v2:172.21.15.38:6800/1779324916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f38ec102ee0 con 0x7f38d80391b0 2024-09-16T00:23:02.422 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.769+0000 7f38f4471640 1 -- 172.21.15.38:0/2189366056 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f38d80391b0 msgr2=0x7f38d803b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:02.422 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.769+0000 7f38f4471640 1 --2- 172.21.15.38:0/2189366056 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f38d80391b0 0x7f38d803b670 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f38dc0098c0 tx=0x7f38dc006db0 comp rx=0 tx=0).stop 2024-09-16T00:23:02.422 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.769+0000 7f38f4471640 1 -- 172.21.15.38:0/2189366056 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38ec108050 msgr2=0x7f38ec19b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:02.423 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.769+0000 7f38f4471640 1 --2- 172.21.15.38:0/2189366056 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38ec108050 0x7f38ec19b590 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f38e8032ce0 tx=0x7f38e8032d10 comp rx=0 tx=0).stop 2024-09-16T00:23:02.423 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.769+0000 7f38f4471640 1 -- 172.21.15.38:0/2189366056 shutdown_connections 2024-09-16T00:23:02.423 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.769+0000 7f38f4471640 1 --2- 172.21.15.38:0/2189366056 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f38d80391b0 0x7f38d803b670 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:02.423 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.769+0000 7f38f4471640 1 --2- 172.21.15.38:0/2189366056 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f38ec108050 0x7f38ec19b590 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:02.423 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.769+0000 7f38f4471640 1 -- 172.21.15.38:0/2189366056 >> 172.21.15.38:0/2189366056 conn(0x7f38ec101c30 msgr2=0x7f38ec1027d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:02.423 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.769+0000 7f38f4471640 1 -- 172.21.15.38:0/2189366056 shutdown_connections 2024-09-16T00:23:02.423 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:01.769+0000 7f38f4471640 1 -- 172.21.15.38:0/2189366056 wait complete. 2024-09-16T00:23:03.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:02 smithi038 ceph-mon[30945]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:23:03.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:02 smithi038 ceph-mon[30945]: Generating ssh key... 2024-09-16T00:23:03.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:02 smithi038 ceph-mon[30945]: from='mgr.14118 172.21.15.38:0/3698772563' entity='mgr.x' 2024-09-16T00:23:03.006 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:02 smithi038 ceph-mon[30945]: from='mgr.14118 172.21.15.38:0/3698772563' entity='mgr.x' 2024-09-16T00:23:03.582 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCf0y82XRDGWCAcNUAxnFIowcLRrsM7vDl9BshzPEeiLlNsv8c48nHD4jbSfEzg7vzle4SEoQFGtCTcAu/9kImKBzX8f4c4Ya4q5P2jtbr1OmyDsz5SlnS/ZSny+r1Jr8nIw3B7LGy7G3y5xh/AScGt3if7UreNISH+OxNf7x/j7feR7KTSn+b7LFryuP6n4BHSyj3xnHJVIMSnbzequuHwZ3SwqbRySgtYI1wenwTsql9PLVr/Oi+kQ0tfo9N8EvCddHWJnIIRVgzRziMAJNd6C2Y0O1Ny5KDJo8lqRwePm90ikJPHjO2SrOHYLaVrsYn+9HFRvuXoBY2qCzisZidnbaQfYW74lrqNT800tdZJU6kf8vF6Bxglaldq38k8iNh+9sZmNSq0hcuf9oj+y0Qj5kpHSIugG3CwJY7sAZBaJXDoTB75HwEzN7GlUB+rLxtblZgY4krveuEyCUupSLQi2fAjiCs/w9lVV139zF4zPuG2ytxteAqJg26ZAEhGg40= ceph-983b2072-73c1-11ef-bceb-c7b262605968 2024-09-16T00:23:03.582 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:02.817+0000 7fc2987a9640 1 Processor -- start 2024-09-16T00:23:03.582 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:02.817+0000 7fc2987a9640 1 -- start start 2024-09-16T00:23:03.582 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:02.818+0000 7fc2987a9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc290107620 0x7fc290107a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:03.582 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:02.818+0000 7fc2987a9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc290107ff0 con 0x7fc290107620 2024-09-16T00:23:03.582 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:02.818+0000 7fc296545640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc290107620 0x7fc290107a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:03.582 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:02.818+0000 7fc296545640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc290107620 0x7fc290107a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34398/0 (socket says 172.21.15.38:34398) 2024-09-16T00:23:03.583 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:02.818+0000 7fc296545640 1 -- 172.21.15.38:0/3806932772 learned_addr learned my addr 172.21.15.38:0/3806932772 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:23:03.583 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:02.818+0000 7fc296545640 1 -- 172.21.15.38:0/3806932772 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc290108820 con 0x7fc290107620 2024-09-16T00:23:03.583 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:02.819+0000 7fc296545640 1 --2- 172.21.15.38:0/3806932772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc290107620 0x7fc290107a20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc28c009820 tx=0x7fc28c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1336293544be9f53 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:03.583 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:02.819+0000 7fc295543640 1 -- 172.21.15.38:0/3806932772 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc28c037070 con 0x7fc290107620 2024-09-16T00:23:03.583 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:02.819+0000 7fc295543640 1 -- 172.21.15.38:0/3806932772 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc28c032900 con 0x7fc290107620 2024-09-16T00:23:03.583 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:02.819+0000 7fc295543640 1 -- 172.21.15.38:0/3806932772 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc28c030850 con 0x7fc290107620 2024-09-16T00:23:03.583 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:02.820+0000 7fc2987a9640 1 -- 172.21.15.38:0/3806932772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc290107620 msgr2=0x7fc290107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:03.583 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:02.820+0000 7fc2987a9640 1 --2- 172.21.15.38:0/3806932772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc290107620 0x7fc290107a20 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc28c009820 tx=0x7fc28c02a270 comp rx=0 tx=0).stop 2024-09-16T00:23:03.583 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:02.820+0000 7fc2987a9640 1 -- 172.21.15.38:0/3806932772 shutdown_connections 2024-09-16T00:23:03.583 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:02.820+0000 7fc2987a9640 1 --2- 172.21.15.38:0/3806932772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc290107620 0x7fc290107a20 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:03.583 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:02.820+0000 7fc2987a9640 1 -- 172.21.15.38:0/3806932772 >> 172.21.15.38:0/3806932772 conn(0x7fc290102ea0 msgr2=0x7fc290105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:03.583 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:02.820+0000 7fc2987a9640 1 -- 172.21.15.38:0/3806932772 shutdown_connections 2024-09-16T00:23:03.583 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:02.820+0000 7fc2987a9640 1 -- 172.21.15.38:0/3806932772 wait complete. 2024-09-16T00:23:03.584 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:02.821+0000 7fc2987a9640 1 Processor -- start 2024-09-16T00:23:03.584 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:02.821+0000 7fc2987a9640 1 -- start start 2024-09-16T00:23:03.584 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:02.821+0000 7fc2987a9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc290107620 0x7fc29019b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:03.584 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:02.821+0000 7fc2987a9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc29010bbd0 con 0x7fc290107620 2024-09-16T00:23:03.584 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:02.821+0000 7fc296545640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc290107620 0x7fc29019b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:03.584 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:02.821+0000 7fc296545640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc290107620 0x7fc29019b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34402/0 (socket says 172.21.15.38:34402) 2024-09-16T00:23:03.584 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:02.821+0000 7fc296545640 1 -- 172.21.15.38:0/3210649124 learned_addr learned my addr 172.21.15.38:0/3210649124 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:23:03.584 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:02.821+0000 7fc296545640 1 -- 172.21.15.38:0/3210649124 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc29019bb50 con 0x7fc290107620 2024-09-16T00:23:03.584 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:02.822+0000 7fc296545640 1 --2- 172.21.15.38:0/3210649124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc290107620 0x7fc29019b610 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc28c009940 tx=0x7fc28c032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:03.584 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:02.822+0000 7fc2877fe640 1 -- 172.21.15.38:0/3210649124 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc28c042070 con 0x7fc290107620 2024-09-16T00:23:03.584 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:02.822+0000 7fc2877fe640 1 -- 172.21.15.38:0/3210649124 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7fc28c0308c0 con 0x7fc290107620 2024-09-16T00:23:03.584 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:02.822+0000 7fc2987a9640 1 -- 172.21.15.38:0/3210649124 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc29019bde0 con 0x7fc290107620 2024-09-16T00:23:03.584 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:02.822+0000 7fc2877fe640 1 -- 172.21.15.38:0/3210649124 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc28c037040 con 0x7fc290107620 2024-09-16T00:23:03.585 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:02.822+0000 7fc2987a9640 1 -- 172.21.15.38:0/3210649124 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc29019ead0 con 0x7fc290107620 2024-09-16T00:23:03.585 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:02.823+0000 7fc2877fe640 1 -- 172.21.15.38:0/3210649124 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fc28c045070 con 0x7fc290107620 2024-09-16T00:23:03.585 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:02.823+0000 7fc2987a9640 1 -- 172.21.15.38:0/3210649124 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc29010bd50 con 0x7fc290107620 2024-09-16T00:23:03.585 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:02.823+0000 7fc2877fe640 1 --2- 172.21.15.38:0/3210649124 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7fc27c039200 0x7fc27c03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:03.585 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:02.823+0000 7fc2877fe640 1 -- 172.21.15.38:0/3210649124 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc28c06c480 con 0x7fc290107620 2024-09-16T00:23:03.585 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:02.823+0000 7fc295d44640 1 --2- 172.21.15.38:0/3210649124 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7fc27c039200 0x7fc27c03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:03.585 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:02.824+0000 7fc295d44640 1 --2- 172.21.15.38:0/3210649124 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7fc27c039200 0x7fc27c03b6c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc2800098c0 tx=0x7fc280006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:03.585 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:02.828+0000 7fc2877fe640 1 -- 172.21.15.38:0/3210649124 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc28c03d430 con 0x7fc290107620 2024-09-16T00:23:03.585 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.005+0000 7fc2987a9640 1 -- 172.21.15.38:0/3210649124 --> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fc2901040a0 con 0x7fc27c039200 2024-09-16T00:23:03.585 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.006+0000 7fc2877fe640 1 -- 172.21.15.38:0/3210649124 <== mgr.14118 v2:172.21.15.38:6800/1779324916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fc2901040a0 con 0x7fc27c039200 2024-09-16T00:23:03.585 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.008+0000 7fc2987a9640 1 -- 172.21.15.38:0/3210649124 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7fc27c039200 msgr2=0x7fc27c03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:03.585 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.008+0000 7fc2987a9640 1 --2- 172.21.15.38:0/3210649124 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7fc27c039200 0x7fc27c03b6c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fc2800098c0 tx=0x7fc280006db0 comp rx=0 tx=0).stop 2024-09-16T00:23:03.585 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.008+0000 7fc2987a9640 1 -- 172.21.15.38:0/3210649124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc290107620 msgr2=0x7fc29019b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:03.586 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.008+0000 7fc2987a9640 1 --2- 172.21.15.38:0/3210649124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc290107620 0x7fc29019b610 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc28c009940 tx=0x7fc28c032ce0 comp rx=0 tx=0).stop 2024-09-16T00:23:03.586 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.008+0000 7fc2987a9640 1 -- 172.21.15.38:0/3210649124 shutdown_connections 2024-09-16T00:23:03.586 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.008+0000 7fc2987a9640 1 --2- 172.21.15.38:0/3210649124 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7fc27c039200 0x7fc27c03b6c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:03.586 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.008+0000 7fc2987a9640 1 --2- 172.21.15.38:0/3210649124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc290107620 0x7fc29019b610 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:03.586 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.008+0000 7fc2987a9640 1 -- 172.21.15.38:0/3210649124 >> 172.21.15.38:0/3210649124 conn(0x7fc290102ea0 msgr2=0x7fc290103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:03.586 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.008+0000 7fc2987a9640 1 -- 172.21.15.38:0/3210649124 shutdown_connections 2024-09-16T00:23:03.586 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.008+0000 7fc2987a9640 1 -- 172.21.15.38:0/3210649124 wait complete. 2024-09-16T00:23:03.586 INFO:teuthology.orchestra.run.smithi038.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-16T00:23:03.586 INFO:teuthology.orchestra.run.smithi038.stdout:Adding key to root@localhost authorized_keys... 2024-09-16T00:23:03.588 INFO:teuthology.orchestra.run.smithi038.stdout:Adding host smithi038... 2024-09-16T00:23:04.184 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:03 smithi038 ceph-mon[30945]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:23:05.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:04 smithi038 ceph-mon[30945]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi038", "addr": "172.21.15.38", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:23:06.149 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:05 smithi038 ceph-mon[30945]: Deploying cephadm binary to smithi038 2024-09-16T00:23:08.706 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout Added host 'smithi038' with addr '172.21.15.38' 2024-09-16T00:23:08.706 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.992+0000 7f87d890c640 1 Processor -- start 2024-09-16T00:23:08.707 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.992+0000 7f87d890c640 1 -- start start 2024-09-16T00:23:08.707 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.992+0000 7f87d890c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87d4107620 0x7f87d4107a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:08.707 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.992+0000 7f87d890c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f87d4107ff0 con 0x7f87d4107620 2024-09-16T00:23:08.707 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.992+0000 7f87d259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87d4107620 0x7f87d4107a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:08.707 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.992+0000 7f87d259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87d4107620 0x7f87d4107a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34416/0 (socket says 172.21.15.38:34416) 2024-09-16T00:23:08.707 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.992+0000 7f87d259c640 1 -- 172.21.15.38:0/3124059902 learned_addr learned my addr 172.21.15.38:0/3124059902 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:23:08.707 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.992+0000 7f87d259c640 1 -- 172.21.15.38:0/3124059902 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d4108820 con 0x7f87d4107620 2024-09-16T00:23:08.707 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.993+0000 7f87d259c640 1 --2- 172.21.15.38:0/3124059902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87d4107620 0x7f87d4107a20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f87c4009820 tx=0x7f87c402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c87592e39c2a3e5f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:08.707 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.993+0000 7f87d1d9b640 1 -- 172.21.15.38:0/3124059902 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87c4037070 con 0x7f87d4107620 2024-09-16T00:23:08.707 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.993+0000 7f87d1d9b640 1 -- 172.21.15.38:0/3124059902 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f87c4032900 con 0x7f87d4107620 2024-09-16T00:23:08.707 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.993+0000 7f87d1d9b640 1 -- 172.21.15.38:0/3124059902 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87c4030850 con 0x7f87d4107620 2024-09-16T00:23:08.707 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.994+0000 7f87d890c640 1 -- 172.21.15.38:0/3124059902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87d4107620 msgr2=0x7f87d4107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:08.708 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.994+0000 7f87d890c640 1 --2- 172.21.15.38:0/3124059902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87d4107620 0x7f87d4107a20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f87c4009820 tx=0x7f87c402a270 comp rx=0 tx=0).stop 2024-09-16T00:23:08.708 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.994+0000 7f87d890c640 1 -- 172.21.15.38:0/3124059902 shutdown_connections 2024-09-16T00:23:08.708 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.994+0000 7f87d890c640 1 --2- 172.21.15.38:0/3124059902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87d4107620 0x7f87d4107a20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:08.708 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.994+0000 7f87d890c640 1 -- 172.21.15.38:0/3124059902 >> 172.21.15.38:0/3124059902 conn(0x7f87d4102ea0 msgr2=0x7f87d4105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:08.708 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.994+0000 7f87d890c640 1 -- 172.21.15.38:0/3124059902 shutdown_connections 2024-09-16T00:23:08.708 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.994+0000 7f87d890c640 1 -- 172.21.15.38:0/3124059902 wait complete. 2024-09-16T00:23:08.708 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.995+0000 7f87d890c640 1 Processor -- start 2024-09-16T00:23:08.708 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.995+0000 7f87d890c640 1 -- start start 2024-09-16T00:23:08.708 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.995+0000 7f87d890c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87d4107620 0x7f87d419b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:08.708 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.995+0000 7f87d890c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f87d410bbd0 con 0x7f87d4107620 2024-09-16T00:23:08.708 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.995+0000 7f87d259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87d4107620 0x7f87d419b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:08.708 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.995+0000 7f87d259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87d4107620 0x7f87d419b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34424/0 (socket says 172.21.15.38:34424) 2024-09-16T00:23:08.708 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.995+0000 7f87d259c640 1 -- 172.21.15.38:0/4081014354 learned_addr learned my addr 172.21.15.38:0/4081014354 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:23:08.709 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.996+0000 7f87d259c640 1 -- 172.21.15.38:0/4081014354 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87d419bb50 con 0x7f87d4107620 2024-09-16T00:23:08.709 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.996+0000 7f87d259c640 1 --2- 172.21.15.38:0/4081014354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87d4107620 0x7f87d419b610 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f87c40097f0 tx=0x7f87c4030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:08.709 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.996+0000 7f87cb7fe640 1 -- 172.21.15.38:0/4081014354 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87c4037040 con 0x7f87d4107620 2024-09-16T00:23:08.709 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.996+0000 7f87cb7fe640 1 -- 172.21.15.38:0/4081014354 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f87c4040070 con 0x7f87d4107620 2024-09-16T00:23:08.709 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.996+0000 7f87cb7fe640 1 -- 172.21.15.38:0/4081014354 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87c403a390 con 0x7f87d4107620 2024-09-16T00:23:08.709 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.996+0000 7f87d890c640 1 -- 172.21.15.38:0/4081014354 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87d419bde0 con 0x7f87d4107620 2024-09-16T00:23:08.709 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.996+0000 7f87d890c640 1 -- 172.21.15.38:0/4081014354 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87d419ead0 con 0x7f87d4107620 2024-09-16T00:23:08.709 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.997+0000 7f87cb7fe640 1 -- 172.21.15.38:0/4081014354 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f87c403b430 con 0x7f87d4107620 2024-09-16T00:23:08.709 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.997+0000 7f87cb7fe640 1 --2- 172.21.15.38:0/4081014354 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f87c00391b0 0x7f87c003b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:08.709 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.997+0000 7f87d890c640 1 -- 172.21.15.38:0/4081014354 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87d4107aa0 con 0x7f87d4107620 2024-09-16T00:23:08.709 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.997+0000 7f87cb7fe640 1 -- 172.21.15.38:0/4081014354 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f87c406c7a0 con 0x7f87d4107620 2024-09-16T00:23:08.709 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.998+0000 7f87cbfff640 1 --2- 172.21.15.38:0/4081014354 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f87c00391b0 0x7f87c003b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:08.709 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:03.998+0000 7f87cbfff640 1 --2- 172.21.15.38:0/4081014354 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f87c00391b0 0x7f87c003b670 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f87bc0098c0 tx=0x7f87bc006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:08.710 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:04.002+0000 7f87cb7fe640 1 -- 172.21.15.38:0/4081014354 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f87c403a5e0 con 0x7f87d4107620 2024-09-16T00:23:08.710 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:04.183+0000 7f87d890c640 1 -- 172.21.15.38:0/4081014354 --> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi038", "addr": "172.21.15.38", "target": ["mon-mgr", ""]}) -- 0x7f87d41040a0 con 0x7f87c00391b0 2024-09-16T00:23:08.710 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:08.099+0000 7f87cb7fe640 1 -- 172.21.15.38:0/4081014354 <== mgr.14118 v2:172.21.15.38:6800/1779324916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f87d41040a0 con 0x7f87c00391b0 2024-09-16T00:23:08.710 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:08.101+0000 7f87d890c640 1 -- 172.21.15.38:0/4081014354 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f87c00391b0 msgr2=0x7f87c003b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:08.710 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:08.101+0000 7f87d890c640 1 --2- 172.21.15.38:0/4081014354 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f87c00391b0 0x7f87c003b670 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f87bc0098c0 tx=0x7f87bc006db0 comp rx=0 tx=0).stop 2024-09-16T00:23:08.710 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:08.101+0000 7f87d890c640 1 -- 172.21.15.38:0/4081014354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87d4107620 msgr2=0x7f87d419b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:08.710 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:08.101+0000 7f87d890c640 1 --2- 172.21.15.38:0/4081014354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87d4107620 0x7f87d419b610 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f87c40097f0 tx=0x7f87c4030d80 comp rx=0 tx=0).stop 2024-09-16T00:23:08.710 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:08.101+0000 7f87d890c640 1 -- 172.21.15.38:0/4081014354 shutdown_connections 2024-09-16T00:23:08.710 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:08.101+0000 7f87d890c640 1 --2- 172.21.15.38:0/4081014354 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f87c00391b0 0x7f87c003b670 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:08.710 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:08.101+0000 7f87d890c640 1 --2- 172.21.15.38:0/4081014354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f87d4107620 0x7f87d419b610 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:08.710 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:08.101+0000 7f87d890c640 1 -- 172.21.15.38:0/4081014354 >> 172.21.15.38:0/4081014354 conn(0x7f87d4102ea0 msgr2=0x7f87d4103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:08.710 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:08.102+0000 7f87d890c640 1 -- 172.21.15.38:0/4081014354 shutdown_connections 2024-09-16T00:23:08.710 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:08.102+0000 7f87d890c640 1 -- 172.21.15.38:0/4081014354 wait complete. 2024-09-16T00:23:08.711 INFO:teuthology.orchestra.run.smithi038.stdout:Deploying unmanaged mon service... 2024-09-16T00:23:09.306 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:09 smithi038 ceph-mon[30945]: from='mgr.14118 172.21.15.38:0/3698772563' entity='mgr.x' 2024-09-16T00:23:09.306 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:09 smithi038 ceph-mon[30945]: Added host smithi038 2024-09-16T00:23:09.306 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:09 smithi038 ceph-mon[30945]: from='mgr.14118 172.21.15.38:0/3698772563' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:23:10.191 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-16T00:23:10.191 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.103+0000 7ffa15815640 1 Processor -- start 2024-09-16T00:23:10.191 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.103+0000 7ffa15815640 1 -- start start 2024-09-16T00:23:10.192 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.103+0000 7ffa15815640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa10107600 0x7ffa10107a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:10.192 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.103+0000 7ffa15815640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ffa10107fd0 con 0x7ffa10107600 2024-09-16T00:23:10.192 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.104+0000 7ffa0effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa10107600 0x7ffa10107a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:10.192 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.104+0000 7ffa0effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa10107600 0x7ffa10107a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35942/0 (socket says 172.21.15.38:35942) 2024-09-16T00:23:10.192 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.104+0000 7ffa0effd640 1 -- 172.21.15.38:0/277186215 learned_addr learned my addr 172.21.15.38:0/277186215 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:23:10.192 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.104+0000 7ffa0effd640 1 -- 172.21.15.38:0/277186215 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa101087b0 con 0x7ffa10107600 2024-09-16T00:23:10.192 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.105+0000 7ffa0effd640 1 --2- 172.21.15.38:0/277186215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa10107600 0x7ffa10107a00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ffa00009820 tx=0x7ffa0002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad813eb7ca77085a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:10.192 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.105+0000 7ffa0dffb640 1 -- 172.21.15.38:0/277186215 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa00037070 con 0x7ffa10107600 2024-09-16T00:23:10.192 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.105+0000 7ffa0dffb640 1 -- 172.21.15.38:0/277186215 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ffa00032900 con 0x7ffa10107600 2024-09-16T00:23:10.192 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.105+0000 7ffa0dffb640 1 -- 172.21.15.38:0/277186215 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa00030850 con 0x7ffa10107600 2024-09-16T00:23:10.192 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.105+0000 7ffa15815640 1 -- 172.21.15.38:0/277186215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa10107600 msgr2=0x7ffa10107a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:10.192 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.105+0000 7ffa15815640 1 --2- 172.21.15.38:0/277186215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa10107600 0x7ffa10107a00 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ffa00009820 tx=0x7ffa0002a270 comp rx=0 tx=0).stop 2024-09-16T00:23:10.193 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.105+0000 7ffa15815640 1 -- 172.21.15.38:0/277186215 shutdown_connections 2024-09-16T00:23:10.193 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.105+0000 7ffa15815640 1 --2- 172.21.15.38:0/277186215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa10107600 0x7ffa10107a00 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:10.193 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.105+0000 7ffa15815640 1 -- 172.21.15.38:0/277186215 >> 172.21.15.38:0/277186215 conn(0x7ffa10102e30 msgr2=0x7ffa10105220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:10.193 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.105+0000 7ffa15815640 1 -- 172.21.15.38:0/277186215 shutdown_connections 2024-09-16T00:23:10.193 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.105+0000 7ffa15815640 1 -- 172.21.15.38:0/277186215 wait complete. 2024-09-16T00:23:10.193 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.106+0000 7ffa15815640 1 Processor -- start 2024-09-16T00:23:10.193 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.106+0000 7ffa15815640 1 -- start start 2024-09-16T00:23:10.193 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.106+0000 7ffa15815640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa10107600 0x7ffa1019b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:10.193 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.106+0000 7ffa15815640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ffa1010bb60 con 0x7ffa10107600 2024-09-16T00:23:10.193 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.107+0000 7ffa0effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa10107600 0x7ffa1019b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:10.193 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.107+0000 7ffa0effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa10107600 0x7ffa1019b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35944/0 (socket says 172.21.15.38:35944) 2024-09-16T00:23:10.193 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.107+0000 7ffa0effd640 1 -- 172.21.15.38:0/3826571288 learned_addr learned my addr 172.21.15.38:0/3826571288 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:23:10.193 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.107+0000 7ffa0effd640 1 -- 172.21.15.38:0/3826571288 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa1019bb10 con 0x7ffa10107600 2024-09-16T00:23:10.194 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.107+0000 7ffa0effd640 1 --2- 172.21.15.38:0/3826571288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa10107600 0x7ffa1019b5d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ffa000097f0 tx=0x7ffa00030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:10.194 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.107+0000 7ffa14813640 1 -- 172.21.15.38:0/3826571288 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa00037040 con 0x7ffa10107600 2024-09-16T00:23:10.194 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.107+0000 7ffa14813640 1 -- 172.21.15.38:0/3826571288 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7ffa00040070 con 0x7ffa10107600 2024-09-16T00:23:10.194 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.107+0000 7ffa14813640 1 -- 172.21.15.38:0/3826571288 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa0003a390 con 0x7ffa10107600 2024-09-16T00:23:10.194 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.107+0000 7ffa15815640 1 -- 172.21.15.38:0/3826571288 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa1019bda0 con 0x7ffa10107600 2024-09-16T00:23:10.194 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.107+0000 7ffa15815640 1 -- 172.21.15.38:0/3826571288 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa1019ea90 con 0x7ffa10107600 2024-09-16T00:23:10.194 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.108+0000 7ffa14813640 1 -- 172.21.15.38:0/3826571288 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7ffa0003a5e0 con 0x7ffa10107600 2024-09-16T00:23:10.194 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.108+0000 7ffa14813640 1 --2- 172.21.15.38:0/3826571288 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7ff9f8039200 0x7ff9f803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:10.194 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.108+0000 7ffa14813640 1 -- 172.21.15.38:0/3826571288 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ffa0006c760 con 0x7ffa10107600 2024-09-16T00:23:10.194 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.108+0000 7ffa0e7fc640 1 --2- 172.21.15.38:0/3826571288 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7ff9f8039200 0x7ff9f803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:10.194 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.108+0000 7ffa15815640 1 -- 172.21.15.38:0/3826571288 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa10107a80 con 0x7ffa10107600 2024-09-16T00:23:10.194 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.109+0000 7ffa0e7fc640 1 --2- 172.21.15.38:0/3826571288 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7ff9f8039200 0x7ff9f803b6c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ffa040098c0 tx=0x7ffa04006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:10.194 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.113+0000 7ffa14813640 1 -- 172.21.15.38:0/3826571288 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffa0003b550 con 0x7ffa10107600 2024-09-16T00:23:10.194 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.305+0000 7ffa15815640 1 -- 172.21.15.38:0/3826571288 --> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7ffa10104050 con 0x7ff9f8039200 2024-09-16T00:23:10.195 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.310+0000 7ffa14813640 1 -- 172.21.15.38:0/3826571288 <== mgr.14118 v2:172.21.15.38:6800/1779324916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7ffa10104050 con 0x7ff9f8039200 2024-09-16T00:23:10.195 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.312+0000 7ffa15815640 1 -- 172.21.15.38:0/3826571288 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7ff9f8039200 msgr2=0x7ff9f803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:10.195 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.312+0000 7ffa15815640 1 --2- 172.21.15.38:0/3826571288 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7ff9f8039200 0x7ff9f803b6c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ffa040098c0 tx=0x7ffa04006db0 comp rx=0 tx=0).stop 2024-09-16T00:23:10.195 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.312+0000 7ffa15815640 1 -- 172.21.15.38:0/3826571288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa10107600 msgr2=0x7ffa1019b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:10.195 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.312+0000 7ffa15815640 1 --2- 172.21.15.38:0/3826571288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa10107600 0x7ffa1019b5d0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ffa000097f0 tx=0x7ffa00030d80 comp rx=0 tx=0).stop 2024-09-16T00:23:10.195 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.312+0000 7ffa15815640 1 -- 172.21.15.38:0/3826571288 shutdown_connections 2024-09-16T00:23:10.195 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.312+0000 7ffa15815640 1 --2- 172.21.15.38:0/3826571288 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7ff9f8039200 0x7ff9f803b6c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:10.195 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.312+0000 7ffa15815640 1 --2- 172.21.15.38:0/3826571288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffa10107600 0x7ffa1019b5d0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:10.195 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.312+0000 7ffa15815640 1 -- 172.21.15.38:0/3826571288 >> 172.21.15.38:0/3826571288 conn(0x7ffa10102e30 msgr2=0x7ffa10103940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:10.195 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.312+0000 7ffa15815640 1 -- 172.21.15.38:0/3826571288 shutdown_connections 2024-09-16T00:23:10.195 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:09.312+0000 7ffa15815640 1 -- 172.21.15.38:0/3826571288 wait complete. 2024-09-16T00:23:10.195 INFO:teuthology.orchestra.run.smithi038.stdout:Deploying unmanaged mgr service... 2024-09-16T00:23:10.411 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:10 smithi038 ceph-mon[30945]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:23:10.411 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:10 smithi038 ceph-mon[30945]: Saving service mon spec with placement count:5 2024-09-16T00:23:10.411 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:10 smithi038 ceph-mon[30945]: from='mgr.14118 172.21.15.38:0/3698772563' entity='mgr.x' 2024-09-16T00:23:11.451 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-16T00:23:11.451 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.602+0000 7f92f5900640 1 Processor -- start 2024-09-16T00:23:11.451 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.602+0000 7f92f5900640 1 -- start start 2024-09-16T00:23:11.451 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.602+0000 7f92f5900640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92f0107620 0x7f92f0107a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:11.451 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.602+0000 7f92f5900640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f92f0107ff0 con 0x7f92f0107620 2024-09-16T00:23:11.451 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.602+0000 7f92eeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92f0107620 0x7f92f0107a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:11.451 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.602+0000 7f92eeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92f0107620 0x7f92f0107a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35954/0 (socket says 172.21.15.38:35954) 2024-09-16T00:23:11.451 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.602+0000 7f92eeffd640 1 -- 172.21.15.38:0/1979517866 learned_addr learned my addr 172.21.15.38:0/1979517866 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:23:11.451 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.603+0000 7f92eeffd640 1 -- 172.21.15.38:0/1979517866 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f0108820 con 0x7f92f0107620 2024-09-16T00:23:11.451 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.603+0000 7f92eeffd640 1 --2- 172.21.15.38:0/1979517866 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92f0107620 0x7f92f0107a20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f92e8009820 tx=0x7f92e802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ec9c60e26ae1385 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:11.451 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.604+0000 7f92edffb640 1 -- 172.21.15.38:0/1979517866 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92e8037070 con 0x7f92f0107620 2024-09-16T00:23:11.452 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.604+0000 7f92edffb640 1 -- 172.21.15.38:0/1979517866 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f92e8032900 con 0x7f92f0107620 2024-09-16T00:23:11.452 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.604+0000 7f92edffb640 1 -- 172.21.15.38:0/1979517866 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92e8030850 con 0x7f92f0107620 2024-09-16T00:23:11.452 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.604+0000 7f92f5900640 1 -- 172.21.15.38:0/1979517866 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92f0107620 msgr2=0x7f92f0107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:11.452 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.604+0000 7f92f5900640 1 --2- 172.21.15.38:0/1979517866 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92f0107620 0x7f92f0107a20 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f92e8009820 tx=0x7f92e802a270 comp rx=0 tx=0).stop 2024-09-16T00:23:11.452 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.604+0000 7f92f5900640 1 -- 172.21.15.38:0/1979517866 shutdown_connections 2024-09-16T00:23:11.452 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.604+0000 7f92f5900640 1 --2- 172.21.15.38:0/1979517866 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92f0107620 0x7f92f0107a20 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:11.452 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.604+0000 7f92f5900640 1 -- 172.21.15.38:0/1979517866 >> 172.21.15.38:0/1979517866 conn(0x7f92f0102ea0 msgr2=0x7f92f0105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:11.452 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.604+0000 7f92f5900640 1 -- 172.21.15.38:0/1979517866 shutdown_connections 2024-09-16T00:23:11.452 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.604+0000 7f92f5900640 1 -- 172.21.15.38:0/1979517866 wait complete. 2024-09-16T00:23:11.452 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.605+0000 7f92f5900640 1 Processor -- start 2024-09-16T00:23:11.452 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.605+0000 7f92f5900640 1 -- start start 2024-09-16T00:23:11.453 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.605+0000 7f92f5900640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92f0107620 0x7f92f019b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:11.453 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.605+0000 7f92f5900640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f92f010bbd0 con 0x7f92f0107620 2024-09-16T00:23:11.453 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.605+0000 7f92eeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92f0107620 0x7f92f019b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:11.453 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.606+0000 7f92eeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92f0107620 0x7f92f019b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35956/0 (socket says 172.21.15.38:35956) 2024-09-16T00:23:11.453 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.606+0000 7f92eeffd640 1 -- 172.21.15.38:0/802517621 learned_addr learned my addr 172.21.15.38:0/802517621 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:23:11.453 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.606+0000 7f92eeffd640 1 -- 172.21.15.38:0/802517621 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92f019bb50 con 0x7f92f0107620 2024-09-16T00:23:11.453 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.606+0000 7f92eeffd640 1 --2- 172.21.15.38:0/802517621 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92f0107620 0x7f92f019b610 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f92e8009940 tx=0x7f92e8032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:11.453 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.606+0000 7f92f48fe640 1 -- 172.21.15.38:0/802517621 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92e8042070 con 0x7f92f0107620 2024-09-16T00:23:11.453 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.606+0000 7f92f48fe640 1 -- 172.21.15.38:0/802517621 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f92e80308c0 con 0x7f92f0107620 2024-09-16T00:23:11.453 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.606+0000 7f92f5900640 1 -- 172.21.15.38:0/802517621 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92f019bde0 con 0x7f92f0107620 2024-09-16T00:23:11.453 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.606+0000 7f92f48fe640 1 -- 172.21.15.38:0/802517621 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f92e8037040 con 0x7f92f0107620 2024-09-16T00:23:11.453 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.606+0000 7f92f5900640 1 -- 172.21.15.38:0/802517621 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92f019ead0 con 0x7f92f0107620 2024-09-16T00:23:11.453 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.607+0000 7f92f48fe640 1 -- 172.21.15.38:0/802517621 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f92e8045070 con 0x7f92f0107620 2024-09-16T00:23:11.454 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.607+0000 7f92f5900640 1 -- 172.21.15.38:0/802517621 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92f010bd50 con 0x7f92f0107620 2024-09-16T00:23:11.454 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.607+0000 7f92f48fe640 1 --2- 172.21.15.38:0/802517621 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f92dc039200 0x7f92dc03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:11.454 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.607+0000 7f92f48fe640 1 -- 172.21.15.38:0/802517621 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f92e806c2e0 con 0x7f92f0107620 2024-09-16T00:23:11.454 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.607+0000 7f92ee7fc640 1 --2- 172.21.15.38:0/802517621 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f92dc039200 0x7f92dc03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:11.454 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.608+0000 7f92ee7fc640 1 --2- 172.21.15.38:0/802517621 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f92dc039200 0x7f92dc03b6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f92d80098c0 tx=0x7f92d8006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:11.454 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.612+0000 7f92f48fe640 1 -- 172.21.15.38:0/802517621 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f92e804b170 con 0x7f92f0107620 2024-09-16T00:23:11.454 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.803+0000 7f92f5900640 1 -- 172.21.15.38:0/802517621 --> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f92f01040a0 con 0x7f92dc039200 2024-09-16T00:23:11.454 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.809+0000 7f92f48fe640 1 -- 172.21.15.38:0/802517621 <== mgr.14118 v2:172.21.15.38:6800/1779324916 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f92f01040a0 con 0x7f92dc039200 2024-09-16T00:23:11.454 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.812+0000 7f92f5900640 1 -- 172.21.15.38:0/802517621 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f92dc039200 msgr2=0x7f92dc03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:11.454 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.812+0000 7f92f5900640 1 --2- 172.21.15.38:0/802517621 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f92dc039200 0x7f92dc03b6c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f92d80098c0 tx=0x7f92d8006db0 comp rx=0 tx=0).stop 2024-09-16T00:23:11.454 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.812+0000 7f92f5900640 1 -- 172.21.15.38:0/802517621 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92f0107620 msgr2=0x7f92f019b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:11.454 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.812+0000 7f92f5900640 1 --2- 172.21.15.38:0/802517621 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92f0107620 0x7f92f019b610 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f92e8009940 tx=0x7f92e8032ce0 comp rx=0 tx=0).stop 2024-09-16T00:23:11.455 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.812+0000 7f92f5900640 1 -- 172.21.15.38:0/802517621 shutdown_connections 2024-09-16T00:23:11.455 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.812+0000 7f92f5900640 1 --2- 172.21.15.38:0/802517621 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f92dc039200 0x7f92dc03b6c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:11.455 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.812+0000 7f92f5900640 1 --2- 172.21.15.38:0/802517621 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f92f0107620 0x7f92f019b610 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:11.455 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.812+0000 7f92f5900640 1 -- 172.21.15.38:0/802517621 >> 172.21.15.38:0/802517621 conn(0x7f92f0102ea0 msgr2=0x7f92f0103990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:11.455 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.812+0000 7f92f5900640 1 -- 172.21.15.38:0/802517621 shutdown_connections 2024-09-16T00:23:11.455 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:10.812+0000 7f92f5900640 1 -- 172.21.15.38:0/802517621 wait complete. 2024-09-16T00:23:11.988 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:11 smithi038 ceph-mon[30945]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:23:11.988 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:11 smithi038 ceph-mon[30945]: Saving service mgr spec with placement count:2 2024-09-16T00:23:11.988 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:11 smithi038 ceph-mon[30945]: from='mgr.14118 172.21.15.38:0/3698772563' entity='mgr.x' 2024-09-16T00:23:13.354 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:13 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/2230017300' entity='client.admin' 2024-09-16T00:23:13.355 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:13 smithi038 ceph-mon[30945]: from='mgr.14118 172.21.15.38:0/3698772563' entity='mgr.x' 2024-09-16T00:23:13.361 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:11.815+0000 7f63abdfa640 1 Processor -- start 2024-09-16T00:23:13.361 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:11.816+0000 7f63abdfa640 1 -- start start 2024-09-16T00:23:13.362 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:11.816+0000 7f63abdfa640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63a4107620 0x7f63a4107a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:13.362 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:11.816+0000 7f63abdfa640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f63a4107ff0 con 0x7f63a4107620 2024-09-16T00:23:13.362 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:11.816+0000 7f63a9b96640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63a4107620 0x7f63a4107a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:13.362 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:11.816+0000 7f63a9b96640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63a4107620 0x7f63a4107a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35970/0 (socket says 172.21.15.38:35970) 2024-09-16T00:23:13.362 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:11.816+0000 7f63a9b96640 1 -- 172.21.15.38:0/2817602297 learned_addr learned my addr 172.21.15.38:0/2817602297 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:23:13.362 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:11.816+0000 7f63a9b96640 1 -- 172.21.15.38:0/2817602297 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a4108820 con 0x7f63a4107620 2024-09-16T00:23:13.362 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:11.817+0000 7f63a9b96640 1 --2- 172.21.15.38:0/2817602297 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63a4107620 0x7f63a4107a20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009820 tx=0x7f63a002a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7efd8d6274ddf7e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:13.362 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:11.817+0000 7f63a8b94640 1 -- 172.21.15.38:0/2817602297 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63a0037070 con 0x7f63a4107620 2024-09-16T00:23:13.362 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:11.817+0000 7f63a8b94640 1 -- 172.21.15.38:0/2817602297 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f63a0032900 con 0x7f63a4107620 2024-09-16T00:23:13.362 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:11.817+0000 7f63a8b94640 1 -- 172.21.15.38:0/2817602297 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63a0030850 con 0x7f63a4107620 2024-09-16T00:23:13.362 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:11.818+0000 7f63abdfa640 1 -- 172.21.15.38:0/2817602297 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63a4107620 msgr2=0x7f63a4107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:13.362 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:11.818+0000 7f63abdfa640 1 --2- 172.21.15.38:0/2817602297 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63a4107620 0x7f63a4107a20 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009820 tx=0x7f63a002a270 comp rx=0 tx=0).stop 2024-09-16T00:23:13.363 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:11.818+0000 7f63abdfa640 1 -- 172.21.15.38:0/2817602297 shutdown_connections 2024-09-16T00:23:13.363 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:11.818+0000 7f63abdfa640 1 --2- 172.21.15.38:0/2817602297 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63a4107620 0x7f63a4107a20 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:13.363 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:11.818+0000 7f63abdfa640 1 -- 172.21.15.38:0/2817602297 >> 172.21.15.38:0/2817602297 conn(0x7f63a4102ea0 msgr2=0x7f63a4105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:13.363 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:11.818+0000 7f63abdfa640 1 -- 172.21.15.38:0/2817602297 shutdown_connections 2024-09-16T00:23:13.363 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:11.818+0000 7f63abdfa640 1 -- 172.21.15.38:0/2817602297 wait complete. 2024-09-16T00:23:13.363 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:11.819+0000 7f63abdfa640 1 Processor -- start 2024-09-16T00:23:13.363 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:11.819+0000 7f63abdfa640 1 -- start start 2024-09-16T00:23:13.363 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:11.819+0000 7f63abdfa640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63a4107620 0x7f63a419b610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:13.363 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:11.819+0000 7f63abdfa640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f63a410bbd0 con 0x7f63a4107620 2024-09-16T00:23:13.363 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:11.819+0000 7f63a9b96640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63a4107620 0x7f63a419b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:13.363 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:11.819+0000 7f63a9b96640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63a4107620 0x7f63a419b610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35982/0 (socket says 172.21.15.38:35982) 2024-09-16T00:23:13.363 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:11.819+0000 7f63a9b96640 1 -- 172.21.15.38:0/2230017300 learned_addr learned my addr 172.21.15.38:0/2230017300 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:23:13.364 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:11.819+0000 7f63a9b96640 1 -- 172.21.15.38:0/2230017300 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63a419bb50 con 0x7f63a4107620 2024-09-16T00:23:13.364 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:11.819+0000 7f63a9b96640 1 --2- 172.21.15.38:0/2230017300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63a4107620 0x7f63a419b610 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f63a00097f0 tx=0x7f63a0030d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:13.364 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:11.819+0000 7f639affd640 1 -- 172.21.15.38:0/2230017300 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63a0037040 con 0x7f63a4107620 2024-09-16T00:23:13.364 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:11.819+0000 7f639affd640 1 -- 172.21.15.38:0/2230017300 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1109+0+0 (secure 0 0 0) 0x7f63a0040070 con 0x7f63a4107620 2024-09-16T00:23:13.364 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:11.819+0000 7f63abdfa640 1 -- 172.21.15.38:0/2230017300 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63a419bde0 con 0x7f63a4107620 2024-09-16T00:23:13.364 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:11.819+0000 7f639affd640 1 -- 172.21.15.38:0/2230017300 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f63a003a390 con 0x7f63a4107620 2024-09-16T00:23:13.364 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:11.820+0000 7f63abdfa640 1 -- 172.21.15.38:0/2230017300 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63a419ead0 con 0x7f63a4107620 2024-09-16T00:23:13.364 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:11.820+0000 7f639affd640 1 -- 172.21.15.38:0/2230017300 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7f63a003a5e0 con 0x7f63a4107620 2024-09-16T00:23:13.364 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:11.821+0000 7f639affd640 1 --2- 172.21.15.38:0/2230017300 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f6390039200 0x7f639003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:13.364 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:11.821+0000 7f639affd640 1 -- 172.21.15.38:0/2230017300 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f63a006c760 con 0x7f63a4107620 2024-09-16T00:23:13.364 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:11.821+0000 7f63abdfa640 1 -- 172.21.15.38:0/2230017300 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63a4107aa0 con 0x7f63a4107620 2024-09-16T00:23:13.364 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:11.821+0000 7f63a9395640 1 --2- 172.21.15.38:0/2230017300 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f6390039200 0x7f639003b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:13.364 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:11.821+0000 7f63a9395640 1 --2- 172.21.15.38:0/2230017300 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f6390039200 0x7f639003b6c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f63940098c0 tx=0x7f6394006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:13.365 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:11.826+0000 7f639affd640 1 -- 172.21.15.38:0/2230017300 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f63a003b550 con 0x7f63a4107620 2024-09-16T00:23:13.365 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:12.002+0000 7f63abdfa640 1 -- 172.21.15.38:0/2230017300 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f63a419c260 con 0x7f63a4107620 2024-09-16T00:23:13.365 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:12.006+0000 7f639affd640 1 -- 172.21.15.38:0/2230017300 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v6)=0 v6) ==== 142+0+0 (secure 0 0 0) 0x7f63a0043760 con 0x7f63a4107620 2024-09-16T00:23:13.365 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:12.008+0000 7f63abdfa640 1 -- 172.21.15.38:0/2230017300 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f6390039200 msgr2=0x7f639003b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:13.365 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:12.008+0000 7f63abdfa640 1 --2- 172.21.15.38:0/2230017300 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f6390039200 0x7f639003b6c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f63940098c0 tx=0x7f6394006db0 comp rx=0 tx=0).stop 2024-09-16T00:23:13.365 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:12.008+0000 7f63abdfa640 1 -- 172.21.15.38:0/2230017300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63a4107620 msgr2=0x7f63a419b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:13.365 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:12.008+0000 7f63abdfa640 1 --2- 172.21.15.38:0/2230017300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63a4107620 0x7f63a419b610 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f63a00097f0 tx=0x7f63a0030d80 comp rx=0 tx=0).stop 2024-09-16T00:23:13.365 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:12.008+0000 7f63abdfa640 1 -- 172.21.15.38:0/2230017300 shutdown_connections 2024-09-16T00:23:13.365 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:12.008+0000 7f63abdfa640 1 --2- 172.21.15.38:0/2230017300 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f6390039200 0x7f639003b6c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:13.365 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:12.008+0000 7f63abdfa640 1 --2- 172.21.15.38:0/2230017300 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f63a4107620 0x7f63a419b610 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:13.365 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:12.008+0000 7f63abdfa640 1 -- 172.21.15.38:0/2230017300 >> 172.21.15.38:0/2230017300 conn(0x7f63a4102ea0 msgr2=0x7f63a41039b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:13.365 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:12.008+0000 7f63abdfa640 1 -- 172.21.15.38:0/2230017300 shutdown_connections 2024-09-16T00:23:13.366 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:12.008+0000 7f63abdfa640 1 -- 172.21.15.38:0/2230017300 wait complete. 2024-09-16T00:23:14.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:14 smithi038 ceph-mon[30945]: from='mgr.14118 172.21.15.38:0/3698772563' entity='mgr.x' 2024-09-16T00:23:14.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:14 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/1479829924' entity='client.admin' 2024-09-16T00:23:14.604 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.757+0000 7fb1d84ef640 1 Processor -- start 2024-09-16T00:23:14.604 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.757+0000 7fb1d84ef640 1 -- start start 2024-09-16T00:23:14.604 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.757+0000 7fb1d84ef640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1d0105850 0x7fb1d0105c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:14.604 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.757+0000 7fb1d84ef640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d0106220 con 0x7fb1d0105850 2024-09-16T00:23:14.604 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.757+0000 7fb1d628b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1d0105850 0x7fb1d0105c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:14.605 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.757+0000 7fb1d628b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1d0105850 0x7fb1d0105c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35990/0 (socket says 172.21.15.38:35990) 2024-09-16T00:23:14.605 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.757+0000 7fb1d628b640 1 -- 172.21.15.38:0/205948767 learned_addr learned my addr 172.21.15.38:0/205948767 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:23:14.605 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.758+0000 7fb1d628b640 1 -- 172.21.15.38:0/205948767 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d0108830 con 0x7fb1d0105850 2024-09-16T00:23:14.605 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.758+0000 7fb1d628b640 1 --2- 172.21.15.38:0/205948767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1d0105850 0x7fb1d0105c50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc009820 tx=0x7fb1cc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda509b5c5afc054 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:14.605 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.759+0000 7fb1d5289640 1 -- 172.21.15.38:0/205948767 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb1cc037070 con 0x7fb1d0105850 2024-09-16T00:23:14.605 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.759+0000 7fb1d5289640 1 -- 172.21.15.38:0/205948767 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1cc032920 con 0x7fb1d0105850 2024-09-16T00:23:14.605 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.759+0000 7fb1d5289640 1 -- 172.21.15.38:0/205948767 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb1cc030810 con 0x7fb1d0105850 2024-09-16T00:23:14.605 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.759+0000 7fb1d84ef640 1 -- 172.21.15.38:0/205948767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1d0105850 msgr2=0x7fb1d0105c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:14.605 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.759+0000 7fb1d84ef640 1 --2- 172.21.15.38:0/205948767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1d0105850 0x7fb1d0105c50 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc009820 tx=0x7fb1cc02a270 comp rx=0 tx=0).stop 2024-09-16T00:23:14.605 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.759+0000 7fb1d84ef640 1 -- 172.21.15.38:0/205948767 shutdown_connections 2024-09-16T00:23:14.605 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.759+0000 7fb1d84ef640 1 --2- 172.21.15.38:0/205948767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1d0105850 0x7fb1d0105c50 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:14.606 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.759+0000 7fb1d84ef640 1 -- 172.21.15.38:0/205948767 >> 172.21.15.38:0/205948767 conn(0x7fb1d0101360 msgr2=0x7fb1d01037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:14.606 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.760+0000 7fb1d84ef640 1 -- 172.21.15.38:0/205948767 shutdown_connections 2024-09-16T00:23:14.606 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.760+0000 7fb1d84ef640 1 -- 172.21.15.38:0/205948767 wait complete. 2024-09-16T00:23:14.606 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.760+0000 7fb1d84ef640 1 Processor -- start 2024-09-16T00:23:14.606 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.760+0000 7fb1d84ef640 1 -- start start 2024-09-16T00:23:14.606 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.761+0000 7fb1d84ef640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1d0105850 0x7fb1d019b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:14.606 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.761+0000 7fb1d84ef640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb1d010ba60 con 0x7fb1d0105850 2024-09-16T00:23:14.606 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.761+0000 7fb1d628b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1d0105850 0x7fb1d019b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:14.606 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.761+0000 7fb1d628b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1d0105850 0x7fb1d019b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35994/0 (socket says 172.21.15.38:35994) 2024-09-16T00:23:14.606 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.761+0000 7fb1d628b640 1 -- 172.21.15.38:0/1479829924 learned_addr learned my addr 172.21.15.38:0/1479829924 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:23:14.606 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.761+0000 7fb1d628b640 1 -- 172.21.15.38:0/1479829924 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1d019bb60 con 0x7fb1d0105850 2024-09-16T00:23:14.606 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.761+0000 7fb1d628b640 1 --2- 172.21.15.38:0/1479829924 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1d0105850 0x7fb1d019b620 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc0097f0 tx=0x7fb1cc0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:14.606 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.761+0000 7fb1c77fe640 1 -- 172.21.15.38:0/1479829924 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb1cc037040 con 0x7fb1d0105850 2024-09-16T00:23:14.606 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.761+0000 7fb1c77fe640 1 -- 172.21.15.38:0/1479829924 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb1cc030cc0 con 0x7fb1d0105850 2024-09-16T00:23:14.607 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.761+0000 7fb1d84ef640 1 -- 172.21.15.38:0/1479829924 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1d019bdf0 con 0x7fb1d0105850 2024-09-16T00:23:14.607 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.761+0000 7fb1c77fe640 1 -- 172.21.15.38:0/1479829924 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb1cc03a640 con 0x7fb1d0105850 2024-09-16T00:23:14.607 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.762+0000 7fb1d84ef640 1 -- 172.21.15.38:0/1479829924 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1d019eae0 con 0x7fb1d0105850 2024-09-16T00:23:14.607 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.763+0000 7fb1c77fe640 1 -- 172.21.15.38:0/1479829924 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fb1cc03a7e0 con 0x7fb1d0105850 2024-09-16T00:23:14.607 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.763+0000 7fb1d84ef640 1 -- 172.21.15.38:0/1479829924 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1d0105cd0 con 0x7fb1d0105850 2024-09-16T00:23:14.607 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.763+0000 7fb1c77fe640 1 --2- 172.21.15.38:0/1479829924 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7fb1bc039200 0x7fb1bc03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:14.607 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.763+0000 7fb1c77fe640 1 -- 172.21.15.38:0/1479829924 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fb1cc06c8f0 con 0x7fb1d0105850 2024-09-16T00:23:14.607 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.763+0000 7fb1d5a8a640 1 --2- 172.21.15.38:0/1479829924 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7fb1bc039200 0x7fb1bc03b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:14.607 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.764+0000 7fb1d5a8a640 1 --2- 172.21.15.38:0/1479829924 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7fb1bc039200 0x7fb1bc03b6c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb1c00098c0 tx=0x7fb1c0006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:14.607 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.768+0000 7fb1c77fe640 1 -- 172.21.15.38:0/1479829924 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb1cc0371e0 con 0x7fb1d0105850 2024-09-16T00:23:14.607 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.948+0000 7fb1d84ef640 1 -- 172.21.15.38:0/1479829924 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fb1d019ecb0 con 0x7fb1d0105850 2024-09-16T00:23:14.608 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.950+0000 7fb1c77fe640 1 -- 172.21.15.38:0/1479829924 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fb1cc06c240 con 0x7fb1d0105850 2024-09-16T00:23:14.608 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.953+0000 7fb1d84ef640 1 -- 172.21.15.38:0/1479829924 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7fb1bc039200 msgr2=0x7fb1bc03b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:14.608 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.953+0000 7fb1d84ef640 1 --2- 172.21.15.38:0/1479829924 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7fb1bc039200 0x7fb1bc03b6c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fb1c00098c0 tx=0x7fb1c0006db0 comp rx=0 tx=0).stop 2024-09-16T00:23:14.608 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.953+0000 7fb1d84ef640 1 -- 172.21.15.38:0/1479829924 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1d0105850 msgr2=0x7fb1d019b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:14.608 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.953+0000 7fb1d84ef640 1 --2- 172.21.15.38:0/1479829924 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1d0105850 0x7fb1d019b620 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc0097f0 tx=0x7fb1cc0327b0 comp rx=0 tx=0).stop 2024-09-16T00:23:14.608 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.953+0000 7fb1d84ef640 1 -- 172.21.15.38:0/1479829924 shutdown_connections 2024-09-16T00:23:14.608 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.953+0000 7fb1d84ef640 1 --2- 172.21.15.38:0/1479829924 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7fb1bc039200 0x7fb1bc03b6c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:14.608 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.953+0000 7fb1d84ef640 1 --2- 172.21.15.38:0/1479829924 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb1d0105850 0x7fb1d019b620 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:14.608 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.953+0000 7fb1d84ef640 1 -- 172.21.15.38:0/1479829924 >> 172.21.15.38:0/1479829924 conn(0x7fb1d0101360 msgr2=0x7fb1d0101f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:14.608 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.953+0000 7fb1d84ef640 1 -- 172.21.15.38:0/1479829924 shutdown_connections 2024-09-16T00:23:14.608 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:13.953+0000 7fb1d84ef640 1 -- 172.21.15.38:0/1479829924 wait complete. 2024-09-16T00:23:14.608 INFO:teuthology.orchestra.run.smithi038.stdout:Enabling the dashboard module... 2024-09-16T00:23:16.695 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:16 smithi038 ceph-mon[30945]: from='mgr.14118 172.21.15.38:0/3698772563' entity='mgr.x' 2024-09-16T00:23:16.695 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:16 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/865242880' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-16T00:23:16.695 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:16 smithi038 ceph-mon[30945]: from='mgr.14118 172.21.15.38:0/3698772563' entity='mgr.x' 2024-09-16T00:23:16.696 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:16 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: ignoring --setuser ceph since I am not root 2024-09-16T00:23:16.696 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:16 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: ignoring --setgroup ceph since I am not root 2024-09-16T00:23:16.696 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:16 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:23:16.479+0000 7f1bf5b0a040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T00:23:16.696 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:16 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:23:16.538+0000 7f1bf5b0a040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T00:23:16.696 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:16 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:23:16.693+0000 7f1bf5b0a040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T00:23:16.965 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:16 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:23:16.763+0000 7f1bf5b0a040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T00:23:16.965 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:16 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:23:16.924+0000 7f1bf5b0a040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:23:17.335 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:17 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:23:17.071+0000 7f1bf5b0a040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T00:23:17.568 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:15.499+0000 7fc1e1a6a640 1 Processor -- start 2024-09-16T00:23:17.568 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:15.500+0000 7fc1e1a6a640 1 -- start start 2024-09-16T00:23:17.568 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:15.500+0000 7fc1e1a6a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1dc07e640 0x7fc1dc07caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:17.568 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:15.500+0000 7fc1e1a6a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc1dc07d030 con 0x7fc1dc07e640 2024-09-16T00:23:17.568 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:15.500+0000 7fc1db7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1dc07e640 0x7fc1dc07caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:17.569 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:15.500+0000 7fc1db7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1dc07e640 0x7fc1dc07caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35996/0 (socket says 172.21.15.38:35996) 2024-09-16T00:23:17.569 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:15.500+0000 7fc1db7fe640 1 -- 172.21.15.38:0/2269475931 learned_addr learned my addr 172.21.15.38:0/2269475931 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:23:17.569 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:15.500+0000 7fc1db7fe640 1 -- 172.21.15.38:0/2269475931 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1dc07d1b0 con 0x7fc1dc07e640 2024-09-16T00:23:17.569 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:15.501+0000 7fc1db7fe640 1 --2- 172.21.15.38:0/2269475931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1dc07e640 0x7fc1dc07caf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc009820 tx=0x7fc1cc02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c624a87a88d2bea5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:17.569 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:15.501+0000 7fc1daffd640 1 -- 172.21.15.38:0/2269475931 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc1cc037070 con 0x7fc1dc07e640 2024-09-16T00:23:17.569 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:15.501+0000 7fc1daffd640 1 -- 172.21.15.38:0/2269475931 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1cc032920 con 0x7fc1dc07e640 2024-09-16T00:23:17.569 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:15.501+0000 7fc1daffd640 1 -- 172.21.15.38:0/2269475931 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc1cc030810 con 0x7fc1dc07e640 2024-09-16T00:23:17.569 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:15.502+0000 7fc1e1a6a640 1 -- 172.21.15.38:0/2269475931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1dc07e640 msgr2=0x7fc1dc07caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:17.569 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:15.502+0000 7fc1e1a6a640 1 --2- 172.21.15.38:0/2269475931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1dc07e640 0x7fc1dc07caf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc009820 tx=0x7fc1cc02a270 comp rx=0 tx=0).stop 2024-09-16T00:23:17.569 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:15.502+0000 7fc1e1a6a640 1 -- 172.21.15.38:0/2269475931 shutdown_connections 2024-09-16T00:23:17.569 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:15.502+0000 7fc1e1a6a640 1 --2- 172.21.15.38:0/2269475931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1dc07e640 0x7fc1dc07caf0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:17.569 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:15.502+0000 7fc1e1a6a640 1 -- 172.21.15.38:0/2269475931 >> 172.21.15.38:0/2269475931 conn(0x7fc1dc102e30 msgr2=0x7fc1dc105220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:17.569 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:15.502+0000 7fc1e1a6a640 1 -- 172.21.15.38:0/2269475931 shutdown_connections 2024-09-16T00:23:17.570 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:15.502+0000 7fc1e1a6a640 1 -- 172.21.15.38:0/2269475931 wait complete. 2024-09-16T00:23:17.570 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:15.503+0000 7fc1e1a6a640 1 Processor -- start 2024-09-16T00:23:17.570 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:15.503+0000 7fc1e1a6a640 1 -- start start 2024-09-16T00:23:17.570 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:15.503+0000 7fc1e1a6a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1dc07e640 0x7fc1dc19d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:17.570 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:15.503+0000 7fc1e1a6a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc1dc10dc30 con 0x7fc1dc07e640 2024-09-16T00:23:17.570 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:15.503+0000 7fc1db7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1dc07e640 0x7fc1dc19d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:17.570 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:15.503+0000 7fc1db7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1dc07e640 0x7fc1dc19d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36002/0 (socket says 172.21.15.38:36002) 2024-09-16T00:23:17.570 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:15.503+0000 7fc1db7fe640 1 -- 172.21.15.38:0/865242880 learned_addr learned my addr 172.21.15.38:0/865242880 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:23:17.570 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:15.504+0000 7fc1db7fe640 1 -- 172.21.15.38:0/865242880 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1dc19dd00 con 0x7fc1dc07e640 2024-09-16T00:23:17.570 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:15.504+0000 7fc1db7fe640 1 --2- 172.21.15.38:0/865242880 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1dc07e640 0x7fc1dc19d7c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc0097f0 tx=0x7fc1cc0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:17.570 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:15.504+0000 7fc1d97fa640 1 -- 172.21.15.38:0/865242880 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc1cc037040 con 0x7fc1dc07e640 2024-09-16T00:23:17.570 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:15.504+0000 7fc1d97fa640 1 -- 172.21.15.38:0/865242880 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc1cc030cc0 con 0x7fc1dc07e640 2024-09-16T00:23:17.570 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:15.504+0000 7fc1e1a6a640 1 -- 172.21.15.38:0/865242880 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1dc19df90 con 0x7fc1dc07e640 2024-09-16T00:23:17.571 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:15.504+0000 7fc1d97fa640 1 -- 172.21.15.38:0/865242880 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc1cc03a6b0 con 0x7fc1dc07e640 2024-09-16T00:23:17.571 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:15.504+0000 7fc1e1a6a640 1 -- 172.21.15.38:0/865242880 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1dc1a0c80 con 0x7fc1dc07e640 2024-09-16T00:23:17.571 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:15.505+0000 7fc1e1a6a640 1 -- 172.21.15.38:0/865242880 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1dc07cb70 con 0x7fc1dc07e640 2024-09-16T00:23:17.571 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:15.505+0000 7fc1d97fa640 1 -- 172.21.15.38:0/865242880 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 8) ==== 46461+0+0 (secure 0 0 0) 0x7fc1cc03baf0 con 0x7fc1dc07e640 2024-09-16T00:23:17.571 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:15.505+0000 7fc1d97fa640 1 --2- 172.21.15.38:0/865242880 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7fc1c8039200 0x7fc1c803b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:17.571 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:15.506+0000 7fc1d97fa640 1 -- 172.21.15.38:0/865242880 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fc1cc06c7a0 con 0x7fc1dc07e640 2024-09-16T00:23:17.571 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:15.506+0000 7fc1d3fff640 1 --2- 172.21.15.38:0/865242880 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7fc1c8039200 0x7fc1c803b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:17.571 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:15.506+0000 7fc1d3fff640 1 --2- 172.21.15.38:0/865242880 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7fc1c8039200 0x7fc1c803b6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc1c40098c0 tx=0x7fc1c4006db0 comp rx=0 tx=0).ready entity=mgr.14118 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:17.571 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:15.510+0000 7fc1d97fa640 1 -- 172.21.15.38:0/865242880 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc1cc042dc0 con 0x7fc1dc07e640 2024-09-16T00:23:17.571 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:15.739+0000 7fc1e1a6a640 1 -- 172.21.15.38:0/865242880 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7fc1dc1a0e50 con 0x7fc1dc07e640 2024-09-16T00:23:17.571 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:16.327+0000 7fc1d97fa640 1 -- 172.21.15.38:0/865242880 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7fc1cc03b4a0 con 0x7fc1dc07e640 2024-09-16T00:23:17.571 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:16.329+0000 7fc1d97fa640 1 -- 172.21.15.38:0/865242880 <== mon.0 v2:172.21.15.38:3300/0 8 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7fc1cc03a850 con 0x7fc1dc07e640 2024-09-16T00:23:17.571 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:16.330+0000 7fc1e1a6a640 1 -- 172.21.15.38:0/865242880 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7fc1c8039200 msgr2=0x7fc1c803b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:17.572 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:16.330+0000 7fc1e1a6a640 1 --2- 172.21.15.38:0/865242880 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7fc1c8039200 0x7fc1c803b6c0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc1c40098c0 tx=0x7fc1c4006db0 comp rx=0 tx=0).stop 2024-09-16T00:23:17.572 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:16.330+0000 7fc1e1a6a640 1 -- 172.21.15.38:0/865242880 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1dc07e640 msgr2=0x7fc1dc19d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:17.572 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:16.330+0000 7fc1e1a6a640 1 --2- 172.21.15.38:0/865242880 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1dc07e640 0x7fc1dc19d7c0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc1cc0097f0 tx=0x7fc1cc0327b0 comp rx=0 tx=0).stop 2024-09-16T00:23:17.572 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:16.330+0000 7fc1e1a6a640 1 -- 172.21.15.38:0/865242880 shutdown_connections 2024-09-16T00:23:17.572 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:16.330+0000 7fc1e1a6a640 1 --2- 172.21.15.38:0/865242880 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7fc1c8039200 0x7fc1c803b6c0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:17.572 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:16.330+0000 7fc1e1a6a640 1 --2- 172.21.15.38:0/865242880 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc1dc07e640 0x7fc1dc19d7c0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:17.572 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:16.330+0000 7fc1e1a6a640 1 -- 172.21.15.38:0/865242880 >> 172.21.15.38:0/865242880 conn(0x7fc1dc102e30 msgr2=0x7fc1dc103970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:17.572 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:16.330+0000 7fc1e1a6a640 1 -- 172.21.15.38:0/865242880 shutdown_connections 2024-09-16T00:23:17.572 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:16.330+0000 7fc1e1a6a640 1 -- 172.21.15.38:0/865242880 wait complete. 2024-09-16T00:23:17.653 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:17 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/865242880' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-16T00:23:17.653 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:17 smithi038 ceph-mon[30945]: mgrmap e9: x(active, since 19s) 2024-09-16T00:23:17.653 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:17 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:23:17.402+0000 7f1bf5b0a040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T00:23:17.653 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:17 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:23:17.466+0000 7f1bf5b0a040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T00:23:17.906 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:17 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:23:17.663+0000 7f1bf5b0a040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T00:23:18.459 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:18 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/874816139' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-16T00:23:18.715 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:18 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:23:18.615+0000 7f1bf5b0a040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T00:23:18.715 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:18 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:23:18.686+0000 7f1bf5b0a040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T00:23:19.045 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-09-16T00:23:19.045 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-16T00:23:19.045 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-16T00:23:19.046 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-16T00:23:19.046 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-16T00:23:19.046 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-09-16T00:23:19.046 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:17.985+0000 7f4917fb8640 1 Processor -- start 2024-09-16T00:23:19.046 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:17.985+0000 7f4917fb8640 1 -- start start 2024-09-16T00:23:19.046 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:17.985+0000 7f4917fb8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4910108100 0x7f491010a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:19.046 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:17.985+0000 7f4917fb8640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f491007c6b0 con 0x7f4910108100 2024-09-16T00:23:19.046 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:17.985+0000 7f4915d54640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4910108100 0x7f491010a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:19.046 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:17.985+0000 7f4915d54640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4910108100 0x7f491010a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40136/0 (socket says 172.21.15.38:40136) 2024-09-16T00:23:19.046 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:17.985+0000 7f4915d54640 1 -- 172.21.15.38:0/4115708588 learned_addr learned my addr 172.21.15.38:0/4115708588 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:23:19.046 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:17.986+0000 7f4915d54640 1 -- 172.21.15.38:0/4115708588 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f491010aa30 con 0x7f4910108100 2024-09-16T00:23:19.046 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:17.986+0000 7f4915d54640 1 --2- 172.21.15.38:0/4115708588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4910108100 0x7f491010a4f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f490c009820 tx=0x7f490c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71556c807938db24 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:19.047 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:17.987+0000 7f4914d52640 1 -- 172.21.15.38:0/4115708588 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f490c037070 con 0x7f4910108100 2024-09-16T00:23:19.047 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:17.987+0000 7f4914d52640 1 -- 172.21.15.38:0/4115708588 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f490c032920 con 0x7f4910108100 2024-09-16T00:23:19.047 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:17.987+0000 7f4914d52640 1 -- 172.21.15.38:0/4115708588 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f490c030810 con 0x7f4910108100 2024-09-16T00:23:19.047 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:17.987+0000 7f4917fb8640 1 -- 172.21.15.38:0/4115708588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4910108100 msgr2=0x7f491010a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:19.047 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:17.987+0000 7f4917fb8640 1 --2- 172.21.15.38:0/4115708588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4910108100 0x7f491010a4f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f490c009820 tx=0x7f490c02a270 comp rx=0 tx=0).stop 2024-09-16T00:23:19.047 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:17.988+0000 7f4917fb8640 1 -- 172.21.15.38:0/4115708588 shutdown_connections 2024-09-16T00:23:19.047 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:17.988+0000 7f4917fb8640 1 --2- 172.21.15.38:0/4115708588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4910108100 0x7f491010a4f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:19.047 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:17.988+0000 7f4917fb8640 1 -- 172.21.15.38:0/4115708588 >> 172.21.15.38:0/4115708588 conn(0x7f4910101cc0 msgr2=0x7f49101040e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:19.047 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:17.988+0000 7f4917fb8640 1 -- 172.21.15.38:0/4115708588 shutdown_connections 2024-09-16T00:23:19.047 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:17.988+0000 7f4917fb8640 1 -- 172.21.15.38:0/4115708588 wait complete. 2024-09-16T00:23:19.047 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:17.988+0000 7f4917fb8640 1 Processor -- start 2024-09-16T00:23:19.047 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:17.988+0000 7f4917fb8640 1 -- start start 2024-09-16T00:23:19.048 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:17.989+0000 7f4917fb8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4910108100 0x7f4910107980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:19.048 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:17.989+0000 7f4917fb8640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f491010ba50 con 0x7f4910108100 2024-09-16T00:23:19.048 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:17.989+0000 7f4915d54640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4910108100 0x7f4910107980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:19.048 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:17.989+0000 7f4915d54640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4910108100 0x7f4910107980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40148/0 (socket says 172.21.15.38:40148) 2024-09-16T00:23:19.048 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:17.989+0000 7f4915d54640 1 -- 172.21.15.38:0/874816139 learned_addr learned my addr 172.21.15.38:0/874816139 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:23:19.048 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:17.989+0000 7f4915d54640 1 -- 172.21.15.38:0/874816139 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4910107ec0 con 0x7f4910108100 2024-09-16T00:23:19.048 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:17.989+0000 7f4915d54640 1 --2- 172.21.15.38:0/874816139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4910108100 0x7f4910107980 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f490c0097f0 tx=0x7f490c0327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:19.048 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:17.989+0000 7f4906ffd640 1 -- 172.21.15.38:0/874816139 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f490c037040 con 0x7f4910108100 2024-09-16T00:23:19.048 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:17.989+0000 7f4906ffd640 1 -- 172.21.15.38:0/874816139 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f490c030cc0 con 0x7f4910108100 2024-09-16T00:23:19.048 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:17.989+0000 7f4917fb8640 1 -- 172.21.15.38:0/874816139 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49101060b0 con 0x7f4910108100 2024-09-16T00:23:19.048 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:17.989+0000 7f4906ffd640 1 -- 172.21.15.38:0/874816139 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f490c03a6b0 con 0x7f4910108100 2024-09-16T00:23:19.048 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:17.989+0000 7f4917fb8640 1 -- 172.21.15.38:0/874816139 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49101065f0 con 0x7f4910108100 2024-09-16T00:23:19.048 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:17.990+0000 7f4917fb8640 1 -- 172.21.15.38:0/874816139 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4910108560 con 0x7f4910108100 2024-09-16T00:23:19.049 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:17.991+0000 7f4906ffd640 1 -- 172.21.15.38:0/874816139 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f490c03a8a0 con 0x7f4910108100 2024-09-16T00:23:19.049 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:17.991+0000 7f4906ffd640 1 --2- 172.21.15.38:0/874816139 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f48fc039200 0x7f48fc03b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:19.049 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:17.991+0000 7f4906ffd640 1 -- 172.21.15.38:0/874816139 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f490c06c970 con 0x7f4910108100 2024-09-16T00:23:19.049 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:17.991+0000 7f4915553640 1 -- 172.21.15.38:0/874816139 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f48fc039200 msgr2=0x7f48fc03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/1779324916 2024-09-16T00:23:19.049 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:17.991+0000 7f4915553640 1 --2- 172.21.15.38:0/874816139 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f48fc039200 0x7f48fc03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T00:23:19.049 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:17.996+0000 7f4906ffd640 1 -- 172.21.15.38:0/874816139 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f490c03ace0 con 0x7f4910108100 2024-09-16T00:23:19.049 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:18.192+0000 7f4915553640 1 -- 172.21.15.38:0/874816139 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f48fc039200 msgr2=0x7f48fc03b6c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/1779324916 2024-09-16T00:23:19.049 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:18.192+0000 7f4915553640 1 --2- 172.21.15.38:0/874816139 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f48fc039200 0x7f48fc03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T00:23:19.049 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:18.224+0000 7f4917fb8640 1 -- 172.21.15.38:0/874816139 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7f4910106870 con 0x7f4910108100 2024-09-16T00:23:19.049 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:18.225+0000 7f4906ffd640 1 -- 172.21.15.38:0/874816139 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7f490c044400 con 0x7f4910108100 2024-09-16T00:23:19.049 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:18.227+0000 7f4917fb8640 1 -- 172.21.15.38:0/874816139 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f48fc039200 msgr2=0x7f48fc03b6c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:23:19.049 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:18.227+0000 7f4917fb8640 1 --2- 172.21.15.38:0/874816139 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f48fc039200 0x7f48fc03b6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:19.050 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:18.227+0000 7f4917fb8640 1 -- 172.21.15.38:0/874816139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4910108100 msgr2=0x7f4910107980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:19.050 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:18.227+0000 7f4917fb8640 1 --2- 172.21.15.38:0/874816139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4910108100 0x7f4910107980 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f490c0097f0 tx=0x7f490c0327b0 comp rx=0 tx=0).stop 2024-09-16T00:23:19.050 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:18.227+0000 7f4917fb8640 1 -- 172.21.15.38:0/874816139 shutdown_connections 2024-09-16T00:23:19.050 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:18.227+0000 7f4917fb8640 1 --2- 172.21.15.38:0/874816139 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f48fc039200 0x7f48fc03b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:19.050 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:18.227+0000 7f4917fb8640 1 --2- 172.21.15.38:0/874816139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4910108100 0x7f4910107980 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:19.050 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:18.227+0000 7f4917fb8640 1 -- 172.21.15.38:0/874816139 >> 172.21.15.38:0/874816139 conn(0x7f4910101cc0 msgr2=0x7f4910102800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:19.050 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:18.227+0000 7f4917fb8640 1 -- 172.21.15.38:0/874816139 shutdown_connections 2024-09-16T00:23:19.050 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:18.227+0000 7f4917fb8640 1 -- 172.21.15.38:0/874816139 wait complete. 2024-09-16T00:23:19.050 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for the mgr to restart... 2024-09-16T00:23:19.050 INFO:teuthology.orchestra.run.smithi038.stdout:Waiting for mgr epoch 9... 2024-09-16T00:23:19.635 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:19 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:23:19.447+0000 7f1bf5b0a040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T00:23:19.636 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:19 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:23:19.568+0000 7f1bf5b0a040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T00:23:19.940 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:19 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:23:19.718+0000 7f1bf5b0a040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:23:19.940 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:19 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:23:19.782+0000 7f1bf5b0a040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T00:23:19.940 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:19 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:23:19.938+0000 7f1bf5b0a040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T00:23:20.215 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:20 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:23:20.009+0000 7f1bf5b0a040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T00:23:20.215 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:20 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:23:20.072+0000 7f1bf5b0a040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T00:23:20.525 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:20 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:23:20.238+0000 7f1bf5b0a040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T00:23:20.526 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:20 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:23:20.306+0000 7f1bf5b0a040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T00:23:20.526 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:20 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:23:20.375+0000 7f1bf5b0a040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T00:23:20.526 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:20 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:23:20.461+0000 7f1bf5b0a040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T00:23:20.526 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:20 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:23:20.524+0000 7f1bf5b0a040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T00:23:20.837 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:20 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:23:20.587+0000 7f1bf5b0a040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T00:23:21.668 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:21 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:23:21.415+0000 7f1bf5b0a040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T00:23:21.669 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:21 smithi038 ceph-mon[30945]: Active manager daemon x restarted 2024-09-16T00:23:21.669 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:21 smithi038 ceph-mon[30945]: Activating manager daemon x 2024-09-16T00:23:21.669 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:21 smithi038 ceph-mon[30945]: osdmap e3: 0 total, 0 up, 0 in 2024-09-16T00:23:21.669 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:21 smithi038 ceph-mon[30945]: mgrmap e10: x(active, starting, since 0.00852535s) 2024-09-16T00:23:21.669 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:21 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:23:21.669 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:21 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-16T00:23:21.669 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:21 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-16T00:23:21.669 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:21 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-16T00:23:21.669 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:21 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-16T00:23:21.669 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:21 smithi038 ceph-mon[30945]: Manager daemon x is now available 2024-09-16T00:23:21.669 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:21 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:23:21.669 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:21 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-09-16T00:23:21.670 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:21 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-09-16T00:23:23.374 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:23 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:23:23.267+0000 7f1b60e1e640 -1 log_channel(cephadm) log [ERR] : [16/Sep/2024:00:23:23] ENGINE Error in HTTPServer.serve 2024-09-16T00:23:23.374 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:23 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: Traceback (most recent call last): 2024-09-16T00:23:23.374 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:23 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T00:23:23.374 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:23 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: self._connections.run(self.expiration_interval) 2024-09-16T00:23:23.375 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:23 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T00:23:23.375 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:23 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: self._run(expiration_interval) 2024-09-16T00:23:23.375 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:23 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T00:23:23.375 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:23 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T00:23:23.375 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:23 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T00:23:23.375 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:23 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T00:23:23.375 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:23 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T00:23:23.375 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:23 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: s = self.context.wrap_socket( 2024-09-16T00:23:23.375 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:23 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T00:23:23.375 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:23 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: return self.sslsocket_class._create( 2024-09-16T00:23:23.375 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:23 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T00:23:23.375 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:23 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: self.do_handshake() 2024-09-16T00:23:23.375 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:23 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T00:23:23.376 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:23 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: self._sslobj.do_handshake() 2024-09-16T00:23:23.376 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:23 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T00:23:23.376 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:23:23 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:23:23.386 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout { 2024-09-16T00:23:23.386 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-16T00:23:23.386 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-16T00:23:23.386 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout } 2024-09-16T00:23:23.387 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:19.427+0000 7f39b0d84640 1 Processor -- start 2024-09-16T00:23:23.387 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:19.427+0000 7f39b0d84640 1 -- start start 2024-09-16T00:23:23.387 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:19.427+0000 7f39b0d84640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39ac108050 0x7f39ac10a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:23.387 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:19.427+0000 7f39b0d84640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f39ac07bea0 con 0x7f39ac108050 2024-09-16T00:23:23.387 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:19.428+0000 7f39aa59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39ac108050 0x7f39ac10a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:23.387 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:19.428+0000 7f39aa59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39ac108050 0x7f39ac10a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40152/0 (socket says 172.21.15.38:40152) 2024-09-16T00:23:23.387 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:19.428+0000 7f39aa59c640 1 -- 172.21.15.38:0/1818555950 learned_addr learned my addr 172.21.15.38:0/1818555950 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:23:23.387 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:19.428+0000 7f39aa59c640 1 -- 172.21.15.38:0/1818555950 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39ac10a980 con 0x7f39ac108050 2024-09-16T00:23:23.388 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:19.429+0000 7f39aa59c640 1 --2- 172.21.15.38:0/1818555950 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39ac108050 0x7f39ac10a440 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f399c009820 tx=0x7f399c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb60d9b7596b36a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:23.388 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:19.429+0000 7f39a9d9b640 1 -- 172.21.15.38:0/1818555950 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f399c037070 con 0x7f39ac108050 2024-09-16T00:23:23.388 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:19.429+0000 7f39a9d9b640 1 -- 172.21.15.38:0/1818555950 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f399c032920 con 0x7f39ac108050 2024-09-16T00:23:23.388 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:19.429+0000 7f39a9d9b640 1 -- 172.21.15.38:0/1818555950 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f399c030810 con 0x7f39ac108050 2024-09-16T00:23:23.388 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:19.429+0000 7f39b0d84640 1 -- 172.21.15.38:0/1818555950 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39ac108050 msgr2=0x7f39ac10a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:23.388 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:19.429+0000 7f39b0d84640 1 --2- 172.21.15.38:0/1818555950 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39ac108050 0x7f39ac10a440 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f399c009820 tx=0x7f399c02a270 comp rx=0 tx=0).stop 2024-09-16T00:23:23.388 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:19.430+0000 7f39b0d84640 1 -- 172.21.15.38:0/1818555950 shutdown_connections 2024-09-16T00:23:23.388 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:19.430+0000 7f39b0d84640 1 --2- 172.21.15.38:0/1818555950 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39ac108050 0x7f39ac10a440 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:23.389 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:19.430+0000 7f39b0d84640 1 -- 172.21.15.38:0/1818555950 >> 172.21.15.38:0/1818555950 conn(0x7f39ac101c30 msgr2=0x7f39ac1040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:23.389 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:19.430+0000 7f39b0d84640 1 -- 172.21.15.38:0/1818555950 shutdown_connections 2024-09-16T00:23:23.389 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:19.430+0000 7f39b0d84640 1 -- 172.21.15.38:0/1818555950 wait complete. 2024-09-16T00:23:23.389 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:19.430+0000 7f39b0d84640 1 Processor -- start 2024-09-16T00:23:23.389 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:19.430+0000 7f39b0d84640 1 -- start start 2024-09-16T00:23:23.389 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:19.430+0000 7f39b0d84640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39ac108050 0x7f39ac19b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:23.389 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:19.430+0000 7f39b0d84640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f39ac10ba60 con 0x7f39ac108050 2024-09-16T00:23:23.389 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:19.431+0000 7f39aa59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39ac108050 0x7f39ac19b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:23.390 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:19.431+0000 7f39aa59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39ac108050 0x7f39ac19b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40160/0 (socket says 172.21.15.38:40160) 2024-09-16T00:23:23.390 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:19.431+0000 7f39aa59c640 1 -- 172.21.15.38:0/4285975360 learned_addr learned my addr 172.21.15.38:0/4285975360 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:23:23.390 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:19.431+0000 7f39aa59c640 1 -- 172.21.15.38:0/4285975360 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39ac19bb40 con 0x7f39ac108050 2024-09-16T00:23:23.390 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:19.431+0000 7f39aa59c640 1 --2- 172.21.15.38:0/4285975360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39ac108050 0x7f39ac19b600 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f399c030420 tx=0x7f399c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:23.390 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:19.431+0000 7f39a37fe640 1 -- 172.21.15.38:0/4285975360 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f399c037040 con 0x7f39ac108050 2024-09-16T00:23:23.390 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:19.431+0000 7f39a37fe640 1 -- 172.21.15.38:0/4285975360 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f399c040070 con 0x7f39ac108050 2024-09-16T00:23:23.390 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:19.431+0000 7f39b0d84640 1 -- 172.21.15.38:0/4285975360 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39ac19bdd0 con 0x7f39ac108050 2024-09-16T00:23:23.390 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:19.431+0000 7f39a37fe640 1 -- 172.21.15.38:0/4285975360 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f399c03a6e0 con 0x7f39ac108050 2024-09-16T00:23:23.390 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:19.431+0000 7f39b0d84640 1 -- 172.21.15.38:0/4285975360 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39ac19eac0 con 0x7f39ac108050 2024-09-16T00:23:23.391 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:19.432+0000 7f39a37fe640 1 -- 172.21.15.38:0/4285975360 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 9) ==== 46474+0+0 (secure 0 0 0) 0x7f399c03a990 con 0x7f39ac108050 2024-09-16T00:23:23.391 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:19.433+0000 7f39a37fe640 1 --2- 172.21.15.38:0/4285975360 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f3998039260 0x7f399803b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:23.391 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:19.433+0000 7f39a37fe640 1 -- 172.21.15.38:0/4285975360 --> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f399803be30 con 0x7f3998039260 2024-09-16T00:23:23.391 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:19.433+0000 7f39a3fff640 1 -- 172.21.15.38:0/4285975360 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f3998039260 msgr2=0x7f399803b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/1779324916 2024-09-16T00:23:23.391 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:19.433+0000 7f39a37fe640 1 -- 172.21.15.38:0/4285975360 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f399c06c890 con 0x7f39ac108050 2024-09-16T00:23:23.391 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:19.433+0000 7f39a3fff640 1 --2- 172.21.15.38:0/4285975360 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f3998039260 0x7f399803b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T00:23:23.391 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:19.634+0000 7f39a3fff640 1 -- 172.21.15.38:0/4285975360 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f3998039260 msgr2=0x7f399803b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/1779324916 2024-09-16T00:23:23.391 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:19.634+0000 7f39a3fff640 1 --2- 172.21.15.38:0/4285975360 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f3998039260 0x7f399803b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-16T00:23:23.392 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:20.035+0000 7f39a3fff640 1 -- 172.21.15.38:0/4285975360 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f3998039260 msgr2=0x7f399803b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/1779324916 2024-09-16T00:23:23.392 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:20.035+0000 7f39a3fff640 1 --2- 172.21.15.38:0/4285975360 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f3998039260 0x7f399803b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-16T00:23:23.392 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:20.837+0000 7f39a3fff640 1 -- 172.21.15.38:0/4285975360 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f3998039260 msgr2=0x7f399803b720 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.38:6800/1779324916 2024-09-16T00:23:23.392 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:20.837+0000 7f39a3fff640 1 --2- 172.21.15.38:0/4285975360 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f3998039260 0x7f399803b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-16T00:23:23.392 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:21.423+0000 7f39a37fe640 1 -- 172.21.15.38:0/4285975360 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mgrmap(e 10) ==== 46241+0+0 (secure 0 0 0) 0x7f399c06b340 con 0x7f39ac108050 2024-09-16T00:23:23.392 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:21.423+0000 7f39a37fe640 1 -- 172.21.15.38:0/4285975360 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f3998039260 msgr2=0x7f399803b720 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:23:23.392 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:21.423+0000 7f39a37fe640 1 --2- 172.21.15.38:0/4285975360 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f3998039260 0x7f399803b720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:23.393 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:22.420+0000 7f39a37fe640 1 -- 172.21.15.38:0/4285975360 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mgrmap(e 11) ==== 46368+0+0 (secure 0 0 0) 0x7f399c06e440 con 0x7f39ac108050 2024-09-16T00:23:23.393 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:22.420+0000 7f39a37fe640 1 --2- 172.21.15.38:0/4285975360 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f399803cd20 0x7f399803f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:23.393 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:22.421+0000 7f39a37fe640 1 -- 172.21.15.38:0/4285975360 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f399c031ad0 con 0x7f399803cd20 2024-09-16T00:23:23.393 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:22.421+0000 7f39a3fff640 1 --2- 172.21.15.38:0/4285975360 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f399803cd20 0x7f399803f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:23.393 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:22.422+0000 7f39a3fff640 1 --2- 172.21.15.38:0/4285975360 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f399803cd20 0x7f399803f110 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3994003980 tx=0x7f39940092b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:23.393 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:22.422+0000 7f39a37fe640 1 -- 172.21.15.38:0/4285975360 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f399c031ad0 con 0x7f399803cd20 2024-09-16T00:23:23.393 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:22.429+0000 7f39b0d84640 1 -- 172.21.15.38:0/4285975360 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f39ac1084b0 con 0x7f399803cd20 2024-09-16T00:23:23.393 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:22.429+0000 7f39a37fe640 1 -- 172.21.15.38:0/4285975360 <== mgr.14150 v2:172.21.15.38:6800/1461245815 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f39ac1084b0 con 0x7f399803cd20 2024-09-16T00:23:23.394 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:22.430+0000 7f39b0d84640 1 -- 172.21.15.38:0/4285975360 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f399803cd20 msgr2=0x7f399803f110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:23.394 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:22.430+0000 7f39b0d84640 1 --2- 172.21.15.38:0/4285975360 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f399803cd20 0x7f399803f110 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3994003980 tx=0x7f39940092b0 comp rx=0 tx=0).stop 2024-09-16T00:23:23.394 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:22.430+0000 7f39b0d84640 1 -- 172.21.15.38:0/4285975360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39ac108050 msgr2=0x7f39ac19b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:23.394 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:22.430+0000 7f39b0d84640 1 --2- 172.21.15.38:0/4285975360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39ac108050 0x7f39ac19b600 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f399c030420 tx=0x7f399c032670 comp rx=0 tx=0).stop 2024-09-16T00:23:23.394 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:22.430+0000 7f39b0d84640 1 -- 172.21.15.38:0/4285975360 shutdown_connections 2024-09-16T00:23:23.394 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:22.430+0000 7f39b0d84640 1 --2- 172.21.15.38:0/4285975360 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f399803cd20 0x7f399803f110 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:23.394 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:22.430+0000 7f39b0d84640 1 --2- 172.21.15.38:0/4285975360 >> [v2:172.21.15.38:6800/1779324916,v1:172.21.15.38:6801/1779324916] conn(0x7f3998039260 0x7f399803b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:23.394 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:22.430+0000 7f39b0d84640 1 --2- 172.21.15.38:0/4285975360 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f39ac108050 0x7f39ac19b600 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:23.394 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:22.430+0000 7f39b0d84640 1 -- 172.21.15.38:0/4285975360 >> 172.21.15.38:0/4285975360 conn(0x7f39ac101c30 msgr2=0x7f39ac102800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:23.395 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:22.430+0000 7f39b0d84640 1 -- 172.21.15.38:0/4285975360 shutdown_connections 2024-09-16T00:23:23.395 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:22.430+0000 7f39b0d84640 1 -- 172.21.15.38:0/4285975360 wait complete. 2024-09-16T00:23:23.395 INFO:teuthology.orchestra.run.smithi038.stdout:mgr epoch 9 is available 2024-09-16T00:23:23.395 INFO:teuthology.orchestra.run.smithi038.stdout:Generating a dashboard self-signed certificate... 2024-09-16T00:23:23.626 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:23 smithi038 ceph-mon[30945]: mgrmap e11: x(active, since 1.00623s) 2024-09-16T00:23:23.626 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:23 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:23.627 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:23 smithi038 ceph-mon[30945]: [16/Sep/2024:00:23:23] ENGINE Bus STARTING 2024-09-16T00:23:23.627 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:23 smithi038 ceph-mon[30945]: [16/Sep/2024:00:23:23] ENGINE Serving on https://172.21.15.38:7150 2024-09-16T00:23:23.627 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:23 smithi038 ceph-mon[30945]: [16/Sep/2024:00:23:23] ENGINE Error in HTTPServer.serve 2024-09-16T00:23:23.627 INFO:journalctl@ceph.mon.a.smithi038.stdout: Traceback (most recent call last): 2024-09-16T00:23:23.627 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-16T00:23:23.627 INFO:journalctl@ceph.mon.a.smithi038.stdout: self._connections.run(self.expiration_interval) 2024-09-16T00:23:23.627 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-16T00:23:23.627 INFO:journalctl@ceph.mon.a.smithi038.stdout: self._run(expiration_interval) 2024-09-16T00:23:23.627 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-16T00:23:23.627 INFO:journalctl@ceph.mon.a.smithi038.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-16T00:23:23.627 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-16T00:23:23.628 INFO:journalctl@ceph.mon.a.smithi038.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-16T00:23:23.628 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-16T00:23:23.628 INFO:journalctl@ceph.mon.a.smithi038.stdout: s = self.context.wrap_socket( 2024-09-16T00:23:23.628 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-16T00:23:23.628 INFO:journalctl@ceph.mon.a.smithi038.stdout: return self.sslsocket_class._create( 2024-09-16T00:23:23.628 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-16T00:23:23.628 INFO:journalctl@ceph.mon.a.smithi038.stdout: self.do_handshake() 2024-09-16T00:23:23.628 INFO:journalctl@ceph.mon.a.smithi038.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-16T00:23:23.628 INFO:journalctl@ceph.mon.a.smithi038.stdout: self._sslobj.do_handshake() 2024-09-16T00:23:23.628 INFO:journalctl@ceph.mon.a.smithi038.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-16T00:23:23.628 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:23 smithi038 ceph-mon[30945]: [16/Sep/2024:00:23:23] ENGINE Serving on http://172.21.15.38:8765 2024-09-16T00:23:23.628 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:23 smithi038 ceph-mon[30945]: [16/Sep/2024:00:23:23] ENGINE Bus STARTED 2024-09-16T00:23:24.747 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-16T00:23:24.747 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:23.791+0000 7f70cfe18640 1 Processor -- start 2024-09-16T00:23:24.747 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:23.791+0000 7f70cfe18640 1 -- start start 2024-09-16T00:23:24.747 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:23.791+0000 7f70cfe18640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70c8107620 0x7f70c8107a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:24.747 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:23.791+0000 7f70cfe18640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f70c8107ff0 con 0x7f70c8107620 2024-09-16T00:23:24.747 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:23.791+0000 7f70cdbb4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70c8107620 0x7f70c8107a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:24.747 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:23.791+0000 7f70cdbb4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70c8107620 0x7f70c8107a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40236/0 (socket says 172.21.15.38:40236) 2024-09-16T00:23:24.748 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:23.791+0000 7f70cdbb4640 1 -- 172.21.15.38:0/3862868943 learned_addr learned my addr 172.21.15.38:0/3862868943 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:23:24.748 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:23.792+0000 7f70cdbb4640 1 -- 172.21.15.38:0/3862868943 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c8108820 con 0x7f70c8107620 2024-09-16T00:23:24.748 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:23.792+0000 7f70cdbb4640 1 --2- 172.21.15.38:0/3862868943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70c8107620 0x7f70c8107a20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f70c4009820 tx=0x7f70c402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9f0b26c0ae436d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:24.748 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:23.792+0000 7f70ccbb2640 1 -- 172.21.15.38:0/3862868943 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70c4037070 con 0x7f70c8107620 2024-09-16T00:23:24.748 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:23.792+0000 7f70ccbb2640 1 -- 172.21.15.38:0/3862868943 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70c4032920 con 0x7f70c8107620 2024-09-16T00:23:24.748 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:23.792+0000 7f70ccbb2640 1 -- 172.21.15.38:0/3862868943 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70c4030810 con 0x7f70c8107620 2024-09-16T00:23:24.748 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:23.793+0000 7f70cfe18640 1 -- 172.21.15.38:0/3862868943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70c8107620 msgr2=0x7f70c8107a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:24.748 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:23.793+0000 7f70cfe18640 1 --2- 172.21.15.38:0/3862868943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70c8107620 0x7f70c8107a20 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f70c4009820 tx=0x7f70c402a270 comp rx=0 tx=0).stop 2024-09-16T00:23:24.748 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:23.793+0000 7f70cfe18640 1 -- 172.21.15.38:0/3862868943 shutdown_connections 2024-09-16T00:23:24.748 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:23.793+0000 7f70cfe18640 1 --2- 172.21.15.38:0/3862868943 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70c8107620 0x7f70c8107a20 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:24.748 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:23.793+0000 7f70cfe18640 1 -- 172.21.15.38:0/3862868943 >> 172.21.15.38:0/3862868943 conn(0x7f70c8102ea0 msgr2=0x7f70c8105290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:24.748 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:23.793+0000 7f70cfe18640 1 -- 172.21.15.38:0/3862868943 shutdown_connections 2024-09-16T00:23:24.748 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:23.793+0000 7f70cfe18640 1 -- 172.21.15.38:0/3862868943 wait complete. 2024-09-16T00:23:24.749 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:23.794+0000 7f70cfe18640 1 Processor -- start 2024-09-16T00:23:24.749 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:23.794+0000 7f70cfe18640 1 -- start start 2024-09-16T00:23:24.749 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:23.794+0000 7f70cfe18640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70c8107620 0x7f70c8199440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:24.749 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:23.794+0000 7f70cfe18640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f70c807cae0 con 0x7f70c8107620 2024-09-16T00:23:24.749 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:23.794+0000 7f70cdbb4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70c8107620 0x7f70c8199440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:24.749 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:23.794+0000 7f70cdbb4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70c8107620 0x7f70c8199440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40250/0 (socket says 172.21.15.38:40250) 2024-09-16T00:23:24.749 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:23.794+0000 7f70cdbb4640 1 -- 172.21.15.38:0/537474695 learned_addr learned my addr 172.21.15.38:0/537474695 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:23:24.749 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:23.794+0000 7f70cdbb4640 1 -- 172.21.15.38:0/537474695 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c8199980 con 0x7f70c8107620 2024-09-16T00:23:24.749 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:23.794+0000 7f70cdbb4640 1 --2- 172.21.15.38:0/537474695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70c8107620 0x7f70c8199440 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f70c40097f0 tx=0x7f70c40327b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:24.749 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:23.795+0000 7f70beffd640 1 -- 172.21.15.38:0/537474695 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70c4037040 con 0x7f70c8107620 2024-09-16T00:23:24.749 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:23.795+0000 7f70beffd640 1 -- 172.21.15.38:0/537474695 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70c4030cc0 con 0x7f70c8107620 2024-09-16T00:23:24.749 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:23.795+0000 7f70cfe18640 1 -- 172.21.15.38:0/537474695 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70c8199c10 con 0x7f70c8107620 2024-09-16T00:23:24.749 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:23.795+0000 7f70beffd640 1 -- 172.21.15.38:0/537474695 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f70c403a640 con 0x7f70c8107620 2024-09-16T00:23:24.749 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:23.795+0000 7f70cfe18640 1 -- 172.21.15.38:0/537474695 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70c819c900 con 0x7f70c8107620 2024-09-16T00:23:24.750 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:23.796+0000 7f70beffd640 1 -- 172.21.15.38:0/537474695 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7f70c403a8f0 con 0x7f70c8107620 2024-09-16T00:23:24.750 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:23.796+0000 7f70beffd640 1 --2- 172.21.15.38:0/537474695 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f70b403d6b0 0x7f70b403fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:24.750 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:23.796+0000 7f70cfe18640 1 -- 172.21.15.38:0/537474695 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70c8107a20 con 0x7f70c8107620 2024-09-16T00:23:24.750 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:23.796+0000 7f70beffd640 1 -- 172.21.15.38:0/537474695 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f70c406cae0 con 0x7f70c8107620 2024-09-16T00:23:24.750 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:23.796+0000 7f70cd3b3640 1 --2- 172.21.15.38:0/537474695 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f70b403d6b0 0x7f70b403fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:24.750 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:23.796+0000 7f70cd3b3640 1 --2- 172.21.15.38:0/537474695 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f70b403d6b0 0x7f70b403fb70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f70b80098c0 tx=0x7f70b8006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:24.750 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:23.801+0000 7f70beffd640 1 -- 172.21.15.38:0/537474695 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f70c402fd60 con 0x7f70c8107620 2024-09-16T00:23:24.750 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:24.004+0000 7f70cfe18640 1 -- 172.21.15.38:0/537474695 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f70c802da20 con 0x7f70b403d6b0 2024-09-16T00:23:24.750 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:24.096+0000 7f70beffd640 1 -- 172.21.15.38:0/537474695 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f70c802da20 con 0x7f70b403d6b0 2024-09-16T00:23:24.750 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:24.098+0000 7f70cfe18640 1 -- 172.21.15.38:0/537474695 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f70b403d6b0 msgr2=0x7f70b403fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:24.750 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:24.098+0000 7f70cfe18640 1 --2- 172.21.15.38:0/537474695 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f70b403d6b0 0x7f70b403fb70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f70b80098c0 tx=0x7f70b8006db0 comp rx=0 tx=0).stop 2024-09-16T00:23:24.750 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:24.098+0000 7f70cfe18640 1 -- 172.21.15.38:0/537474695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70c8107620 msgr2=0x7f70c8199440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:24.750 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:24.098+0000 7f70cfe18640 1 --2- 172.21.15.38:0/537474695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70c8107620 0x7f70c8199440 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f70c40097f0 tx=0x7f70c40327b0 comp rx=0 tx=0).stop 2024-09-16T00:23:24.751 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:24.099+0000 7f70cfe18640 1 -- 172.21.15.38:0/537474695 shutdown_connections 2024-09-16T00:23:24.751 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:24.099+0000 7f70cfe18640 1 --2- 172.21.15.38:0/537474695 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f70b403d6b0 0x7f70b403fb70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:24.751 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:24.099+0000 7f70cfe18640 1 --2- 172.21.15.38:0/537474695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70c8107620 0x7f70c8199440 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:24.751 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:24.099+0000 7f70cfe18640 1 -- 172.21.15.38:0/537474695 >> 172.21.15.38:0/537474695 conn(0x7f70c8102ea0 msgr2=0x7f70c8104110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:24.751 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:24.099+0000 7f70cfe18640 1 -- 172.21.15.38:0/537474695 shutdown_connections 2024-09-16T00:23:24.751 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:24.099+0000 7f70cfe18640 1 -- 172.21.15.38:0/537474695 wait complete. 2024-09-16T00:23:24.751 INFO:teuthology.orchestra.run.smithi038.stdout:Creating initial admin user... 2024-09-16T00:23:25.032 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:24 smithi038 ceph-mon[30945]: mgrmap e12: x(active, since 2s) 2024-09-16T00:23:25.032 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:24 smithi038 ceph-mon[30945]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:23:25.032 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:24 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:25.032 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:24 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:25.033 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:24 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:27.129 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:26 smithi038 ceph-mon[30945]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:23:27.129 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:26 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:27.136 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$vW7GRnxOSlbnA0e6B.uyPeo7eXoQhDyhAbItoqb6FL1WEkYACcDQa", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726446205, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-16T00:23:27.136 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.356+0000 7fcf38a4a640 1 Processor -- start 2024-09-16T00:23:27.137 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.356+0000 7fcf38a4a640 1 -- start start 2024-09-16T00:23:27.137 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.357+0000 7fcf38a4a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf341064e0 0x7fcf341068e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:27.137 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.357+0000 7fcf38a4a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcf34106eb0 con 0x7fcf341064e0 2024-09-16T00:23:27.137 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.357+0000 7fcf3259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf341064e0 0x7fcf341068e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:27.137 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.357+0000 7fcf3259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf341064e0 0x7fcf341068e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40266/0 (socket says 172.21.15.38:40266) 2024-09-16T00:23:27.137 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.357+0000 7fcf3259c640 1 -- 172.21.15.38:0/1067714946 learned_addr learned my addr 172.21.15.38:0/1067714946 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:23:27.137 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.357+0000 7fcf3259c640 1 -- 172.21.15.38:0/1067714946 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf34107690 con 0x7fcf341064e0 2024-09-16T00:23:27.137 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.358+0000 7fcf3259c640 1 --2- 172.21.15.38:0/1067714946 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf341064e0 0x7fcf341068e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcf24009820 tx=0x7fcf2402a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa2ba020a390ff4a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:27.138 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.358+0000 7fcf3159a640 1 -- 172.21.15.38:0/1067714946 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf24037070 con 0x7fcf341064e0 2024-09-16T00:23:27.138 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.358+0000 7fcf3159a640 1 -- 172.21.15.38:0/1067714946 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf24032920 con 0x7fcf341064e0 2024-09-16T00:23:27.138 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.358+0000 7fcf3159a640 1 -- 172.21.15.38:0/1067714946 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf24030810 con 0x7fcf341064e0 2024-09-16T00:23:27.138 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.359+0000 7fcf38a4a640 1 -- 172.21.15.38:0/1067714946 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf341064e0 msgr2=0x7fcf341068e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:27.138 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.359+0000 7fcf38a4a640 1 --2- 172.21.15.38:0/1067714946 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf341064e0 0x7fcf341068e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcf24009820 tx=0x7fcf2402a270 comp rx=0 tx=0).stop 2024-09-16T00:23:27.138 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.359+0000 7fcf38a4a640 1 -- 172.21.15.38:0/1067714946 shutdown_connections 2024-09-16T00:23:27.138 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.359+0000 7fcf38a4a640 1 --2- 172.21.15.38:0/1067714946 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf341064e0 0x7fcf341068e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:27.138 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.359+0000 7fcf38a4a640 1 -- 172.21.15.38:0/1067714946 >> 172.21.15.38:0/1067714946 conn(0x7fcf34101c30 msgr2=0x7fcf341040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:27.138 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.359+0000 7fcf38a4a640 1 -- 172.21.15.38:0/1067714946 shutdown_connections 2024-09-16T00:23:27.138 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.359+0000 7fcf38a4a640 1 -- 172.21.15.38:0/1067714946 wait complete. 2024-09-16T00:23:27.138 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.360+0000 7fcf38a4a640 1 Processor -- start 2024-09-16T00:23:27.138 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.360+0000 7fcf38a4a640 1 -- start start 2024-09-16T00:23:27.138 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.360+0000 7fcf38a4a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf341064e0 0x7fcf3419b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:27.139 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.360+0000 7fcf38a4a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcf3410aa40 con 0x7fcf341064e0 2024-09-16T00:23:27.139 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.361+0000 7fcf3259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf341064e0 0x7fcf3419b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:27.139 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.361+0000 7fcf3259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf341064e0 0x7fcf3419b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40282/0 (socket says 172.21.15.38:40282) 2024-09-16T00:23:27.139 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.361+0000 7fcf3259c640 1 -- 172.21.15.38:0/587491107 learned_addr learned my addr 172.21.15.38:0/587491107 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:23:27.139 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.361+0000 7fcf3259c640 1 -- 172.21.15.38:0/587491107 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf3419bb60 con 0x7fcf341064e0 2024-09-16T00:23:27.139 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.361+0000 7fcf3259c640 1 --2- 172.21.15.38:0/587491107 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf341064e0 0x7fcf3419b620 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcf24030560 tx=0x7fcf24032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:27.139 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.361+0000 7fcf237fe640 1 -- 172.21.15.38:0/587491107 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf24037040 con 0x7fcf341064e0 2024-09-16T00:23:27.139 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.361+0000 7fcf38a4a640 1 -- 172.21.15.38:0/587491107 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf3419bdf0 con 0x7fcf341064e0 2024-09-16T00:23:27.139 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.362+0000 7fcf38a4a640 1 -- 172.21.15.38:0/587491107 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf3419eae0 con 0x7fcf341064e0 2024-09-16T00:23:27.139 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.362+0000 7fcf237fe640 1 -- 172.21.15.38:0/587491107 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcf24040070 con 0x7fcf341064e0 2024-09-16T00:23:27.139 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.362+0000 7fcf237fe640 1 -- 172.21.15.38:0/587491107 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcf2403a740 con 0x7fcf341064e0 2024-09-16T00:23:27.139 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.363+0000 7fcf237fe640 1 -- 172.21.15.38:0/587491107 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7fcf2403a8e0 con 0x7fcf341064e0 2024-09-16T00:23:27.140 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.363+0000 7fcf237fe640 1 --2- 172.21.15.38:0/587491107 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fcf1c039260 0x7fcf1c03b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:27.140 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.363+0000 7fcf237fe640 1 -- 172.21.15.38:0/587491107 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcf2406c9c0 con 0x7fcf341064e0 2024-09-16T00:23:27.140 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.363+0000 7fcf38a4a640 1 -- 172.21.15.38:0/587491107 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf3410abc0 con 0x7fcf341064e0 2024-09-16T00:23:27.140 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.363+0000 7fcf31d9b640 1 --2- 172.21.15.38:0/587491107 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fcf1c039260 0x7fcf1c03b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:27.140 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.364+0000 7fcf31d9b640 1 --2- 172.21.15.38:0/587491107 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fcf1c039260 0x7fcf1c03b720 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcf280098c0 tx=0x7fcf28006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:27.140 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.369+0000 7fcf237fe640 1 -- 172.21.15.38:0/587491107 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcf24041270 con 0x7fcf341064e0 2024-09-16T00:23:27.140 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.571+0000 7fcf38a4a640 1 -- 172.21.15.38:0/587491107 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7fcf3402d890 con 0x7fcf1c039260 2024-09-16T00:23:27.140 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.821+0000 7fcf237fe640 1 -- 172.21.15.38:0/587491107 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7fcf3402d890 con 0x7fcf1c039260 2024-09-16T00:23:27.140 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.823+0000 7fcf38a4a640 1 -- 172.21.15.38:0/587491107 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fcf1c039260 msgr2=0x7fcf1c03b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:27.140 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.823+0000 7fcf38a4a640 1 --2- 172.21.15.38:0/587491107 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fcf1c039260 0x7fcf1c03b720 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcf280098c0 tx=0x7fcf28006db0 comp rx=0 tx=0).stop 2024-09-16T00:23:27.140 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.823+0000 7fcf38a4a640 1 -- 172.21.15.38:0/587491107 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf341064e0 msgr2=0x7fcf3419b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:27.140 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.823+0000 7fcf38a4a640 1 --2- 172.21.15.38:0/587491107 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf341064e0 0x7fcf3419b620 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcf24030560 tx=0x7fcf24032670 comp rx=0 tx=0).stop 2024-09-16T00:23:27.140 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.823+0000 7fcf38a4a640 1 -- 172.21.15.38:0/587491107 shutdown_connections 2024-09-16T00:23:27.141 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.823+0000 7fcf38a4a640 1 --2- 172.21.15.38:0/587491107 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fcf1c039260 0x7fcf1c03b720 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:27.141 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.823+0000 7fcf38a4a640 1 --2- 172.21.15.38:0/587491107 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcf341064e0 0x7fcf3419b620 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:27.141 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.823+0000 7fcf38a4a640 1 -- 172.21.15.38:0/587491107 >> 172.21.15.38:0/587491107 conn(0x7fcf34101c30 msgr2=0x7fcf34102be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:27.141 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.823+0000 7fcf38a4a640 1 -- 172.21.15.38:0/587491107 shutdown_connections 2024-09-16T00:23:27.141 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:25.823+0000 7fcf38a4a640 1 -- 172.21.15.38:0/587491107 wait complete. 2024-09-16T00:23:27.141 INFO:teuthology.orchestra.run.smithi038.stdout:Fetching dashboard port number... 2024-09-16T00:23:28.035 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:27 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/3773977695' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-16T00:23:28.327 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stdout 8443 2024-09-16T00:23:28.327 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.536+0000 7fc638def640 1 Processor -- start 2024-09-16T00:23:28.327 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.536+0000 7fc638def640 1 -- start start 2024-09-16T00:23:28.327 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.537+0000 7fc638def640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6341080b0 0x7fc63410a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:28.327 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.537+0000 7fc638def640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc63407bdf0 con 0x7fc6341080b0 2024-09-16T00:23:28.327 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.537+0000 7fc63259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6341080b0 0x7fc63410a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:28.327 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.537+0000 7fc63259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6341080b0 0x7fc63410a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45308/0 (socket says 172.21.15.38:45308) 2024-09-16T00:23:28.327 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.537+0000 7fc63259c640 1 -- 172.21.15.38:0/1208934390 learned_addr learned my addr 172.21.15.38:0/1208934390 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:23:28.327 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.537+0000 7fc63259c640 1 -- 172.21.15.38:0/1208934390 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63410a9e0 con 0x7fc6341080b0 2024-09-16T00:23:28.328 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.538+0000 7fc63259c640 1 --2- 172.21.15.38:0/1208934390 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6341080b0 0x7fc63410a4a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc62c009820 tx=0x7fc62c02a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb172e50716961de server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:28.328 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.538+0000 7fc63159a640 1 -- 172.21.15.38:0/1208934390 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc62c037070 con 0x7fc6341080b0 2024-09-16T00:23:28.328 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.538+0000 7fc63159a640 1 -- 172.21.15.38:0/1208934390 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc62c032920 con 0x7fc6341080b0 2024-09-16T00:23:28.328 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.538+0000 7fc63159a640 1 -- 172.21.15.38:0/1208934390 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc62c030810 con 0x7fc6341080b0 2024-09-16T00:23:28.328 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.539+0000 7fc638def640 1 -- 172.21.15.38:0/1208934390 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6341080b0 msgr2=0x7fc63410a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:28.328 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.539+0000 7fc638def640 1 --2- 172.21.15.38:0/1208934390 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6341080b0 0x7fc63410a4a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc62c009820 tx=0x7fc62c02a270 comp rx=0 tx=0).stop 2024-09-16T00:23:28.328 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.539+0000 7fc638def640 1 -- 172.21.15.38:0/1208934390 shutdown_connections 2024-09-16T00:23:28.328 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.539+0000 7fc638def640 1 --2- 172.21.15.38:0/1208934390 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6341080b0 0x7fc63410a4a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:28.328 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.539+0000 7fc638def640 1 -- 172.21.15.38:0/1208934390 >> 172.21.15.38:0/1208934390 conn(0x7fc634101c30 msgr2=0x7fc6341040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:28.328 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.539+0000 7fc638def640 1 -- 172.21.15.38:0/1208934390 shutdown_connections 2024-09-16T00:23:28.328 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.539+0000 7fc638def640 1 -- 172.21.15.38:0/1208934390 wait complete. 2024-09-16T00:23:28.328 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.540+0000 7fc638def640 1 Processor -- start 2024-09-16T00:23:28.328 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.540+0000 7fc638def640 1 -- start start 2024-09-16T00:23:28.329 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.540+0000 7fc638def640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6341080b0 0x7fc63419b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:28.329 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.540+0000 7fc638def640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc63410ba00 con 0x7fc6341080b0 2024-09-16T00:23:28.329 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.540+0000 7fc63259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6341080b0 0x7fc63419b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:28.329 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.540+0000 7fc63259c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6341080b0 0x7fc63419b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45310/0 (socket says 172.21.15.38:45310) 2024-09-16T00:23:28.329 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.540+0000 7fc63259c640 1 -- 172.21.15.38:0/3773977695 learned_addr learned my addr 172.21.15.38:0/3773977695 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:23:28.329 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.540+0000 7fc63259c640 1 -- 172.21.15.38:0/3773977695 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc63419baf0 con 0x7fc6341080b0 2024-09-16T00:23:28.329 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.541+0000 7fc63259c640 1 --2- 172.21.15.38:0/3773977695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6341080b0 0x7fc63419b5b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc62c030380 tx=0x7fc62c032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:28.329 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.541+0000 7fc62b7fe640 1 -- 172.21.15.38:0/3773977695 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc62c037040 con 0x7fc6341080b0 2024-09-16T00:23:28.329 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.541+0000 7fc62b7fe640 1 -- 172.21.15.38:0/3773977695 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc62c040070 con 0x7fc6341080b0 2024-09-16T00:23:28.329 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.541+0000 7fc638def640 1 -- 172.21.15.38:0/3773977695 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc63419bd80 con 0x7fc6341080b0 2024-09-16T00:23:28.329 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.541+0000 7fc62b7fe640 1 -- 172.21.15.38:0/3773977695 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc62c03a7b0 con 0x7fc6341080b0 2024-09-16T00:23:28.329 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.541+0000 7fc638def640 1 -- 172.21.15.38:0/3773977695 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc63419ea70 con 0x7fc6341080b0 2024-09-16T00:23:28.329 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.542+0000 7fc62b7fe640 1 -- 172.21.15.38:0/3773977695 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 12) ==== 46474+0+0 (secure 0 0 0) 0x7fc62c03b420 con 0x7fc6341080b0 2024-09-16T00:23:28.330 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.542+0000 7fc638def640 1 -- 172.21.15.38:0/3773977695 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc634108510 con 0x7fc6341080b0 2024-09-16T00:23:28.330 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.542+0000 7fc62b7fe640 1 --2- 172.21.15.38:0/3773977695 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fc620039200 0x7fc62003b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:28.330 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.542+0000 7fc62b7fe640 1 -- 172.21.15.38:0/3773977695 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fc62c06ca30 con 0x7fc6341080b0 2024-09-16T00:23:28.330 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.542+0000 7fc631d9b640 1 --2- 172.21.15.38:0/3773977695 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fc620039200 0x7fc62003b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:28.330 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.543+0000 7fc631d9b640 1 --2- 172.21.15.38:0/3773977695 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fc620039200 0x7fc62003b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc61c0098c0 tx=0x7fc61c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:28.330 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.547+0000 7fc62b7fe640 1 -- 172.21.15.38:0/3773977695 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc62c043360 con 0x7fc6341080b0 2024-09-16T00:23:28.330 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.726+0000 7fc638def640 1 -- 172.21.15.38:0/3773977695 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fc63419ec40 con 0x7fc6341080b0 2024-09-16T00:23:28.330 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.726+0000 7fc62b7fe640 1 -- 172.21.15.38:0/3773977695 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fc62c040220 con 0x7fc6341080b0 2024-09-16T00:23:28.330 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.728+0000 7fc638def640 1 -- 172.21.15.38:0/3773977695 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fc620039200 msgr2=0x7fc62003b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:28.330 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.728+0000 7fc638def640 1 --2- 172.21.15.38:0/3773977695 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fc620039200 0x7fc62003b6c0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc61c0098c0 tx=0x7fc61c006db0 comp rx=0 tx=0).stop 2024-09-16T00:23:28.330 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.728+0000 7fc638def640 1 -- 172.21.15.38:0/3773977695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6341080b0 msgr2=0x7fc63419b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:28.330 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.728+0000 7fc638def640 1 --2- 172.21.15.38:0/3773977695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6341080b0 0x7fc63419b5b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fc62c030380 tx=0x7fc62c032670 comp rx=0 tx=0).stop 2024-09-16T00:23:28.330 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.728+0000 7fc638def640 1 -- 172.21.15.38:0/3773977695 shutdown_connections 2024-09-16T00:23:28.330 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.729+0000 7fc638def640 1 --2- 172.21.15.38:0/3773977695 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fc620039200 0x7fc62003b6c0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:28.331 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.729+0000 7fc638def640 1 --2- 172.21.15.38:0/3773977695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc6341080b0 0x7fc63419b5b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:28.331 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.729+0000 7fc638def640 1 -- 172.21.15.38:0/3773977695 >> 172.21.15.38:0/3773977695 conn(0x7fc634101c30 msgr2=0x7fc6341027e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:28.331 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.729+0000 7fc638def640 1 -- 172.21.15.38:0/3773977695 shutdown_connections 2024-09-16T00:23:28.331 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:27.729+0000 7fc638def640 1 -- 172.21.15.38:0/3773977695 wait complete. 2024-09-16T00:23:28.337 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-16T00:23:28.337 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout disabled 2024-09-16T00:23:28.345 INFO:teuthology.orchestra.run.smithi038.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-16T00:23:28.346 INFO:teuthology.orchestra.run.smithi038.stdout:systemctl: stdout inactive 2024-09-16T00:23:28.346 INFO:teuthology.orchestra.run.smithi038.stdout:firewalld.service is not enabled 2024-09-16T00:23:28.346 INFO:teuthology.orchestra.run.smithi038.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-16T00:23:28.347 INFO:teuthology.orchestra.run.smithi038.stdout:Ceph Dashboard is now available at: 2024-09-16T00:23:28.347 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:23:28.348 INFO:teuthology.orchestra.run.smithi038.stdout: URL: https://smithi038.front.sepia.ceph.com:8443/ 2024-09-16T00:23:28.348 INFO:teuthology.orchestra.run.smithi038.stdout: User: admin 2024-09-16T00:23:28.348 INFO:teuthology.orchestra.run.smithi038.stdout: Password: 9nfpucsoas 2024-09-16T00:23:28.348 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:23:28.348 INFO:teuthology.orchestra.run.smithi038.stdout:Saving cluster configuration to /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/config directory 2024-09-16T00:23:29.214 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:28 smithi038 ceph-mon[30945]: mgrmap e13: x(active, since 6s) 2024-09-16T00:23:29.676 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:28.775+0000 7fe937252640 1 Processor -- start 2024-09-16T00:23:29.676 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:28.775+0000 7fe937252640 1 -- start start 2024-09-16T00:23:29.676 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:28.775+0000 7fe937252640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe930108110 0x7fe93010a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:29.676 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:28.775+0000 7fe937252640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe93007bd90 con 0x7fe930108110 2024-09-16T00:23:29.676 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:28.775+0000 7fe934fee640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe930108110 0x7fe93010a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:29.677 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:28.775+0000 7fe934fee640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe930108110 0x7fe93010a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45324/0 (socket says 172.21.15.38:45324) 2024-09-16T00:23:29.677 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:28.775+0000 7fe934fee640 1 -- 172.21.15.38:0/622261882 learned_addr learned my addr 172.21.15.38:0/622261882 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:23:29.677 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:28.776+0000 7fe934fee640 1 -- 172.21.15.38:0/622261882 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe93010aa40 con 0x7fe930108110 2024-09-16T00:23:29.677 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:28.776+0000 7fe934fee640 1 --2- 172.21.15.38:0/622261882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe930108110 0x7fe93010a500 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe918009820 tx=0x7fe91802a270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6108be1505cd5cd9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:29.677 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:28.777+0000 7fe92f7fe640 1 -- 172.21.15.38:0/622261882 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe918037070 con 0x7fe930108110 2024-09-16T00:23:29.677 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:28.777+0000 7fe92f7fe640 1 -- 172.21.15.38:0/622261882 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe918032920 con 0x7fe930108110 2024-09-16T00:23:29.677 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:28.777+0000 7fe92f7fe640 1 -- 172.21.15.38:0/622261882 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe918030810 con 0x7fe930108110 2024-09-16T00:23:29.677 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:28.777+0000 7fe937252640 1 -- 172.21.15.38:0/622261882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe930108110 msgr2=0x7fe93010a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:29.677 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:28.777+0000 7fe937252640 1 --2- 172.21.15.38:0/622261882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe930108110 0x7fe93010a500 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe918009820 tx=0x7fe91802a270 comp rx=0 tx=0).stop 2024-09-16T00:23:29.677 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:28.777+0000 7fe937252640 1 -- 172.21.15.38:0/622261882 shutdown_connections 2024-09-16T00:23:29.677 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:28.777+0000 7fe937252640 1 --2- 172.21.15.38:0/622261882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe930108110 0x7fe93010a500 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:29.677 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:28.777+0000 7fe937252640 1 -- 172.21.15.38:0/622261882 >> 172.21.15.38:0/622261882 conn(0x7fe930101c30 msgr2=0x7fe9301040b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:29.678 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:28.777+0000 7fe937252640 1 -- 172.21.15.38:0/622261882 shutdown_connections 2024-09-16T00:23:29.678 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:28.778+0000 7fe937252640 1 -- 172.21.15.38:0/622261882 wait complete. 2024-09-16T00:23:29.678 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:28.778+0000 7fe937252640 1 Processor -- start 2024-09-16T00:23:29.678 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:28.778+0000 7fe937252640 1 -- start start 2024-09-16T00:23:29.678 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:28.778+0000 7fe937252640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe930108110 0x7fe930107990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:29.678 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:28.778+0000 7fe937252640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe93010ba00 con 0x7fe930108110 2024-09-16T00:23:29.678 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:28.778+0000 7fe934fee640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe930108110 0x7fe930107990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:29.678 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:28.778+0000 7fe934fee640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe930108110 0x7fe930107990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45340/0 (socket says 172.21.15.38:45340) 2024-09-16T00:23:29.678 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:28.779+0000 7fe934fee640 1 -- 172.21.15.38:0/702414218 learned_addr learned my addr 172.21.15.38:0/702414218 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:23:29.678 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:28.779+0000 7fe934fee640 1 -- 172.21.15.38:0/702414218 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe930107ed0 con 0x7fe930108110 2024-09-16T00:23:29.678 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:28.779+0000 7fe934fee640 1 --2- 172.21.15.38:0/702414218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe930108110 0x7fe930107990 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe918030380 tx=0x7fe918032670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:29.678 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:28.779+0000 7fe92dffb640 1 -- 172.21.15.38:0/702414218 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe918037040 con 0x7fe930108110 2024-09-16T00:23:29.678 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:28.779+0000 7fe92dffb640 1 -- 172.21.15.38:0/702414218 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe918040070 con 0x7fe930108110 2024-09-16T00:23:29.679 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:28.779+0000 7fe937252640 1 -- 172.21.15.38:0/702414218 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9301060c0 con 0x7fe930108110 2024-09-16T00:23:29.679 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:28.779+0000 7fe92dffb640 1 -- 172.21.15.38:0/702414218 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe91803a7b0 con 0x7fe930108110 2024-09-16T00:23:29.679 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:28.779+0000 7fe937252640 1 -- 172.21.15.38:0/702414218 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe930106600 con 0x7fe930108110 2024-09-16T00:23:29.679 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:28.780+0000 7fe92dffb640 1 -- 172.21.15.38:0/702414218 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fe91803b420 con 0x7fe930108110 2024-09-16T00:23:29.679 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:28.780+0000 7fe937252640 1 -- 172.21.15.38:0/702414218 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe930108570 con 0x7fe930108110 2024-09-16T00:23:29.679 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:28.780+0000 7fe92dffb640 1 --2- 172.21.15.38:0/702414218 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fe91c039300 0x7fe91c03b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:29.679 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:28.781+0000 7fe92dffb640 1 -- 172.21.15.38:0/702414218 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe91806caa0 con 0x7fe930108110 2024-09-16T00:23:29.679 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:28.781+0000 7fe92ffff640 1 --2- 172.21.15.38:0/702414218 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fe91c039300 0x7fe91c03b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:29.679 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:28.781+0000 7fe92ffff640 1 --2- 172.21.15.38:0/702414218 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fe91c039300 0x7fe91c03b7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe92000ad30 tx=0x7fe9200093f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:29.679 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:28.785+0000 7fe92dffb640 1 -- 172.21.15.38:0/702414218 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe918042b50 con 0x7fe930108110 2024-09-16T00:23:29.679 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:29.051+0000 7fe937252640 1 -- 172.21.15.38:0/702414218 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fe930106880 con 0x7fe930108110 2024-09-16T00:23:29.679 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:29.053+0000 7fe92dffb640 1 -- 172.21.15.38:0/702414218 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fe918043360 con 0x7fe930108110 2024-09-16T00:23:29.679 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-16T00:23:29.679 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:29.055+0000 7fe937252640 1 -- 172.21.15.38:0/702414218 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fe91c039300 msgr2=0x7fe91c03b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:29.680 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:29.055+0000 7fe937252640 1 --2- 172.21.15.38:0/702414218 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fe91c039300 0x7fe91c03b7c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe92000ad30 tx=0x7fe9200093f0 comp rx=0 tx=0).stop 2024-09-16T00:23:29.680 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:29.055+0000 7fe937252640 1 -- 172.21.15.38:0/702414218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe930108110 msgr2=0x7fe930107990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:29.680 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:29.055+0000 7fe937252640 1 --2- 172.21.15.38:0/702414218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe930108110 0x7fe930107990 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe918030380 tx=0x7fe918032670 comp rx=0 tx=0).stop 2024-09-16T00:23:29.680 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:29.055+0000 7fe937252640 1 -- 172.21.15.38:0/702414218 shutdown_connections 2024-09-16T00:23:29.680 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:29.055+0000 7fe937252640 1 --2- 172.21.15.38:0/702414218 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fe91c039300 0x7fe91c03b7c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:29.680 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:29.055+0000 7fe937252640 1 --2- 172.21.15.38:0/702414218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe930108110 0x7fe930107990 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:29.680 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:29.055+0000 7fe937252640 1 -- 172.21.15.38:0/702414218 >> 172.21.15.38:0/702414218 conn(0x7fe930101c30 msgr2=0x7fe930102800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:29.680 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:29.055+0000 7fe937252640 1 -- 172.21.15.38:0/702414218 shutdown_connections 2024-09-16T00:23:29.680 INFO:teuthology.orchestra.run.smithi038.stdout:/usr/bin/ceph: stderr 2024-09-16T00:23:29.055+0000 7fe937252640 1 -- 172.21.15.38:0/702414218 wait complete. 2024-09-16T00:23:29.680 INFO:teuthology.orchestra.run.smithi038.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-16T00:23:29.680 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:23:29.680 INFO:teuthology.orchestra.run.smithi038.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:23:29.680 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:23:29.680 INFO:teuthology.orchestra.run.smithi038.stdout:Or, if you are only running a single cluster on this host: 2024-09-16T00:23:29.681 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:23:29.681 INFO:teuthology.orchestra.run.smithi038.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-16T00:23:29.681 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:23:29.681 INFO:teuthology.orchestra.run.smithi038.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-16T00:23:29.681 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:23:29.681 INFO:teuthology.orchestra.run.smithi038.stdout: ceph telemetry on 2024-09-16T00:23:29.681 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:23:29.681 INFO:teuthology.orchestra.run.smithi038.stdout:For more information see: 2024-09-16T00:23:29.681 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:23:29.681 INFO:teuthology.orchestra.run.smithi038.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-16T00:23:29.681 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:23:29.681 INFO:teuthology.orchestra.run.smithi038.stdout:Bootstrap complete. 2024-09-16T00:23:29.720 INFO:tasks.cephadm:Fetching config... 2024-09-16T00:23:29.721 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-16T00:23:29.721 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-16T00:23:29.739 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-16T00:23:29.740 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-16T00:23:29.740 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-16T00:23:29.796 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-16T00:23:29.796 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-16T00:23:29.796 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-16T00:23:29.863 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-16T00:23:29.863 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-16T00:23:29.863 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-16T00:23:29.919 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-16T00:23:29.920 DEBUG:teuthology.orchestra.run.smithi038:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCf0y82XRDGWCAcNUAxnFIowcLRrsM7vDl9BshzPEeiLlNsv8c48nHD4jbSfEzg7vzle4SEoQFGtCTcAu/9kImKBzX8f4c4Ya4q5P2jtbr1OmyDsz5SlnS/ZSny+r1Jr8nIw3B7LGy7G3y5xh/AScGt3if7UreNISH+OxNf7x/j7feR7KTSn+b7LFryuP6n4BHSyj3xnHJVIMSnbzequuHwZ3SwqbRySgtYI1wenwTsql9PLVr/Oi+kQ0tfo9N8EvCddHWJnIIRVgzRziMAJNd6C2Y0O1Ny5KDJo8lqRwePm90ikJPHjO2SrOHYLaVrsYn+9HFRvuXoBY2qCzisZidnbaQfYW74lrqNT800tdZJU6kf8vF6Bxglaldq38k8iNh+9sZmNSq0hcuf9oj+y0Qj5kpHSIugG3CwJY7sAZBaJXDoTB75HwEzN7GlUB+rLxtblZgY4krveuEyCUupSLQi2fAjiCs/w9lVV139zF4zPuG2ytxteAqJg26ZAEhGg40= ceph-983b2072-73c1-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-16T00:23:29.995 INFO:teuthology.orchestra.run.smithi038.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCf0y82XRDGWCAcNUAxnFIowcLRrsM7vDl9BshzPEeiLlNsv8c48nHD4jbSfEzg7vzle4SEoQFGtCTcAu/9kImKBzX8f4c4Ya4q5P2jtbr1OmyDsz5SlnS/ZSny+r1Jr8nIw3B7LGy7G3y5xh/AScGt3if7UreNISH+OxNf7x/j7feR7KTSn+b7LFryuP6n4BHSyj3xnHJVIMSnbzequuHwZ3SwqbRySgtYI1wenwTsql9PLVr/Oi+kQ0tfo9N8EvCddHWJnIIRVgzRziMAJNd6C2Y0O1Ny5KDJo8lqRwePm90ikJPHjO2SrOHYLaVrsYn+9HFRvuXoBY2qCzisZidnbaQfYW74lrqNT800tdZJU6kf8vF6Bxglaldq38k8iNh+9sZmNSq0hcuf9oj+y0Qj5kpHSIugG3CwJY7sAZBaJXDoTB75HwEzN7GlUB+rLxtblZgY4krveuEyCUupSLQi2fAjiCs/w9lVV139zF4zPuG2ytxteAqJg26ZAEhGg40= ceph-983b2072-73c1-11ef-bceb-c7b262605968 2024-09-16T00:23:30.008 DEBUG:teuthology.orchestra.run.smithi073:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCf0y82XRDGWCAcNUAxnFIowcLRrsM7vDl9BshzPEeiLlNsv8c48nHD4jbSfEzg7vzle4SEoQFGtCTcAu/9kImKBzX8f4c4Ya4q5P2jtbr1OmyDsz5SlnS/ZSny+r1Jr8nIw3B7LGy7G3y5xh/AScGt3if7UreNISH+OxNf7x/j7feR7KTSn+b7LFryuP6n4BHSyj3xnHJVIMSnbzequuHwZ3SwqbRySgtYI1wenwTsql9PLVr/Oi+kQ0tfo9N8EvCddHWJnIIRVgzRziMAJNd6C2Y0O1Ny5KDJo8lqRwePm90ikJPHjO2SrOHYLaVrsYn+9HFRvuXoBY2qCzisZidnbaQfYW74lrqNT800tdZJU6kf8vF6Bxglaldq38k8iNh+9sZmNSq0hcuf9oj+y0Qj5kpHSIugG3CwJY7sAZBaJXDoTB75HwEzN7GlUB+rLxtblZgY4krveuEyCUupSLQi2fAjiCs/w9lVV139zF4zPuG2ytxteAqJg26ZAEhGg40= ceph-983b2072-73c1-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-16T00:23:30.050 INFO:teuthology.orchestra.run.smithi073.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCf0y82XRDGWCAcNUAxnFIowcLRrsM7vDl9BshzPEeiLlNsv8c48nHD4jbSfEzg7vzle4SEoQFGtCTcAu/9kImKBzX8f4c4Ya4q5P2jtbr1OmyDsz5SlnS/ZSny+r1Jr8nIw3B7LGy7G3y5xh/AScGt3if7UreNISH+OxNf7x/j7feR7KTSn+b7LFryuP6n4BHSyj3xnHJVIMSnbzequuHwZ3SwqbRySgtYI1wenwTsql9PLVr/Oi+kQ0tfo9N8EvCddHWJnIIRVgzRziMAJNd6C2Y0O1Ny5KDJo8lqRwePm90ikJPHjO2SrOHYLaVrsYn+9HFRvuXoBY2qCzisZidnbaQfYW74lrqNT800tdZJU6kf8vF6Bxglaldq38k8iNh+9sZmNSq0hcuf9oj+y0Qj5kpHSIugG3CwJY7sAZBaJXDoTB75HwEzN7GlUB+rLxtblZgY4krveuEyCUupSLQi2fAjiCs/w9lVV139zF4zPuG2ytxteAqJg26ZAEhGg40= ceph-983b2072-73c1-11ef-bceb-c7b262605968 2024-09-16T00:23:30.062 DEBUG:teuthology.orchestra.run.smithi122:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCf0y82XRDGWCAcNUAxnFIowcLRrsM7vDl9BshzPEeiLlNsv8c48nHD4jbSfEzg7vzle4SEoQFGtCTcAu/9kImKBzX8f4c4Ya4q5P2jtbr1OmyDsz5SlnS/ZSny+r1Jr8nIw3B7LGy7G3y5xh/AScGt3if7UreNISH+OxNf7x/j7feR7KTSn+b7LFryuP6n4BHSyj3xnHJVIMSnbzequuHwZ3SwqbRySgtYI1wenwTsql9PLVr/Oi+kQ0tfo9N8EvCddHWJnIIRVgzRziMAJNd6C2Y0O1Ny5KDJo8lqRwePm90ikJPHjO2SrOHYLaVrsYn+9HFRvuXoBY2qCzisZidnbaQfYW74lrqNT800tdZJU6kf8vF6Bxglaldq38k8iNh+9sZmNSq0hcuf9oj+y0Qj5kpHSIugG3CwJY7sAZBaJXDoTB75HwEzN7GlUB+rLxtblZgY4krveuEyCUupSLQi2fAjiCs/w9lVV139zF4zPuG2ytxteAqJg26ZAEhGg40= ceph-983b2072-73c1-11ef-bceb-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-16T00:23:30.103 INFO:teuthology.orchestra.run.smithi122.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCf0y82XRDGWCAcNUAxnFIowcLRrsM7vDl9BshzPEeiLlNsv8c48nHD4jbSfEzg7vzle4SEoQFGtCTcAu/9kImKBzX8f4c4Ya4q5P2jtbr1OmyDsz5SlnS/ZSny+r1Jr8nIw3B7LGy7G3y5xh/AScGt3if7UreNISH+OxNf7x/j7feR7KTSn+b7LFryuP6n4BHSyj3xnHJVIMSnbzequuHwZ3SwqbRySgtYI1wenwTsql9PLVr/Oi+kQ0tfo9N8EvCddHWJnIIRVgzRziMAJNd6C2Y0O1Ny5KDJo8lqRwePm90ikJPHjO2SrOHYLaVrsYn+9HFRvuXoBY2qCzisZidnbaQfYW74lrqNT800tdZJU6kf8vF6Bxglaldq38k8iNh+9sZmNSq0hcuf9oj+y0Qj5kpHSIugG3CwJY7sAZBaJXDoTB75HwEzN7GlUB+rLxtblZgY4krveuEyCUupSLQi2fAjiCs/w9lVV139zF4zPuG2ytxteAqJg26ZAEhGg40= ceph-983b2072-73c1-11ef-bceb-c7b262605968 2024-09-16T00:23:30.116 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-16T00:23:30.247 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:30 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/702414218' entity='client.admin' 2024-09-16T00:23:30.368 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:23:30.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:30.888+0000 7f4f8c819640 1 -- 172.21.15.38:0/957767235 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f88101600 msgr2=0x7f4f88101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:30.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:30.888+0000 7f4f8c819640 1 --2- 172.21.15.38:0/957767235 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f88101600 0x7f4f88101a00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4f78009880 tx=0x7f4f7802a5d0 comp rx=0 tx=0).stop 2024-09-16T00:23:30.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:30.888+0000 7f4f8c819640 1 -- 172.21.15.38:0/957767235 shutdown_connections 2024-09-16T00:23:30.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:30.888+0000 7f4f8c819640 1 --2- 172.21.15.38:0/957767235 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f88101600 0x7f4f88101a00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:30.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:30.888+0000 7f4f8c819640 1 -- 172.21.15.38:0/957767235 >> 172.21.15.38:0/957767235 conn(0x7f4f880fcdb0 msgr2=0x7f4f880ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:30.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:30.888+0000 7f4f8c819640 1 -- 172.21.15.38:0/957767235 shutdown_connections 2024-09-16T00:23:30.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:30.888+0000 7f4f8c819640 1 -- 172.21.15.38:0/957767235 wait complete. 2024-09-16T00:23:30.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:30.889+0000 7f4f8c819640 1 Processor -- start 2024-09-16T00:23:30.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:30.889+0000 7f4f8c819640 1 -- start start 2024-09-16T00:23:30.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:30.889+0000 7f4f8c819640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f88101600 0x7f4f88196db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:30.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:30.889+0000 7f4f8c819640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4f88107710 con 0x7f4f88101600 2024-09-16T00:23:30.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:30.889+0000 7f4f8659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f88101600 0x7f4f88196db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:30.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:30.889+0000 7f4f8659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f88101600 0x7f4f88196db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45358/0 (socket says 172.21.15.38:45358) 2024-09-16T00:23:30.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:30.889+0000 7f4f8659c640 1 -- 172.21.15.38:0/2600614915 learned_addr learned my addr 172.21.15.38:0/2600614915 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:23:30.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:30.890+0000 7f4f8659c640 1 -- 172.21.15.38:0/2600614915 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f881972f0 con 0x7f4f88101600 2024-09-16T00:23:30.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:30.890+0000 7f4f8659c640 1 --2- 172.21.15.38:0/2600614915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f88101600 0x7f4f88196db0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4f78004f60 tx=0x7f4f780048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:30.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:30.890+0000 7f4f777fe640 1 -- 172.21.15.38:0/2600614915 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f7803f070 con 0x7f4f88101600 2024-09-16T00:23:30.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:30.890+0000 7f4f8c819640 1 -- 172.21.15.38:0/2600614915 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f88197580 con 0x7f4f88101600 2024-09-16T00:23:30.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:30.890+0000 7f4f777fe640 1 -- 172.21.15.38:0/2600614915 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4f78004cc0 con 0x7f4f88101600 2024-09-16T00:23:30.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:30.890+0000 7f4f777fe640 1 -- 172.21.15.38:0/2600614915 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4f78035030 con 0x7f4f88101600 2024-09-16T00:23:30.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:30.890+0000 7f4f8c819640 1 -- 172.21.15.38:0/2600614915 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f8819a230 con 0x7f4f88101600 2024-09-16T00:23:30.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:30.891+0000 7f4f777fe640 1 -- 172.21.15.38:0/2600614915 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f4f78042070 con 0x7f4f88101600 2024-09-16T00:23:30.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:30.891+0000 7f4f8c819640 1 -- 172.21.15.38:0/2600614915 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f88107e90 con 0x7f4f88101600 2024-09-16T00:23:30.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:30.891+0000 7f4f777fe640 1 --2- 172.21.15.38:0/2600614915 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f4f700392a0 0x7f4f7003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:30.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:30.891+0000 7f4f777fe640 1 -- 172.21.15.38:0/2600614915 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f4f7806d070 con 0x7f4f88101600 2024-09-16T00:23:30.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:30.892+0000 7f4f85d9b640 1 --2- 172.21.15.38:0/2600614915 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f4f700392a0 0x7f4f7003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:30.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:30.892+0000 7f4f85d9b640 1 --2- 172.21.15.38:0/2600614915 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f4f700392a0 0x7f4f7003b760 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c0098c0 tx=0x7f4f7c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:30.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:30.896+0000 7f4f777fe640 1 -- 172.21.15.38:0/2600614915 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4f78036da0 con 0x7f4f88101600 2024-09-16T00:23:31.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:31.075+0000 7f4f8c819640 1 -- 172.21.15.38:0/2600614915 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f4f8802d940 con 0x7f4f88101600 2024-09-16T00:23:31.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:31.078+0000 7f4f777fe640 1 -- 172.21.15.38:0/2600614915 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f4f78030d60 con 0x7f4f88101600 2024-09-16T00:23:31.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:31.081+0000 7f4f8c819640 1 -- 172.21.15.38:0/2600614915 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f4f700392a0 msgr2=0x7f4f7003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:31.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:31.081+0000 7f4f8c819640 1 --2- 172.21.15.38:0/2600614915 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f4f700392a0 0x7f4f7003b760 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4f7c0098c0 tx=0x7f4f7c006db0 comp rx=0 tx=0).stop 2024-09-16T00:23:31.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:31.081+0000 7f4f8c819640 1 -- 172.21.15.38:0/2600614915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f88101600 msgr2=0x7f4f88196db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:31.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:31.081+0000 7f4f8c819640 1 --2- 172.21.15.38:0/2600614915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f88101600 0x7f4f88196db0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4f78004f60 tx=0x7f4f780048f0 comp rx=0 tx=0).stop 2024-09-16T00:23:31.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:31.081+0000 7f4f8c819640 1 -- 172.21.15.38:0/2600614915 shutdown_connections 2024-09-16T00:23:31.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:31.081+0000 7f4f8c819640 1 --2- 172.21.15.38:0/2600614915 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f4f700392a0 0x7f4f7003b760 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:31.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:31.081+0000 7f4f8c819640 1 --2- 172.21.15.38:0/2600614915 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4f88101600 0x7f4f88196db0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:31.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:31.081+0000 7f4f8c819640 1 -- 172.21.15.38:0/2600614915 >> 172.21.15.38:0/2600614915 conn(0x7f4f880fcdb0 msgr2=0x7f4f880fd770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:31.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:31.082+0000 7f4f8c819640 1 -- 172.21.15.38:0/2600614915 shutdown_connections 2024-09-16T00:23:31.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:31.082+0000 7f4f8c819640 1 -- 172.21.15.38:0/2600614915 wait complete. 2024-09-16T00:23:32.402 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:32 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/2600614915' entity='client.admin' 2024-09-16T00:23:32.589 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-16T00:23:32.590 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-16T00:23:33.312 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:23:33.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:33.749+0000 7f5d0d932640 1 -- 172.21.15.38:0/2554454569 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d08101600 msgr2=0x7f5d08101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:33.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:33.749+0000 7f5d0d932640 1 --2- 172.21.15.38:0/2554454569 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d08101600 0x7f5d08101a00 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8009880 tx=0x7f5cf802a5d0 comp rx=0 tx=0).stop 2024-09-16T00:23:33.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:33.749+0000 7f5d0d932640 1 -- 172.21.15.38:0/2554454569 shutdown_connections 2024-09-16T00:23:33.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:33.749+0000 7f5d0d932640 1 --2- 172.21.15.38:0/2554454569 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d08101600 0x7f5d08101a00 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:33.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:33.749+0000 7f5d0d932640 1 -- 172.21.15.38:0/2554454569 >> 172.21.15.38:0/2554454569 conn(0x7f5d080fcdb0 msgr2=0x7f5d080ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:33.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:33.749+0000 7f5d0d932640 1 -- 172.21.15.38:0/2554454569 shutdown_connections 2024-09-16T00:23:33.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:33.749+0000 7f5d0d932640 1 -- 172.21.15.38:0/2554454569 wait complete. 2024-09-16T00:23:33.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:33.750+0000 7f5d0d932640 1 Processor -- start 2024-09-16T00:23:33.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:33.750+0000 7f5d0d932640 1 -- start start 2024-09-16T00:23:33.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:33.750+0000 7f5d0d932640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d08101600 0x7f5d08075eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:33.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:33.750+0000 7f5d0d932640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5d08107710 con 0x7f5d08101600 2024-09-16T00:23:33.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:33.750+0000 7f5d06ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d08101600 0x7f5d08075eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:33.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:33.750+0000 7f5d06ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d08101600 0x7f5d08075eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45366/0 (socket says 172.21.15.38:45366) 2024-09-16T00:23:33.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:33.750+0000 7f5d06ffd640 1 -- 172.21.15.38:0/196822766 learned_addr learned my addr 172.21.15.38:0/196822766 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:23:33.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:33.750+0000 7f5d06ffd640 1 -- 172.21.15.38:0/196822766 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d080763f0 con 0x7f5d08101600 2024-09-16T00:23:33.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:33.750+0000 7f5d06ffd640 1 --2- 172.21.15.38:0/196822766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d08101600 0x7f5d08075eb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8004f60 tx=0x7f5cf80048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:33.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:33.751+0000 7f5d0c930640 1 -- 172.21.15.38:0/196822766 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cf803f070 con 0x7f5d08101600 2024-09-16T00:23:33.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:33.751+0000 7f5d0c930640 1 -- 172.21.15.38:0/196822766 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5cf8004d40 con 0x7f5d08101600 2024-09-16T00:23:33.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:33.751+0000 7f5d0d932640 1 -- 172.21.15.38:0/196822766 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d080745e0 con 0x7f5d08101600 2024-09-16T00:23:33.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:33.751+0000 7f5d0c930640 1 -- 172.21.15.38:0/196822766 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5cf8035030 con 0x7f5d08101600 2024-09-16T00:23:33.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:33.751+0000 7f5d0d932640 1 -- 172.21.15.38:0/196822766 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d08074b20 con 0x7f5d08101600 2024-09-16T00:23:33.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:33.752+0000 7f5d0c930640 1 -- 172.21.15.38:0/196822766 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f5cf8042070 con 0x7f5d08101600 2024-09-16T00:23:33.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:33.752+0000 7f5d0d932640 1 -- 172.21.15.38:0/196822766 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d08107e90 con 0x7f5d08101600 2024-09-16T00:23:33.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:33.752+0000 7f5d0c930640 1 --2- 172.21.15.38:0/196822766 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f5cf4039300 0x7f5cf403b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:33.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:33.752+0000 7f5d0c930640 1 -- 172.21.15.38:0/196822766 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5cf806cf90 con 0x7f5d08101600 2024-09-16T00:23:33.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:33.752+0000 7f5cfffff640 1 --2- 172.21.15.38:0/196822766 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f5cf4039300 0x7f5cf403b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:33.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:33.752+0000 7f5cfffff640 1 --2- 172.21.15.38:0/196822766 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f5cf4039300 0x7f5cf403b7c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5cf00098c0 tx=0x7f5cf0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:33.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:33.757+0000 7f5d0c930640 1 -- 172.21.15.38:0/196822766 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5cf804a260 con 0x7f5d08101600 2024-09-16T00:23:33.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:33.934+0000 7f5d0d932640 1 -- 172.21.15.38:0/196822766 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f5d081901b0 con 0x7f5cf4039300 2024-09-16T00:23:33.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:33.938+0000 7f5d0c930640 1 -- 172.21.15.38:0/196822766 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5d081901b0 con 0x7f5cf4039300 2024-09-16T00:23:33.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:33.940+0000 7f5d0d932640 1 -- 172.21.15.38:0/196822766 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f5cf4039300 msgr2=0x7f5cf403b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:33.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:33.940+0000 7f5d0d932640 1 --2- 172.21.15.38:0/196822766 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f5cf4039300 0x7f5cf403b7c0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5cf00098c0 tx=0x7f5cf0006db0 comp rx=0 tx=0).stop 2024-09-16T00:23:33.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:33.940+0000 7f5d0d932640 1 -- 172.21.15.38:0/196822766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d08101600 msgr2=0x7f5d08075eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:33.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:33.940+0000 7f5d0d932640 1 --2- 172.21.15.38:0/196822766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d08101600 0x7f5d08075eb0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5cf8004f60 tx=0x7f5cf80048f0 comp rx=0 tx=0).stop 2024-09-16T00:23:33.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:33.940+0000 7f5d0d932640 1 -- 172.21.15.38:0/196822766 shutdown_connections 2024-09-16T00:23:33.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:33.940+0000 7f5d0d932640 1 --2- 172.21.15.38:0/196822766 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f5cf4039300 0x7f5cf403b7c0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:33.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:33.940+0000 7f5d0d932640 1 --2- 172.21.15.38:0/196822766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5d08101600 0x7f5d08075eb0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:33.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:33.940+0000 7f5d0d932640 1 -- 172.21.15.38:0/196822766 >> 172.21.15.38:0/196822766 conn(0x7f5d080fcdb0 msgr2=0x7f5d080fd720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:33.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:33.940+0000 7f5d0d932640 1 -- 172.21.15.38:0/196822766 shutdown_connections 2024-09-16T00:23:33.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:33.940+0000 7f5d0d932640 1 -- 172.21.15.38:0/196822766 wait complete. 2024-09-16T00:23:34.695 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:34 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:34.695 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:34 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:34.695 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:34 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi038", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:23:34.695 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:34 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:34.696 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:34 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:23:34.696 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:34 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:34.696 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:34 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:34.696 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:34 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:23:34.696 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:34 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:34.696 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:34 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:23:34.725 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi073 2024-09-16T00:23:34.726 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-16T00:23:34.726 DEBUG:teuthology.orchestra.run.smithi073:> dd of=/etc/ceph/ceph.conf 2024-09-16T00:23:34.746 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-16T00:23:34.746 DEBUG:teuthology.orchestra.run.smithi073:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:23:34.804 INFO:tasks.cephadm:Adding host smithi073 to orchestrator... 2024-09-16T00:23:34.804 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph orch host add smithi073 2024-09-16T00:23:35.053 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:23:35.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:35 smithi038 ceph-mon[30945]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:23:35.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:35 smithi038 ceph-mon[30945]: Updating smithi038:/etc/ceph/ceph.conf 2024-09-16T00:23:35.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:35 smithi038 ceph-mon[30945]: Updating smithi038:/var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:23:35.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:35 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:35.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:35 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:35.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:35 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:35.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:35.591+0000 7f347ee57640 1 -- 172.21.15.38:0/3651999986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3478107660 msgr2=0x7f3478107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:35.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:35.591+0000 7f347ee57640 1 --2- 172.21.15.38:0/3651999986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3478107660 0x7f3478107a40 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f346c009880 tx=0x7f346c02a5d0 comp rx=0 tx=0).stop 2024-09-16T00:23:35.593 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:35.591+0000 7f347ee57640 1 -- 172.21.15.38:0/3651999986 shutdown_connections 2024-09-16T00:23:35.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:35.591+0000 7f347ee57640 1 --2- 172.21.15.38:0/3651999986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3478107660 0x7f3478107a40 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:35.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:35.591+0000 7f347ee57640 1 -- 172.21.15.38:0/3651999986 >> 172.21.15.38:0/3651999986 conn(0x7f34780fd290 msgr2=0x7f34780ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:35.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:35.591+0000 7f347ee57640 1 -- 172.21.15.38:0/3651999986 shutdown_connections 2024-09-16T00:23:35.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:35.591+0000 7f347ee57640 1 -- 172.21.15.38:0/3651999986 wait complete. 2024-09-16T00:23:35.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:35.592+0000 7f347ee57640 1 Processor -- start 2024-09-16T00:23:35.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:35.592+0000 7f347ee57640 1 -- start start 2024-09-16T00:23:35.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:35.592+0000 7f347ee57640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3478107660 0x7f34781970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:35.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:35.592+0000 7f347ee57640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f347810bc90 con 0x7f3478107660 2024-09-16T00:23:35.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:35.592+0000 7f347cbf3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3478107660 0x7f34781970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:35.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:35.592+0000 7f347cbf3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3478107660 0x7f34781970c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45384/0 (socket says 172.21.15.38:45384) 2024-09-16T00:23:35.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:35.592+0000 7f347cbf3640 1 -- 172.21.15.38:0/1243376087 learned_addr learned my addr 172.21.15.38:0/1243376087 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:23:35.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:35.593+0000 7f347cbf3640 1 -- 172.21.15.38:0/1243376087 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3478197600 con 0x7f3478107660 2024-09-16T00:23:35.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:35.593+0000 7f347cbf3640 1 --2- 172.21.15.38:0/1243376087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3478107660 0x7f34781970c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f346c009230 tx=0x7f346c003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:35.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:35.593+0000 7f3475ffb640 1 -- 172.21.15.38:0/1243376087 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f346c03f070 con 0x7f3478107660 2024-09-16T00:23:35.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:35.593+0000 7f3475ffb640 1 -- 172.21.15.38:0/1243376087 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f346c03bc80 con 0x7f3478107660 2024-09-16T00:23:35.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:35.593+0000 7f347ee57640 1 -- 172.21.15.38:0/1243376087 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3478197890 con 0x7f3478107660 2024-09-16T00:23:35.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:35.593+0000 7f3475ffb640 1 -- 172.21.15.38:0/1243376087 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f346c035030 con 0x7f3478107660 2024-09-16T00:23:35.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:35.593+0000 7f347ee57640 1 -- 172.21.15.38:0/1243376087 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f347819b990 con 0x7f3478107660 2024-09-16T00:23:35.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:35.594+0000 7f3475ffb640 1 -- 172.21.15.38:0/1243376087 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f346c042070 con 0x7f3478107660 2024-09-16T00:23:35.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:35.594+0000 7f3475ffb640 1 --2- 172.21.15.38:0/1243376087 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f3468039300 0x7f346803b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:35.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:35.594+0000 7f347ee57640 1 -- 172.21.15.38:0/1243376087 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3478102cd0 con 0x7f3478107660 2024-09-16T00:23:35.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:35.594+0000 7f3475ffb640 1 -- 172.21.15.38:0/1243376087 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f346c06d1c0 con 0x7f3478107660 2024-09-16T00:23:35.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:35.594+0000 7f3477fff640 1 --2- 172.21.15.38:0/1243376087 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f3468039300 0x7f346803b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:35.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:35.595+0000 7f3477fff640 1 --2- 172.21.15.38:0/1243376087 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f3468039300 0x7f346803b7c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f34640098c0 tx=0x7f3464006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:35.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:35.599+0000 7f3475ffb640 1 -- 172.21.15.38:0/1243376087 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f346c0314c0 con 0x7f3478107660 2024-09-16T00:23:35.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:35.781+0000 7f347ee57640 1 -- 172.21.15.38:0/1243376087 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi073", "target": ["mon-mgr", ""]}) -- 0x7f3478107b20 con 0x7f3468039300 2024-09-16T00:23:36.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:36 smithi038 ceph-mon[30945]: Updating smithi038:/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:23:36.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:36 smithi038 ceph-mon[30945]: Updating smithi038:/var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T00:23:37.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:37 smithi038 ceph-mon[30945]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi073", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:23:38.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:38 smithi038 ceph-mon[30945]: Deploying cephadm binary to smithi073 2024-09-16T00:23:39.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:39.690+0000 7f3475ffb640 1 -- 172.21.15.38:0/1243376087 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f3478107b20 con 0x7f3468039300 2024-09-16T00:23:39.691 INFO:teuthology.orchestra.run.smithi038.stdout:Added host 'smithi073' with addr '172.21.15.73' 2024-09-16T00:23:39.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:39.692+0000 7f347ee57640 1 -- 172.21.15.38:0/1243376087 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f3468039300 msgr2=0x7f346803b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:39.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:39.692+0000 7f347ee57640 1 --2- 172.21.15.38:0/1243376087 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f3468039300 0x7f346803b7c0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f34640098c0 tx=0x7f3464006db0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:39.692+0000 7f347ee57640 1 -- 172.21.15.38:0/1243376087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3478107660 msgr2=0x7f34781970c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:39.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:39.692+0000 7f347ee57640 1 --2- 172.21.15.38:0/1243376087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3478107660 0x7f34781970c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f346c009230 tx=0x7f346c003d40 comp rx=0 tx=0).stop 2024-09-16T00:23:39.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:39.692+0000 7f347ee57640 1 -- 172.21.15.38:0/1243376087 shutdown_connections 2024-09-16T00:23:39.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:39.692+0000 7f347ee57640 1 --2- 172.21.15.38:0/1243376087 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f3468039300 0x7f346803b7c0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:39.692+0000 7f347ee57640 1 --2- 172.21.15.38:0/1243376087 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3478107660 0x7f34781970c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:39.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:39.692+0000 7f347ee57640 1 -- 172.21.15.38:0/1243376087 >> 172.21.15.38:0/1243376087 conn(0x7f34780fd290 msgr2=0x7f34780febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:39.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:39.693+0000 7f347ee57640 1 -- 172.21.15.38:0/1243376087 shutdown_connections 2024-09-16T00:23:39.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:39.693+0000 7f347ee57640 1 -- 172.21.15.38:0/1243376087 wait complete. 2024-09-16T00:23:40.325 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-16T00:23:40.573 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:23:40.955 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:40 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:40.956 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:40 smithi038 ceph-mon[30945]: Added host smithi073 2024-09-16T00:23:40.956 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:40 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:23:40.956 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:40 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:40.956 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:40 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:41.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:41.038+0000 7f775d0bf640 1 -- 172.21.15.38:0/2189547212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7758102750 msgr2=0x7f7758102b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:41.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:41.038+0000 7f775d0bf640 1 --2- 172.21.15.38:0/2189547212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7758102750 0x7f7758102b30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f7748009880 tx=0x7f774802a670 comp rx=0 tx=0).stop 2024-09-16T00:23:41.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:41.038+0000 7f775d0bf640 1 -- 172.21.15.38:0/2189547212 shutdown_connections 2024-09-16T00:23:41.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:41.038+0000 7f775d0bf640 1 --2- 172.21.15.38:0/2189547212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7758102750 0x7f7758102b30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:41.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:41.038+0000 7f775d0bf640 1 -- 172.21.15.38:0/2189547212 >> 172.21.15.38:0/2189547212 conn(0x7f77580fc1b0 msgr2=0x7f77580fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:41.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:41.038+0000 7f775d0bf640 1 -- 172.21.15.38:0/2189547212 shutdown_connections 2024-09-16T00:23:41.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:41.038+0000 7f775d0bf640 1 -- 172.21.15.38:0/2189547212 wait complete. 2024-09-16T00:23:41.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:41.039+0000 7f775d0bf640 1 Processor -- start 2024-09-16T00:23:41.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:41.039+0000 7f775d0bf640 1 -- start start 2024-09-16T00:23:41.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:41.039+0000 7f775d0bf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7758102750 0x7f77581971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:41.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:41.040+0000 7f775d0bf640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f775810bcc0 con 0x7f7758102750 2024-09-16T00:23:41.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:41.040+0000 7f7756d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7758102750 0x7f77581971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:41.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:41.040+0000 7f7756d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7758102750 0x7f77581971c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60060/0 (socket says 172.21.15.38:60060) 2024-09-16T00:23:41.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:41.040+0000 7f7756d9d640 1 -- 172.21.15.38:0/2781548528 learned_addr learned my addr 172.21.15.38:0/2781548528 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:23:41.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:41.040+0000 7f7756d9d640 1 -- 172.21.15.38:0/2781548528 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7758197700 con 0x7f7758102750 2024-09-16T00:23:41.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:41.040+0000 7f7756d9d640 1 --2- 172.21.15.38:0/2781548528 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7758102750 0x7f77581971c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7748009230 tx=0x7f7748003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:41.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:41.040+0000 7f773ffff640 1 -- 172.21.15.38:0/2781548528 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f774803f070 con 0x7f7758102750 2024-09-16T00:23:41.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:41.040+0000 7f773ffff640 1 -- 172.21.15.38:0/2781548528 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f774803bc80 con 0x7f7758102750 2024-09-16T00:23:41.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:41.040+0000 7f775d0bf640 1 -- 172.21.15.38:0/2781548528 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7758197990 con 0x7f7758102750 2024-09-16T00:23:41.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:41.040+0000 7f773ffff640 1 -- 172.21.15.38:0/2781548528 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7748035030 con 0x7f7758102750 2024-09-16T00:23:41.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:41.040+0000 7f775d0bf640 1 -- 172.21.15.38:0/2781548528 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7758197df0 con 0x7f7758102750 2024-09-16T00:23:41.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:41.041+0000 7f773ffff640 1 -- 172.21.15.38:0/2781548528 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f7748042070 con 0x7f7758102750 2024-09-16T00:23:41.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:41.042+0000 7f773ffff640 1 --2- 172.21.15.38:0/2781548528 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f77400392a0 0x7f774003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:41.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:41.042+0000 7f775d0bf640 1 -- 172.21.15.38:0/2781548528 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f775806a7b0 con 0x7f7758102750 2024-09-16T00:23:41.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:41.042+0000 7f773ffff640 1 -- 172.21.15.38:0/2781548528 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f774806cd40 con 0x7f7758102750 2024-09-16T00:23:41.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:41.042+0000 7f775659c640 1 --2- 172.21.15.38:0/2781548528 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f77400392a0 0x7f774003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:41.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:41.042+0000 7f775659c640 1 --2- 172.21.15.38:0/2781548528 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f77400392a0 0x7f774003b760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f774c0098c0 tx=0x7f774c006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:41.050 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:41.046+0000 7f773ffff640 1 -- 172.21.15.38:0/2781548528 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7748072020 con 0x7f7758102750 2024-09-16T00:23:41.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:41.226+0000 7f775d0bf640 1 -- 172.21.15.38:0/2781548528 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7758198520 con 0x7f77400392a0 2024-09-16T00:23:41.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:41.227+0000 7f773ffff640 1 -- 172.21.15.38:0/2781548528 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f7758198520 con 0x7f77400392a0 2024-09-16T00:23:41.227 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:23:41.227 INFO:teuthology.orchestra.run.smithi038.stdout:[{"addr": "172.21.15.38", "hostname": "smithi038", "labels": [], "status": ""}, {"addr": "172.21.15.73", "hostname": "smithi073", "labels": [], "status": ""}] 2024-09-16T00:23:41.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:41.229+0000 7f775d0bf640 1 -- 172.21.15.38:0/2781548528 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f77400392a0 msgr2=0x7f774003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:41.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:41.229+0000 7f775d0bf640 1 --2- 172.21.15.38:0/2781548528 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f77400392a0 0x7f774003b760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f774c0098c0 tx=0x7f774c006db0 comp rx=0 tx=0).stop 2024-09-16T00:23:41.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:41.229+0000 7f775d0bf640 1 -- 172.21.15.38:0/2781548528 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7758102750 msgr2=0x7f77581971c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:41.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:41.229+0000 7f775d0bf640 1 --2- 172.21.15.38:0/2781548528 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7758102750 0x7f77581971c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7748009230 tx=0x7f7748003d40 comp rx=0 tx=0).stop 2024-09-16T00:23:41.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:41.229+0000 7f775d0bf640 1 -- 172.21.15.38:0/2781548528 shutdown_connections 2024-09-16T00:23:41.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:41.229+0000 7f775d0bf640 1 --2- 172.21.15.38:0/2781548528 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f77400392a0 0x7f774003b760 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:41.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:41.229+0000 7f775d0bf640 1 --2- 172.21.15.38:0/2781548528 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7758102750 0x7f77581971c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:41.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:41.229+0000 7f775d0bf640 1 -- 172.21.15.38:0/2781548528 >> 172.21.15.38:0/2781548528 conn(0x7f77580fc1b0 msgr2=0x7f7758103500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:41.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:41.229+0000 7f775d0bf640 1 -- 172.21.15.38:0/2781548528 shutdown_connections 2024-09-16T00:23:41.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:41.229+0000 7f775d0bf640 1 -- 172.21.15.38:0/2781548528 wait complete. 2024-09-16T00:23:41.879 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi122 2024-09-16T00:23:41.879 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-16T00:23:41.879 DEBUG:teuthology.orchestra.run.smithi122:> dd of=/etc/ceph/ceph.conf 2024-09-16T00:23:41.895 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-16T00:23:41.895 DEBUG:teuthology.orchestra.run.smithi122:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:23:41.950 INFO:tasks.cephadm:Adding host smithi122 to orchestrator... 2024-09-16T00:23:41.950 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph orch host add smithi122 2024-09-16T00:23:42.211 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:23:42.214 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:42 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:42.214 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:42 smithi038 ceph-mon[30945]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:23:42.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:42.662+0000 7fd5885fd640 1 -- 172.21.15.38:0/588534110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd580106560 msgr2=0x7fd580106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:42.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:42.662+0000 7fd5885fd640 1 --2- 172.21.15.38:0/588534110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd580106560 0x7fd580106940 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009880 tx=0x7fd57c02a5d0 comp rx=0 tx=0).stop 2024-09-16T00:23:42.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:42.662+0000 7fd5885fd640 1 -- 172.21.15.38:0/588534110 shutdown_connections 2024-09-16T00:23:42.672 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:42.662+0000 7fd5885fd640 1 --2- 172.21.15.38:0/588534110 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd580106560 0x7fd580106940 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:42.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:42.662+0000 7fd5885fd640 1 -- 172.21.15.38:0/588534110 >> 172.21.15.38:0/588534110 conn(0x7fd5800fc1b0 msgr2=0x7fd5800fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:42.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:42.662+0000 7fd5885fd640 1 -- 172.21.15.38:0/588534110 shutdown_connections 2024-09-16T00:23:42.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:42.662+0000 7fd5885fd640 1 -- 172.21.15.38:0/588534110 wait complete. 2024-09-16T00:23:42.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:42.662+0000 7fd5885fd640 1 Processor -- start 2024-09-16T00:23:42.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:42.663+0000 7fd5885fd640 1 -- start start 2024-09-16T00:23:42.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:42.663+0000 7fd5885fd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd580106560 0x7fd5800fc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:42.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:42.663+0000 7fd5885fd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd58010ab90 con 0x7fd580106560 2024-09-16T00:23:42.673 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:42.663+0000 7fd586399640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd580106560 0x7fd5800fc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:42.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:42.663+0000 7fd586399640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd580106560 0x7fd5800fc660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:60092/0 (socket says 172.21.15.38:60092) 2024-09-16T00:23:42.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:42.663+0000 7fd586399640 1 -- 172.21.15.38:0/2262828881 learned_addr learned my addr 172.21.15.38:0/2262828881 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:23:42.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:42.663+0000 7fd586399640 1 -- 172.21.15.38:0/2262828881 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5800fe550 con 0x7fd580106560 2024-09-16T00:23:42.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:42.663+0000 7fd586399640 1 --2- 172.21.15.38:0/2262828881 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd580106560 0x7fd5800fc660 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009230 tx=0x7fd57c003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:42.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:42.663+0000 7fd5777fe640 1 -- 172.21.15.38:0/2262828881 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd57c03f070 con 0x7fd580106560 2024-09-16T00:23:42.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:42.664+0000 7fd5777fe640 1 -- 172.21.15.38:0/2262828881 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd57c03bc80 con 0x7fd580106560 2024-09-16T00:23:42.674 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:42.664+0000 7fd5885fd640 1 -- 172.21.15.38:0/2262828881 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5800fcba0 con 0x7fd580106560 2024-09-16T00:23:42.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:42.664+0000 7fd5777fe640 1 -- 172.21.15.38:0/2262828881 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd57c035030 con 0x7fd580106560 2024-09-16T00:23:42.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:42.664+0000 7fd5885fd640 1 -- 172.21.15.38:0/2262828881 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5800fd060 con 0x7fd580106560 2024-09-16T00:23:42.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:42.664+0000 7fd5777fe640 1 -- 172.21.15.38:0/2262828881 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fd57c042070 con 0x7fd580106560 2024-09-16T00:23:42.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:42.664+0000 7fd5885fd640 1 -- 172.21.15.38:0/2262828881 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd580101bd0 con 0x7fd580106560 2024-09-16T00:23:42.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:42.665+0000 7fd5777fe640 1 --2- 172.21.15.38:0/2262828881 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd56c039300 0x7fd56c03b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:42.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:42.665+0000 7fd5777fe640 1 -- 172.21.15.38:0/2262828881 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd57c06cbe0 con 0x7fd580106560 2024-09-16T00:23:42.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:42.665+0000 7fd585b98640 1 --2- 172.21.15.38:0/2262828881 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd56c039300 0x7fd56c03b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:42.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:42.665+0000 7fd585b98640 1 --2- 172.21.15.38:0/2262828881 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd56c039300 0x7fd56c03b7c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd5700098c0 tx=0x7fd570006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:42.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:42.669+0000 7fd5777fe640 1 -- 172.21.15.38:0/2262828881 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd57c0314c0 con 0x7fd580106560 2024-09-16T00:23:42.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:42.850+0000 7fd5885fd640 1 -- 172.21.15.38:0/2262828881 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi122", "target": ["mon-mgr", ""]}) -- 0x7fd5800fdc00 con 0x7fd56c039300 2024-09-16T00:23:43.214 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:43 smithi038 ceph-mon[30945]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:23:44.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:44 smithi038 ceph-mon[30945]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi122", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:23:44.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:44 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:44.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:44 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:44.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:44 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:44.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:44 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:44.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:44 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi073", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:23:44.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:44 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:44.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:44 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:23:45.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:45 smithi038 ceph-mon[30945]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:23:45.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:45 smithi038 ceph-mon[30945]: Updating smithi073:/etc/ceph/ceph.conf 2024-09-16T00:23:45.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:45 smithi038 ceph-mon[30945]: Deploying cephadm binary to smithi122 2024-09-16T00:23:45.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:45 smithi038 ceph-mon[30945]: Updating smithi073:/var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:23:45.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:45 smithi038 ceph-mon[30945]: Updating smithi073:/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:23:45.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:45 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:45.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:45 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:45.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:45 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:46.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:46 smithi038 ceph-mon[30945]: Updating smithi073:/var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T00:23:46.787 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:46.787+0000 7fd5777fe640 1 -- 172.21.15.38:0/2262828881 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fd5800fdc00 con 0x7fd56c039300 2024-09-16T00:23:46.788 INFO:teuthology.orchestra.run.smithi038.stdout:Added host 'smithi122' with addr '172.21.15.122' 2024-09-16T00:23:46.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:46.790+0000 7fd5885fd640 1 -- 172.21.15.38:0/2262828881 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd56c039300 msgr2=0x7fd56c03b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:46.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:46.790+0000 7fd5885fd640 1 --2- 172.21.15.38:0/2262828881 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd56c039300 0x7fd56c03b7c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd5700098c0 tx=0x7fd570006db0 comp rx=0 tx=0).stop 2024-09-16T00:23:46.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:46.790+0000 7fd5885fd640 1 -- 172.21.15.38:0/2262828881 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd580106560 msgr2=0x7fd5800fc660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:46.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:46.790+0000 7fd5885fd640 1 --2- 172.21.15.38:0/2262828881 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd580106560 0x7fd5800fc660 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd57c009230 tx=0x7fd57c003d40 comp rx=0 tx=0).stop 2024-09-16T00:23:46.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:46.790+0000 7fd5885fd640 1 -- 172.21.15.38:0/2262828881 shutdown_connections 2024-09-16T00:23:46.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:46.790+0000 7fd5885fd640 1 --2- 172.21.15.38:0/2262828881 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd56c039300 0x7fd56c03b7c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:46.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:46.790+0000 7fd5885fd640 1 --2- 172.21.15.38:0/2262828881 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd580106560 0x7fd5800fc660 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:46.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:46.790+0000 7fd5885fd640 1 -- 172.21.15.38:0/2262828881 >> 172.21.15.38:0/2262828881 conn(0x7fd5800fc1b0 msgr2=0x7fd580069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:46.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:46.790+0000 7fd5885fd640 1 -- 172.21.15.38:0/2262828881 shutdown_connections 2024-09-16T00:23:46.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:46.790+0000 7fd5885fd640 1 -- 172.21.15.38:0/2262828881 wait complete. 2024-09-16T00:23:47.392 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:47 smithi038 ceph-mon[30945]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:23:47.392 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:47 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:47.392 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:47 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:23:47.426 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph orch host ls --format=json 2024-09-16T00:23:47.678 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:23:48.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:48.137+0000 7f3934c65640 1 -- 172.21.15.38:0/589304455 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3930106540 msgr2=0x7f3930106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:48.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:48.137+0000 7f3934c65640 1 --2- 172.21.15.38:0/589304455 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3930106540 0x7f3930106920 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f3920009810 tx=0x7f392002a540 comp rx=0 tx=0).stop 2024-09-16T00:23:48.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:48.137+0000 7f3934c65640 1 -- 172.21.15.38:0/589304455 shutdown_connections 2024-09-16T00:23:48.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:48.137+0000 7f3934c65640 1 --2- 172.21.15.38:0/589304455 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3930106540 0x7f3930106920 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:48.137+0000 7f3934c65640 1 -- 172.21.15.38:0/589304455 >> 172.21.15.38:0/589304455 conn(0x7f3930077310 msgr2=0x7f39300fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:48.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:48.137+0000 7f3934c65640 1 -- 172.21.15.38:0/589304455 shutdown_connections 2024-09-16T00:23:48.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:48.137+0000 7f3934c65640 1 -- 172.21.15.38:0/589304455 wait complete. 2024-09-16T00:23:48.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:48.138+0000 7f3934c65640 1 Processor -- start 2024-09-16T00:23:48.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:48.138+0000 7f3934c65640 1 -- start start 2024-09-16T00:23:48.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:48.138+0000 7f3934c65640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3930106540 0x7f39301993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:48.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:48.138+0000 7f3934c65640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f393010ab70 con 0x7f3930106540 2024-09-16T00:23:48.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:48.138+0000 7f392e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3930106540 0x7f39301993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:48.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:48.138+0000 7f392e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3930106540 0x7f39301993c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49234/0 (socket says 172.21.15.38:49234) 2024-09-16T00:23:48.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:48.138+0000 7f392e59c640 1 -- 172.21.15.38:0/696702742 learned_addr learned my addr 172.21.15.38:0/696702742 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:23:48.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:48.138+0000 7f392e59c640 1 -- 172.21.15.38:0/696702742 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3930199900 con 0x7f3930106540 2024-09-16T00:23:48.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:48.139+0000 7f392e59c640 1 --2- 172.21.15.38:0/696702742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3930106540 0x7f39301993c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3920009930 tx=0x7f3920004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:48.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:48.139+0000 7f391f7fe640 1 -- 172.21.15.38:0/696702742 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3920031400 con 0x7f3930106540 2024-09-16T00:23:48.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:48.139+0000 7f391f7fe640 1 -- 172.21.15.38:0/696702742 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f39200035f0 con 0x7f3930106540 2024-09-16T00:23:48.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:48.139+0000 7f3934c65640 1 -- 172.21.15.38:0/696702742 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3930199b90 con 0x7f3930106540 2024-09-16T00:23:48.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:48.139+0000 7f391f7fe640 1 -- 172.21.15.38:0/696702742 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f392003c440 con 0x7f3930106540 2024-09-16T00:23:48.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:48.139+0000 7f3934c65640 1 -- 172.21.15.38:0/696702742 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3930199ff0 con 0x7f3930106540 2024-09-16T00:23:48.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:48.140+0000 7f3934c65640 1 -- 172.21.15.38:0/696702742 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3930101bb0 con 0x7f3930106540 2024-09-16T00:23:48.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:48.140+0000 7f391f7fe640 1 -- 172.21.15.38:0/696702742 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f3920039040 con 0x7f3930106540 2024-09-16T00:23:48.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:48.140+0000 7f391f7fe640 1 --2- 172.21.15.38:0/696702742 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f39180392a0 0x7f391803b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:48.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:48.140+0000 7f391f7fe640 1 -- 172.21.15.38:0/696702742 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f392006da40 con 0x7f3930106540 2024-09-16T00:23:48.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:48.141+0000 7f392dd9b640 1 --2- 172.21.15.38:0/696702742 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f39180392a0 0x7f391803b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:48.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:48.141+0000 7f392dd9b640 1 --2- 172.21.15.38:0/696702742 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f39180392a0 0x7f391803b760 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f39240098c0 tx=0x7f3924006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:48.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:48.145+0000 7f391f7fe640 1 -- 172.21.15.38:0/696702742 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3920040030 con 0x7f3930106540 2024-09-16T00:23:48.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:48.324+0000 7f3934c65640 1 -- 172.21.15.38:0/696702742 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f393019a720 con 0x7f39180392a0 2024-09-16T00:23:48.325 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:48 smithi038 ceph-mon[30945]: Added host smithi122 2024-09-16T00:23:48.325 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:48 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:48.325 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:48 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:48.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:48.325+0000 7f391f7fe640 1 -- 172.21.15.38:0/696702742 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f393019a720 con 0x7f39180392a0 2024-09-16T00:23:48.325 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:23:48.325 INFO:teuthology.orchestra.run.smithi038.stdout:[{"addr": "172.21.15.38", "hostname": "smithi038", "labels": [], "status": ""}, {"addr": "172.21.15.73", "hostname": "smithi073", "labels": [], "status": ""}, {"addr": "172.21.15.122", "hostname": "smithi122", "labels": [], "status": ""}] 2024-09-16T00:23:48.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:48.327+0000 7f3934c65640 1 -- 172.21.15.38:0/696702742 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f39180392a0 msgr2=0x7f391803b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:48.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:48.327+0000 7f3934c65640 1 --2- 172.21.15.38:0/696702742 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f39180392a0 0x7f391803b760 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f39240098c0 tx=0x7f3924006db0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:48.327+0000 7f3934c65640 1 -- 172.21.15.38:0/696702742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3930106540 msgr2=0x7f39301993c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:48.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:48.327+0000 7f3934c65640 1 --2- 172.21.15.38:0/696702742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3930106540 0x7f39301993c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f3920009930 tx=0x7f3920004570 comp rx=0 tx=0).stop 2024-09-16T00:23:48.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:48.327+0000 7f3934c65640 1 -- 172.21.15.38:0/696702742 shutdown_connections 2024-09-16T00:23:48.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:48.327+0000 7f3934c65640 1 --2- 172.21.15.38:0/696702742 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f39180392a0 0x7f391803b760 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:48.327+0000 7f3934c65640 1 --2- 172.21.15.38:0/696702742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3930106540 0x7f39301993c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:48.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:48.327+0000 7f3934c65640 1 -- 172.21.15.38:0/696702742 >> 172.21.15.38:0/696702742 conn(0x7f3930077310 msgr2=0x7f39300fdc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:48.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:48.327+0000 7f3934c65640 1 -- 172.21.15.38:0/696702742 shutdown_connections 2024-09-16T00:23:48.328 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:48.327+0000 7f3934c65640 1 -- 172.21.15.38:0/696702742 wait complete. 2024-09-16T00:23:48.957 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-16T00:23:48.957 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd crush tunables default 2024-09-16T00:23:49.207 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:23:49.214 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:49 smithi038 ceph-mon[30945]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:23:49.214 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:49 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:49.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:49 smithi038 ceph-mon[30945]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:23:49.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:49.651+0000 7fbb87cc1640 1 -- 172.21.15.38:0/2218927383 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb80106560 msgr2=0x7fbb80106940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:49.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:49.651+0000 7fbb87cc1640 1 --2- 172.21.15.38:0/2218927383 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb80106560 0x7fbb80106940 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c009880 tx=0x7fbb7c02a5d0 comp rx=0 tx=0).stop 2024-09-16T00:23:49.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:49.651+0000 7fbb87cc1640 1 -- 172.21.15.38:0/2218927383 shutdown_connections 2024-09-16T00:23:49.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:49.651+0000 7fbb87cc1640 1 --2- 172.21.15.38:0/2218927383 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb80106560 0x7fbb80106940 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:49.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:49.651+0000 7fbb87cc1640 1 -- 172.21.15.38:0/2218927383 >> 172.21.15.38:0/2218927383 conn(0x7fbb800fc1b0 msgr2=0x7fbb800fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:49.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:49.651+0000 7fbb87cc1640 1 -- 172.21.15.38:0/2218927383 shutdown_connections 2024-09-16T00:23:49.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:49.651+0000 7fbb87cc1640 1 -- 172.21.15.38:0/2218927383 wait complete. 2024-09-16T00:23:49.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:49.652+0000 7fbb87cc1640 1 Processor -- start 2024-09-16T00:23:49.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:49.652+0000 7fbb87cc1640 1 -- start start 2024-09-16T00:23:49.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:49.652+0000 7fbb87cc1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb80106560 0x7fbb80194ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:49.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:49.652+0000 7fbb87cc1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbb8010ab90 con 0x7fbb80106560 2024-09-16T00:23:49.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:49.652+0000 7fbb85a5d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb80106560 0x7fbb80194ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:49.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:49.652+0000 7fbb85a5d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb80106560 0x7fbb80194ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49250/0 (socket says 172.21.15.38:49250) 2024-09-16T00:23:49.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:49.653+0000 7fbb85a5d640 1 -- 172.21.15.38:0/1343175778 learned_addr learned my addr 172.21.15.38:0/1343175778 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:23:49.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:49.653+0000 7fbb85a5d640 1 -- 172.21.15.38:0/1343175778 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb80195410 con 0x7fbb80106560 2024-09-16T00:23:49.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:49.653+0000 7fbb85a5d640 1 --2- 172.21.15.38:0/1343175778 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb80106560 0x7fbb80194ed0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c009230 tx=0x7fbb7c003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:49.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:49.653+0000 7fbb76ffd640 1 -- 172.21.15.38:0/1343175778 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb7c03f070 con 0x7fbb80106560 2024-09-16T00:23:49.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:49.653+0000 7fbb87cc1640 1 -- 172.21.15.38:0/1343175778 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb801956a0 con 0x7fbb80106560 2024-09-16T00:23:49.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:49.653+0000 7fbb76ffd640 1 -- 172.21.15.38:0/1343175778 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbb7c03bc80 con 0x7fbb80106560 2024-09-16T00:23:49.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:49.653+0000 7fbb76ffd640 1 -- 172.21.15.38:0/1343175778 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb7c035030 con 0x7fbb80106560 2024-09-16T00:23:49.655 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:49.653+0000 7fbb87cc1640 1 -- 172.21.15.38:0/1343175778 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb80195b00 con 0x7fbb80106560 2024-09-16T00:23:49.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:49.654+0000 7fbb76ffd640 1 -- 172.21.15.38:0/1343175778 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fbb7c042070 con 0x7fbb80106560 2024-09-16T00:23:49.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:49.654+0000 7fbb87cc1640 1 -- 172.21.15.38:0/1343175778 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbb80101bd0 con 0x7fbb80106560 2024-09-16T00:23:49.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:49.655+0000 7fbb76ffd640 1 --2- 172.21.15.38:0/1343175778 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fbb6c0392a0 0x7fbb6c03b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:49.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:49.655+0000 7fbb76ffd640 1 -- 172.21.15.38:0/1343175778 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbb7c06d1d0 con 0x7fbb80106560 2024-09-16T00:23:49.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:49.655+0000 7fbb8525c640 1 --2- 172.21.15.38:0/1343175778 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fbb6c0392a0 0x7fbb6c03b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:49.656 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:49.655+0000 7fbb8525c640 1 --2- 172.21.15.38:0/1343175778 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fbb6c0392a0 0x7fbb6c03b760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbb700098c0 tx=0x7fbb70006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:49.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:49.659+0000 7fbb76ffd640 1 -- 172.21.15.38:0/1343175778 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbb7c04c350 con 0x7fbb80106560 2024-09-16T00:23:49.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:49.839+0000 7fbb87cc1640 1 -- 172.21.15.38:0/1343175778 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fbb80196230 con 0x7fbb80106560 2024-09-16T00:23:50.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:50.164+0000 7fbb76ffd640 1 -- 172.21.15.38:0/1343175778 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fbb7c031e20 con 0x7fbb80106560 2024-09-16T00:23:50.165 INFO:teuthology.orchestra.run.smithi038.stderr:adjusted tunables profile to default 2024-09-16T00:23:50.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:50.166+0000 7fbb87cc1640 1 -- 172.21.15.38:0/1343175778 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fbb6c0392a0 msgr2=0x7fbb6c03b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:50.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:50.166+0000 7fbb87cc1640 1 --2- 172.21.15.38:0/1343175778 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fbb6c0392a0 0x7fbb6c03b760 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbb700098c0 tx=0x7fbb70006db0 comp rx=0 tx=0).stop 2024-09-16T00:23:50.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:50.166+0000 7fbb87cc1640 1 -- 172.21.15.38:0/1343175778 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb80106560 msgr2=0x7fbb80194ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:50.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:50.166+0000 7fbb87cc1640 1 --2- 172.21.15.38:0/1343175778 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb80106560 0x7fbb80194ed0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c009230 tx=0x7fbb7c003d40 comp rx=0 tx=0).stop 2024-09-16T00:23:50.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:50.166+0000 7fbb87cc1640 1 -- 172.21.15.38:0/1343175778 shutdown_connections 2024-09-16T00:23:50.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:50.166+0000 7fbb87cc1640 1 --2- 172.21.15.38:0/1343175778 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fbb6c0392a0 0x7fbb6c03b760 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:50.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:50.166+0000 7fbb87cc1640 1 --2- 172.21.15.38:0/1343175778 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbb80106560 0x7fbb80194ed0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:50.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:50.166+0000 7fbb87cc1640 1 -- 172.21.15.38:0/1343175778 >> 172.21.15.38:0/1343175778 conn(0x7fbb800fc1b0 msgr2=0x7fbb800fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:50.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:50.166+0000 7fbb87cc1640 1 -- 172.21.15.38:0/1343175778 shutdown_connections 2024-09-16T00:23:50.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:23:50.166+0000 7fbb87cc1640 1 -- 172.21.15.38:0/1343175778 wait complete. 2024-09-16T00:23:50.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:50 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/1343175778' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-16T00:23:50.758 INFO:tasks.cephadm:Adding mon.a on smithi038 2024-09-16T00:23:50.758 INFO:tasks.cephadm:Adding mon.b on smithi073 2024-09-16T00:23:50.758 INFO:tasks.cephadm:Adding mon.c on smithi122 2024-09-16T00:23:50.759 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph orch apply mon '3;smithi038:172.21.15.38=a;smithi073:172.21.15.73=b;smithi122:172.21.15.122=c' 2024-09-16T00:23:51.073 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T00:23:51.134 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-16T00:23:51.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:51 smithi038 ceph-mon[30945]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:23:51.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:51 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/1343175778' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-16T00:23:51.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:51 smithi038 ceph-mon[30945]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:23:51.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:51 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:51.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:51 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:51.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:51 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:51.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:51 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:51.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:51 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi122", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:23:51.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:51 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:51.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:51 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:23:51.643 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:51.609+0000 7f5df0844640 1 -- 172.21.15.122:0/1780356389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5dec1004f0 msgr2=0x7f5dec1008f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:51.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:51.609+0000 7f5df0844640 1 --2- 172.21.15.122:0/1780356389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5dec1004f0 0x7f5dec1008f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc009880 tx=0x7f5ddc02a5d0 comp rx=0 tx=0).stop 2024-09-16T00:23:51.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:51.609+0000 7f5df0844640 1 -- 172.21.15.122:0/1780356389 shutdown_connections 2024-09-16T00:23:51.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:51.609+0000 7f5df0844640 1 --2- 172.21.15.122:0/1780356389 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5dec1004f0 0x7f5dec1008f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:51.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:51.609+0000 7f5df0844640 1 -- 172.21.15.122:0/1780356389 >> 172.21.15.122:0/1780356389 conn(0x7f5dec0fbc60 msgr2=0x7f5dec0fe0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:51.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:51.609+0000 7f5df0844640 1 -- 172.21.15.122:0/1780356389 shutdown_connections 2024-09-16T00:23:51.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:51.609+0000 7f5df0844640 1 -- 172.21.15.122:0/1780356389 wait complete. 2024-09-16T00:23:51.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:51.610+0000 7f5df0844640 1 Processor -- start 2024-09-16T00:23:51.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:51.610+0000 7f5df0844640 1 -- start start 2024-09-16T00:23:51.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:51.610+0000 7f5df0844640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5dec1004f0 0x7f5dec196da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:51.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:51.610+0000 7f5df0844640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5dec106600 con 0x7f5dec1004f0 2024-09-16T00:23:51.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:51.610+0000 7f5dea59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5dec1004f0 0x7f5dec196da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:51.644 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:51.610+0000 7f5dea59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5dec1004f0 0x7f5dec196da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.122:54468/0 (socket says 172.21.15.122:54468) 2024-09-16T00:23:51.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:51.610+0000 7f5dea59c640 1 -- 172.21.15.122:0/3051832533 learned_addr learned my addr 172.21.15.122:0/3051832533 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-16T00:23:51.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:51.611+0000 7f5dea59c640 1 -- 172.21.15.122:0/3051832533 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dec1972e0 con 0x7f5dec1004f0 2024-09-16T00:23:51.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:51.611+0000 7f5dea59c640 1 --2- 172.21.15.122:0/3051832533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5dec1004f0 0x7f5dec196da0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc004f60 tx=0x7f5ddc0048f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:51.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:51.611+0000 7f5ddb7fe640 1 -- 172.21.15.122:0/3051832533 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ddc03f070 con 0x7f5dec1004f0 2024-09-16T00:23:51.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:51.611+0000 7f5ddb7fe640 1 -- 172.21.15.122:0/3051832533 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5ddc004d40 con 0x7f5dec1004f0 2024-09-16T00:23:51.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:51.611+0000 7f5df0844640 1 -- 172.21.15.122:0/3051832533 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dec197570 con 0x7f5dec1004f0 2024-09-16T00:23:51.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:51.611+0000 7f5ddb7fe640 1 -- 172.21.15.122:0/3051832533 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ddc035030 con 0x7f5dec1004f0 2024-09-16T00:23:51.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:51.611+0000 7f5df0844640 1 -- 172.21.15.122:0/3051832533 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dec19a260 con 0x7f5dec1004f0 2024-09-16T00:23:51.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:51.612+0000 7f5ddb7fe640 1 -- 172.21.15.122:0/3051832533 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f5ddc042070 con 0x7f5dec1004f0 2024-09-16T00:23:51.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:51.612+0000 7f5df0844640 1 -- 172.21.15.122:0/3051832533 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dec106d80 con 0x7f5dec1004f0 2024-09-16T00:23:51.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:51.613+0000 7f5ddb7fe640 1 --2- 172.21.15.122:0/3051832533 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f5dd40392a0 0x7f5dd403b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:51.645 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:51.613+0000 7f5ddb7fe640 1 -- 172.21.15.122:0/3051832533 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5ddc06d070 con 0x7f5dec1004f0 2024-09-16T00:23:51.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:51.613+0000 7f5de9d9b640 1 --2- 172.21.15.122:0/3051832533 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f5dd40392a0 0x7f5dd403b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:51.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:51.613+0000 7f5de9d9b640 1 --2- 172.21.15.122:0/3051832533 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f5dd40392a0 0x7f5dd403b760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5de00098c0 tx=0x7f5de0006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:51.646 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:51.618+0000 7f5ddb7fe640 1 -- 172.21.15.122:0/3051832533 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5ddc042360 con 0x7f5dec1004f0 2024-09-16T00:23:51.815 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:51.814+0000 7f5df0844640 1 -- 172.21.15.122:0/3051832533 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi038:172.21.15.38=a;smithi073:172.21.15.73=b;smithi122:172.21.15.122=c", "target": ["mon-mgr", ""]}) -- 0x7f5dec0fd570 con 0x7f5dd40392a0 2024-09-16T00:23:51.825 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:51.825+0000 7f5ddb7fe640 1 -- 172.21.15.122:0/3051832533 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f5dec0fd570 con 0x7f5dd40392a0 2024-09-16T00:23:51.825 INFO:teuthology.orchestra.run.smithi122.stdout:Scheduled mon update... 2024-09-16T00:23:51.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:51.827+0000 7f5df0844640 1 -- 172.21.15.122:0/3051832533 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f5dd40392a0 msgr2=0x7f5dd403b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:51.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:51.827+0000 7f5df0844640 1 --2- 172.21.15.122:0/3051832533 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f5dd40392a0 0x7f5dd403b760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5de00098c0 tx=0x7f5de0006db0 comp rx=0 tx=0).stop 2024-09-16T00:23:51.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:51.827+0000 7f5df0844640 1 -- 172.21.15.122:0/3051832533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5dec1004f0 msgr2=0x7f5dec196da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:51.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:51.827+0000 7f5df0844640 1 --2- 172.21.15.122:0/3051832533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5dec1004f0 0x7f5dec196da0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc004f60 tx=0x7f5ddc0048f0 comp rx=0 tx=0).stop 2024-09-16T00:23:51.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:51.827+0000 7f5df0844640 1 -- 172.21.15.122:0/3051832533 shutdown_connections 2024-09-16T00:23:51.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:51.827+0000 7f5df0844640 1 --2- 172.21.15.122:0/3051832533 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f5dd40392a0 0x7f5dd403b760 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:51.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:51.827+0000 7f5df0844640 1 --2- 172.21.15.122:0/3051832533 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5dec1004f0 0x7f5dec196da0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:51.828 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:51.827+0000 7f5df0844640 1 -- 172.21.15.122:0/3051832533 >> 172.21.15.122:0/3051832533 conn(0x7f5dec0fbc60 msgr2=0x7f5dec0fc890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:51.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:51.827+0000 7f5df0844640 1 -- 172.21.15.122:0/3051832533 shutdown_connections 2024-09-16T00:23:51.829 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:51.827+0000 7f5df0844640 1 -- 172.21.15.122:0/3051832533 wait complete. 2024-09-16T00:23:52.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:52 smithi038 ceph-mon[30945]: Updating smithi122:/etc/ceph/ceph.conf 2024-09-16T00:23:52.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:52 smithi038 ceph-mon[30945]: Updating smithi122:/var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:23:52.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:52 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:52.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:52 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:52.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:52 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:52.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:52 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:52.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:52 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:23:52.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:52 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:52.475 DEBUG:teuthology.orchestra.run.smithi073:mon.b> sudo journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@mon.b.service 2024-09-16T00:23:52.478 DEBUG:teuthology.orchestra.run.smithi122:mon.c> sudo journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@mon.c.service 2024-09-16T00:23:52.481 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T00:23:52.481 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T00:23:53.198 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:23:53.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:53 smithi038 ceph-mon[30945]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:23:53.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:53 smithi038 ceph-mon[30945]: Updating smithi122:/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:23:53.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:53 smithi038 ceph-mon[30945]: Updating smithi122:/var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T00:23:53.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:53 smithi038 ceph-mon[30945]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi038:172.21.15.38=a;smithi073:172.21.15.73=b;smithi122:172.21.15.122=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:23:53.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:53 smithi038 ceph-mon[30945]: Saving service mon spec with placement smithi038:172.21.15.38=a;smithi073:172.21.15.73=b;smithi122:172.21.15.122=c;count:3 2024-09-16T00:23:53.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:53 smithi038 ceph-mon[30945]: Deploying daemon mon.c on smithi122 2024-09-16T00:23:53.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:53.598+0000 7f47e90e6640 1 -- 172.21.15.122:0/3449733238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47e4107660 msgr2=0x7f47e4107a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:53.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:53.598+0000 7f47e90e6640 1 --2- 172.21.15.122:0/3449733238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47e4107660 0x7f47e4107a40 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f47cc009880 tx=0x7f47cc02a5d0 comp rx=0 tx=0).stop 2024-09-16T00:23:53.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:53.598+0000 7f47e90e6640 1 -- 172.21.15.122:0/3449733238 shutdown_connections 2024-09-16T00:23:53.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:53.598+0000 7f47e90e6640 1 --2- 172.21.15.122:0/3449733238 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47e4107660 0x7f47e4107a40 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:53.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:53.598+0000 7f47e90e6640 1 -- 172.21.15.122:0/3449733238 >> 172.21.15.122:0/3449733238 conn(0x7f47e40fd290 msgr2=0x7f47e40ff6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:53.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:53.598+0000 7f47e90e6640 1 -- 172.21.15.122:0/3449733238 shutdown_connections 2024-09-16T00:23:53.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:53.598+0000 7f47e90e6640 1 -- 172.21.15.122:0/3449733238 wait complete. 2024-09-16T00:23:53.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:53.599+0000 7f47e90e6640 1 Processor -- start 2024-09-16T00:23:53.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:53.599+0000 7f47e90e6640 1 -- start start 2024-09-16T00:23:53.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:53.599+0000 7f47e90e6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47e4107660 0x7f47e4197120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:53.600 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:53.599+0000 7f47e90e6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f47e410bc90 con 0x7f47e4107660 2024-09-16T00:23:53.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:53.600+0000 7f47e2d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47e4107660 0x7f47e4197120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:53.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:53.600+0000 7f47e2d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47e4107660 0x7f47e4197120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.122:54472/0 (socket says 172.21.15.122:54472) 2024-09-16T00:23:53.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:53.600+0000 7f47e2d9d640 1 -- 172.21.15.122:0/2592937766 learned_addr learned my addr 172.21.15.122:0/2592937766 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-16T00:23:53.602 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:53.600+0000 7f47e2d9d640 1 -- 172.21.15.122:0/2592937766 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47e4197660 con 0x7f47e4107660 2024-09-16T00:23:53.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:53.600+0000 7f47e2d9d640 1 --2- 172.21.15.122:0/2592937766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47e4107660 0x7f47e4197120 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f47cc009230 tx=0x7f47cc003d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:53.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:53.601+0000 7f47dbfff640 1 -- 172.21.15.122:0/2592937766 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47cc03f070 con 0x7f47e4107660 2024-09-16T00:23:53.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:53.601+0000 7f47e90e6640 1 -- 172.21.15.122:0/2592937766 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47e41978f0 con 0x7f47e4107660 2024-09-16T00:23:53.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:53.601+0000 7f47dbfff640 1 -- 172.21.15.122:0/2592937766 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47cc03bc80 con 0x7f47e4107660 2024-09-16T00:23:53.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:53.601+0000 7f47e90e6640 1 -- 172.21.15.122:0/2592937766 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47e419b9f0 con 0x7f47e4107660 2024-09-16T00:23:53.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:53.601+0000 7f47dbfff640 1 -- 172.21.15.122:0/2592937766 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f47cc049060 con 0x7f47e4107660 2024-09-16T00:23:53.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:53.602+0000 7f47e90e6640 1 -- 172.21.15.122:0/2592937766 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47e4102cd0 con 0x7f47e4107660 2024-09-16T00:23:53.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:53.602+0000 7f47dbfff640 1 -- 172.21.15.122:0/2592937766 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f47cc042070 con 0x7f47e4107660 2024-09-16T00:23:53.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:53.602+0000 7f47dbfff640 1 --2- 172.21.15.122:0/2592937766 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f47d00392a0 0x7f47d003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:53.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:53.602+0000 7f47dbfff640 1 -- 172.21.15.122:0/2592937766 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f47cc06d070 con 0x7f47e4107660 2024-09-16T00:23:53.603 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:53.602+0000 7f47e259c640 1 --2- 172.21.15.122:0/2592937766 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f47d00392a0 0x7f47d003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:53.604 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:53.603+0000 7f47e259c640 1 --2- 172.21.15.122:0/2592937766 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f47d00392a0 0x7f47d003b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f47d40098c0 tx=0x7f47d4006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:53.607 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:53.607+0000 7f47dbfff640 1 -- 172.21.15.122:0/2592937766 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f47cc037e60 con 0x7f47e4107660 2024-09-16T00:23:53.843 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:53.843+0000 7f47e90e6640 1 -- 172.21.15.122:0/2592937766 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f47e4107ac0 con 0x7f47e4107660 2024-09-16T00:23:53.844 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:53.843+0000 7f47dbfff640 1 -- 172.21.15.122:0/2592937766 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f47cc031570 con 0x7f47e4107660 2024-09-16T00:23:53.844 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-16T00:23:53.844 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":1,"fsid":"983b2072-73c1-11ef-bceb-c7b262605968","modified":"2024-09-16T00:22:31.653387Z","created":"2024-09-16T00:22:31.653387Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T00:23:53.844 INFO:teuthology.orchestra.run.smithi122.stderr:dumped monmap epoch 1 2024-09-16T00:23:53.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:53.845+0000 7f47e90e6640 1 -- 172.21.15.122:0/2592937766 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f47d00392a0 msgr2=0x7f47d003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:53.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:53.845+0000 7f47e90e6640 1 --2- 172.21.15.122:0/2592937766 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f47d00392a0 0x7f47d003b760 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f47d40098c0 tx=0x7f47d4006db0 comp rx=0 tx=0).stop 2024-09-16T00:23:53.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:53.845+0000 7f47e90e6640 1 -- 172.21.15.122:0/2592937766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47e4107660 msgr2=0x7f47e4197120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:53.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:53.845+0000 7f47e90e6640 1 --2- 172.21.15.122:0/2592937766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47e4107660 0x7f47e4197120 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f47cc009230 tx=0x7f47cc003d40 comp rx=0 tx=0).stop 2024-09-16T00:23:53.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:53.846+0000 7f47e90e6640 1 -- 172.21.15.122:0/2592937766 shutdown_connections 2024-09-16T00:23:53.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:53.846+0000 7f47e90e6640 1 --2- 172.21.15.122:0/2592937766 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f47d00392a0 0x7f47d003b760 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:53.846 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:53.846+0000 7f47e90e6640 1 --2- 172.21.15.122:0/2592937766 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47e4107660 0x7f47e4197120 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:53.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:53.846+0000 7f47e90e6640 1 -- 172.21.15.122:0/2592937766 >> 172.21.15.122:0/2592937766 conn(0x7f47e40fd290 msgr2=0x7f47e40fd670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:53.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:53.846+0000 7f47e90e6640 1 -- 172.21.15.122:0/2592937766 shutdown_connections 2024-09-16T00:23:53.847 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:53.846+0000 7f47e90e6640 1 -- 172.21.15.122:0/2592937766 wait complete. 2024-09-16T00:23:54.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:54 smithi038 ceph-mon[30945]: from='client.? 172.21.15.122:0/2592937766' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:23:55.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:55 smithi038 ceph-mon[30945]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:23:55.696 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T00:23:55.697 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T00:23:56.025 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:23:56.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:56.668+0000 7f348be9a640 1 -- 172.21.15.122:0/4022706998 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34841028c0 msgr2=0x7f3484102ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:56.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:56.668+0000 7f348be9a640 1 --2- 172.21.15.122:0/4022706998 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34841028c0 0x7f3484102ca0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3480009880 tx=0x7f348002a670 comp rx=0 tx=0).stop 2024-09-16T00:23:56.670 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:56.669+0000 7f348be9a640 1 -- 172.21.15.122:0/4022706998 shutdown_connections 2024-09-16T00:23:56.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:56.669+0000 7f348be9a640 1 --2- 172.21.15.122:0/4022706998 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34841028c0 0x7f3484102ca0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:56.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:56.669+0000 7f348be9a640 1 -- 172.21.15.122:0/4022706998 >> 172.21.15.122:0/4022706998 conn(0x7f34840fc280 msgr2=0x7f34840fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:56.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:56.669+0000 7f348be9a640 1 -- 172.21.15.122:0/4022706998 shutdown_connections 2024-09-16T00:23:56.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:56.669+0000 7f348be9a640 1 -- 172.21.15.122:0/4022706998 wait complete. 2024-09-16T00:23:56.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:56.669+0000 7f348be9a640 1 Processor -- start 2024-09-16T00:23:56.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:56.669+0000 7f348be9a640 1 -- start start 2024-09-16T00:23:56.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:56.670+0000 7f348be9a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34841028c0 0x7f3484197350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:56.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:56.670+0000 7f348be9a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f348410bda0 con 0x7f34841028c0 2024-09-16T00:23:56.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:56.670+0000 7f3489c36640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34841028c0 0x7f3484197350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:56.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:56.670+0000 7f3489c36640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34841028c0 0x7f3484197350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.122:37100/0 (socket says 172.21.15.122:37100) 2024-09-16T00:23:56.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:56.670+0000 7f3489c36640 1 -- 172.21.15.122:0/542757212 learned_addr learned my addr 172.21.15.122:0/542757212 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-16T00:23:56.671 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:56.670+0000 7f3489c36640 1 -- 172.21.15.122:0/542757212 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3484197890 con 0x7f34841028c0 2024-09-16T00:23:56.672 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:56.670+0000 7f3489c36640 1 --2- 172.21.15.122:0/542757212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34841028c0 0x7f3484197350 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3480009230 tx=0x7f3480003d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:56.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:56.670+0000 7f347affd640 1 -- 172.21.15.122:0/542757212 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f348003f070 con 0x7f34841028c0 2024-09-16T00:23:56.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:56.671+0000 7f347affd640 1 -- 172.21.15.122:0/542757212 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f348003bd40 con 0x7f34841028c0 2024-09-16T00:23:56.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:56.671+0000 7f348be9a640 1 -- 172.21.15.122:0/542757212 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3484197b20 con 0x7f34841028c0 2024-09-16T00:23:56.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:56.671+0000 7f347affd640 1 -- 172.21.15.122:0/542757212 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3480035030 con 0x7f34841028c0 2024-09-16T00:23:56.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:56.671+0000 7f348be9a640 1 -- 172.21.15.122:0/542757212 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3484197f80 con 0x7f34841028c0 2024-09-16T00:23:56.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:56.672+0000 7f348be9a640 1 -- 172.21.15.122:0/542757212 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f348406a470 con 0x7f34841028c0 2024-09-16T00:23:56.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:56.672+0000 7f347affd640 1 -- 172.21.15.122:0/542757212 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f3480042070 con 0x7f34841028c0 2024-09-16T00:23:56.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:56.672+0000 7f347affd640 1 --2- 172.21.15.122:0/542757212 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f34700392a0 0x7f347003b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:23:56.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:56.672+0000 7f347affd640 1 -- 172.21.15.122:0/542757212 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f348006d130 con 0x7f34841028c0 2024-09-16T00:23:56.673 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:56.672+0000 7f3489435640 1 --2- 172.21.15.122:0/542757212 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f34700392a0 0x7f347003b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:23:56.674 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:56.673+0000 7f3489435640 1 --2- 172.21.15.122:0/542757212 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f34700392a0 0x7f347003b760 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f34740098c0 tx=0x7f3474006db0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:23:56.677 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:56.677+0000 7f347affd640 1 -- 172.21.15.122:0/542757212 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f348003a890 con 0x7f34841028c0 2024-09-16T00:23:56.926 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:56 smithi122 systemd[1]: Starting Ceph mon.c for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:23:56.932 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:56.932+0000 7f348be9a640 1 -- 172.21.15.122:0/542757212 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f34841986b0 con 0x7f34841028c0 2024-09-16T00:23:56.933 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:56.932+0000 7f347affd640 1 -- 172.21.15.122:0/542757212 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f348003a3c0 con 0x7f34841028c0 2024-09-16T00:23:56.933 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-16T00:23:56.933 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":1,"fsid":"983b2072-73c1-11ef-bceb-c7b262605968","modified":"2024-09-16T00:22:31.653387Z","created":"2024-09-16T00:22:31.653387Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-16T00:23:56.933 INFO:teuthology.orchestra.run.smithi122.stderr:dumped monmap epoch 1 2024-09-16T00:23:56.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:56.934+0000 7f348be9a640 1 -- 172.21.15.122:0/542757212 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f34700392a0 msgr2=0x7f347003b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:56.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:56.934+0000 7f348be9a640 1 --2- 172.21.15.122:0/542757212 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f34700392a0 0x7f347003b760 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f34740098c0 tx=0x7f3474006db0 comp rx=0 tx=0).stop 2024-09-16T00:23:56.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:56.934+0000 7f348be9a640 1 -- 172.21.15.122:0/542757212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34841028c0 msgr2=0x7f3484197350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:23:56.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:56.934+0000 7f348be9a640 1 --2- 172.21.15.122:0/542757212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34841028c0 0x7f3484197350 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3480009230 tx=0x7f3480003d80 comp rx=0 tx=0).stop 2024-09-16T00:23:56.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:56.935+0000 7f348be9a640 1 -- 172.21.15.122:0/542757212 shutdown_connections 2024-09-16T00:23:56.935 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:56.935+0000 7f348be9a640 1 --2- 172.21.15.122:0/542757212 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f34700392a0 0x7f347003b760 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:56.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:56.935+0000 7f348be9a640 1 --2- 172.21.15.122:0/542757212 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34841028c0 0x7f3484197350 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:23:56.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:56.935+0000 7f348be9a640 1 -- 172.21.15.122:0/542757212 >> 172.21.15.122:0/542757212 conn(0x7f34840fc280 msgr2=0x7f3484105680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:23:56.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:56.935+0000 7f348be9a640 1 -- 172.21.15.122:0/542757212 shutdown_connections 2024-09-16T00:23:56.936 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:23:56.935+0000 7f348be9a640 1 -- 172.21.15.122:0/542757212 wait complete. 2024-09-16T00:23:57.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:56 smithi122 podman[35382]: 2024-09-16 00:23:56.830601084 +0000 UTC m=+0.017794199 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:23:57.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:56 smithi122 podman[35382]: 2024-09-16 00:23:56.947832037 +0000 UTC m=+0.135025169 container create ae3c68d8f84e1f7808d41fbac12ef7695511523d18d1529edb70bde76306f494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mon-c, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, ceph=True) 2024-09-16T00:23:57.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:57 smithi038 ceph-mon[30945]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:23:57.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:23:57 smithi038 ceph-mon[30945]: from='client.? 172.21.15.122:0/542757212' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:23:57.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 podman[35382]: 2024-09-16 00:23:57.358751271 +0000 UTC m=+0.545944361 container init ae3c68d8f84e1f7808d41fbac12ef7695511523d18d1529edb70bde76306f494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True) 2024-09-16T00:23:57.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 podman[35382]: 2024-09-16 00:23:57.362114097 +0000 UTC m=+0.549307187 container start ae3c68d8f84e1f7808d41fbac12ef7695511523d18d1529edb70bde76306f494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mon-c, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-16T00:23:57.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: set uid:gid to 167:167 (ceph:ceph) 2024-09-16T00:23:57.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable), process ceph-mon, pid 2 2024-09-16T00:23:57.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: pidfile_write: ignore empty --pid-file 2024-09-16T00:23:57.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: load: jerasure load: lrc 2024-09-16T00:23:57.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: RocksDB version: 7.9.2 2024-09-16T00:23:57.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Git sha 0 2024-09-16T00:23:57.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Compile date 2024-09-15 15:04:35 2024-09-16T00:23:57.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: DB SUMMARY 2024-09-16T00:23:57.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: DB Session ID: VQPTW90JVB9YIDHQ4V8O 2024-09-16T00:23:57.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: CURRENT file: CURRENT 2024-09-16T00:23:57.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: IDENTITY file: IDENTITY 2024-09-16T00:23:57.719 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-16T00:23:57.719 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-16T00:23:57.719 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-16T00:23:57.719 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.error_if_exists: 0 2024-09-16T00:23:57.719 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.create_if_missing: 0 2024-09-16T00:23:57.719 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.paranoid_checks: 1 2024-09-16T00:23:57.719 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-16T00:23:57.719 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-16T00:23:57.719 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-16T00:23:57.719 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.env: 0x5555ce4ab940 2024-09-16T00:23:57.719 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.fs: PosixFileSystem 2024-09-16T00:23:57.719 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.info_log: 0x5555cfeb76e0 2024-09-16T00:23:57.720 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.max_file_opening_threads: 16 2024-09-16T00:23:57.720 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.statistics: (nil) 2024-09-16T00:23:57.720 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.use_fsync: 0 2024-09-16T00:23:57.720 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.max_log_file_size: 0 2024-09-16T00:23:57.720 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-16T00:23:57.720 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-16T00:23:57.720 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.keep_log_file_num: 1000 2024-09-16T00:23:57.720 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.recycle_log_file_num: 0 2024-09-16T00:23:57.720 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.allow_fallocate: 1 2024-09-16T00:23:57.720 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.allow_mmap_reads: 0 2024-09-16T00:23:57.720 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.allow_mmap_writes: 0 2024-09-16T00:23:57.721 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.use_direct_reads: 0 2024-09-16T00:23:57.721 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-16T00:23:57.721 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.create_missing_column_families: 0 2024-09-16T00:23:57.721 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.db_log_dir: 2024-09-16T00:23:57.721 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.wal_dir: 2024-09-16T00:23:57.721 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-16T00:23:57.721 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-16T00:23:57.721 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-16T00:23:57.721 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-16T00:23:57.721 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-16T00:23:57.721 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-16T00:23:57.721 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.advise_random_on_open: 1 2024-09-16T00:23:57.722 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.db_write_buffer_size: 0 2024-09-16T00:23:57.722 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.write_buffer_manager: 0x5555cfec6dc0 2024-09-16T00:23:57.722 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-16T00:23:57.722 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-16T00:23:57.722 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-16T00:23:57.722 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.rate_limiter: (nil) 2024-09-16T00:23:57.722 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-16T00:23:57.722 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.wal_recovery_mode: 2 2024-09-16T00:23:57.722 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.enable_thread_tracking: 0 2024-09-16T00:23:57.722 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.enable_pipelined_write: 0 2024-09-16T00:23:57.722 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.unordered_write: 0 2024-09-16T00:23:57.722 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-16T00:23:57.723 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-16T00:23:57.723 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-16T00:23:57.723 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-16T00:23:57.723 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.row_cache: None 2024-09-16T00:23:57.723 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.wal_filter: None 2024-09-16T00:23:57.725 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-16T00:23:57.725 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.allow_ingest_behind: 0 2024-09-16T00:23:57.725 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.two_write_queues: 0 2024-09-16T00:23:57.725 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.manual_wal_flush: 0 2024-09-16T00:23:57.725 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.wal_compression: 0 2024-09-16T00:23:57.725 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.atomic_flush: 0 2024-09-16T00:23:57.725 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-16T00:23:57.725 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-16T00:23:57.725 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-16T00:23:57.725 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.log_readahead_size: 0 2024-09-16T00:23:57.726 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-16T00:23:57.726 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.best_efforts_recovery: 0 2024-09-16T00:23:57.726 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-16T00:23:57.726 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-16T00:23:57.726 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.allow_data_in_errors: 0 2024-09-16T00:23:57.726 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.db_host_id: __hostname__ 2024-09-16T00:23:57.726 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-16T00:23:57.726 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.max_background_jobs: 2 2024-09-16T00:23:57.726 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.max_background_compactions: -1 2024-09-16T00:23:57.726 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.max_subcompactions: 1 2024-09-16T00:23:57.726 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-16T00:23:57.727 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-16T00:23:57.727 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-16T00:23:57.727 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.max_total_wal_size: 0 2024-09-16T00:23:57.727 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-16T00:23:57.727 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-16T00:23:57.727 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-16T00:23:57.727 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-16T00:23:57.727 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.max_open_files: -1 2024-09-16T00:23:57.727 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.bytes_per_sync: 0 2024-09-16T00:23:57.727 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-16T00:23:57.727 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-16T00:23:57.728 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.compaction_readahead_size: 0 2024-09-16T00:23:57.728 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.max_background_flushes: -1 2024-09-16T00:23:57.728 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Compression algorithms supported: 2024-09-16T00:23:57.728 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: kZSTD supported: 0 2024-09-16T00:23:57.728 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: kXpressCompression supported: 0 2024-09-16T00:23:57.728 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: kBZip2Compression supported: 0 2024-09-16T00:23:57.728 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-16T00:23:57.728 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: kLZ4Compression supported: 1 2024-09-16T00:23:57.728 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: kZlibCompression supported: 1 2024-09-16T00:23:57.728 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: kLZ4HCCompression supported: 1 2024-09-16T00:23:57.728 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: kSnappyCompression supported: 1 2024-09-16T00:23:57.729 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-16T00:23:57.729 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-16T00:23:57.729 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-16T00:23:57.729 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-16T00:23:57.729 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-16T00:23:57.729 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.merge_operator: 2024-09-16T00:23:57.729 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.compaction_filter: None 2024-09-16T00:23:57.729 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.compaction_filter_factory: None 2024-09-16T00:23:57.729 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.sst_partitioner_factory: None 2024-09-16T00:23:57.729 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-16T00:23:57.729 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-16T00:23:57.729 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5555cfeb7320) 2024-09-16T00:23:57.730 INFO:journalctl@ceph.mon.c.smithi122.stdout: cache_index_and_filter_blocks: 1 2024-09-16T00:23:57.730 INFO:journalctl@ceph.mon.c.smithi122.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-16T00:23:57.730 INFO:journalctl@ceph.mon.c.smithi122.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-16T00:23:57.730 INFO:journalctl@ceph.mon.c.smithi122.stdout: pin_top_level_index_and_filter: 1 2024-09-16T00:23:57.730 INFO:journalctl@ceph.mon.c.smithi122.stdout: index_type: 0 2024-09-16T00:23:57.730 INFO:journalctl@ceph.mon.c.smithi122.stdout: data_block_index_type: 0 2024-09-16T00:23:57.730 INFO:journalctl@ceph.mon.c.smithi122.stdout: index_shortening: 1 2024-09-16T00:23:57.730 INFO:journalctl@ceph.mon.c.smithi122.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-16T00:23:57.730 INFO:journalctl@ceph.mon.c.smithi122.stdout: checksum: 4 2024-09-16T00:23:57.730 INFO:journalctl@ceph.mon.c.smithi122.stdout: no_block_cache: 0 2024-09-16T00:23:57.730 INFO:journalctl@ceph.mon.c.smithi122.stdout: block_cache: 0x5555cfeab1f0 2024-09-16T00:23:57.730 INFO:journalctl@ceph.mon.c.smithi122.stdout: block_cache_name: BinnedLRUCache 2024-09-16T00:23:57.731 INFO:journalctl@ceph.mon.c.smithi122.stdout: block_cache_options: 2024-09-16T00:23:57.731 INFO:journalctl@ceph.mon.c.smithi122.stdout: capacity : 536870912 2024-09-16T00:23:57.731 INFO:journalctl@ceph.mon.c.smithi122.stdout: num_shard_bits : 4 2024-09-16T00:23:57.731 INFO:journalctl@ceph.mon.c.smithi122.stdout: strict_capacity_limit : 0 2024-09-16T00:23:57.731 INFO:journalctl@ceph.mon.c.smithi122.stdout: high_pri_pool_ratio: 0.000 2024-09-16T00:23:57.731 INFO:journalctl@ceph.mon.c.smithi122.stdout: block_cache_compressed: (nil) 2024-09-16T00:23:57.731 INFO:journalctl@ceph.mon.c.smithi122.stdout: persistent_cache: (nil) 2024-09-16T00:23:57.731 INFO:journalctl@ceph.mon.c.smithi122.stdout: block_size: 4096 2024-09-16T00:23:57.731 INFO:journalctl@ceph.mon.c.smithi122.stdout: block_size_deviation: 10 2024-09-16T00:23:57.731 INFO:journalctl@ceph.mon.c.smithi122.stdout: block_restart_interval: 16 2024-09-16T00:23:57.731 INFO:journalctl@ceph.mon.c.smithi122.stdout: index_block_restart_interval: 1 2024-09-16T00:23:57.731 INFO:journalctl@ceph.mon.c.smithi122.stdout: metadata_block_size: 4096 2024-09-16T00:23:57.731 INFO:journalctl@ceph.mon.c.smithi122.stdout: partition_filters: 0 2024-09-16T00:23:57.732 INFO:journalctl@ceph.mon.c.smithi122.stdout: use_delta_encoding: 1 2024-09-16T00:23:57.732 INFO:journalctl@ceph.mon.c.smithi122.stdout: filter_policy: bloomfilter 2024-09-16T00:23:57.732 INFO:journalctl@ceph.mon.c.smithi122.stdout: whole_key_filtering: 1 2024-09-16T00:23:57.732 INFO:journalctl@ceph.mon.c.smithi122.stdout: verify_compression: 0 2024-09-16T00:23:57.732 INFO:journalctl@ceph.mon.c.smithi122.stdout: read_amp_bytes_per_bit: 0 2024-09-16T00:23:57.732 INFO:journalctl@ceph.mon.c.smithi122.stdout: format_version: 5 2024-09-16T00:23:57.732 INFO:journalctl@ceph.mon.c.smithi122.stdout: enable_index_compression: 1 2024-09-16T00:23:57.732 INFO:journalctl@ceph.mon.c.smithi122.stdout: block_align: 0 2024-09-16T00:23:57.732 INFO:journalctl@ceph.mon.c.smithi122.stdout: max_auto_readahead_size: 262144 2024-09-16T00:23:57.732 INFO:journalctl@ceph.mon.c.smithi122.stdout: prepopulate_block_cache: 0 2024-09-16T00:23:57.732 INFO:journalctl@ceph.mon.c.smithi122.stdout: initial_auto_readahead_size: 8192 2024-09-16T00:23:57.732 INFO:journalctl@ceph.mon.c.smithi122.stdout: num_file_reads_for_auto_readahead: 2 2024-09-16T00:23:57.733 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.write_buffer_size: 33554432 2024-09-16T00:23:57.733 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.max_write_buffer_number: 2 2024-09-16T00:23:57.733 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.compression: NoCompression 2024-09-16T00:23:57.733 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.bottommost_compression: Disabled 2024-09-16T00:23:57.733 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.prefix_extractor: nullptr 2024-09-16T00:23:57.733 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-16T00:23:57.733 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.num_levels: 7 2024-09-16T00:23:57.733 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-16T00:23:57.733 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-16T00:23:57.733 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-16T00:23:57.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-16T00:23:57.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-16T00:23:57.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-16T00:23:57.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-16T00:23:57.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-16T00:23:57.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-16T00:23:57.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-16T00:23:57.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-16T00:23:57.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-16T00:23:57.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-16T00:23:57.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.compression_opts.level: 32767 2024-09-16T00:23:57.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.compression_opts.strategy: 0 2024-09-16T00:23:57.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-16T00:23:57.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-16T00:23:57.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-16T00:23:57.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-16T00:23:57.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.compression_opts.enabled: false 2024-09-16T00:23:57.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-16T00:23:57.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-16T00:23:57.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-16T00:23:57.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-16T00:23:57.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.target_file_size_base: 67108864 2024-09-16T00:23:57.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-16T00:23:57.736 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-16T00:23:57.736 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-16T00:23:57.736 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-16T00:23:57.736 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-16T00:23:57.736 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-16T00:23:57.736 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-16T00:23:57.736 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-16T00:23:57.736 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-16T00:23:57.736 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-16T00:23:57.736 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-16T00:23:57.736 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-16T00:23:57.737 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-16T00:23:57.737 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-16T00:23:57.737 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.arena_block_size: 1048576 2024-09-16T00:23:57.737 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-16T00:23:57.737 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-16T00:23:57.737 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.disable_auto_compactions: 0 2024-09-16T00:23:57.737 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-16T00:23:57.737 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-16T00:23:57.737 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-16T00:23:57.737 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-16T00:23:57.737 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-16T00:23:57.737 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-16T00:23:57.738 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-16T00:23:57.738 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-16T00:23:57.738 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-16T00:23:57.738 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-16T00:23:57.738 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-16T00:23:57.738 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.inplace_update_support: 0 2024-09-16T00:23:57.738 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-16T00:23:57.738 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-16T00:23:57.738 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-16T00:23:57.738 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-16T00:23:57.738 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.bloom_locality: 0 2024-09-16T00:23:57.738 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.max_successive_merges: 0 2024-09-16T00:23:57.738 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-16T00:23:57.739 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.paranoid_file_checks: 0 2024-09-16T00:23:57.739 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.force_consistency_checks: 1 2024-09-16T00:23:57.739 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.report_bg_io_stats: 0 2024-09-16T00:23:57.739 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.ttl: 2592000 2024-09-16T00:23:57.739 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-16T00:23:57.739 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-16T00:23:57.739 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-16T00:23:57.739 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.enable_blob_files: false 2024-09-16T00:23:57.739 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.min_blob_size: 0 2024-09-16T00:23:57.739 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.blob_file_size: 268435456 2024-09-16T00:23:57.739 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-16T00:23:57.739 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-16T00:23:57.739 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-16T00:23:57.740 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-16T00:23:57.740 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-16T00:23:57.740 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.blob_file_starting_level: 0 2024-09-16T00:23:57.740 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-16T00:23:57.740 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-16T00:23:57.740 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-16T00:23:57.740 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: d653aa16-d764-4e0f-b71b-b6dc160253db 2024-09-16T00:23:57.740 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726446237389370, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-16T00:23:57.740 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-16T00:23:57.740 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726446237389772, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726446237, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "d653aa16-d764-4e0f-b71b-b6dc160253db", "db_session_id": "VQPTW90JVB9YIDHQ4V8O", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-16T00:23:57.740 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726446237389872, "job": 1, "event": "recovery_finished"} 2024-09-16T00:23:57.740 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-16T00:23:57.741 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-16T00:23:57.741 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5555cfed8e00 2024-09-16T00:23:57.741 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: DB pointer 0x5555cffd4000 2024-09-16T00:23:57.741 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-16T00:23:57.741 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-16T00:23:57.741 INFO:journalctl@ceph.mon.c.smithi122.stdout: ** DB Stats ** 2024-09-16T00:23:57.741 INFO:journalctl@ceph.mon.c.smithi122.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T00:23:57.741 INFO:journalctl@ceph.mon.c.smithi122.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-16T00:23:57.741 INFO:journalctl@ceph.mon.c.smithi122.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T00:23:57.741 INFO:journalctl@ceph.mon.c.smithi122.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T00:23:57.741 INFO:journalctl@ceph.mon.c.smithi122.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-16T00:23:57.741 INFO:journalctl@ceph.mon.c.smithi122.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-16T00:23:57.742 INFO:journalctl@ceph.mon.c.smithi122.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-16T00:23:57.742 INFO:journalctl@ceph.mon.c.smithi122.stdout: 2024-09-16T00:23:57.742 INFO:journalctl@ceph.mon.c.smithi122.stdout: ** Compaction Stats [default] ** 2024-09-16T00:23:57.742 INFO:journalctl@ceph.mon.c.smithi122.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T00:23:57.742 INFO:journalctl@ceph.mon.c.smithi122.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-16T00:23:57.742 INFO:journalctl@ceph.mon.c.smithi122.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T00:23:57.742 INFO:journalctl@ceph.mon.c.smithi122.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T00:23:57.742 INFO:journalctl@ceph.mon.c.smithi122.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T00:23:57.742 INFO:journalctl@ceph.mon.c.smithi122.stdout: 2024-09-16T00:23:57.742 INFO:journalctl@ceph.mon.c.smithi122.stdout: ** Compaction Stats [default] ** 2024-09-16T00:23:57.742 INFO:journalctl@ceph.mon.c.smithi122.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-16T00:23:57.742 INFO:journalctl@ceph.mon.c.smithi122.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-16T00:23:57.742 INFO:journalctl@ceph.mon.c.smithi122.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.1 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-09-16T00:23:57.743 INFO:journalctl@ceph.mon.c.smithi122.stdout: 2024-09-16T00:23:57.743 INFO:journalctl@ceph.mon.c.smithi122.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-16T00:23:57.743 INFO:journalctl@ceph.mon.c.smithi122.stdout: 2024-09-16T00:23:57.743 INFO:journalctl@ceph.mon.c.smithi122.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-16T00:23:57.743 INFO:journalctl@ceph.mon.c.smithi122.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-16T00:23:57.743 INFO:journalctl@ceph.mon.c.smithi122.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-16T00:23:57.743 INFO:journalctl@ceph.mon.c.smithi122.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-16T00:23:57.743 INFO:journalctl@ceph.mon.c.smithi122.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-16T00:23:57.743 INFO:journalctl@ceph.mon.c.smithi122.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-16T00:23:57.743 INFO:journalctl@ceph.mon.c.smithi122.stdout: Cumulative compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T00:23:57.743 INFO:journalctl@ceph.mon.c.smithi122.stdout: Interval compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-16T00:23:57.743 INFO:journalctl@ceph.mon.c.smithi122.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-16T00:23:57.744 INFO:journalctl@ceph.mon.c.smithi122.stdout: Block cache BinnedLRUCache@0x5555cfeab1f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-09-16T00:23:57.744 INFO:journalctl@ceph.mon.c.smithi122.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-16T00:23:57.744 INFO:journalctl@ceph.mon.c.smithi122.stdout: 2024-09-16T00:23:57.744 INFO:journalctl@ceph.mon.c.smithi122.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-16T00:23:57.744 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-16T00:23:57.744 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: using public_addr v2:172.21.15.122:0/0 -> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] 2024-09-16T00:23:57.744 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: starting mon.c rank -1 at public addrs [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] at bind addrs [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 983b2072-73c1-11ef-bceb-c7b262605968 2024-09-16T00:23:57.744 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: mon.c@-1(???) e0 preinit fsid 983b2072-73c1-11ef-bceb-c7b262605968 2024-09-16T00:23:57.744 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: mon.c@-1(synchronizing).mds e1 new map 2024-09-16T00:23:57.744 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-16T00:23:57.744 INFO:journalctl@ceph.mon.c.smithi122.stdout: e1 2024-09-16T00:23:57.745 INFO:journalctl@ceph.mon.c.smithi122.stdout: btime 2024-09-16T00:22:34:896635+0000 2024-09-16T00:23:57.745 INFO:journalctl@ceph.mon.c.smithi122.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:23:57.745 INFO:journalctl@ceph.mon.c.smithi122.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-16T00:23:57.745 INFO:journalctl@ceph.mon.c.smithi122.stdout: legacy client fscid: -1 2024-09-16T00:23:57.745 INFO:journalctl@ceph.mon.c.smithi122.stdout: 2024-09-16T00:23:57.745 INFO:journalctl@ceph.mon.c.smithi122.stdout: No filesystems configured 2024-09-16T00:23:57.745 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-16T00:23:57.745 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-16T00:23:57.745 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-16T00:23:57.745 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-16T00:23:57.745 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-16T00:23:57.745 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-16T00:23:57.746 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-16T00:23:57.746 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T00:23:57.746 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T00:23:57.746 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-16T00:23:57.746 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: Updating smithi073:/var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T00:23:57.746 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:23:57.746 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:57.746 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:23:57.746 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: Added host smithi122 2024-09-16T00:23:57.746 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:57.746 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:57.746 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:23:57.746 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:57.747 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:23:57.747 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/1343175778' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-16T00:23:57.747 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:23:57.747 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/1343175778' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-16T00:23:57.747 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:23:57.747 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:57.747 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:57.747 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:57.747 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:57.747 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi122", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:23:57.747 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:57.747 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:23:57.748 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: Updating smithi122:/etc/ceph/ceph.conf 2024-09-16T00:23:57.748 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: Updating smithi122:/var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:23:57.748 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:57.748 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:57.748 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:57.748 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:23:57.748 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:23:57.748 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:23:57.748 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:23:57.748 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: Updating smithi122:/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:23:57.748 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: Updating smithi122:/var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/config/ceph.client.admin.keyring 2024-09-16T00:23:57.748 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi038:172.21.15.38=a;smithi073:172.21.15.73=b;smithi122:172.21.15.122=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:23:57.748 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: Saving service mon spec with placement smithi038:172.21.15.38=a;smithi073:172.21.15.73=b;smithi122:172.21.15.122=c;count:3 2024-09-16T00:23:57.749 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: Deploying daemon mon.c on smithi122 2024-09-16T00:23:57.749 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: from='client.? 172.21.15.122:0/2592937766' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:23:57.749 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:23:57.749 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:23:57.749 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: from='client.? 172.21.15.122:0/542757212' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:23:57.749 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 ceph-mon[35412]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-16T00:23:57.749 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 bash[35382]: ae3c68d8f84e1f7808d41fbac12ef7695511523d18d1529edb70bde76306f494 2024-09-16T00:23:57.749 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:23:57 smithi122 systemd[1]: Started Ceph mon.c for 983b2072-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:23:58.809 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T00:23:58.809 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T00:23:59.090 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:24:02.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.479+0000 7f27ba2af640 1 -- 172.21.15.122:0/2258059662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f279c003530 msgr2=0x7f279c0059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:02.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.479+0000 7f27ba2af640 1 --2- 172.21.15.122:0/2258059662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f279c003530 0x7f279c0059c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f27b406aa60 tx=0x7f27a402bce0 comp rx=0 tx=0).stop 2024-09-16T00:24:02.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.479+0000 7f27ba2af640 1 -- 172.21.15.122:0/2258059662 shutdown_connections 2024-09-16T00:24:02.482 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.479+0000 7f27ba2af640 1 --2- 172.21.15.122:0/2258059662 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f279c003530 0x7f279c0059c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:02.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.479+0000 7f27ba2af640 1 -- 172.21.15.122:0/2258059662 >> 172.21.15.122:0/2258059662 conn(0x7f27b40fc220 msgr2=0x7f27b40fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:02.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.479+0000 7f27ba2af640 1 -- 172.21.15.122:0/2258059662 shutdown_connections 2024-09-16T00:24:02.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.479+0000 7f27ba2af640 1 -- 172.21.15.122:0/2258059662 wait complete. 2024-09-16T00:24:02.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.480+0000 7f27ba2af640 1 Processor -- start 2024-09-16T00:24:02.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.480+0000 7f27ba2af640 1 -- start start 2024-09-16T00:24:02.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.481+0000 7f27ba2af640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f279c003530 0x7f27b4197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:02.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.481+0000 7f27ba2af640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27b41978c0 0x7f27b419bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:02.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.481+0000 7f27ba2af640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f27b4109780 con 0x7f27b41978c0 2024-09-16T00:24:02.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.481+0000 7f27ba2af640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f27b4109600 con 0x7f279c003530 2024-09-16T00:24:02.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.481+0000 7f27b3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f279c003530 0x7f27b4197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:02.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.481+0000 7f27b3fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f279c003530 0x7f27b4197380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:59870/0 (socket says 172.21.15.122:59870) 2024-09-16T00:24:02.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.481+0000 7f27b3fff640 1 -- 172.21.15.122:0/2805098813 learned_addr learned my addr 172.21.15.122:0/2805098813 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-16T00:24:02.483 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.481+0000 7f27b3fff640 1 -- 172.21.15.122:0/2805098813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f279c003530 msgr2=0x7f27b4197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-16T00:24:02.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.481+0000 7f27b3fff640 1 -- 172.21.15.122:0/2805098813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f279c003530 msgr2=0x7f27b4197380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T00:24:02.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.481+0000 7f27b3fff640 1 --2- 172.21.15.122:0/2805098813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f279c003530 0x7f27b4197380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T00:24:02.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.481+0000 7f27b3fff640 1 --2- 172.21.15.122:0/2805098813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f279c003530 0x7f27b4197380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T00:24:02.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.481+0000 7f27b37fe640 1 --2- 172.21.15.122:0/2805098813 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27b41978c0 0x7f27b419bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:02.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.482+0000 7f27b37fe640 1 -- 172.21.15.122:0/2805098813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f279c003530 msgr2=0x7f27b4197380 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:24:02.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.482+0000 7f27b37fe640 1 --2- 172.21.15.122:0/2805098813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f279c003530 0x7f27b4197380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:02.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.482+0000 7f27b37fe640 1 -- 172.21.15.122:0/2805098813 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27b4197ee0 con 0x7f27b41978c0 2024-09-16T00:24:02.484 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.482+0000 7f27b37fe640 1 --2- 172.21.15.122:0/2805098813 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27b41978c0 0x7f27b419bc00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f27a800b3d0 tx=0x7f27a800b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:02.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.482+0000 7f27b17fa640 1 -- 172.21.15.122:0/2805098813 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f27a8011630 con 0x7f27b41978c0 2024-09-16T00:24:02.485 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.482+0000 7f27b17fa640 1 -- 172.21.15.122:0/2805098813 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f27a8011cb0 con 0x7f27b41978c0 2024-09-16T00:24:02.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.482+0000 7f27ba2af640 1 -- 172.21.15.122:0/2805098813 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27b419c140 con 0x7f27b41978c0 2024-09-16T00:24:02.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.482+0000 7f27b17fa640 1 -- 172.21.15.122:0/2805098813 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f27a8010df0 con 0x7f27b41978c0 2024-09-16T00:24:02.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.482+0000 7f27ba2af640 1 -- 172.21.15.122:0/2805098813 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27b419c680 con 0x7f27b41978c0 2024-09-16T00:24:02.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.484+0000 7f27ba2af640 1 -- 172.21.15.122:0/2805098813 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27b406ad10 con 0x7f27b41978c0 2024-09-16T00:24:02.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.484+0000 7f27b17fa640 1 -- 172.21.15.122:0/2805098813 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f27a8018020 con 0x7f27b41978c0 2024-09-16T00:24:02.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.484+0000 7f27b17fa640 1 --2- 172.21.15.122:0/2805098813 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f27a0039090 0x7f27a003b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:02.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.484+0000 7f27b17fa640 1 -- 172.21.15.122:0/2805098813 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f27a804c010 con 0x7f27b41978c0 2024-09-16T00:24:02.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.484+0000 7f27b3fff640 1 --2- 172.21.15.122:0/2805098813 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f27a0039090 0x7f27a003b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:02.486 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.485+0000 7f27b3fff640 1 --2- 172.21.15.122:0/2805098813 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f27a0039090 0x7f27a003b550 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f27a4005f70 tx=0x7f27a402c6e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:02.493 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.493+0000 7f27b17fa640 1 -- 172.21.15.122:0/2805098813 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f27a801f020 con 0x7f27b41978c0 2024-09-16T00:24:02.734 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.733+0000 7f27ba2af640 1 -- 172.21.15.122:0/2805098813 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f27b4190f70 con 0x7f27b41978c0 2024-09-16T00:24:02.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:02 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:24:02.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:02 smithi122 ceph-mon[35412]: mon.a calling monitor election 2024-09-16T00:24:02.734 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:02 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:24:02.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:02 smithi122 ceph-mon[35412]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:02.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:02 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:24:02.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:02 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:24:02.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:02 smithi122 ceph-mon[35412]: mon.c calling monitor election 2024-09-16T00:24:02.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:02 smithi122 ceph-mon[35412]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:02.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:02 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:24:02.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:02 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:24:02.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:02 smithi122 ceph-mon[35412]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:02.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:02 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:24:02.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:02 smithi122 ceph-mon[35412]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-16T00:24:02.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:02 smithi122 ceph-mon[35412]: monmap epoch 2 2024-09-16T00:24:02.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:02 smithi122 ceph-mon[35412]: fsid 983b2072-73c1-11ef-bceb-c7b262605968 2024-09-16T00:24:02.735 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:02 smithi122 ceph-mon[35412]: last_changed 2024-09-16T00:23:57.413904+0000 2024-09-16T00:24:02.736 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:02 smithi122 ceph-mon[35412]: created 2024-09-16T00:22:31.653387+0000 2024-09-16T00:24:02.736 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:02 smithi122 ceph-mon[35412]: min_mon_release 19 (squid) 2024-09-16T00:24:02.736 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:02 smithi122 ceph-mon[35412]: election_strategy: 1 2024-09-16T00:24:02.736 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:02 smithi122 ceph-mon[35412]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-09-16T00:24:02.736 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:02 smithi122 ceph-mon[35412]: 1: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.c 2024-09-16T00:24:02.736 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:02 smithi122 ceph-mon[35412]: fsmap 2024-09-16T00:24:02.736 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:02 smithi122 ceph-mon[35412]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:24:02.736 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:02 smithi122 ceph-mon[35412]: mgrmap e13: x(active, since 41s) 2024-09-16T00:24:02.736 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:02 smithi122 ceph-mon[35412]: overall HEALTH_OK 2024-09-16T00:24:02.736 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:02 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:02.736 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:02 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:02.736 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:02 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:02.736 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:02 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:24:02.737 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:02 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:02.737 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.734+0000 7f27b17fa640 1 -- 172.21.15.122:0/2805098813 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f27a80172c0 con 0x7f27b41978c0 2024-09-16T00:24:02.737 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-16T00:24:02.737 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":2,"fsid":"983b2072-73c1-11ef-bceb-c7b262605968","modified":"2024-09-16T00:23:57.413904Z","created":"2024-09-16T00:22:31.653387Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:3300","nonce":0},{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-16T00:24:02.738 INFO:teuthology.orchestra.run.smithi122.stderr:dumped monmap epoch 2 2024-09-16T00:24:02.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.736+0000 7f27ba2af640 1 -- 172.21.15.122:0/2805098813 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f27a0039090 msgr2=0x7f27a003b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:02.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.736+0000 7f27ba2af640 1 --2- 172.21.15.122:0/2805098813 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f27a0039090 0x7f27a003b550 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f27a4005f70 tx=0x7f27a402c6e0 comp rx=0 tx=0).stop 2024-09-16T00:24:02.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.736+0000 7f27ba2af640 1 -- 172.21.15.122:0/2805098813 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27b41978c0 msgr2=0x7f27b419bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:02.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.736+0000 7f27ba2af640 1 --2- 172.21.15.122:0/2805098813 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27b41978c0 0x7f27b419bc00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f27a800b3d0 tx=0x7f27a800b890 comp rx=0 tx=0).stop 2024-09-16T00:24:02.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.736+0000 7f27ba2af640 1 -- 172.21.15.122:0/2805098813 shutdown_connections 2024-09-16T00:24:02.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.736+0000 7f27ba2af640 1 --2- 172.21.15.122:0/2805098813 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f27a0039090 0x7f27a003b550 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:02.738 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.736+0000 7f27ba2af640 1 --2- 172.21.15.122:0/2805098813 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f27b41978c0 0x7f27b419bc00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:02.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.736+0000 7f27ba2af640 1 --2- 172.21.15.122:0/2805098813 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f279c003530 0x7f27b4197380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:02.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.736+0000 7f27ba2af640 1 -- 172.21.15.122:0/2805098813 >> 172.21.15.122:0/2805098813 conn(0x7f27b40fc220 msgr2=0x7f27b40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:02.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.736+0000 7f27ba2af640 1 -- 172.21.15.122:0/2805098813 shutdown_connections 2024-09-16T00:24:02.739 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:02.736+0000 7f27ba2af640 1 -- 172.21.15.122:0/2805098813 wait complete. 2024-09-16T00:24:02.872 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:02 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:24:02.872 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:02 smithi038 ceph-mon[30945]: mon.a calling monitor election 2024-09-16T00:24:02.872 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:02 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:24:02.872 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:02 smithi038 ceph-mon[30945]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:02.872 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:02 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:24:02.872 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:02 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:24:02.872 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:02 smithi038 ceph-mon[30945]: mon.c calling monitor election 2024-09-16T00:24:02.872 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:02 smithi038 ceph-mon[30945]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:02.873 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:02 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:24:02.873 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:02 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:24:02.873 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:02 smithi038 ceph-mon[30945]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:02.873 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:02 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:24:02.873 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:02 smithi038 ceph-mon[30945]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-16T00:24:02.873 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:02 smithi038 ceph-mon[30945]: monmap epoch 2 2024-09-16T00:24:02.873 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:02 smithi038 ceph-mon[30945]: fsid 983b2072-73c1-11ef-bceb-c7b262605968 2024-09-16T00:24:02.873 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:02 smithi038 ceph-mon[30945]: last_changed 2024-09-16T00:23:57.413904+0000 2024-09-16T00:24:02.873 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:02 smithi038 ceph-mon[30945]: created 2024-09-16T00:22:31.653387+0000 2024-09-16T00:24:02.873 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:02 smithi038 ceph-mon[30945]: min_mon_release 19 (squid) 2024-09-16T00:24:02.873 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:02 smithi038 ceph-mon[30945]: election_strategy: 1 2024-09-16T00:24:02.873 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:02 smithi038 ceph-mon[30945]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-09-16T00:24:02.873 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:02 smithi038 ceph-mon[30945]: 1: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.c 2024-09-16T00:24:02.874 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:02 smithi038 ceph-mon[30945]: fsmap 2024-09-16T00:24:02.874 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:02 smithi038 ceph-mon[30945]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:24:02.874 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:02 smithi038 ceph-mon[30945]: mgrmap e13: x(active, since 41s) 2024-09-16T00:24:02.874 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:02 smithi038 ceph-mon[30945]: overall HEALTH_OK 2024-09-16T00:24:02.874 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:02 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:02.874 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:02 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:02.874 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:02 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:02.874 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:02 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:24:02.874 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:02 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:03.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:03 smithi122 ceph-mon[35412]: Deploying daemon mon.b on smithi073 2024-09-16T00:24:03.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:03 smithi122 ceph-mon[35412]: from='client.? 172.21.15.122:0/2805098813' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:24:03.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:03 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:24:03.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:03 smithi038 ceph-mon[30945]: Deploying daemon mon.b on smithi073 2024-09-16T00:24:03.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:03 smithi038 ceph-mon[30945]: from='client.? 172.21.15.122:0/2805098813' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:24:03.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:03 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:24:04.305 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T00:24:04.305 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T00:24:04.554 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:24:04.674 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:04 smithi122 ceph-mon[35412]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:04.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:04 smithi038 ceph-mon[30945]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:04.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:04.982+0000 7fc4fd08c640 1 -- 172.21.15.122:0/4021991108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc4f8106650 msgr2=0x7fc4f8106a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:04.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:04.982+0000 7fc4fd08c640 1 --2- 172.21.15.122:0/4021991108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc4f8106650 0x7fc4f8106a30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0009880 tx=0x7fc4f002a5d0 comp rx=0 tx=0).stop 2024-09-16T00:24:04.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:04.982+0000 7fc4fd08c640 1 -- 172.21.15.122:0/4021991108 shutdown_connections 2024-09-16T00:24:04.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:04.982+0000 7fc4fd08c640 1 --2- 172.21.15.122:0/4021991108 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc4f8106650 0x7fc4f8106a30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:04.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:04.982+0000 7fc4fd08c640 1 -- 172.21.15.122:0/4021991108 >> 172.21.15.122:0/4021991108 conn(0x7fc4f80fc280 msgr2=0x7fc4f80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:04.987 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:04.982+0000 7fc4fd08c640 1 -- 172.21.15.122:0/4021991108 shutdown_connections 2024-09-16T00:24:04.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:04.982+0000 7fc4fd08c640 1 -- 172.21.15.122:0/4021991108 wait complete. 2024-09-16T00:24:04.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:04.983+0000 7fc4fd08c640 1 Processor -- start 2024-09-16T00:24:04.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:04.983+0000 7fc4fd08c640 1 -- start start 2024-09-16T00:24:04.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:04.983+0000 7fc4fd08c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc4f8106650 0x7fc4f81973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:04.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:04.983+0000 7fc4fd08c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4f81978f0 0x7fc4f819bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:04.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:04.983+0000 7fc4fd08c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f810ac80 con 0x7fc4f8106650 2024-09-16T00:24:04.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:04.983+0000 7fc4fd08c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc4f810ab00 con 0x7fc4f81978f0 2024-09-16T00:24:04.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:04.983+0000 7fc4f659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4f81978f0 0x7fc4f819bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:04.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:04.983+0000 7fc4f659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4f81978f0 0x7fc4f819bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38766/0 (socket says 172.21.15.122:38766) 2024-09-16T00:24:04.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:04.983+0000 7fc4f659c640 1 -- 172.21.15.122:0/1630117718 learned_addr learned my addr 172.21.15.122:0/1630117718 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-16T00:24:04.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:04.983+0000 7fc4f659c640 1 -- 172.21.15.122:0/1630117718 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4f81978f0 msgr2=0x7fc4f819bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-09-16T00:24:04.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:04.983+0000 7fc4f659c640 1 -- 172.21.15.122:0/1630117718 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4f81978f0 msgr2=0x7fc4f819bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T00:24:04.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:04.983+0000 7fc4f659c640 1 --2- 172.21.15.122:0/1630117718 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4f81978f0 0x7fc4f819bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T00:24:04.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:04.983+0000 7fc4f6d9d640 1 --2- 172.21.15.122:0/1630117718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc4f8106650 0x7fc4f81973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:04.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:04.983+0000 7fc4f659c640 1 --2- 172.21.15.122:0/1630117718 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4f81978f0 0x7fc4f819bc80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T00:24:04.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:04.984+0000 7fc4f6d9d640 1 -- 172.21.15.122:0/1630117718 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4f81978f0 msgr2=0x7fc4f819bc80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:24:04.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:04.984+0000 7fc4f6d9d640 1 --2- 172.21.15.122:0/1630117718 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4f81978f0 0x7fc4f819bc80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:04.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:04.984+0000 7fc4f6d9d640 1 -- 172.21.15.122:0/1630117718 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4f8197f10 con 0x7fc4f8106650 2024-09-16T00:24:04.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:04.984+0000 7fc4f6d9d640 1 --2- 172.21.15.122:0/1630117718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc4f8106650 0x7fc4f81973b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0006010 tx=0x7fc4f0032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:04.989 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:04.984+0000 7fc4dffff640 1 -- 172.21.15.122:0/1630117718 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fc4f0035070 con 0x7fc4f8106650 2024-09-16T00:24:04.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:04.984+0000 7fc4dffff640 1 -- 172.21.15.122:0/1630117718 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc4f0032e40 con 0x7fc4f8106650 2024-09-16T00:24:04.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:04.984+0000 7fc4fd08c640 1 -- 172.21.15.122:0/1630117718 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4f819c1c0 con 0x7fc4f8106650 2024-09-16T00:24:04.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:04.984+0000 7fc4dffff640 1 -- 172.21.15.122:0/1630117718 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fc4f003bb50 con 0x7fc4f8106650 2024-09-16T00:24:04.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:04.984+0000 7fc4fd08c640 1 -- 172.21.15.122:0/1630117718 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4f819c720 con 0x7fc4f8106650 2024-09-16T00:24:04.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:04.985+0000 7fc4dffff640 1 -- 172.21.15.122:0/1630117718 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fc4f0042030 con 0x7fc4f8106650 2024-09-16T00:24:04.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:04.985+0000 7fc4dffff640 1 --2- 172.21.15.122:0/1630117718 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fc4e0039040 0x7fc4e003b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:04.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:04.985+0000 7fc4fd08c640 1 -- 172.21.15.122:0/1630117718 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4f8101cc0 con 0x7fc4f8106650 2024-09-16T00:24:04.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:04.985+0000 7fc4dffff640 1 -- 172.21.15.122:0/1630117718 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fc4f0040070 con 0x7fc4f8106650 2024-09-16T00:24:04.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:04.986+0000 7fc4f659c640 1 --2- 172.21.15.122:0/1630117718 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fc4e0039040 0x7fc4e003b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:04.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:04.986+0000 7fc4f659c640 1 --2- 172.21.15.122:0/1630117718 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fc4e0039040 0x7fc4e003b500 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec009400 tx=0x7fc4ec007040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:04.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:04.991+0000 7fc4dffff640 1 -- 172.21.15.122:0/1630117718 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc4f003b460 con 0x7fc4f8106650 2024-09-16T00:24:05.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:05.220+0000 7fc4fd08c640 1 -- 172.21.15.122:0/1630117718 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fc4f819c9d0 con 0x7fc4f8106650 2024-09-16T00:24:05.221 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:05.221+0000 7fc4dffff640 1 -- 172.21.15.122:0/1630117718 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fc4f0036da0 con 0x7fc4f8106650 2024-09-16T00:24:05.222 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-16T00:24:05.222 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":2,"fsid":"983b2072-73c1-11ef-bceb-c7b262605968","modified":"2024-09-16T00:23:57.413904Z","created":"2024-09-16T00:22:31.653387Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:3300","nonce":0},{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-16T00:24:05.222 INFO:teuthology.orchestra.run.smithi122.stderr:dumped monmap epoch 2 2024-09-16T00:24:05.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:05.223+0000 7fc4fd08c640 1 -- 172.21.15.122:0/1630117718 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fc4e0039040 msgr2=0x7fc4e003b500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:05.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:05.223+0000 7fc4fd08c640 1 --2- 172.21.15.122:0/1630117718 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fc4e0039040 0x7fc4e003b500 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc4ec009400 tx=0x7fc4ec007040 comp rx=0 tx=0).stop 2024-09-16T00:24:05.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:05.223+0000 7fc4fd08c640 1 -- 172.21.15.122:0/1630117718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc4f8106650 msgr2=0x7fc4f81973b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:05.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:05.223+0000 7fc4fd08c640 1 --2- 172.21.15.122:0/1630117718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc4f8106650 0x7fc4f81973b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fc4f0006010 tx=0x7fc4f0032c50 comp rx=0 tx=0).stop 2024-09-16T00:24:05.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:05.223+0000 7fc4fd08c640 1 -- 172.21.15.122:0/1630117718 shutdown_connections 2024-09-16T00:24:05.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:05.223+0000 7fc4fd08c640 1 --2- 172.21.15.122:0/1630117718 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fc4e0039040 0x7fc4e003b500 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:05.224 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:05.223+0000 7fc4fd08c640 1 --2- 172.21.15.122:0/1630117718 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc4f81978f0 0x7fc4f819bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:05.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:05.223+0000 7fc4fd08c640 1 --2- 172.21.15.122:0/1630117718 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc4f8106650 0x7fc4f81973b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:05.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:05.223+0000 7fc4fd08c640 1 -- 172.21.15.122:0/1630117718 >> 172.21.15.122:0/1630117718 conn(0x7fc4f80fc280 msgr2=0x7fc4f8104150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:05.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:05.224+0000 7fc4fd08c640 1 -- 172.21.15.122:0/1630117718 shutdown_connections 2024-09-16T00:24:05.225 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:05.224+0000 7fc4fd08c640 1 -- 172.21.15.122:0/1630117718 wait complete. 2024-09-16T00:24:05.766 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:05 smithi122 ceph-mon[35412]: from='client.? 172.21.15.122:0/1630117718' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:24:05.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:05 smithi038 ceph-mon[30945]: from='client.? 172.21.15.122:0/1630117718' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:24:06.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:06 smithi073 systemd[1]: Started Ceph mon.b for 983b2072-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:24:06.792 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-16T00:24:06.792 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph mon dump -f json 2024-09-16T00:24:07.062 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:24:11.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:11 smithi038 ceph-mon[30945]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:11.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:11 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:24:11.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:11 smithi038 ceph-mon[30945]: mon.a calling monitor election 2024-09-16T00:24:11.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:11 smithi038 ceph-mon[30945]: mon.c calling monitor election 2024-09-16T00:24:11.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:11 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:24:11.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:11 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:24:11.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:11 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:24:11.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:11 smithi038 ceph-mon[30945]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:11.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:11 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:24:11.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:11 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:24:11.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:11 smithi038 ceph-mon[30945]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:11.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:11 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:24:11.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:11 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:24:11.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:11 smithi038 ceph-mon[30945]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-16T00:24:11.467 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:11 smithi038 ceph-mon[30945]: monmap epoch 3 2024-09-16T00:24:11.467 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:11 smithi038 ceph-mon[30945]: fsid 983b2072-73c1-11ef-bceb-c7b262605968 2024-09-16T00:24:11.467 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:11 smithi038 ceph-mon[30945]: last_changed 2024-09-16T00:24:06.147017+0000 2024-09-16T00:24:11.467 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:11 smithi038 ceph-mon[30945]: created 2024-09-16T00:22:31.653387+0000 2024-09-16T00:24:11.467 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:11 smithi038 ceph-mon[30945]: min_mon_release 19 (squid) 2024-09-16T00:24:11.467 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:11 smithi038 ceph-mon[30945]: election_strategy: 1 2024-09-16T00:24:11.467 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:11 smithi038 ceph-mon[30945]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-09-16T00:24:11.467 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:11 smithi038 ceph-mon[30945]: 1: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.c 2024-09-16T00:24:11.467 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:11 smithi038 ceph-mon[30945]: 2: [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] mon.b 2024-09-16T00:24:11.467 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:11 smithi038 ceph-mon[30945]: fsmap 2024-09-16T00:24:11.467 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:11 smithi038 ceph-mon[30945]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:24:11.467 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:11 smithi038 ceph-mon[30945]: mgrmap e13: x(active, since 49s) 2024-09-16T00:24:11.467 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:11 smithi038 ceph-mon[30945]: overall HEALTH_OK 2024-09-16T00:24:11.468 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:11 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:11.468 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:11 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:11.468 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:11 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:11.468 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:11 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:11.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:11 smithi122 ceph-mon[35412]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:11.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:11 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:24:11.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:11 smithi122 ceph-mon[35412]: mon.a calling monitor election 2024-09-16T00:24:11.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:11 smithi122 ceph-mon[35412]: mon.c calling monitor election 2024-09-16T00:24:11.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:11 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:24:11.469 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:11 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:24:11.469 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:11 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:24:11.469 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:11 smithi122 ceph-mon[35412]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:11.469 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:11 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:24:11.469 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:11 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:24:11.469 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:11 smithi122 ceph-mon[35412]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:11.469 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:11 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:24:11.469 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:11 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:24:11.469 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:11 smithi122 ceph-mon[35412]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-16T00:24:11.469 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:11 smithi122 ceph-mon[35412]: monmap epoch 3 2024-09-16T00:24:11.469 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:11 smithi122 ceph-mon[35412]: fsid 983b2072-73c1-11ef-bceb-c7b262605968 2024-09-16T00:24:11.469 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:11 smithi122 ceph-mon[35412]: last_changed 2024-09-16T00:24:06.147017+0000 2024-09-16T00:24:11.469 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:11 smithi122 ceph-mon[35412]: created 2024-09-16T00:22:31.653387+0000 2024-09-16T00:24:11.470 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:11 smithi122 ceph-mon[35412]: min_mon_release 19 (squid) 2024-09-16T00:24:11.470 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:11 smithi122 ceph-mon[35412]: election_strategy: 1 2024-09-16T00:24:11.470 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:11 smithi122 ceph-mon[35412]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-09-16T00:24:11.470 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:11 smithi122 ceph-mon[35412]: 1: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.c 2024-09-16T00:24:11.470 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:11 smithi122 ceph-mon[35412]: 2: [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] mon.b 2024-09-16T00:24:11.470 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:11 smithi122 ceph-mon[35412]: fsmap 2024-09-16T00:24:11.470 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:11 smithi122 ceph-mon[35412]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:24:11.470 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:11 smithi122 ceph-mon[35412]: mgrmap e13: x(active, since 49s) 2024-09-16T00:24:11.470 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:11 smithi122 ceph-mon[35412]: overall HEALTH_OK 2024-09-16T00:24:11.470 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:11 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:11.470 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:11 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:11.470 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:11 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:11.471 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:11 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:11.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.851+0000 7fd108c2c640 1 -- 172.21.15.122:0/573618037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0ec005740 msgr2=0x7fd0ec007bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:11.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.851+0000 7fd108c2c640 1 --2- 172.21.15.122:0/573618037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0ec005740 0x7fd0ec007bd0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd1041016c0 tx=0x7fd0f402c490 comp rx=0 tx=0).stop 2024-09-16T00:24:11.852 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.851+0000 7fd108c2c640 1 -- 172.21.15.122:0/573618037 shutdown_connections 2024-09-16T00:24:11.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.851+0000 7fd108c2c640 1 --2- 172.21.15.122:0/573618037 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd0ec005740 0x7fd0ec007bd0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:11.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.851+0000 7fd108c2c640 1 -- 172.21.15.122:0/573618037 >> 172.21.15.122:0/573618037 conn(0x7fd1040fc280 msgr2=0x7fd1040fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:11.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.851+0000 7fd108c2c640 1 -- 172.21.15.122:0/573618037 shutdown_connections 2024-09-16T00:24:11.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.851+0000 7fd108c2c640 1 -- 172.21.15.122:0/573618037 wait complete. 2024-09-16T00:24:11.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.852+0000 7fd108c2c640 1 Processor -- start 2024-09-16T00:24:11.853 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.852+0000 7fd108c2c640 1 -- start start 2024-09-16T00:24:11.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.852+0000 7fd108c2c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0ec005740 0x7fd104199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:11.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.852+0000 7fd108c2c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd104199aa0 0x7fd10419de30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:11.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.852+0000 7fd108c2c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd10419e370 0x7fd10419e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:11.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.852+0000 7fd108c2c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd1041075c0 con 0x7fd104199aa0 2024-09-16T00:24:11.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.852+0000 7fd108c2c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd104107440 con 0x7fd0ec005740 2024-09-16T00:24:11.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.852+0000 7fd108c2c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd104107740 con 0x7fd10419e370 2024-09-16T00:24:11.855 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.853+0000 7fd102d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd10419e370 0x7fd10419e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:11.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.853+0000 7fd101d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd104199aa0 0x7fd10419de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:11.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.853+0000 7fd102d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd10419e370 0x7fd10419e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:38780/0 (socket says 172.21.15.122:38780) 2024-09-16T00:24:11.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.853+0000 7fd102d9d640 1 -- 172.21.15.122:0/2659540991 learned_addr learned my addr 172.21.15.122:0/2659540991 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-16T00:24:11.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.853+0000 7fd102d9d640 1 -- 172.21.15.122:0/2659540991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd10419e370 msgr2=0x7fd10419e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-16T00:24:11.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.853+0000 7fd102d9d640 1 -- 172.21.15.122:0/2659540991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd10419e370 msgr2=0x7fd10419e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-16T00:24:11.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.853+0000 7fd102d9d640 1 --2- 172.21.15.122:0/2659540991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd10419e370 0x7fd10419e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-09-16T00:24:11.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.853+0000 7fd102d9d640 1 --2- 172.21.15.122:0/2659540991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd10419e370 0x7fd10419e820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-16T00:24:11.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.853+0000 7fd10259c640 1 --2- 172.21.15.122:0/2659540991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0ec005740 0x7fd104199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:11.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.853+0000 7fd101d9b640 1 -- 172.21.15.122:0/2659540991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd10419e370 msgr2=0x7fd10419e820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-16T00:24:11.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.853+0000 7fd101d9b640 1 --2- 172.21.15.122:0/2659540991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd10419e370 0x7fd10419e820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:11.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.853+0000 7fd101d9b640 1 -- 172.21.15.122:0/2659540991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0ec005740 msgr2=0x7fd104199560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:11.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.853+0000 7fd101d9b640 1 --2- 172.21.15.122:0/2659540991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0ec005740 0x7fd104199560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:11.856 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.853+0000 7fd101d9b640 1 -- 172.21.15.122:0/2659540991 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1041a60e0 con 0x7fd104199aa0 2024-09-16T00:24:11.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.853+0000 7fd101d9b640 1 --2- 172.21.15.122:0/2659540991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd104199aa0 0x7fd10419de30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd0f800b450 tx=0x7fd0f800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:11.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.853+0000 7fd0f37fe640 1 -- 172.21.15.122:0/2659540991 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f8004230 con 0x7fd104199aa0 2024-09-16T00:24:11.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.853+0000 7fd108c2c640 1 -- 172.21.15.122:0/2659540991 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1041a6310 con 0x7fd104199aa0 2024-09-16T00:24:11.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.853+0000 7fd0f37fe640 1 -- 172.21.15.122:0/2659540991 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd0f80043d0 con 0x7fd104199aa0 2024-09-16T00:24:11.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.853+0000 7fd0f37fe640 1 -- 172.21.15.122:0/2659540991 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd0f8003cb0 con 0x7fd104199aa0 2024-09-16T00:24:11.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.853+0000 7fd108c2c640 1 -- 172.21.15.122:0/2659540991 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1041a68a0 con 0x7fd104199aa0 2024-09-16T00:24:11.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.854+0000 7fd0f37fe640 1 -- 172.21.15.122:0/2659540991 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fd0f800f050 con 0x7fd104199aa0 2024-09-16T00:24:11.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.855+0000 7fd0f37fe640 1 --2- 172.21.15.122:0/2659540991 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd0e4039730 0x7fd0e403bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:11.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.855+0000 7fd0f37fe640 1 -- 172.21.15.122:0/2659540991 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fd0f804c130 con 0x7fd104199aa0 2024-09-16T00:24:11.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.855+0000 7fd108c2c640 1 -- 172.21.15.122:0/2659540991 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1041086d0 con 0x7fd104199aa0 2024-09-16T00:24:11.857 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.855+0000 7fd10259c640 1 --2- 172.21.15.122:0/2659540991 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd0e4039730 0x7fd0e403bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:11.858 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.855+0000 7fd10259c640 1 --2- 172.21.15.122:0/2659540991 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd0e4039730 0x7fd0e403bbf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd0f402ca00 tx=0x7fd0f402c990 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:11.860 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:11.860+0000 7fd0f37fe640 1 -- 172.21.15.122:0/2659540991 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd0f800d040 con 0x7fd104199aa0 2024-09-16T00:24:12.102 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:12.102+0000 7fd108c2c640 1 -- 172.21.15.122:0/2659540991 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fd1041a6b30 con 0x7fd104199aa0 2024-09-16T00:24:12.103 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:12.102+0000 7fd0f37fe640 1 -- 172.21.15.122:0/2659540991 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7fd0f8011d00 con 0x7fd104199aa0 2024-09-16T00:24:12.103 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-16T00:24:12.103 INFO:teuthology.orchestra.run.smithi122.stdout:{"epoch":3,"fsid":"983b2072-73c1-11ef-bceb-c7b262605968","modified":"2024-09-16T00:24:06.147017Z","created":"2024-09-16T00:22:31.653387Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:3300","nonce":0},{"type":"v1","addr":"172.21.15.38:6789","nonce":0}]},"addr":"172.21.15.38:6789/0","public_addr":"172.21.15.38:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:3300","nonce":0},{"type":"v1","addr":"172.21.15.122:6789","nonce":0}]},"addr":"172.21.15.122:6789/0","public_addr":"172.21.15.122:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:3300","nonce":0},{"type":"v1","addr":"172.21.15.73:6789","nonce":0}]},"addr":"172.21.15.73:6789/0","public_addr":"172.21.15.73:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-16T00:24:12.103 INFO:teuthology.orchestra.run.smithi122.stderr:dumped monmap epoch 3 2024-09-16T00:24:12.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:12.104+0000 7fd108c2c640 1 -- 172.21.15.122:0/2659540991 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd0e4039730 msgr2=0x7fd0e403bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:12.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:12.104+0000 7fd108c2c640 1 --2- 172.21.15.122:0/2659540991 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd0e4039730 0x7fd0e403bbf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fd0f402ca00 tx=0x7fd0f402c990 comp rx=0 tx=0).stop 2024-09-16T00:24:12.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:12.104+0000 7fd108c2c640 1 -- 172.21.15.122:0/2659540991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd104199aa0 msgr2=0x7fd10419de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:12.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:12.104+0000 7fd108c2c640 1 --2- 172.21.15.122:0/2659540991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd104199aa0 0x7fd10419de30 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd0f800b450 tx=0x7fd0f800b910 comp rx=0 tx=0).stop 2024-09-16T00:24:12.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:12.104+0000 7fd108c2c640 1 -- 172.21.15.122:0/2659540991 shutdown_connections 2024-09-16T00:24:12.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:12.104+0000 7fd108c2c640 1 --2- 172.21.15.122:0/2659540991 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd0e4039730 0x7fd0e403bbf0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:12.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:12.104+0000 7fd108c2c640 1 --2- 172.21.15.122:0/2659540991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd10419e370 0x7fd10419e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:12.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:12.104+0000 7fd108c2c640 1 --2- 172.21.15.122:0/2659540991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd104199aa0 0x7fd10419de30 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:12.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:12.104+0000 7fd108c2c640 1 --2- 172.21.15.122:0/2659540991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd0ec005740 0x7fd104199560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:12.105 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:12.104+0000 7fd108c2c640 1 -- 172.21.15.122:0/2659540991 >> 172.21.15.122:0/2659540991 conn(0x7fd1040fc280 msgr2=0x7fd1040fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:12.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:12.104+0000 7fd108c2c640 1 -- 172.21.15.122:0/2659540991 shutdown_connections 2024-09-16T00:24:12.106 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:12.104+0000 7fd108c2c640 1 -- 172.21.15.122:0/2659540991 wait complete. 2024-09-16T00:24:12.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:12 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:24:12.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:12 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:12.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:12 smithi122 ceph-mon[35412]: from='client.? 172.21.15.122:0/2659540991' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:24:12.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:12 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:24:12.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:12 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:24:12.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:12 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:12.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:12 smithi038 ceph-mon[30945]: from='client.? 172.21.15.122:0/2659540991' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:24:12.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:12 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:24:12.955 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-16T00:24:12.955 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph config generate-minimal-conf 2024-09-16T00:24:13.218 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:24:13.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:13.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:24:13.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: mon.a calling monitor election 2024-09-16T00:24:13.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: mon.c calling monitor election 2024-09-16T00:24:13.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:24:13.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:24:13.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:24:13.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:13.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:24:13.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:24:13.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:13.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:24:13.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:24:13.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-16T00:24:13.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: monmap epoch 3 2024-09-16T00:24:13.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: fsid 983b2072-73c1-11ef-bceb-c7b262605968 2024-09-16T00:24:13.546 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: last_changed 2024-09-16T00:24:06.147017+0000 2024-09-16T00:24:13.546 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: created 2024-09-16T00:22:31.653387+0000 2024-09-16T00:24:13.546 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: min_mon_release 19 (squid) 2024-09-16T00:24:13.546 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: election_strategy: 1 2024-09-16T00:24:13.546 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-09-16T00:24:13.546 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: 1: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.c 2024-09-16T00:24:13.546 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: 2: [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] mon.b 2024-09-16T00:24:13.546 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: fsmap 2024-09-16T00:24:13.546 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:24:13.546 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: mgrmap e13: x(active, since 49s) 2024-09-16T00:24:13.546 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: overall HEALTH_OK 2024-09-16T00:24:13.546 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:13.546 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:13.547 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:13.547 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:13.547 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:24:13.547 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:13.547 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: from='client.? 172.21.15.122:0/2659540991' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-16T00:24:13.547 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:24:13.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:13.934+0000 7f99ec650640 1 -- 172.21.15.38:0/3417553836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e4106540 msgr2=0x7f99e4106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:13.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:13.934+0000 7f99ec650640 1 --2- 172.21.15.38:0/3417553836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e4106540 0x7f99e4106920 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f99e0009810 tx=0x7f99e002a540 comp rx=0 tx=0).stop 2024-09-16T00:24:13.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:13.935+0000 7f99ec650640 1 -- 172.21.15.38:0/3417553836 shutdown_connections 2024-09-16T00:24:13.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:13.935+0000 7f99ec650640 1 --2- 172.21.15.38:0/3417553836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e4106540 0x7f99e4106920 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:13.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:13.935+0000 7f99ec650640 1 -- 172.21.15.38:0/3417553836 >> 172.21.15.38:0/3417553836 conn(0x7f99e4077310 msgr2=0x7f99e40fe640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:13.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:13.935+0000 7f99ec650640 1 -- 172.21.15.38:0/3417553836 shutdown_connections 2024-09-16T00:24:13.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:13.935+0000 7f99ec650640 1 -- 172.21.15.38:0/3417553836 wait complete. 2024-09-16T00:24:13.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:13.935+0000 7f99ec650640 1 Processor -- start 2024-09-16T00:24:13.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:13.936+0000 7f99ec650640 1 -- start start 2024-09-16T00:24:13.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:13.936+0000 7f99ec650640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e4106540 0x7f99e4199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:13.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:13.936+0000 7f99ec650640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99e4199a50 0x7f99e419dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:13.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:13.936+0000 7f99ec650640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99e419e320 0x7f99e419e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:13.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:13.936+0000 7f99ec650640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f99e410ab20 con 0x7f99e4106540 2024-09-16T00:24:13.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:13.936+0000 7f99ec650640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f99e410a9a0 con 0x7f99e4199a50 2024-09-16T00:24:13.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:13.936+0000 7f99ec650640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f99e410aca0 con 0x7f99e419e320 2024-09-16T00:24:13.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:13.936+0000 7f99ea3ec640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e4106540 0x7f99e4199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:13.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:13.936+0000 7f99ea3ec640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e4106540 0x7f99e4199510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47546/0 (socket says 172.21.15.38:47546) 2024-09-16T00:24:13.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:13.936+0000 7f99ea3ec640 1 -- 172.21.15.38:0/76390111 learned_addr learned my addr 172.21.15.38:0/76390111 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:24:13.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:13.936+0000 7f99e9beb640 1 --2- 172.21.15.38:0/76390111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99e4199a50 0x7f99e419dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:13.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:13.936+0000 7f99ea3ec640 1 -- 172.21.15.38:0/76390111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99e419e320 msgr2=0x7f99e419e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:13.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:13.936+0000 7f99eabed640 1 --2- 172.21.15.38:0/76390111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99e419e320 0x7f99e419e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:13.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:13.936+0000 7f99ea3ec640 1 --2- 172.21.15.38:0/76390111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99e419e320 0x7f99e419e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:13.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:13.936+0000 7f99ea3ec640 1 -- 172.21.15.38:0/76390111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99e4199a50 msgr2=0x7f99e419dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:13.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:13.936+0000 7f99ea3ec640 1 --2- 172.21.15.38:0/76390111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99e4199a50 0x7f99e419dde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:13.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:13.936+0000 7f99ea3ec640 1 -- 172.21.15.38:0/76390111 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99e41a6090 con 0x7f99e4106540 2024-09-16T00:24:13.939 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:13.937+0000 7f99e9beb640 1 --2- 172.21.15.38:0/76390111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99e4199a50 0x7f99e419dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:24:13.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:13.937+0000 7f99eabed640 1 --2- 172.21.15.38:0/76390111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99e419e320 0x7f99e419e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:24:13.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:13.938+0000 7f99ea3ec640 1 --2- 172.21.15.38:0/76390111 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e4106540 0x7f99e4199510 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f99e00307a0 tx=0x7f99e00307d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:13.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:13.938+0000 7f99db7fe640 1 -- 172.21.15.38:0/76390111 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e0040070 con 0x7f99e4106540 2024-09-16T00:24:13.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:13.938+0000 7f99db7fe640 1 -- 172.21.15.38:0/76390111 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f99e0004c70 con 0x7f99e4106540 2024-09-16T00:24:13.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:13.938+0000 7f99ec650640 1 -- 172.21.15.38:0/76390111 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99e41a6260 con 0x7f99e4106540 2024-09-16T00:24:13.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:13.938+0000 7f99db7fe640 1 -- 172.21.15.38:0/76390111 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e002f570 con 0x7f99e4106540 2024-09-16T00:24:13.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:13.938+0000 7f99ec650640 1 -- 172.21.15.38:0/76390111 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99e41a67e0 con 0x7f99e4106540 2024-09-16T00:24:13.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:13.939+0000 7f99db7fe640 1 -- 172.21.15.38:0/76390111 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7f99e0039040 con 0x7f99e4106540 2024-09-16T00:24:13.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:13.939+0000 7f99ec650640 1 -- 172.21.15.38:0/76390111 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99e4106920 con 0x7f99e4106540 2024-09-16T00:24:13.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:13.940+0000 7f99db7fe640 1 --2- 172.21.15.38:0/76390111 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f99d0039730 0x7f99d003bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:13.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:13.940+0000 7f99db7fe640 1 -- 172.21.15.38:0/76390111 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f99e006db60 con 0x7f99e4106540 2024-09-16T00:24:13.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:13.940+0000 7f99e9beb640 1 --2- 172.21.15.38:0/76390111 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f99d0039730 0x7f99d003bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:13.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:13.940+0000 7f99e9beb640 1 --2- 172.21.15.38:0/76390111 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f99d0039730 0x7f99d003bbf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f99d4007920 tx=0x7f99d4008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:13.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:13.945+0000 7f99db7fe640 1 -- 172.21.15.38:0/76390111 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f99e0031a70 con 0x7f99e4106540 2024-09-16T00:24:14.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:14.124+0000 7f99ec650640 1 -- 172.21.15.38:0/76390111 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f99e4104e90 con 0x7f99e4106540 2024-09-16T00:24:14.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:14.124+0000 7f99db7fe640 1 -- 172.21.15.38:0/76390111 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f99e0043360 con 0x7f99e4106540 2024-09-16T00:24:14.125 INFO:teuthology.orchestra.run.smithi038.stdout:# minimal ceph.conf for 983b2072-73c1-11ef-bceb-c7b262605968 2024-09-16T00:24:14.125 INFO:teuthology.orchestra.run.smithi038.stdout:[global] 2024-09-16T00:24:14.125 INFO:teuthology.orchestra.run.smithi038.stdout: fsid = 983b2072-73c1-11ef-bceb-c7b262605968 2024-09-16T00:24:14.125 INFO:teuthology.orchestra.run.smithi038.stdout: mon_host = [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] 2024-09-16T00:24:14.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:14.126+0000 7f99ec650640 1 -- 172.21.15.38:0/76390111 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f99d0039730 msgr2=0x7f99d003bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:14.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:14.126+0000 7f99ec650640 1 --2- 172.21.15.38:0/76390111 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f99d0039730 0x7f99d003bbf0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f99d4007920 tx=0x7f99d4008040 comp rx=0 tx=0).stop 2024-09-16T00:24:14.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:14.126+0000 7f99ec650640 1 -- 172.21.15.38:0/76390111 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e4106540 msgr2=0x7f99e4199510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:14.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:14.126+0000 7f99ec650640 1 --2- 172.21.15.38:0/76390111 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e4106540 0x7f99e4199510 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f99e00307a0 tx=0x7f99e00307d0 comp rx=0 tx=0).stop 2024-09-16T00:24:14.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:14.126+0000 7f99ec650640 1 -- 172.21.15.38:0/76390111 shutdown_connections 2024-09-16T00:24:14.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:14.126+0000 7f99ec650640 1 --2- 172.21.15.38:0/76390111 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f99d0039730 0x7f99d003bbf0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:14.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:14.126+0000 7f99ec650640 1 --2- 172.21.15.38:0/76390111 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f99e419e320 0x7f99e419e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:14.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:14.126+0000 7f99ec650640 1 --2- 172.21.15.38:0/76390111 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99e4199a50 0x7f99e419dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:14.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:14.126+0000 7f99ec650640 1 --2- 172.21.15.38:0/76390111 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f99e4106540 0x7f99e4199510 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:14.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:14.126+0000 7f99ec650640 1 -- 172.21.15.38:0/76390111 >> 172.21.15.38:0/76390111 conn(0x7f99e4077310 msgr2=0x7f99e4102030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:14.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:14.127+0000 7f99ec650640 1 -- 172.21.15.38:0/76390111 shutdown_connections 2024-09-16T00:24:14.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:14.127+0000 7f99ec650640 1 -- 172.21.15.38:0/76390111 wait complete. 2024-09-16T00:24:14.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:13 smithi038 ceph-mon[30945]: mon.b calling monitor election 2024-09-16T00:24:14.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:13 smithi038 ceph-mon[30945]: Updating smithi073:/var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:24:14.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:13 smithi038 ceph-mon[30945]: Updating smithi122:/var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:24:14.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:13 smithi038 ceph-mon[30945]: Updating smithi038:/var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:24:14.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:13 smithi038 ceph-mon[30945]: Reconfiguring mon.a (unknown last config time)... 2024-09-16T00:24:14.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:13 smithi038 ceph-mon[30945]: Reconfiguring daemon mon.a on smithi038 2024-09-16T00:24:14.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:13 smithi038 ceph-mon[30945]: mon.b calling monitor election 2024-09-16T00:24:14.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:13 smithi038 ceph-mon[30945]: mon.c calling monitor election 2024-09-16T00:24:14.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:13 smithi038 ceph-mon[30945]: mon.a calling monitor election 2024-09-16T00:24:14.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:13 smithi038 ceph-mon[30945]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-16T00:24:14.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:13 smithi038 ceph-mon[30945]: monmap epoch 3 2024-09-16T00:24:14.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:13 smithi038 ceph-mon[30945]: fsid 983b2072-73c1-11ef-bceb-c7b262605968 2024-09-16T00:24:14.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:13 smithi038 ceph-mon[30945]: last_changed 2024-09-16T00:24:06.147017+0000 2024-09-16T00:24:14.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:13 smithi038 ceph-mon[30945]: created 2024-09-16T00:22:31.653387+0000 2024-09-16T00:24:14.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:13 smithi038 ceph-mon[30945]: min_mon_release 19 (squid) 2024-09-16T00:24:14.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:13 smithi038 ceph-mon[30945]: election_strategy: 1 2024-09-16T00:24:14.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:13 smithi038 ceph-mon[30945]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-09-16T00:24:14.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:13 smithi038 ceph-mon[30945]: 1: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.c 2024-09-16T00:24:14.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:13 smithi038 ceph-mon[30945]: 2: [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] mon.b 2024-09-16T00:24:14.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:13 smithi038 ceph-mon[30945]: fsmap 2024-09-16T00:24:14.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:13 smithi038 ceph-mon[30945]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:24:14.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:13 smithi038 ceph-mon[30945]: mgrmap e13: x(active, since 51s) 2024-09-16T00:24:14.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:13 smithi038 ceph-mon[30945]: overall HEALTH_OK 2024-09-16T00:24:14.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:13 smithi122 ceph-mon[35412]: mon.b calling monitor election 2024-09-16T00:24:14.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:13 smithi122 ceph-mon[35412]: Updating smithi073:/var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:24:14.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:13 smithi122 ceph-mon[35412]: Updating smithi122:/var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:24:14.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:13 smithi122 ceph-mon[35412]: Updating smithi038:/var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:24:14.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:13 smithi122 ceph-mon[35412]: Reconfiguring mon.a (unknown last config time)... 2024-09-16T00:24:14.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:13 smithi122 ceph-mon[35412]: Reconfiguring daemon mon.a on smithi038 2024-09-16T00:24:14.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:13 smithi122 ceph-mon[35412]: mon.b calling monitor election 2024-09-16T00:24:14.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:13 smithi122 ceph-mon[35412]: mon.c calling monitor election 2024-09-16T00:24:14.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:13 smithi122 ceph-mon[35412]: mon.a calling monitor election 2024-09-16T00:24:14.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:13 smithi122 ceph-mon[35412]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-16T00:24:14.218 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:13 smithi122 ceph-mon[35412]: monmap epoch 3 2024-09-16T00:24:14.218 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:13 smithi122 ceph-mon[35412]: fsid 983b2072-73c1-11ef-bceb-c7b262605968 2024-09-16T00:24:14.218 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:13 smithi122 ceph-mon[35412]: last_changed 2024-09-16T00:24:06.147017+0000 2024-09-16T00:24:14.218 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:13 smithi122 ceph-mon[35412]: created 2024-09-16T00:22:31.653387+0000 2024-09-16T00:24:14.218 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:13 smithi122 ceph-mon[35412]: min_mon_release 19 (squid) 2024-09-16T00:24:14.218 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:13 smithi122 ceph-mon[35412]: election_strategy: 1 2024-09-16T00:24:14.218 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:13 smithi122 ceph-mon[35412]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-09-16T00:24:14.218 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:13 smithi122 ceph-mon[35412]: 1: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.c 2024-09-16T00:24:14.218 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:13 smithi122 ceph-mon[35412]: 2: [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] mon.b 2024-09-16T00:24:14.218 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:13 smithi122 ceph-mon[35412]: fsmap 2024-09-16T00:24:14.218 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:13 smithi122 ceph-mon[35412]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:24:14.218 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:13 smithi122 ceph-mon[35412]: mgrmap e13: x(active, since 51s) 2024-09-16T00:24:14.218 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:13 smithi122 ceph-mon[35412]: overall HEALTH_OK 2024-09-16T00:24:14.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: mon.b calling monitor election 2024-09-16T00:24:14.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: Updating smithi073:/var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:24:14.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: Updating smithi122:/var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:24:14.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: Updating smithi038:/var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/config/ceph.conf 2024-09-16T00:24:14.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: Reconfiguring mon.a (unknown last config time)... 2024-09-16T00:24:14.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: Reconfiguring daemon mon.a on smithi038 2024-09-16T00:24:14.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: mon.b calling monitor election 2024-09-16T00:24:14.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: mon.c calling monitor election 2024-09-16T00:24:14.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: mon.a calling monitor election 2024-09-16T00:24:14.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-16T00:24:14.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: monmap epoch 3 2024-09-16T00:24:14.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: fsid 983b2072-73c1-11ef-bceb-c7b262605968 2024-09-16T00:24:14.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: last_changed 2024-09-16T00:24:06.147017+0000 2024-09-16T00:24:14.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: created 2024-09-16T00:22:31.653387+0000 2024-09-16T00:24:14.296 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: min_mon_release 19 (squid) 2024-09-16T00:24:14.296 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: election_strategy: 1 2024-09-16T00:24:14.296 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: 0: [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] mon.a 2024-09-16T00:24:14.296 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: 1: [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] mon.c 2024-09-16T00:24:14.296 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: 2: [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] mon.b 2024-09-16T00:24:14.296 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: fsmap 2024-09-16T00:24:14.296 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: osdmap e4: 0 total, 0 up, 0 in 2024-09-16T00:24:14.296 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: mgrmap e13: x(active, since 51s) 2024-09-16T00:24:14.296 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:13 smithi073 ceph-mon[35254]: overall HEALTH_OK 2024-09-16T00:24:15.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:14 smithi038 ceph-mon[30945]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:15.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:14 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/76390111' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:15.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:14 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:24:15.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:14 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:15.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:14 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:15.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:14 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:24:15.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:14 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:24:15.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:14 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:15.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:14 smithi122 ceph-mon[35412]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:15.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:14 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/76390111' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:15.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:14 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:24:15.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:14 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:15.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:14 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:15.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:14 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:24:15.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:14 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:24:15.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:14 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:15.245 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:14 smithi073 ceph-mon[35254]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:15.246 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:14 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/76390111' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:15.246 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:14 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:24:15.246 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:14 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:15.246 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:14 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:15.246 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:14 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:24:15.246 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:14 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:24:15.246 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:14 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:15.349 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-16T00:24:15.350 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-16T00:24:15.350 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/ceph/ceph.conf 2024-09-16T00:24:15.378 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-16T00:24:15.378 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:24:15.444 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-16T00:24:15.445 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/ceph/ceph.conf 2024-09-16T00:24:15.472 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-16T00:24:15.472 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:24:15.537 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-16T00:24:15.537 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/ceph/ceph.conf 2024-09-16T00:24:15.569 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-16T00:24:15.569 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-16T00:24:15.637 INFO:tasks.cephadm:Adding mgr.x on smithi038 2024-09-16T00:24:15.637 INFO:tasks.cephadm:Adding mgr.y on smithi073 2024-09-16T00:24:15.638 INFO:tasks.cephadm:Adding mgr.z on smithi122 2024-09-16T00:24:15.638 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph orch apply mgr '3;smithi038=x;smithi073=y;smithi122=z' 2024-09-16T00:24:15.919 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:24:16.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:15 smithi038 ceph-mon[30945]: Reconfiguring mon.b (monmap changed)... 2024-09-16T00:24:16.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:15 smithi038 ceph-mon[30945]: Reconfiguring daemon mon.b on smithi073 2024-09-16T00:24:16.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:15 smithi122 ceph-mon[35412]: Reconfiguring mon.b (monmap changed)... 2024-09-16T00:24:16.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:15 smithi122 ceph-mon[35412]: Reconfiguring daemon mon.b on smithi073 2024-09-16T00:24:16.296 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:15 smithi073 ceph-mon[35254]: Reconfiguring mon.b (monmap changed)... 2024-09-16T00:24:16.296 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:15 smithi073 ceph-mon[35254]: Reconfiguring daemon mon.b on smithi073 2024-09-16T00:24:16.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.419+0000 7fb408afe640 1 -- 172.21.15.122:0/1432681433 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4041005f0 msgr2=0x7fb4041009f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:16.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.419+0000 7fb408afe640 1 --2- 172.21.15.122:0/1432681433 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4041005f0 0x7fb4041009f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4009880 tx=0x7fb3f402a5d0 comp rx=0 tx=0).stop 2024-09-16T00:24:16.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.419+0000 7fb408afe640 1 -- 172.21.15.122:0/1432681433 shutdown_connections 2024-09-16T00:24:16.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.419+0000 7fb408afe640 1 --2- 172.21.15.122:0/1432681433 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb4041005f0 0x7fb4041009f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:16.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.419+0000 7fb408afe640 1 -- 172.21.15.122:0/1432681433 >> 172.21.15.122:0/1432681433 conn(0x7fb4040fbda0 msgr2=0x7fb4040fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:16.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.419+0000 7fb408afe640 1 -- 172.21.15.122:0/1432681433 shutdown_connections 2024-09-16T00:24:16.433 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.419+0000 7fb408afe640 1 -- 172.21.15.122:0/1432681433 wait complete. 2024-09-16T00:24:16.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.420+0000 7fb408afe640 1 Processor -- start 2024-09-16T00:24:16.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.420+0000 7fb408afe640 1 -- start start 2024-09-16T00:24:16.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.420+0000 7fb408afe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4041005f0 0x7fb40418d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:16.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.420+0000 7fb408afe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb40418bf40 0x7fb40418c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:16.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.420+0000 7fb408afe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb40418c900 0x7fb4041a1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:16.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.420+0000 7fb408afe640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb4041066e0 con 0x7fb40418bf40 2024-09-16T00:24:16.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.420+0000 7fb408afe640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb404106560 con 0x7fb4041005f0 2024-09-16T00:24:16.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.420+0000 7fb408afe640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb404106860 con 0x7fb40418c900 2024-09-16T00:24:16.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.420+0000 7fb402d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb40418c900 0x7fb4041a1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:16.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.420+0000 7fb3fbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb40418bf40 0x7fb40418c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:16.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.420+0000 7fb402d9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb40418c900 0x7fb4041a1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:45908/0 (socket says 172.21.15.122:45908) 2024-09-16T00:24:16.434 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.420+0000 7fb3fbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb40418bf40 0x7fb40418c3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.122:44408/0 (socket says 172.21.15.122:44408) 2024-09-16T00:24:16.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.420+0000 7fb402d9d640 1 -- 172.21.15.122:0/1112377081 learned_addr learned my addr 172.21.15.122:0/1112377081 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-16T00:24:16.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.420+0000 7fb40259c640 1 --2- 172.21.15.122:0/1112377081 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4041005f0 0x7fb40418d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:16.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.420+0000 7fb402d9d640 1 -- 172.21.15.122:0/1112377081 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4041005f0 msgr2=0x7fb40418d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:16.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.420+0000 7fb402d9d640 1 --2- 172.21.15.122:0/1112377081 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4041005f0 0x7fb40418d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:16.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.420+0000 7fb402d9d640 1 -- 172.21.15.122:0/1112377081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb40418bf40 msgr2=0x7fb40418c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:16.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.420+0000 7fb402d9d640 1 --2- 172.21.15.122:0/1112377081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb40418bf40 0x7fb40418c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:16.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.421+0000 7fb402d9d640 1 -- 172.21.15.122:0/1112377081 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4041a1600 con 0x7fb40418c900 2024-09-16T00:24:16.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.421+0000 7fb3fbfff640 1 --2- 172.21.15.122:0/1112377081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb40418bf40 0x7fb40418c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:24:16.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.421+0000 7fb402d9d640 1 --2- 172.21.15.122:0/1112377081 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb40418c900 0x7fb4041a1070 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc00ba50 tx=0x7fb3fc00bf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:16.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.421+0000 7fb3fb7fe640 1 -- 172.21.15.122:0/1112377081 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc019070 con 0x7fb40418c900 2024-09-16T00:24:16.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.421+0000 7fb3fb7fe640 1 -- 172.21.15.122:0/1112377081 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb3fc00cbe0 con 0x7fb40418c900 2024-09-16T00:24:16.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.421+0000 7fb408afe640 1 -- 172.21.15.122:0/1112377081 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4041a18f0 con 0x7fb40418c900 2024-09-16T00:24:16.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.421+0000 7fb3fb7fe640 1 -- 172.21.15.122:0/1112377081 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3fc0076a0 con 0x7fb40418c900 2024-09-16T00:24:16.435 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.421+0000 7fb408afe640 1 -- 172.21.15.122:0/1112377081 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4041a1eb0 con 0x7fb40418c900 2024-09-16T00:24:16.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.422+0000 7fb3fb7fe640 1 -- 172.21.15.122:0/1112377081 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7fb3fc01c070 con 0x7fb40418c900 2024-09-16T00:24:16.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.422+0000 7fb408afe640 1 -- 172.21.15.122:0/1112377081 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4041a2140 con 0x7fb40418c900 2024-09-16T00:24:16.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.422+0000 7fb3fb7fe640 1 --2- 172.21.15.122:0/1112377081 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb3f0039730 0x7fb3f003bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:16.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.422+0000 7fb3fb7fe640 1 -- 172.21.15.122:0/1112377081 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fb3fc04bd10 con 0x7fb40418c900 2024-09-16T00:24:16.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.422+0000 7fb40259c640 1 --2- 172.21.15.122:0/1112377081 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb3f0039730 0x7fb3f003bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:16.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.423+0000 7fb40259c640 1 --2- 172.21.15.122:0/1112377081 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb3f0039730 0x7fb3f003bbf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4000c00 tx=0x7fb3f4031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:16.436 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.427+0000 7fb3fb7fe640 1 -- 172.21.15.122:0/1112377081 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb3fc012e50 con 0x7fb40418c900 2024-09-16T00:24:16.609 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.609+0000 7fb408afe640 1 -- 172.21.15.122:0/1112377081 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi038=x;smithi073=y;smithi122=z", "target": ["mon-mgr", ""]}) -- 0x7fb40406ab40 con 0x7fb3f0039730 2024-09-16T00:24:16.615 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.615+0000 7fb3fb7fe640 1 -- 172.21.15.122:0/1112377081 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fb40406ab40 con 0x7fb3f0039730 2024-09-16T00:24:16.615 INFO:teuthology.orchestra.run.smithi122.stdout:Scheduled mgr update... 2024-09-16T00:24:16.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.617+0000 7fb408afe640 1 -- 172.21.15.122:0/1112377081 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb3f0039730 msgr2=0x7fb3f003bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:16.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.617+0000 7fb408afe640 1 --2- 172.21.15.122:0/1112377081 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb3f0039730 0x7fb3f003bbf0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4000c00 tx=0x7fb3f4031400 comp rx=0 tx=0).stop 2024-09-16T00:24:16.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.617+0000 7fb408afe640 1 -- 172.21.15.122:0/1112377081 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb40418c900 msgr2=0x7fb4041a1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:16.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.617+0000 7fb408afe640 1 --2- 172.21.15.122:0/1112377081 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb40418c900 0x7fb4041a1070 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc00ba50 tx=0x7fb3fc00bf10 comp rx=0 tx=0).stop 2024-09-16T00:24:16.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.617+0000 7fb408afe640 1 -- 172.21.15.122:0/1112377081 shutdown_connections 2024-09-16T00:24:16.617 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.617+0000 7fb408afe640 1 --2- 172.21.15.122:0/1112377081 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb3f0039730 0x7fb3f003bbf0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:16.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.617+0000 7fb408afe640 1 --2- 172.21.15.122:0/1112377081 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb40418c900 0x7fb4041a1070 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:16.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.617+0000 7fb408afe640 1 --2- 172.21.15.122:0/1112377081 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb40418bf40 0x7fb40418c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:16.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.617+0000 7fb408afe640 1 --2- 172.21.15.122:0/1112377081 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb4041005f0 0x7fb40418d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:16.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.617+0000 7fb408afe640 1 -- 172.21.15.122:0/1112377081 >> 172.21.15.122:0/1112377081 conn(0x7fb4040fbda0 msgr2=0x7fb40406a430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:16.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.617+0000 7fb408afe640 1 -- 172.21.15.122:0/1112377081 shutdown_connections 2024-09-16T00:24:16.618 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:24:16.617+0000 7fb408afe640 1 -- 172.21.15.122:0/1112377081 wait complete. 2024-09-16T00:24:17.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:17 smithi073 ceph-mon[35254]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:17.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:17 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:17.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:17 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:17.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:17 smithi073 ceph-mon[35254]: Reconfiguring mon.c (monmap changed)... 2024-09-16T00:24:17.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:17 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:24:17.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:17 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:24:17.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:17 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:17.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:17 smithi073 ceph-mon[35254]: Reconfiguring daemon mon.c on smithi122 2024-09-16T00:24:17.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:17 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:17.302 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:17 smithi122 ceph-mon[35412]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:17.302 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:17 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:17.302 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:17 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:17.302 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:17 smithi122 ceph-mon[35412]: Reconfiguring mon.c (monmap changed)... 2024-09-16T00:24:17.302 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:17 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:24:17.302 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:17 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:24:17.303 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:17 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:17.303 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:17 smithi122 ceph-mon[35412]: Reconfiguring daemon mon.c on smithi122 2024-09-16T00:24:17.303 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:17 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:17.329 DEBUG:teuthology.orchestra.run.smithi073:mgr.y> sudo journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@mgr.y.service 2024-09-16T00:24:17.332 DEBUG:teuthology.orchestra.run.smithi122:mgr.z> sudo journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@mgr.z.service 2024-09-16T00:24:17.334 INFO:tasks.cephadm:Deploying OSDs... 2024-09-16T00:24:17.334 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-16T00:24:17.334 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T00:24:17.355 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T00:24:17.355 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_1 2024-09-16T00:24:17.415 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T00:24:17.415 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:24:17.415 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-09-16T00:24:17.415 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:24:17.415 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:24:17.415 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-09-16 00:23:32.498642503 +0000 2024-09-16T00:24:17.415 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-09-16 00:18:57.921004378 +0000 2024-09-16T00:24:17.415 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-09-16 00:18:57.921004378 +0000 2024-09-16T00:24:17.416 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-09-16 00:18:57.921004378 +0000 2024-09-16T00:24:17.416 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T00:24:17.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:17 smithi038 ceph-mon[30945]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:17.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:17 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:17.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:17 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:17.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:17 smithi038 ceph-mon[30945]: Reconfiguring mon.c (monmap changed)... 2024-09-16T00:24:17.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:17 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-16T00:24:17.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:17 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-16T00:24:17.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:17 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:17.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:17 smithi038 ceph-mon[30945]: Reconfiguring daemon mon.c on smithi122 2024-09-16T00:24:17.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:17 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:17.485 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-09-16T00:24:17.485 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-09-16T00:24:17.485 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000305621 s, 1.7 MB/s 2024-09-16T00:24:17.487 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T00:24:17.547 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_2 2024-09-16T00:24:17.605 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T00:24:17.605 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:24:17.605 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-16T00:24:17.605 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:24:17.605 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:24:17.606 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-09-16 00:23:32.531641869 +0000 2024-09-16T00:24:17.606 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-09-16 00:18:58.263997808 +0000 2024-09-16T00:24:17.606 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-09-16 00:18:58.263997808 +0000 2024-09-16T00:24:17.606 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-09-16 00:18:58.263997808 +0000 2024-09-16T00:24:17.606 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T00:24:17.672 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-09-16T00:24:17.672 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-09-16T00:24:17.672 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000393592 s, 1.3 MB/s 2024-09-16T00:24:17.673 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T00:24:17.733 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_3 2024-09-16T00:24:17.790 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T00:24:17.790 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:24:17.790 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-16T00:24:17.790 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:24:17.790 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:24:17.790 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-09-16 00:23:32.564641235 +0000 2024-09-16T00:24:17.790 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-09-16 00:18:58.598991391 +0000 2024-09-16T00:24:17.791 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-09-16 00:18:58.598991391 +0000 2024-09-16T00:24:17.791 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-09-16 00:18:58.598991391 +0000 2024-09-16T00:24:17.791 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T00:24:17.856 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-09-16T00:24:17.856 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-09-16T00:24:17.856 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000268159 s, 1.9 MB/s 2024-09-16T00:24:17.857 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T00:24:17.914 DEBUG:teuthology.orchestra.run.smithi038:> stat /dev/vg_nvme/lv_4 2024-09-16T00:24:17.972 INFO:teuthology.orchestra.run.smithi038.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T00:24:17.972 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:24:17.972 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 5h/5d Inode: 724 Links: 1 2024-09-16T00:24:17.972 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:24:17.972 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:24:17.972 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-09-16 00:23:32.599640563 +0000 2024-09-16T00:24:17.972 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-09-16 00:18:58.954984572 +0000 2024-09-16T00:24:17.972 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-09-16 00:18:58.954984572 +0000 2024-09-16T00:24:17.972 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-09-16 00:18:58.954984572 +0000 2024-09-16T00:24:17.972 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T00:24:18.037 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records in 2024-09-16T00:24:18.038 INFO:teuthology.orchestra.run.smithi038.stderr:1+0 records out 2024-09-16T00:24:18.038 INFO:teuthology.orchestra.run.smithi038.stderr:512 bytes copied, 0.000286843 s, 1.8 MB/s 2024-09-16T00:24:18.039 DEBUG:teuthology.orchestra.run.smithi038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T00:24:18.096 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-09-16T00:24:18.097 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T00:24:18.116 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T00:24:18.117 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_1 2024-09-16T00:24:18.175 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T00:24:18.175 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:24:18.175 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-16T00:24:18.175 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:24:18.176 INFO:teuthology.orchestra.run.smithi073.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:24:18.176 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-09-16 00:24:05.653773742 +0000 2024-09-16T00:24:18.176 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-09-16 00:19:07.703863683 +0000 2024-09-16T00:24:18.176 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-09-16 00:19:07.703863683 +0000 2024-09-16T00:24:18.176 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: 2024-09-16 00:19:07.703863683 +0000 2024-09-16T00:24:18.176 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T00:24:18.214 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:18 smithi038 ceph-mon[30945]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi038=x;smithi073=y;smithi122=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:24:18.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:18 smithi038 ceph-mon[30945]: Saving service mgr spec with placement smithi038=x;smithi073=y;smithi122=z;count:3 2024-09-16T00:24:18.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:18 smithi122 ceph-mon[35412]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi038=x;smithi073=y;smithi122=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:24:18.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:18 smithi122 ceph-mon[35412]: Saving service mgr spec with placement smithi038=x;smithi073=y;smithi122=z;count:3 2024-09-16T00:24:18.242 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-09-16T00:24:18.243 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-09-16T00:24:18.243 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000287024 s, 1.8 MB/s 2024-09-16T00:24:18.244 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T00:24:18.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:18 smithi073 ceph-mon[35254]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi038=x;smithi073=y;smithi122=z", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:24:18.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:18 smithi073 ceph-mon[35254]: Saving service mgr spec with placement smithi038=x;smithi073=y;smithi122=z;count:3 2024-09-16T00:24:18.303 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_2 2024-09-16T00:24:18.360 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T00:24:18.360 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:24:18.360 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-16T00:24:18.361 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:24:18.361 INFO:teuthology.orchestra.run.smithi073.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:24:18.361 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-09-16 00:24:05.653773742 +0000 2024-09-16T00:24:18.361 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-09-16 00:19:08.044855870 +0000 2024-09-16T00:24:18.361 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-09-16 00:19:08.044855870 +0000 2024-09-16T00:24:18.361 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: 2024-09-16 00:19:08.044855870 +0000 2024-09-16T00:24:18.361 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T00:24:18.426 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-09-16T00:24:18.426 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-09-16T00:24:18.426 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000232838 s, 2.2 MB/s 2024-09-16T00:24:18.427 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T00:24:18.484 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_3 2024-09-16T00:24:18.542 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T00:24:18.542 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:24:18.542 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-16T00:24:18.542 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:24:18.542 INFO:teuthology.orchestra.run.smithi073.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:24:18.543 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-09-16 00:24:05.653773742 +0000 2024-09-16T00:24:18.543 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-09-16 00:19:08.361848606 +0000 2024-09-16T00:24:18.543 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-09-16 00:19:08.361848606 +0000 2024-09-16T00:24:18.543 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: 2024-09-16 00:19:08.361848606 +0000 2024-09-16T00:24:18.544 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T00:24:18.607 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-09-16T00:24:18.607 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-09-16T00:24:18.607 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000256069 s, 2.0 MB/s 2024-09-16T00:24:18.608 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T00:24:18.665 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_4 2024-09-16T00:24:18.721 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T00:24:18.721 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:24:18.721 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-09-16T00:24:18.721 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:24:18.722 INFO:teuthology.orchestra.run.smithi073.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:24:18.722 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-09-16 00:24:05.654773719 +0000 2024-09-16T00:24:18.722 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-09-16 00:19:08.689841090 +0000 2024-09-16T00:24:18.722 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-09-16 00:19:08.689841090 +0000 2024-09-16T00:24:18.722 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: 2024-09-16 00:19:08.689841090 +0000 2024-09-16T00:24:18.722 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T00:24:18.785 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-09-16T00:24:18.785 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-09-16T00:24:18.785 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000201622 s, 2.5 MB/s 2024-09-16T00:24:18.786 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T00:24:18.842 DEBUG:teuthology.orchestra.run.smithi122:> set -ex 2024-09-16T00:24:18.842 DEBUG:teuthology.orchestra.run.smithi122:> dd if=/scratch_devs of=/dev/stdout 2024-09-16T00:24:18.859 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-16T00:24:18.859 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_1 2024-09-16T00:24:18.914 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-16T00:24:18.914 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:24:18.914 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-09-16T00:24:18.915 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:24:18.915 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:24:18.915 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-09-16 00:23:56.624505271 +0000 2024-09-16T00:24:18.915 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-09-16 00:19:09.560923775 +0000 2024-09-16T00:24:18.915 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-09-16 00:19:09.560923775 +0000 2024-09-16T00:24:18.915 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-09-16 00:19:09.560923775 +0000 2024-09-16T00:24:18.915 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-16T00:24:18.978 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-09-16T00:24:18.978 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-09-16T00:24:18.978 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000215087 s, 2.4 MB/s 2024-09-16T00:24:18.980 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-16T00:24:19.036 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_2 2024-09-16T00:24:19.091 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-16T00:24:19.091 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:24:19.091 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-09-16T00:24:19.091 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:24:19.091 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:24:19.091 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-09-16 00:23:56.624505271 +0000 2024-09-16T00:24:19.092 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-09-16 00:19:09.884916037 +0000 2024-09-16T00:24:19.092 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-09-16 00:19:09.884916037 +0000 2024-09-16T00:24:19.092 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-09-16 00:19:09.884916037 +0000 2024-09-16T00:24:19.092 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-16T00:24:19.155 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-09-16T00:24:19.155 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-09-16T00:24:19.155 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000216563 s, 2.4 MB/s 2024-09-16T00:24:19.156 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-16T00:24:19.213 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_3 2024-09-16T00:24:19.268 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-16T00:24:19.268 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:24:19.268 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-09-16T00:24:19.269 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:24:19.269 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:24:19.269 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-09-16 00:23:56.625505247 +0000 2024-09-16T00:24:19.269 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-09-16 00:19:10.203908418 +0000 2024-09-16T00:24:19.269 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-09-16 00:19:10.203908418 +0000 2024-09-16T00:24:19.269 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-09-16 00:19:10.203908418 +0000 2024-09-16T00:24:19.269 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-16T00:24:19.332 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-09-16T00:24:19.332 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-09-16T00:24:19.332 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000181135 s, 2.8 MB/s 2024-09-16T00:24:19.334 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-16T00:24:19.390 DEBUG:teuthology.orchestra.run.smithi122:> stat /dev/vg_nvme/lv_4 2024-09-16T00:24:19.445 INFO:teuthology.orchestra.run.smithi122.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-16T00:24:19.445 INFO:teuthology.orchestra.run.smithi122.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-16T00:24:19.445 INFO:teuthology.orchestra.run.smithi122.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-16T00:24:19.445 INFO:teuthology.orchestra.run.smithi122.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:24:19.445 INFO:teuthology.orchestra.run.smithi122.stdout:Context: system_u:object_r:device_t:s0 2024-09-16T00:24:19.445 INFO:teuthology.orchestra.run.smithi122.stdout:Access: 2024-09-16 00:23:56.625505247 +0000 2024-09-16T00:24:19.445 INFO:teuthology.orchestra.run.smithi122.stdout:Modify: 2024-09-16 00:19:10.527900679 +0000 2024-09-16T00:24:19.446 INFO:teuthology.orchestra.run.smithi122.stdout:Change: 2024-09-16 00:19:10.527900679 +0000 2024-09-16T00:24:19.446 INFO:teuthology.orchestra.run.smithi122.stdout: Birth: 2024-09-16 00:19:10.527900679 +0000 2024-09-16T00:24:19.446 DEBUG:teuthology.orchestra.run.smithi122:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-16T00:24:19.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:19 smithi038 ceph-mon[30945]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:19.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:19 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:19.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:19 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:19.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:19 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:24:19.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:19 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:19.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:19 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:24:19.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:19 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:19.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:19 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:24:19.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:19 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T00:24:19.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:19 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:24:19.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:19 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:19.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:19 smithi038 ceph-mon[30945]: Deploying daemon mgr.z on smithi122 2024-09-16T00:24:19.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:19 smithi122 ceph-mon[35412]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:19.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:19 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:19.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:19 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:19.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:19 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:24:19.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:19 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:19.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:19 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:24:19.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:19 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:19.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:19 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:24:19.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:19 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T00:24:19.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:19 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:24:19.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:19 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:19.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:19 smithi122 ceph-mon[35412]: Deploying daemon mgr.z on smithi122 2024-09-16T00:24:19.489 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records in 2024-09-16T00:24:19.489 INFO:teuthology.orchestra.run.smithi122.stderr:1+0 records out 2024-09-16T00:24:19.490 INFO:teuthology.orchestra.run.smithi122.stderr:512 bytes copied, 0.000169786 s, 3.0 MB/s 2024-09-16T00:24:19.491 DEBUG:teuthology.orchestra.run.smithi122:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-16T00:24:19.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:19 smithi073 ceph-mon[35254]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:19.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:19 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:19.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:19 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:19.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:19 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:24:19.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:19 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:19.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:19 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:24:19.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:19 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:19.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:19 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:24:19.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:19 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T00:24:19.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:19 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:24:19.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:19 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:19.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:19 smithi073 ceph-mon[35254]: Deploying daemon mgr.z on smithi122 2024-09-16T00:24:19.547 INFO:tasks.cephadm:Deploying osd.0 on smithi038 with /dev/vg_nvme/lv_4... 2024-09-16T00:24:19.547 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-16T00:24:19.810 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:24:20.217 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Sep 16 00:24:20 smithi122 systemd[1]: Starting Ceph mgr.z for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:24:20.512 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Sep 16 00:24:20 smithi122 podman[37207]: 2024-09-16 00:24:20.145583347 +0000 UTC m=+0.015607016 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:24:20.512 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Sep 16 00:24:20 smithi122 podman[37207]: 2024-09-16 00:24:20.257486877 +0000 UTC m=+0.127510533 container create b280204398a613416c1a5dc3826ed883f7f2efc8815b61e4c7d14c68224b469e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-16T00:24:20.512 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Sep 16 00:24:20 smithi122 podman[37207]: 2024-09-16 00:24:20.484608905 +0000 UTC m=+0.354632561 container init b280204398a613416c1a5dc3826ed883f7f2efc8815b61e4c7d14c68224b469e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-z, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e) 2024-09-16T00:24:20.512 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Sep 16 00:24:20 smithi122 podman[37207]: 2024-09-16 00:24:20.487244617 +0000 UTC m=+0.357268276 container start b280204398a613416c1a5dc3826ed883f7f2efc8815b61e4c7d14c68224b469e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-z, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=) 2024-09-16T00:24:20.512 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Sep 16 00:24:20 smithi122 ceph-mgr[37226]: -- 172.21.15.122:0/119860473 <== mon.1 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c6734890e0 con 0x55c673466800 2024-09-16T00:24:20.816 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Sep 16 00:24:20 smithi122 bash[37207]: b280204398a613416c1a5dc3826ed883f7f2efc8815b61e4c7d14c68224b469e 2024-09-16T00:24:20.816 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Sep 16 00:24:20 smithi122 systemd[1]: Started Ceph mgr.z for 983b2072-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:24:20.816 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Sep 16 00:24:20 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-z[37222]: 2024-09-16T00:24:20.607+0000 7feaf2f2d040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T00:24:20.816 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Sep 16 00:24:20 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-z[37222]: 2024-09-16T00:24:20.667+0000 7feaf2f2d040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T00:24:20.816 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Sep 16 00:24:20 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-z[37222]: 2024-09-16T00:24:20.814+0000 7feaf2f2d040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T00:24:21.166 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Sep 16 00:24:20 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-z[37222]: 2024-09-16T00:24:20.879+0000 7feaf2f2d040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T00:24:21.166 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Sep 16 00:24:21 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-z[37222]: 2024-09-16T00:24:21.025+0000 7feaf2f2d040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:24:21.166 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Sep 16 00:24:21 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-z[37222]: 2024-09-16T00:24:21.164+0000 7feaf2f2d040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T00:24:21.190 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:21 smithi038 ceph-mon[30945]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:21.191 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:21 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:21.191 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:21 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:21.191 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:21 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:21.191 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:21 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:24:21.191 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:21 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T00:24:21.191 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:21 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:24:21.191 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:21 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:21.327 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:21 smithi073 ceph-mon[35254]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:21.327 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:21 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:21.327 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:21 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:21.327 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:21 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:21.328 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:21 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:24:21.328 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:21 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T00:24:21.328 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:21 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:24:21.328 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:21 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:21.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:21 smithi122 ceph-mon[35412]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:21.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:21 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:21.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:21 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:21.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:21 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:21.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:21 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:24:21.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:21 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-16T00:24:21.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:21 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:24:21.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:21 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:21.966 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Sep 16 00:24:21 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-z[37222]: 2024-09-16T00:24:21.477+0000 7feaf2f2d040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T00:24:21.967 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Sep 16 00:24:21 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-z[37222]: 2024-09-16T00:24:21.538+0000 7feaf2f2d040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T00:24:21.967 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Sep 16 00:24:21 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-z[37222]: 2024-09-16T00:24:21.721+0000 7feaf2f2d040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T00:24:22.301 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:22 smithi038 ceph-mon[30945]: Deploying daemon mgr.y on smithi073 2024-09-16T00:24:22.305 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:24:22.335 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi038:vg_nvme/lv_4 2024-09-16T00:24:22.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:22 smithi122 ceph-mon[35412]: Deploying daemon mgr.y on smithi073 2024-09-16T00:24:22.496 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:22 smithi073 ceph-mon[35254]: Deploying daemon mgr.y on smithi073 2024-09-16T00:24:22.496 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Sep 16 00:24:22 smithi073 systemd[1]: Starting Ceph mgr.y for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:24:22.590 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:24:22.763 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Sep 16 00:24:22 smithi073 podman[36606]: 2024-09-16 00:24:22.398103593 +0000 UTC m=+0.018948683 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:24:22.763 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Sep 16 00:24:22 smithi073 podman[36606]: 2024-09-16 00:24:22.524411596 +0000 UTC m=+0.145256721 container create 33aa75fcc7b01fe2263720841db8a4ebf0904298dc43ebab53cb0be2ffbdd7bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-y, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-16T00:24:22.763 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Sep 16 00:24:22 smithi073 podman[36606]: 2024-09-16 00:24:22.735303047 +0000 UTC m=+0.356148123 container init 33aa75fcc7b01fe2263720841db8a4ebf0904298dc43ebab53cb0be2ffbdd7bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-y, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-16T00:24:22.763 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Sep 16 00:24:22 smithi073 podman[36606]: 2024-09-16 00:24:22.738116228 +0000 UTC m=+0.358961313 container start 33aa75fcc7b01fe2263720841db8a4ebf0904298dc43ebab53cb0be2ffbdd7bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-y, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-16T00:24:22.763 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Sep 16 00:24:22 smithi073 ceph-mgr[36625]: -- 172.21.15.73:0/3571238112 <== mon.2 v2:172.21.15.73:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5577835330e0 con 0x557783510800 2024-09-16T00:24:22.966 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Sep 16 00:24:22 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-z[37222]: 2024-09-16T00:24:22.594+0000 7feaf2f2d040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T00:24:22.966 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Sep 16 00:24:22 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-z[37222]: 2024-09-16T00:24:22.660+0000 7feaf2f2d040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T00:24:23.044 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Sep 16 00:24:22 smithi073 bash[36606]: 33aa75fcc7b01fe2263720841db8a4ebf0904298dc43ebab53cb0be2ffbdd7bd 2024-09-16T00:24:23.044 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Sep 16 00:24:22 smithi073 systemd[1]: Started Ceph mgr.y for 983b2072-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:24:23.044 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Sep 16 00:24:22 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-y[36621]: 2024-09-16T00:24:22.859+0000 7f00bfaac040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-16T00:24:23.045 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Sep 16 00:24:22 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-y[36621]: 2024-09-16T00:24:22.919+0000 7f00bfaac040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-16T00:24:23.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.099+0000 7ff00dcb1640 1 -- 172.21.15.38:0/3011794382 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0081026a0 msgr2=0x7ff008104a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:23.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.099+0000 7ff00dcb1640 1 --2- 172.21.15.38:0/3011794382 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0081026a0 0x7ff008104a90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7feffc009860 tx=0x7feffc02a340 comp rx=0 tx=0).stop 2024-09-16T00:24:23.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.099+0000 7ff00dcb1640 1 -- 172.21.15.38:0/3011794382 shutdown_connections 2024-09-16T00:24:23.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.099+0000 7ff00dcb1640 1 --2- 172.21.15.38:0/3011794382 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff008107900 0x7ff008109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:23.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.099+0000 7ff00dcb1640 1 --2- 172.21.15.38:0/3011794382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff008104fd0 0x7ff0081073c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:23.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.099+0000 7ff00dcb1640 1 --2- 172.21.15.38:0/3011794382 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0081026a0 0x7ff008104a90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:23.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.099+0000 7ff00dcb1640 1 -- 172.21.15.38:0/3011794382 >> 172.21.15.38:0/3011794382 conn(0x7ff0080fc2a0 msgr2=0x7ff0080fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:23.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.099+0000 7ff00dcb1640 1 -- 172.21.15.38:0/3011794382 shutdown_connections 2024-09-16T00:24:23.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.099+0000 7ff00dcb1640 1 -- 172.21.15.38:0/3011794382 wait complete. 2024-09-16T00:24:23.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.100+0000 7ff00dcb1640 1 Processor -- start 2024-09-16T00:24:23.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.100+0000 7ff00dcb1640 1 -- start start 2024-09-16T00:24:23.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.100+0000 7ff00dcb1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0081026a0 0x7ff008197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:23.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.100+0000 7ff00dcb1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff008104fd0 0x7ff008198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:23.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.100+0000 7ff00dcb1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff008107900 0x7ff00819f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:23.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.100+0000 7ff00dcb1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff00810d8b0 con 0x7ff008107900 2024-09-16T00:24:23.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.100+0000 7ff00dcb1640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff00810d730 con 0x7ff0081026a0 2024-09-16T00:24:23.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.100+0000 7ff00dcb1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff00810da30 con 0x7ff008104fd0 2024-09-16T00:24:23.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.100+0000 7ff007fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff008107900 0x7ff00819f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:23.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.100+0000 7ff006ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff008104fd0 0x7ff008198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:23.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.100+0000 7ff007fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff008107900 0x7ff00819f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34886/0 (socket says 172.21.15.38:34886) 2024-09-16T00:24:23.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.100+0000 7ff007fff640 1 -- 172.21.15.38:0/2281102549 learned_addr learned my addr 172.21.15.38:0/2281102549 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:24:23.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.100+0000 7ff006ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff008104fd0 0x7ff008198470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.38:54142/0 (socket says 172.21.15.38:54142) 2024-09-16T00:24:23.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.100+0000 7ff0077fe640 1 --2- 172.21.15.38:0/2281102549 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0081026a0 0x7ff008197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:23.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.101+0000 7ff007fff640 1 -- 172.21.15.38:0/2281102549 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff008104fd0 msgr2=0x7ff008198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:23.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.101+0000 7ff007fff640 1 --2- 172.21.15.38:0/2281102549 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff008104fd0 0x7ff008198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:23.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.101+0000 7ff007fff640 1 -- 172.21.15.38:0/2281102549 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0081026a0 msgr2=0x7ff008197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:23.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.101+0000 7ff007fff640 1 --2- 172.21.15.38:0/2281102549 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0081026a0 0x7ff008197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:23.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.101+0000 7ff007fff640 1 -- 172.21.15.38:0/2281102549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff00819fb60 con 0x7ff008107900 2024-09-16T00:24:23.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.101+0000 7ff006ffd640 1 --2- 172.21.15.38:0/2281102549 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff008104fd0 0x7ff008198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:24:23.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.101+0000 7ff007fff640 1 --2- 172.21.15.38:0/2281102549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff008107900 0x7ff00819f4f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff000009fd0 tx=0x7ff00000c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:23.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.101+0000 7ff004ff9640 1 -- 172.21.15.38:0/2281102549 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff000004b20 con 0x7ff008107900 2024-09-16T00:24:23.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.101+0000 7ff004ff9640 1 -- 172.21.15.38:0/2281102549 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff000003e50 con 0x7ff008107900 2024-09-16T00:24:23.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.101+0000 7ff00dcb1640 1 -- 172.21.15.38:0/2281102549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff00819fe50 con 0x7ff008107900 2024-09-16T00:24:23.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.101+0000 7ff004ff9640 1 -- 172.21.15.38:0/2281102549 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00000b630 con 0x7ff008107900 2024-09-16T00:24:23.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.101+0000 7ff00dcb1640 1 -- 172.21.15.38:0/2281102549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0081a02b0 con 0x7ff008107900 2024-09-16T00:24:23.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.102+0000 7ff004ff9640 1 -- 172.21.15.38:0/2281102549 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 13) ==== 46517+0+0 (secure 0 0 0) 0x7ff00001d020 con 0x7ff008107900 2024-09-16T00:24:23.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.102+0000 7ff004ff9640 1 --2- 172.21.15.38:0/2281102549 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7feff0039500 0x7feff003b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:23.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.102+0000 7ff00dcb1640 1 -- 172.21.15.38:0/2281102549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0080fdd90 con 0x7ff008107900 2024-09-16T00:24:23.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.102+0000 7ff004ff9640 1 -- 172.21.15.38:0/2281102549 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff00004c5f0 con 0x7ff008107900 2024-09-16T00:24:23.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.102+0000 7ff0077fe640 1 --2- 172.21.15.38:0/2281102549 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7feff0039500 0x7feff003b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:23.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.103+0000 7ff0077fe640 1 --2- 172.21.15.38:0/2281102549 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7feff0039500 0x7feff003b9c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7feffc02a930 tx=0x7feffc030570 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:23.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.107+0000 7ff004ff9640 1 -- 172.21.15.38:0/2281102549 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff00001d310 con 0x7ff008107900 2024-09-16T00:24:23.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:23.292+0000 7ff00dcb1640 1 -- 172.21.15.38:0/2281102549 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ff008062630 con 0x7feff0039500 2024-09-16T00:24:23.293 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:23 smithi038 ceph-mon[30945]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:23.293 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:23 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:23.293 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:23 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:23.293 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:23 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:23.293 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:23 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:23.293 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:23 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:24:23.454 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:23 smithi073 ceph-mon[35254]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:23.455 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:23 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:23.455 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:23 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:23.455 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:23 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:23.455 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:23 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:23.455 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:23 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:24:23.455 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Sep 16 00:24:23 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-y[36621]: 2024-09-16T00:24:23.074+0000 7f00bfaac040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-09-16T00:24:23.455 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Sep 16 00:24:23 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-y[36621]: 2024-09-16T00:24:23.144+0000 7f00bfaac040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-16T00:24:23.455 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Sep 16 00:24:23 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-y[36621]: 2024-09-16T00:24:23.299+0000 7f00bfaac040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:24:23.455 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Sep 16 00:24:23 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-y[36621]: 2024-09-16T00:24:23.454+0000 7f00bfaac040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-16T00:24:23.468 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Sep 16 00:24:23 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-z[37222]: 2024-09-16T00:24:23.412+0000 7feaf2f2d040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T00:24:23.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:23 smithi122 ceph-mon[35412]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:23.469 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:23 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:23.469 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:23 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:23.469 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:23 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:23.469 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:23 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:23.469 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:23 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:24:23.846 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Sep 16 00:24:23 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-z[37222]: 2024-09-16T00:24:23.532+0000 7feaf2f2d040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T00:24:23.846 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Sep 16 00:24:23 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-z[37222]: 2024-09-16T00:24:23.680+0000 7feaf2f2d040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:24:23.846 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Sep 16 00:24:23 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-z[37222]: 2024-09-16T00:24:23.746+0000 7feaf2f2d040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T00:24:24.037 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Sep 16 00:24:23 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-y[36621]: 2024-09-16T00:24:23.779+0000 7f00bfaac040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-16T00:24:24.037 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Sep 16 00:24:23 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-y[36621]: 2024-09-16T00:24:23.840+0000 7f00bfaac040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-16T00:24:24.184 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Sep 16 00:24:23 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-z[37222]: 2024-09-16T00:24:23.898+0000 7feaf2f2d040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T00:24:24.184 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Sep 16 00:24:23 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-z[37222]: 2024-09-16T00:24:23.967+0000 7feaf2f2d040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T00:24:24.184 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Sep 16 00:24:24 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-z[37222]: 2024-09-16T00:24:24.027+0000 7feaf2f2d040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T00:24:24.294 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Sep 16 00:24:24 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-y[36621]: 2024-09-16T00:24:24.036+0000 7f00bfaac040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-16T00:24:24.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:24 smithi073 ceph-mon[35254]: from='client.14232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:24:24.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:24 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:24:24.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:24 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:24:24.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:24 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:24.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:24 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:24.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:24 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:24.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:24 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:24.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:24 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:24.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:24 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:24.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:24 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:24:24.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:24 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:24.387 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:24 smithi038 ceph-mon[30945]: from='client.14232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:24:24.387 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:24 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:24:24.387 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:24 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:24:24.387 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:24 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:24.387 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:24 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:24.388 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:24 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:24.388 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:24 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:24.388 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:24 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:24.388 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:24 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:24.388 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:24 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:24:24.388 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:24 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:24.466 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Sep 16 00:24:24 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-z[37222]: 2024-09-16T00:24:24.196+0000 7feaf2f2d040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T00:24:24.467 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Sep 16 00:24:24 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-z[37222]: 2024-09-16T00:24:24.264+0000 7feaf2f2d040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T00:24:24.467 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Sep 16 00:24:24 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-z[37222]: 2024-09-16T00:24:24.330+0000 7feaf2f2d040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T00:24:24.467 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Sep 16 00:24:24 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-z[37222]: 2024-09-16T00:24:24.411+0000 7feaf2f2d040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T00:24:24.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:24 smithi122 ceph-mon[35412]: from='client.14232 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:24:24.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:24 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:24:24.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:24 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:24:24.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:24 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:24.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:24 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:24.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:24 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:24.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:24 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:24.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:24 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:24.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:24 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:24.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:24 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:24:24.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:24 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:24.966 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Sep 16 00:24:24 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-z[37222]: 2024-09-16T00:24:24.469+0000 7feaf2f2d040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T00:24:24.967 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Sep 16 00:24:24 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-z[37222]: 2024-09-16T00:24:24.529+0000 7feaf2f2d040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T00:24:25.185 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Sep 16 00:24:24 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-y[36621]: 2024-09-16T00:24:24.920+0000 7f00bfaac040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-16T00:24:25.192 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Sep 16 00:24:24 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-y[36621]: 2024-09-16T00:24:24.987+0000 7f00bfaac040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-16T00:24:25.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:25 smithi038 ceph-mon[30945]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:25.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:25 smithi038 ceph-mon[30945]: Reconfiguring mgr.x (unknown last config time)... 2024-09-16T00:24:25.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:25 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:24:25.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:25 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:24:25.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:25 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:25.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:25 smithi038 ceph-mon[30945]: Reconfiguring daemon mgr.x on smithi038 2024-09-16T00:24:25.467 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Sep 16 00:24:25 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-z[37222]: 2024-09-16T00:24:25.314+0000 7feaf2f2d040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T00:24:25.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:25 smithi122 ceph-mon[35412]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:25.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:25 smithi122 ceph-mon[35412]: Reconfiguring mgr.x (unknown last config time)... 2024-09-16T00:24:25.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:25 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:24:25.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:25 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:24:25.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:25 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:25.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:25 smithi122 ceph-mon[35412]: Reconfiguring daemon mgr.x on smithi038 2024-09-16T00:24:25.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:25 smithi073 ceph-mon[35254]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:25.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:25 smithi073 ceph-mon[35254]: Reconfiguring mgr.x (unknown last config time)... 2024-09-16T00:24:25.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:25 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-16T00:24:25.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:25 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-16T00:24:25.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:25 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:25.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:25 smithi073 ceph-mon[35254]: Reconfiguring daemon mgr.x on smithi038 2024-09-16T00:24:25.965 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Sep 16 00:24:25 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-y[36621]: 2024-09-16T00:24:25.706+0000 7f00bfaac040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-16T00:24:25.965 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Sep 16 00:24:25 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-y[36621]: 2024-09-16T00:24:25.822+0000 7f00bfaac040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-16T00:24:25.965 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Sep 16 00:24:25 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-y[36621]: 2024-09-16T00:24:25.964+0000 7f00bfaac040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-16T00:24:26.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:26.193+0000 7ff004ff9640 1 -- 172.21.15.38:0/2281102549 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mgrmap(e 14) ==== 92408+0+0 (secure 0 0 0) 0x7ff000005030 con 0x7ff008107900 2024-09-16T00:24:26.246 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Sep 16 00:24:26 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-y[36621]: 2024-09-16T00:24:26.027+0000 7f00bfaac040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-16T00:24:26.247 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Sep 16 00:24:26 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-y[36621]: 2024-09-16T00:24:26.176+0000 7f00bfaac040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-16T00:24:26.247 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Sep 16 00:24:26 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-y[36621]: 2024-09-16T00:24:26.246+0000 7f00bfaac040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-16T00:24:26.247 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:26 smithi073 ceph-mon[35254]: Standby manager daemon z started 2024-09-16T00:24:26.247 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:26 smithi073 ceph-mon[35254]: from='mgr.? 172.21.15.122:0/1538347124' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-16T00:24:26.247 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:26 smithi073 ceph-mon[35254]: from='mgr.? 172.21.15.122:0/1538347124' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:24:26.247 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:26 smithi073 ceph-mon[35254]: from='mgr.? 172.21.15.122:0/1538347124' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-16T00:24:26.247 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:26 smithi073 ceph-mon[35254]: from='mgr.? 172.21.15.122:0/1538347124' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:24:26.247 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:26 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/3044643110' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3afc20e3-869f-43a9-a911-b0e8d07de86f"}]: dispatch 2024-09-16T00:24:26.247 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:26 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/3044643110' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3afc20e3-869f-43a9-a911-b0e8d07de86f"}]': finished 2024-09-16T00:24:26.247 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:26 smithi073 ceph-mon[35254]: osdmap e5: 1 total, 0 up, 1 in 2024-09-16T00:24:26.247 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:26 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:24:26.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:26 smithi038 ceph-mon[30945]: Standby manager daemon z started 2024-09-16T00:24:26.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:26 smithi038 ceph-mon[30945]: from='mgr.? 172.21.15.122:0/1538347124' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-16T00:24:26.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:26 smithi038 ceph-mon[30945]: from='mgr.? 172.21.15.122:0/1538347124' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:24:26.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:26 smithi038 ceph-mon[30945]: from='mgr.? 172.21.15.122:0/1538347124' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-16T00:24:26.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:26 smithi038 ceph-mon[30945]: from='mgr.? 172.21.15.122:0/1538347124' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:24:26.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:26 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/3044643110' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3afc20e3-869f-43a9-a911-b0e8d07de86f"}]: dispatch 2024-09-16T00:24:26.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:26 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/3044643110' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3afc20e3-869f-43a9-a911-b0e8d07de86f"}]': finished 2024-09-16T00:24:26.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:26 smithi038 ceph-mon[30945]: osdmap e5: 1 total, 0 up, 1 in 2024-09-16T00:24:26.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:26 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:24:26.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:26 smithi122 ceph-mon[35412]: Standby manager daemon z started 2024-09-16T00:24:26.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:26 smithi122 ceph-mon[35412]: from='mgr.? 172.21.15.122:0/1538347124' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-09-16T00:24:26.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:26 smithi122 ceph-mon[35412]: from='mgr.? 172.21.15.122:0/1538347124' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:24:26.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:26 smithi122 ceph-mon[35412]: from='mgr.? 172.21.15.122:0/1538347124' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-09-16T00:24:26.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:26 smithi122 ceph-mon[35412]: from='mgr.? 172.21.15.122:0/1538347124' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:24:26.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:26 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/3044643110' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3afc20e3-869f-43a9-a911-b0e8d07de86f"}]: dispatch 2024-09-16T00:24:26.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:26 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/3044643110' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3afc20e3-869f-43a9-a911-b0e8d07de86f"}]': finished 2024-09-16T00:24:26.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:26 smithi122 ceph-mon[35412]: osdmap e5: 1 total, 0 up, 1 in 2024-09-16T00:24:26.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:26 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:24:26.530 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Sep 16 00:24:26 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-y[36621]: 2024-09-16T00:24:26.306+0000 7f00bfaac040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-16T00:24:26.530 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Sep 16 00:24:26 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-y[36621]: 2024-09-16T00:24:26.465+0000 7f00bfaac040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-16T00:24:26.531 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Sep 16 00:24:26 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-y[36621]: 2024-09-16T00:24:26.530+0000 7f00bfaac040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-16T00:24:26.794 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Sep 16 00:24:26 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-y[36621]: 2024-09-16T00:24:26.596+0000 7f00bfaac040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-16T00:24:26.794 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Sep 16 00:24:26 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-y[36621]: 2024-09-16T00:24:26.677+0000 7f00bfaac040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-16T00:24:26.794 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Sep 16 00:24:26 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-y[36621]: 2024-09-16T00:24:26.736+0000 7f00bfaac040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-16T00:24:27.192 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Sep 16 00:24:26 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-y[36621]: 2024-09-16T00:24:26.795+0000 7f00bfaac040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-16T00:24:27.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:27 smithi038 ceph-mon[30945]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:27.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:27 smithi038 ceph-mon[30945]: mgrmap e14: x(active, since 64s), standbys: z 2024-09-16T00:24:27.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:27 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-16T00:24:27.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:27 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/1362849220' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:24:27.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:27 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:27.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:27 smithi122 ceph-mon[35412]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:27.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:27 smithi122 ceph-mon[35412]: mgrmap e14: x(active, since 64s), standbys: z 2024-09-16T00:24:27.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:27 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-16T00:24:27.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:27 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/1362849220' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:24:27.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:27 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:27.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:27 smithi073 ceph-mon[35254]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:27.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:27 smithi073 ceph-mon[35254]: mgrmap e14: x(active, since 64s), standbys: z 2024-09-16T00:24:27.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:27 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-09-16T00:24:27.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:27 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/1362849220' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:24:27.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:27 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:28.044 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Sep 16 00:24:27 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-y[36621]: 2024-09-16T00:24:27.583+0000 7f00bfaac040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-16T00:24:28.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:28.206+0000 7ff004ff9640 1 -- 172.21.15.38:0/2281102549 <== mon.0 v2:172.21.15.38:3300/0 8 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7ff000015cf0 con 0x7ff008107900 2024-09-16T00:24:28.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:28 smithi038 ceph-mon[30945]: Standby manager daemon y started 2024-09-16T00:24:28.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:28 smithi038 ceph-mon[30945]: from='mgr.? 172.21.15.73:0/1448408258' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-16T00:24:28.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:28 smithi038 ceph-mon[30945]: from='mgr.? 172.21.15.73:0/1448408258' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:24:28.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:28 smithi038 ceph-mon[30945]: from='mgr.? 172.21.15.73:0/1448408258' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-16T00:24:28.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:28 smithi038 ceph-mon[30945]: from='mgr.? 172.21.15.73:0/1448408258' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:24:28.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:28 smithi122 ceph-mon[35412]: Standby manager daemon y started 2024-09-16T00:24:28.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:28 smithi122 ceph-mon[35412]: from='mgr.? 172.21.15.73:0/1448408258' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-16T00:24:28.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:28 smithi122 ceph-mon[35412]: from='mgr.? 172.21.15.73:0/1448408258' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:24:28.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:28 smithi122 ceph-mon[35412]: from='mgr.? 172.21.15.73:0/1448408258' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-16T00:24:28.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:28 smithi122 ceph-mon[35412]: from='mgr.? 172.21.15.73:0/1448408258' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:24:28.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:28 smithi073 ceph-mon[35254]: Standby manager daemon y started 2024-09-16T00:24:28.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:28 smithi073 ceph-mon[35254]: from='mgr.? 172.21.15.73:0/1448408258' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-09-16T00:24:28.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:28 smithi073 ceph-mon[35254]: from='mgr.? 172.21.15.73:0/1448408258' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-16T00:24:28.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:28 smithi073 ceph-mon[35254]: from='mgr.? 172.21.15.73:0/1448408258' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-09-16T00:24:28.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:28 smithi073 ceph-mon[35254]: from='mgr.? 172.21.15.73:0/1448408258' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-16T00:24:29.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:29 smithi038 ceph-mon[30945]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:29.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:29 smithi038 ceph-mon[30945]: mgrmap e15: x(active, since 66s), standbys: z, y 2024-09-16T00:24:29.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:29 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T00:24:29.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:29 smithi122 ceph-mon[35412]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:29.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:29 smithi122 ceph-mon[35412]: mgrmap e15: x(active, since 66s), standbys: z, y 2024-09-16T00:24:29.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:29 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T00:24:29.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:29 smithi073 ceph-mon[35254]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:29.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:29 smithi073 ceph-mon[35254]: mgrmap e15: x(active, since 66s), standbys: z, y 2024-09-16T00:24:29.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:29 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-16T00:24:31.268 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:31 smithi038 ceph-mon[30945]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:31.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:31 smithi122 ceph-mon[35412]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:31.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:31 smithi073 ceph-mon[35254]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:33.328 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:33 smithi038 ceph-mon[30945]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:33.329 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:33 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:33.329 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:33 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:33.329 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:33 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:24:33.329 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:33 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:33.329 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:33 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:24:33.329 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:33 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:33.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:33 smithi122 ceph-mon[35412]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:33.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:33 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:33.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:33 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:33.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:33 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:24:33.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:33 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:33.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:33 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:24:33.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:33 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:33.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:33 smithi073 ceph-mon[35254]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:33.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:33 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:33.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:33 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:33.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:33 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:24:33.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:33 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:33.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:33 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:24:33.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:33 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:35.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:35 smithi038 ceph-mon[30945]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:35.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:35 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T00:24:35.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:35 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:35.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:35 smithi122 ceph-mon[35412]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:35.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:35 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T00:24:35.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:35 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:35.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:35 smithi073 ceph-mon[35254]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:35.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:35 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-16T00:24:35.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:35 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:36.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:36 smithi038 ceph-mon[30945]: Deploying daemon osd.0 on smithi038 2024-09-16T00:24:36.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:36 smithi122 ceph-mon[35412]: Deploying daemon osd.0 on smithi038 2024-09-16T00:24:36.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:36 smithi073 ceph-mon[35254]: Deploying daemon osd.0 on smithi038 2024-09-16T00:24:37.455 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:37 smithi038 ceph-mon[30945]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:37.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:37 smithi122 ceph-mon[35412]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:37.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:37 smithi073 ceph-mon[35254]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:39.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:39 smithi038 ceph-mon[30945]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:39.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:39 smithi122 ceph-mon[35412]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:39.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:39 smithi073 ceph-mon[35254]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:41.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:41 smithi038 ceph-mon[30945]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:41.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:41 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:24:41.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:41 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:41.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:41 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:41.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:41 smithi122 ceph-mon[35412]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:41.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:41 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:24:41.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:41 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:41.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:41 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:41.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:41 smithi073 ceph-mon[35254]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:41.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:41 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:24:41.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:41 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:41.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:41 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:43.450 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:43 smithi038 ceph-mon[30945]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:43.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:43 smithi122 ceph-mon[35412]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:43.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:43 smithi073 ceph-mon[35254]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:44.938 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:44 smithi038 ceph-mon[30945]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:44.938 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:44 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:44.938 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:44 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:44.938 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:44 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:44.938 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:44 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:24:44.938 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:44 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:44.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:44 smithi122 ceph-mon[35412]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:44.967 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:44 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:44.967 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:44 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:44.967 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:44 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:44.967 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:44 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:24:44.967 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:44 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:45.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:44 smithi073 ceph-mon[35254]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:45.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:44 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:45.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:44 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:45.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:44 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:45.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:44 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:24:45.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:44 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:45.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:45.225+0000 7ff004ff9640 1 -- 172.21.15.38:0/2281102549 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff008062630 con 0x7feff0039500 2024-09-16T00:24:45.226 INFO:teuthology.orchestra.run.smithi038.stdout:Created osd(s) 0 on host 'smithi038' 2024-09-16T00:24:45.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:45.227+0000 7ff00dcb1640 1 -- 172.21.15.38:0/2281102549 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7feff0039500 msgr2=0x7feff003b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:45.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:45.227+0000 7ff00dcb1640 1 --2- 172.21.15.38:0/2281102549 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7feff0039500 0x7feff003b9c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7feffc02a930 tx=0x7feffc030570 comp rx=0 tx=0).stop 2024-09-16T00:24:45.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:45.227+0000 7ff00dcb1640 1 -- 172.21.15.38:0/2281102549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff008107900 msgr2=0x7ff00819f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:45.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:45.227+0000 7ff00dcb1640 1 --2- 172.21.15.38:0/2281102549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff008107900 0x7ff00819f4f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff000009fd0 tx=0x7ff00000c800 comp rx=0 tx=0).stop 2024-09-16T00:24:45.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:45.227+0000 7ff00dcb1640 1 -- 172.21.15.38:0/2281102549 shutdown_connections 2024-09-16T00:24:45.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:45.227+0000 7ff00dcb1640 1 --2- 172.21.15.38:0/2281102549 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7feff0039500 0x7feff003b9c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:45.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:45.227+0000 7ff00dcb1640 1 --2- 172.21.15.38:0/2281102549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff008107900 0x7ff00819f4f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:45.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:45.227+0000 7ff00dcb1640 1 --2- 172.21.15.38:0/2281102549 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff008104fd0 0x7ff008198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:45.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:45.227+0000 7ff00dcb1640 1 --2- 172.21.15.38:0/2281102549 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0081026a0 0x7ff008197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:45.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:45.227+0000 7ff00dcb1640 1 -- 172.21.15.38:0/2281102549 >> 172.21.15.38:0/2281102549 conn(0x7ff0080fc2a0 msgr2=0x7ff008103100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:45.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:45.228+0000 7ff00dcb1640 1 -- 172.21.15.38:0/2281102549 shutdown_connections 2024-09-16T00:24:45.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:24:45.228+0000 7ff00dcb1640 1 -- 172.21.15.38:0/2281102549 wait complete. 2024-09-16T00:24:45.842 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:45 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:24:45.842 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:45 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:45.843 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:45 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:45.870 DEBUG:teuthology.orchestra.run.smithi038:osd.0> sudo journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@osd.0.service 2024-09-16T00:24:45.873 INFO:tasks.cephadm:Deploying osd.1 on smithi073 with /dev/vg_nvme/lv_4... 2024-09-16T00:24:45.873 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-16T00:24:45.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:45 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:24:45.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:45 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:45.896 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:45 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:45.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:45 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:24:45.967 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:45 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:45.967 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:45 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:46.110 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:24:46.465 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 16 00:24:46 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-0[45103]: 2024-09-16T00:24:46.356+0000 7f0f4df90640 -1 osd.0 0 log_to_monitors true 2024-09-16T00:24:47.337 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:47 smithi073 ceph-mon[35254]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:47.337 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:47 smithi073 ceph-mon[35254]: from='osd.0 [v2:172.21.15.38:6802/1622028921,v1:172.21.15.38:6803/1622028921]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T00:24:47.380 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:47 smithi038 ceph-mon[30945]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:47.381 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:47 smithi038 ceph-mon[30945]: from='osd.0 [v2:172.21.15.38:6802/1622028921,v1:172.21.15.38:6803/1622028921]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T00:24:47.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:47 smithi122 ceph-mon[35412]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:47.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:47 smithi122 ceph-mon[35412]: from='osd.0 [v2:172.21.15.38:6802/1622028921,v1:172.21.15.38:6803/1622028921]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-16T00:24:48.370 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-16T00:24:48.395 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi073:vg_nvme/lv_4 2024-09-16T00:24:48.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:48 smithi038 ceph-mon[30945]: from='osd.0 [v2:172.21.15.38:6802/1622028921,v1:172.21.15.38:6803/1622028921]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T00:24:48.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:48 smithi038 ceph-mon[30945]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T00:24:48.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:48 smithi038 ceph-mon[30945]: from='osd.0 [v2:172.21.15.38:6802/1622028921,v1:172.21.15.38:6803/1622028921]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-16T00:24:48.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:48 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:24:48.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:48 smithi073 ceph-mon[35254]: from='osd.0 [v2:172.21.15.38:6802/1622028921,v1:172.21.15.38:6803/1622028921]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T00:24:48.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:48 smithi073 ceph-mon[35254]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T00:24:48.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:48 smithi073 ceph-mon[35254]: from='osd.0 [v2:172.21.15.38:6802/1622028921,v1:172.21.15.38:6803/1622028921]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-16T00:24:48.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:48 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:24:48.681 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:24:48.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:48 smithi122 ceph-mon[35412]: from='osd.0 [v2:172.21.15.38:6802/1622028921,v1:172.21.15.38:6803/1622028921]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-16T00:24:48.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:48 smithi122 ceph-mon[35412]: osdmap e6: 1 total, 0 up, 1 in 2024-09-16T00:24:48.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:48 smithi122 ceph-mon[35412]: from='osd.0 [v2:172.21.15.38:6802/1622028921,v1:172.21.15.38:6803/1622028921]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-16T00:24:48.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:48 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:24:49.188 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.173+0000 7eff0b322640 1 -- 172.21.15.73:0/372663606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff04101cd0 msgr2=0x7eff04102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:49.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.173+0000 7eff0b322640 1 --2- 172.21.15.73:0/372663606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff04101cd0 0x7eff04102150 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7eff00009900 tx=0x7eff0002a410 comp rx=0 tx=0).stop 2024-09-16T00:24:49.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.173+0000 7eff0b322640 1 -- 172.21.15.73:0/372663606 shutdown_connections 2024-09-16T00:24:49.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.173+0000 7eff0b322640 1 --2- 172.21.15.73:0/372663606 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff04102690 0x7eff04108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:49.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.173+0000 7eff0b322640 1 --2- 172.21.15.73:0/372663606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff04101cd0 0x7eff04102150 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:49.189 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.173+0000 7eff0b322640 1 --2- 172.21.15.73:0/372663606 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff04100ad0 0x7eff04100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:49.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.173+0000 7eff0b322640 1 -- 172.21.15.73:0/372663606 >> 172.21.15.73:0/372663606 conn(0x7eff040fc280 msgr2=0x7eff040fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:24:49.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.173+0000 7eff0b322640 1 -- 172.21.15.73:0/372663606 shutdown_connections 2024-09-16T00:24:49.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.173+0000 7eff0b322640 1 -- 172.21.15.73:0/372663606 wait complete. 2024-09-16T00:24:49.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.173+0000 7eff0b322640 1 Processor -- start 2024-09-16T00:24:49.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.173+0000 7eff0b322640 1 -- start start 2024-09-16T00:24:49.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.174+0000 7eff0b322640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff04100ad0 0x7eff04196de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:49.190 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.174+0000 7eff0b322640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff04101cd0 0x7eff04197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:49.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.174+0000 7eff0b322640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff04102690 0x7eff0419e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:49.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.174+0000 7eff0b322640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7eff0410c730 con 0x7eff04101cd0 2024-09-16T00:24:49.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.174+0000 7eff0b322640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7eff0410c5b0 con 0x7eff04102690 2024-09-16T00:24:49.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.174+0000 7eff0b322640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7eff0410c8b0 con 0x7eff04100ad0 2024-09-16T00:24:49.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.174+0000 7eff088bd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff04101cd0 0x7eff04197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:49.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.174+0000 7eff098bf640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff04102690 0x7eff0419e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:49.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.174+0000 7eff088bd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff04101cd0 0x7eff04197320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.73:36450/0 (socket says 172.21.15.73:36450) 2024-09-16T00:24:49.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.174+0000 7eff088bd640 1 -- 172.21.15.73:0/1081297547 learned_addr learned my addr 172.21.15.73:0/1081297547 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-09-16T00:24:49.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.174+0000 7eff098bf640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff04102690 0x7eff0419e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41202/0 (socket says 172.21.15.73:41202) 2024-09-16T00:24:49.191 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.174+0000 7eff090be640 1 --2- 172.21.15.73:0/1081297547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff04100ad0 0x7eff04196de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:49.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.174+0000 7eff098bf640 1 -- 172.21.15.73:0/1081297547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff04100ad0 msgr2=0x7eff04196de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:49.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.174+0000 7eff098bf640 1 --2- 172.21.15.73:0/1081297547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff04100ad0 0x7eff04196de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:49.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.174+0000 7eff098bf640 1 -- 172.21.15.73:0/1081297547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff04101cd0 msgr2=0x7eff04197320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:24:49.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.174+0000 7eff098bf640 1 --2- 172.21.15.73:0/1081297547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff04101cd0 0x7eff04197320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:24:49.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.174+0000 7eff098bf640 1 -- 172.21.15.73:0/1081297547 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff0419ea10 con 0x7eff04102690 2024-09-16T00:24:49.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.174+0000 7eff088bd640 1 --2- 172.21.15.73:0/1081297547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff04101cd0 0x7eff04197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:24:49.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.174+0000 7eff098bf640 1 --2- 172.21.15.73:0/1081297547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff04102690 0x7eff0419e3a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efefc00ea50 tx=0x7efefc00ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:49.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.174+0000 7efefa7fc640 1 -- 172.21.15.73:0/1081297547 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efefc00e2e0 con 0x7eff04102690 2024-09-16T00:24:49.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.174+0000 7efefa7fc640 1 -- 172.21.15.73:0/1081297547 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efefc007900 con 0x7eff04102690 2024-09-16T00:24:49.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.175+0000 7eff0b322640 1 -- 172.21.15.73:0/1081297547 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff04069f70 con 0x7eff04102690 2024-09-16T00:24:49.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.175+0000 7efefa7fc640 1 -- 172.21.15.73:0/1081297547 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efefc005750 con 0x7eff04102690 2024-09-16T00:24:49.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.175+0000 7eff0b322640 1 -- 172.21.15.73:0/1081297547 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff0406a430 con 0x7eff04102690 2024-09-16T00:24:49.192 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.176+0000 7eff0b322640 1 -- 172.21.15.73:0/1081297547 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff0406ae50 con 0x7eff04102690 2024-09-16T00:24:49.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.177+0000 7efefa7fc640 1 -- 172.21.15.73:0/1081297547 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7efefc007ab0 con 0x7eff04102690 2024-09-16T00:24:49.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.177+0000 7efefa7fc640 1 --2- 172.21.15.73:0/1081297547 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efef00a31e0 0x7efef00a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:24:49.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.177+0000 7efefa7fc640 1 -- 172.21.15.73:0/1081297547 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1487+0+0 (secure 0 0 0) 0x7efefc0cf0e0 con 0x7eff04102690 2024-09-16T00:24:49.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.177+0000 7eff090be640 1 --2- 172.21.15.73:0/1081297547 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efef00a31e0 0x7efef00a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:24:49.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.178+0000 7eff090be640 1 --2- 172.21.15.73:0/1081297547 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efef00a31e0 0x7efef00a56a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7eff04101b30 tx=0x7efef400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:24:49.193 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.186+0000 7efefa7fc640 1 -- 172.21.15.73:0/1081297547 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efefc09d340 con 0x7eff04102690 2024-09-16T00:24:49.215 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 16 00:24:48 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-0[45103]: 2024-09-16T00:24:48.968+0000 7f0f49f13640 -1 osd.0 0 waiting for initial osdmap 2024-09-16T00:24:49.215 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 16 00:24:48 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-0[45103]: 2024-09-16T00:24:48.972+0000 7f0f41d33640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:24:49.366 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:24:49.366+0000 7eff0b322640 1 -- 172.21.15.73:0/1081297547 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7eff04107fa0 con 0x7efef00a31e0 2024-09-16T00:24:49.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:49 smithi073 ceph-mon[35254]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:49.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:49 smithi073 ceph-mon[35254]: from='osd.0 [v2:172.21.15.38:6802/1622028921,v1:172.21.15.38:6803/1622028921]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-16T00:24:49.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:49 smithi073 ceph-mon[35254]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T00:24:49.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:49 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:24:49.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:49 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:24:49.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:49 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:49.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:49 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:49.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:49 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:24:49.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:49 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:49.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:49 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:49.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:49 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:24:49.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:49 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:49.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:49 smithi073 ceph-mon[35254]: from='osd.0 [v2:172.21.15.38:6802/1622028921,v1:172.21.15.38:6803/1622028921]' entity='osd.0' 2024-09-16T00:24:49.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:49 smithi038 ceph-mon[30945]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:49.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:49 smithi038 ceph-mon[30945]: from='osd.0 [v2:172.21.15.38:6802/1622028921,v1:172.21.15.38:6803/1622028921]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-16T00:24:49.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:49 smithi038 ceph-mon[30945]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T00:24:49.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:49 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:24:49.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:49 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:24:49.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:49 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:49.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:49 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:49.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:49 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:24:49.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:49 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:49.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:49 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:49.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:49 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:24:49.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:49 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:49.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:49 smithi038 ceph-mon[30945]: from='osd.0 [v2:172.21.15.38:6802/1622028921,v1:172.21.15.38:6803/1622028921]' entity='osd.0' 2024-09-16T00:24:49.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:49 smithi122 ceph-mon[35412]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:49.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:49 smithi122 ceph-mon[35412]: from='osd.0 [v2:172.21.15.38:6802/1622028921,v1:172.21.15.38:6803/1622028921]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-16T00:24:49.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:49 smithi122 ceph-mon[35412]: osdmap e7: 1 total, 0 up, 1 in 2024-09-16T00:24:49.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:49 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:24:49.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:49 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:24:49.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:49 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:49.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:49 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:49.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:49 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:24:49.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:49 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:49.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:49 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:49.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:49 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:24:49.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:49 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:24:49.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:49 smithi122 ceph-mon[35412]: from='osd.0 [v2:172.21.15.38:6802/1622028921,v1:172.21.15.38:6803/1622028921]' entity='osd.0' 2024-09-16T00:24:50.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:50 smithi073 ceph-mon[35254]: purged_snaps scrub starts 2024-09-16T00:24:50.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:50 smithi073 ceph-mon[35254]: purged_snaps scrub ok 2024-09-16T00:24:50.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:50 smithi073 ceph-mon[35254]: Detected new or changed devices on smithi038 2024-09-16T00:24:50.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:50 smithi073 ceph-mon[35254]: Adjusting osd_memory_target on smithi038 to 17092M 2024-09-16T00:24:50.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:50 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:24:50.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:50 smithi073 ceph-mon[35254]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:24:50.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:50 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:24:50.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:50 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:24:50.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:50 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:50.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:50 smithi073 ceph-mon[35254]: osd.0 [v2:172.21.15.38:6802/1622028921,v1:172.21.15.38:6803/1622028921] boot 2024-09-16T00:24:50.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:50 smithi073 ceph-mon[35254]: osdmap e8: 1 total, 1 up, 1 in 2024-09-16T00:24:50.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:50 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:24:50.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:50 smithi038 ceph-mon[30945]: purged_snaps scrub starts 2024-09-16T00:24:50.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:50 smithi038 ceph-mon[30945]: purged_snaps scrub ok 2024-09-16T00:24:50.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:50 smithi038 ceph-mon[30945]: Detected new or changed devices on smithi038 2024-09-16T00:24:50.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:50 smithi038 ceph-mon[30945]: Adjusting osd_memory_target on smithi038 to 17092M 2024-09-16T00:24:50.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:50 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:24:50.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:50 smithi038 ceph-mon[30945]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:24:50.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:50 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:24:50.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:50 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:24:50.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:50 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:50.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:50 smithi038 ceph-mon[30945]: osd.0 [v2:172.21.15.38:6802/1622028921,v1:172.21.15.38:6803/1622028921] boot 2024-09-16T00:24:50.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:50 smithi038 ceph-mon[30945]: osdmap e8: 1 total, 1 up, 1 in 2024-09-16T00:24:50.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:50 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:24:50.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:50 smithi122 ceph-mon[35412]: purged_snaps scrub starts 2024-09-16T00:24:50.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:50 smithi122 ceph-mon[35412]: purged_snaps scrub ok 2024-09-16T00:24:50.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:50 smithi122 ceph-mon[35412]: Detected new or changed devices on smithi038 2024-09-16T00:24:50.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:50 smithi122 ceph-mon[35412]: Adjusting osd_memory_target on smithi038 to 17092M 2024-09-16T00:24:50.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:50 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:24:50.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:50 smithi122 ceph-mon[35412]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:24:50.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:50 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:24:50.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:50 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:24:50.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:50 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:24:50.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:50 smithi122 ceph-mon[35412]: osd.0 [v2:172.21.15.38:6802/1622028921,v1:172.21.15.38:6803/1622028921] boot 2024-09-16T00:24:50.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:50 smithi122 ceph-mon[35412]: osdmap e8: 1 total, 1 up, 1 in 2024-09-16T00:24:50.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:50 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-16T00:24:51.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:51 smithi073 ceph-mon[35254]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:51.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:51 smithi038 ceph-mon[30945]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:51.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:51 smithi122 ceph-mon[35412]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-16T00:24:52.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:52 smithi073 ceph-mon[35254]: osdmap e9: 1 total, 1 up, 1 in 2024-09-16T00:24:52.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:52 smithi073 ceph-mon[35254]: from='client.? 172.21.15.73:0/1129150589' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d9d7acb-45ed-4543-8afc-6433fc7f096f"}]: dispatch 2024-09-16T00:24:52.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:52 smithi073 ceph-mon[35254]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d9d7acb-45ed-4543-8afc-6433fc7f096f"}]: dispatch 2024-09-16T00:24:52.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:52 smithi073 ceph-mon[35254]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d9d7acb-45ed-4543-8afc-6433fc7f096f"}]': finished 2024-09-16T00:24:52.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:52 smithi073 ceph-mon[35254]: osdmap e10: 2 total, 1 up, 2 in 2024-09-16T00:24:52.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:52 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:24:52.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:52 smithi073 ceph-mon[35254]: from='client.? 172.21.15.73:0/1575552913' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:24:52.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:52 smithi038 ceph-mon[30945]: osdmap e9: 1 total, 1 up, 1 in 2024-09-16T00:24:52.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:52 smithi038 ceph-mon[30945]: from='client.? 172.21.15.73:0/1129150589' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d9d7acb-45ed-4543-8afc-6433fc7f096f"}]: dispatch 2024-09-16T00:24:52.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:52 smithi038 ceph-mon[30945]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d9d7acb-45ed-4543-8afc-6433fc7f096f"}]: dispatch 2024-09-16T00:24:52.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:52 smithi038 ceph-mon[30945]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d9d7acb-45ed-4543-8afc-6433fc7f096f"}]': finished 2024-09-16T00:24:52.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:52 smithi038 ceph-mon[30945]: osdmap e10: 2 total, 1 up, 2 in 2024-09-16T00:24:52.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:52 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:24:52.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:52 smithi038 ceph-mon[30945]: from='client.? 172.21.15.73:0/1575552913' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:24:52.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:52 smithi122 ceph-mon[35412]: osdmap e9: 1 total, 1 up, 1 in 2024-09-16T00:24:52.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:52 smithi122 ceph-mon[35412]: from='client.? 172.21.15.73:0/1129150589' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d9d7acb-45ed-4543-8afc-6433fc7f096f"}]: dispatch 2024-09-16T00:24:52.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:52 smithi122 ceph-mon[35412]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6d9d7acb-45ed-4543-8afc-6433fc7f096f"}]: dispatch 2024-09-16T00:24:52.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:52 smithi122 ceph-mon[35412]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6d9d7acb-45ed-4543-8afc-6433fc7f096f"}]': finished 2024-09-16T00:24:52.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:52 smithi122 ceph-mon[35412]: osdmap e10: 2 total, 1 up, 2 in 2024-09-16T00:24:52.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:52 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:24:52.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:52 smithi122 ceph-mon[35412]: from='client.? 172.21.15.73:0/1575552913' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:24:53.538 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:53 smithi038 ceph-mon[30945]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:24:53.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:53 smithi073 ceph-mon[35254]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:24:53.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:53 smithi122 ceph-mon[35412]: pgmap v44: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:24:55.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:55 smithi073 ceph-mon[35254]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:24:55.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:55 smithi038 ceph-mon[30945]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:24:55.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:55 smithi122 ceph-mon[35412]: pgmap v46: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:24:57.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:57 smithi073 ceph-mon[35254]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:24:57.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:57 smithi038 ceph-mon[30945]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:24:57.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:57 smithi122 ceph-mon[35412]: pgmap v47: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:24:59.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:24:59 smithi073 ceph-mon[35254]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:24:59.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:24:59 smithi038 ceph-mon[30945]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:24:59.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:24:59 smithi122 ceph-mon[35412]: pgmap v48: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:25:00.635 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:00 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T00:25:00.635 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:00 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:00.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:00 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T00:25:00.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:00 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:00.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:00 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-16T00:25:00.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:00 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:01.540 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:01 smithi073 ceph-mon[35254]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:25:01.541 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:01 smithi073 ceph-mon[35254]: Deploying daemon osd.1 on smithi073 2024-09-16T00:25:01.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:01 smithi038 ceph-mon[30945]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:25:01.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:01 smithi038 ceph-mon[30945]: Deploying daemon osd.1 on smithi073 2024-09-16T00:25:01.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:01 smithi122 ceph-mon[35412]: pgmap v49: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:25:01.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:01 smithi122 ceph-mon[35412]: Deploying daemon osd.1 on smithi073 2024-09-16T00:25:03.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:03 smithi038 ceph-mon[30945]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:25:03.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:03 smithi122 ceph-mon[35412]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:25:03.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:03 smithi073 ceph-mon[35254]: pgmap v50: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:25:05.678 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:05 smithi073 ceph-mon[35254]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:25:05.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:05 smithi038 ceph-mon[30945]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:25:05.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:05 smithi122 ceph-mon[35412]: pgmap v51: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:25:06.715 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:06 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:25:06.715 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:06 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:06.715 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:06 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:06.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:06 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:25:06.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:06 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:06.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:06 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:06.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:06 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:25:06.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:06 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:06.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:06 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:07.713 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:07 smithi073 ceph-mon[35254]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:25:07.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:07 smithi038 ceph-mon[30945]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:25:07.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:07 smithi122 ceph-mon[35412]: pgmap v52: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:25:09.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:09 smithi122 ceph-mon[35412]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:25:09.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:09 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:09.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:09 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:09.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:09 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:09.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:09 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:25:09.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:09 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:09.718 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:09 smithi038 ceph-mon[30945]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:25:09.718 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:09 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:09.718 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:09 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:09.718 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:09 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:09.719 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:09 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:25:09.719 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:09 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:09.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:09 smithi073 ceph-mon[35254]: pgmap v53: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:25:09.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:09 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:09.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:09 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:09.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:09 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:09.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:09 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:25:09.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:09 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:10.513 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:25:10.513+0000 7efefa7fc640 1 -- 172.21.15.73:0/1081297547 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7eff04107fa0 con 0x7efef00a31e0 2024-09-16T00:25:10.513 INFO:teuthology.orchestra.run.smithi073.stdout:Created osd(s) 1 on host 'smithi073' 2024-09-16T00:25:10.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:25:10.515+0000 7eff0b322640 1 -- 172.21.15.73:0/1081297547 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efef00a31e0 msgr2=0x7efef00a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:10.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:25:10.515+0000 7eff0b322640 1 --2- 172.21.15.73:0/1081297547 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efef00a31e0 0x7efef00a56a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7eff04101b30 tx=0x7efef400b390 comp rx=0 tx=0).stop 2024-09-16T00:25:10.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:25:10.515+0000 7eff0b322640 1 -- 172.21.15.73:0/1081297547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff04102690 msgr2=0x7eff0419e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:10.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:25:10.515+0000 7eff0b322640 1 --2- 172.21.15.73:0/1081297547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff04102690 0x7eff0419e3a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efefc00ea50 tx=0x7efefc00ef10 comp rx=0 tx=0).stop 2024-09-16T00:25:10.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:25:10.515+0000 7eff0b322640 1 -- 172.21.15.73:0/1081297547 shutdown_connections 2024-09-16T00:25:10.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:25:10.515+0000 7eff0b322640 1 --2- 172.21.15.73:0/1081297547 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efef00a31e0 0x7efef00a56a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:10.516 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:25:10.515+0000 7eff0b322640 1 --2- 172.21.15.73:0/1081297547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff04102690 0x7eff0419e3a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:10.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:25:10.515+0000 7eff0b322640 1 --2- 172.21.15.73:0/1081297547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7eff04101cd0 0x7eff04197320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:10.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:25:10.515+0000 7eff0b322640 1 --2- 172.21.15.73:0/1081297547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7eff04100ad0 0x7eff04196de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:10.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:25:10.515+0000 7eff0b322640 1 -- 172.21.15.73:0/1081297547 >> 172.21.15.73:0/1081297547 conn(0x7eff040fc280 msgr2=0x7eff041050f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:10.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:25:10.516+0000 7eff0b322640 1 -- 172.21.15.73:0/1081297547 shutdown_connections 2024-09-16T00:25:10.517 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:25:10.516+0000 7eff0b322640 1 -- 172.21.15.73:0/1081297547 wait complete. 2024-09-16T00:25:11.145 DEBUG:teuthology.orchestra.run.smithi073:osd.1> sudo journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@osd.1.service 2024-09-16T00:25:11.147 INFO:tasks.cephadm:Deploying osd.2 on smithi122 with /dev/vg_nvme/lv_4... 2024-09-16T00:25:11.148 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_4 2024-09-16T00:25:11.378 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:11 smithi073 ceph-mon[35254]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:25:11.378 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:11 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:25:11.378 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:11 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:11.378 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:11 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:11.411 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:25:11.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:11 smithi122 ceph-mon[35412]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:25:11.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:11 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:25:11.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:11 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:11.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:11 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:11.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:11 smithi038 ceph-mon[30945]: pgmap v54: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:25:11.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:11 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:25:11.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:11 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:11.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:11 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:12.025 INFO:journalctl@ceph.osd.1.smithi073.stdout:Sep 16 00:25:11 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-1[39704]: 2024-09-16T00:25:11.792+0000 7f82f43fa640 -1 osd.1 0 log_to_monitors true 2024-09-16T00:25:12.534 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:12 smithi122 ceph-mon[35412]: from='osd.1 [v2:172.21.15.73:6800/3811690336,v1:172.21.15.73:6801/3811690336]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T00:25:12.534 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:12 smithi122 ceph-mon[35412]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T00:25:12.585 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:12 smithi073 ceph-mon[35254]: from='osd.1 [v2:172.21.15.73:6800/3811690336,v1:172.21.15.73:6801/3811690336]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T00:25:12.585 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:12 smithi073 ceph-mon[35254]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T00:25:12.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:12 smithi038 ceph-mon[30945]: from='osd.1 [v2:172.21.15.73:6800/3811690336,v1:172.21.15.73:6801/3811690336]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T00:25:12.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:12 smithi038 ceph-mon[30945]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-16T00:25:13.455 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:13 smithi122 ceph-mon[35412]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:25:13.455 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:13 smithi122 ceph-mon[35412]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T00:25:13.455 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:13 smithi122 ceph-mon[35412]: from='osd.1 [v2:172.21.15.73:6800/3811690336,v1:172.21.15.73:6801/3811690336]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-09-16T00:25:13.455 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:13 smithi122 ceph-mon[35412]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T00:25:13.455 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:13 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:25:13.455 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:13 smithi122 ceph-mon[35412]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-09-16T00:25:13.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:13 smithi038 ceph-mon[30945]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:25:13.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:13 smithi038 ceph-mon[30945]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T00:25:13.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:13 smithi038 ceph-mon[30945]: from='osd.1 [v2:172.21.15.73:6800/3811690336,v1:172.21.15.73:6801/3811690336]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-09-16T00:25:13.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:13 smithi038 ceph-mon[30945]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T00:25:13.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:13 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:25:13.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:13 smithi038 ceph-mon[30945]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-09-16T00:25:13.783 INFO:journalctl@ceph.osd.1.smithi073.stdout:Sep 16 00:25:13 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-1[39704]: 2024-09-16T00:25:13.349+0000 7f82f037d640 -1 osd.1 0 waiting for initial osdmap 2024-09-16T00:25:13.783 INFO:journalctl@ceph.osd.1.smithi073.stdout:Sep 16 00:25:13 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-1[39704]: 2024-09-16T00:25:13.352+0000 7f82ec1a5640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:25:13.784 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:13 smithi073 ceph-mon[35254]: pgmap v55: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:25:13.784 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:13 smithi073 ceph-mon[35254]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-16T00:25:13.784 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:13 smithi073 ceph-mon[35254]: from='osd.1 [v2:172.21.15.73:6800/3811690336,v1:172.21.15.73:6801/3811690336]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-09-16T00:25:13.784 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:13 smithi073 ceph-mon[35254]: osdmap e11: 2 total, 1 up, 2 in 2024-09-16T00:25:13.784 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:13 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:25:13.784 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:13 smithi073 ceph-mon[35254]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-09-16T00:25:13.977 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-16T00:25:14.002 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi122:vg_nvme/lv_4 2024-09-16T00:25:14.250 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:25:14.619 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:14 smithi122 ceph-mon[35412]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-09-16T00:25:14.619 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:14 smithi122 ceph-mon[35412]: osdmap e12: 2 total, 1 up, 2 in 2024-09-16T00:25:14.619 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:14 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:25:14.619 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:14 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:25:14.619 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:14 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:14.619 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:14 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:14.619 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:14 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:25:14.619 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:14 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:14.619 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:14 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:14.619 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:14 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:25:14.620 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:14 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:14.620 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:14 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:25:14.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:14.701+0000 7f58023da640 1 -- 172.21.15.122:0/2076458732 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57fc101c00 msgr2=0x7f57fc102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:14.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:14.701+0000 7f58023da640 1 --2- 172.21.15.122:0/2076458732 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57fc101c00 0x7f57fc102000 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009900 tx=0x7f57ec02a410 comp rx=0 tx=0).stop 2024-09-16T00:25:14.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:14.701+0000 7f58023da640 1 -- 172.21.15.122:0/2076458732 shutdown_connections 2024-09-16T00:25:14.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:14.701+0000 7f58023da640 1 --2- 172.21.15.122:0/2076458732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57fc1037c0 0x7f57fc10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:14.704 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:14.701+0000 7f58023da640 1 --2- 172.21.15.122:0/2076458732 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57fc102e00 0x7f57fc103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:14.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:14.701+0000 7f58023da640 1 --2- 172.21.15.122:0/2076458732 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57fc101c00 0x7f57fc102000 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:14.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:14.701+0000 7f58023da640 1 -- 172.21.15.122:0/2076458732 >> 172.21.15.122:0/2076458732 conn(0x7f57fc0fd390 msgr2=0x7f57fc0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:14.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:14.701+0000 7f58023da640 1 -- 172.21.15.122:0/2076458732 shutdown_connections 2024-09-16T00:25:14.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:14.701+0000 7f58023da640 1 -- 172.21.15.122:0/2076458732 wait complete. 2024-09-16T00:25:14.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:14.702+0000 7f58023da640 1 Processor -- start 2024-09-16T00:25:14.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:14.702+0000 7f58023da640 1 -- start start 2024-09-16T00:25:14.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:14.702+0000 7f58023da640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57fc101c00 0x7f57fc197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:14.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:14.702+0000 7f58023da640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57fc102e00 0x7f57fc1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:14.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:14.702+0000 7f58023da640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57fc1037c0 0x7f57fc19f440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:14.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:14.702+0000 7f58023da640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f57fc10d8b0 con 0x7f57fc1037c0 2024-09-16T00:25:14.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:14.702+0000 7f58023da640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f57fc10d730 con 0x7f57fc101c00 2024-09-16T00:25:14.705 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:14.703+0000 7f58023da640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f57fc10da30 con 0x7f57fc102e00 2024-09-16T00:25:14.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:14.703+0000 7f57fb7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57fc102e00 0x7f57fc1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:14.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:14.703+0000 7f57fb7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57fc102e00 0x7f57fc1983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:48820/0 (socket says 172.21.15.122:48820) 2024-09-16T00:25:14.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:14.703+0000 7f57fb7fe640 1 -- 172.21.15.122:0/1468703196 learned_addr learned my addr 172.21.15.122:0/1468703196 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-16T00:25:14.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:14.703+0000 7f57fbfff640 1 --2- 172.21.15.122:0/1468703196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57fc101c00 0x7f57fc197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:14.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:14.703+0000 7f5800977640 1 --2- 172.21.15.122:0/1468703196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57fc1037c0 0x7f57fc19f440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:14.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:14.703+0000 7f57fb7fe640 1 -- 172.21.15.122:0/1468703196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57fc101c00 msgr2=0x7f57fc197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:14.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:14.703+0000 7f57fb7fe640 1 --2- 172.21.15.122:0/1468703196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57fc101c00 0x7f57fc197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:14.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:14.703+0000 7f57fb7fe640 1 -- 172.21.15.122:0/1468703196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57fc1037c0 msgr2=0x7f57fc19f440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:14.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:14.703+0000 7f57fb7fe640 1 --2- 172.21.15.122:0/1468703196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57fc1037c0 0x7f57fc19f440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:14.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:14.703+0000 7f57fb7fe640 1 -- 172.21.15.122:0/1468703196 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57fc19fab0 con 0x7f57fc102e00 2024-09-16T00:25:14.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:14.703+0000 7f57fb7fe640 1 --2- 172.21.15.122:0/1468703196 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57fc102e00 0x7f57fc1983c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f57f000c920 tx=0x7f57f000cde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:14.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:14.703+0000 7f57f97fa640 1 -- 172.21.15.122:0/1468703196 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f0007c70 con 0x7f57fc102e00 2024-09-16T00:25:14.706 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:14.703+0000 7f57f97fa640 1 -- 172.21.15.122:0/1468703196 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57f0007e10 con 0x7f57fc102e00 2024-09-16T00:25:14.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:14.703+0000 7f58023da640 1 -- 172.21.15.122:0/1468703196 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57fc19fda0 con 0x7f57fc102e00 2024-09-16T00:25:14.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:14.703+0000 7f57f97fa640 1 -- 172.21.15.122:0/1468703196 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f0018890 con 0x7f57fc102e00 2024-09-16T00:25:14.707 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:14.703+0000 7f58023da640 1 -- 172.21.15.122:0/1468703196 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57fc1a0250 con 0x7f57fc102e00 2024-09-16T00:25:14.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:14.704+0000 7f58023da640 1 -- 172.21.15.122:0/1468703196 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57fc108f30 con 0x7f57fc102e00 2024-09-16T00:25:14.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:14.705+0000 7f57f97fa640 1 -- 172.21.15.122:0/1468703196 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7f57f0018af0 con 0x7f57fc102e00 2024-09-16T00:25:14.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:14.706+0000 7f57f97fa640 1 --2- 172.21.15.122:0/1468703196 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f57e40a31e0 0x7f57e40a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:14.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:14.706+0000 7f57f97fa640 1 -- 172.21.15.122:0/1468703196 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2309+0+0 (secure 0 0 0) 0x7f57f00ce190 con 0x7f57fc102e00 2024-09-16T00:25:14.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:14.706+0000 7f57fbfff640 1 --2- 172.21.15.122:0/1468703196 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f57e40a31e0 0x7f57e40a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:14.708 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:14.707+0000 7f57fbfff640 1 --2- 172.21.15.122:0/1468703196 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f57e40a31e0 0x7f57e40a56a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f57ec02aa00 tx=0x7f57ec0315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:14.710 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:14.710+0000 7f57f97fa640 1 -- 172.21.15.122:0/1468703196 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f57f0014030 con 0x7f57fc102e00 2024-09-16T00:25:14.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:14 smithi038 ceph-mon[30945]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-09-16T00:25:14.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:14 smithi038 ceph-mon[30945]: osdmap e12: 2 total, 1 up, 2 in 2024-09-16T00:25:14.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:14 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:25:14.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:14 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:25:14.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:14 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:14.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:14 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:14.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:14 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:25:14.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:14 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:14.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:14 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:14.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:14 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:25:14.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:14 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:14.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:14 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:25:14.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:14 smithi073 ceph-mon[35254]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-09-16T00:25:14.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:14 smithi073 ceph-mon[35254]: osdmap e12: 2 total, 1 up, 2 in 2024-09-16T00:25:14.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:14 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:25:14.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:14 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:25:14.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:14 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:14.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:14 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:14.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:14 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:25:14.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:14 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:14.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:14 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:14.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:14 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:25:14.795 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:14 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:14.795 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:14 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:25:14.884 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:14.883+0000 7f58023da640 1 -- 172.21.15.122:0/1468703196 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f57fc062630 con 0x7f57e40a31e0 2024-09-16T00:25:15.580 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:15 smithi122 ceph-mon[35412]: purged_snaps scrub starts 2024-09-16T00:25:15.581 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:15 smithi122 ceph-mon[35412]: purged_snaps scrub ok 2024-09-16T00:25:15.581 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:15 smithi122 ceph-mon[35412]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:25:15.581 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:15 smithi122 ceph-mon[35412]: Detected new or changed devices on smithi073 2024-09-16T00:25:15.581 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:15 smithi122 ceph-mon[35412]: Adjusting osd_memory_target on smithi073 to 17088M 2024-09-16T00:25:15.581 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:15 smithi122 ceph-mon[35412]: osd.1 [v2:172.21.15.73:6800/3811690336,v1:172.21.15.73:6801/3811690336] boot 2024-09-16T00:25:15.581 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:15 smithi122 ceph-mon[35412]: osdmap e13: 2 total, 2 up, 2 in 2024-09-16T00:25:15.581 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:15 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:25:15.581 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:15 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:25:15.581 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:15 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:25:15.581 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:15 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:15.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:15 smithi038 ceph-mon[30945]: purged_snaps scrub starts 2024-09-16T00:25:15.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:15 smithi038 ceph-mon[30945]: purged_snaps scrub ok 2024-09-16T00:25:15.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:15 smithi038 ceph-mon[30945]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:25:15.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:15 smithi038 ceph-mon[30945]: Detected new or changed devices on smithi073 2024-09-16T00:25:15.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:15 smithi038 ceph-mon[30945]: Adjusting osd_memory_target on smithi073 to 17088M 2024-09-16T00:25:15.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:15 smithi038 ceph-mon[30945]: osd.1 [v2:172.21.15.73:6800/3811690336,v1:172.21.15.73:6801/3811690336] boot 2024-09-16T00:25:15.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:15 smithi038 ceph-mon[30945]: osdmap e13: 2 total, 2 up, 2 in 2024-09-16T00:25:15.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:15 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:25:15.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:15 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:25:15.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:15 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:25:15.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:15 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:15.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:15 smithi073 ceph-mon[35254]: purged_snaps scrub starts 2024-09-16T00:25:15.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:15 smithi073 ceph-mon[35254]: purged_snaps scrub ok 2024-09-16T00:25:15.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:15 smithi073 ceph-mon[35254]: pgmap v58: 0 pgs: ; 0 B data, 31 MiB used, 89 GiB / 89 GiB avail 2024-09-16T00:25:15.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:15 smithi073 ceph-mon[35254]: Detected new or changed devices on smithi073 2024-09-16T00:25:15.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:15 smithi073 ceph-mon[35254]: Adjusting osd_memory_target on smithi073 to 17088M 2024-09-16T00:25:15.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:15 smithi073 ceph-mon[35254]: osd.1 [v2:172.21.15.73:6800/3811690336,v1:172.21.15.73:6801/3811690336] boot 2024-09-16T00:25:15.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:15 smithi073 ceph-mon[35254]: osdmap e13: 2 total, 2 up, 2 in 2024-09-16T00:25:15.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:15 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-16T00:25:15.795 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:15 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:25:15.795 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:15 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:25:15.795 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:15 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:16.453 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:16 smithi122 ceph-mon[35412]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:25:16.454 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:16 smithi122 ceph-mon[35412]: osdmap e14: 2 total, 2 up, 2 in 2024-09-16T00:25:16.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:16 smithi038 ceph-mon[30945]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:25:16.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:16 smithi038 ceph-mon[30945]: osdmap e14: 2 total, 2 up, 2 in 2024-09-16T00:25:16.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:16 smithi073 ceph-mon[35254]: from='client.24160 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:25:16.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:16 smithi073 ceph-mon[35254]: osdmap e14: 2 total, 2 up, 2 in 2024-09-16T00:25:17.669 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:17 smithi122 ceph-mon[35412]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:25:17.669 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:17 smithi122 ceph-mon[35412]: from='client.? 172.21.15.122:0/631901866' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48690f35-99e9-4f45-9bee-487857caa8b8"}]: dispatch 2024-09-16T00:25:17.669 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:17 smithi122 ceph-mon[35412]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48690f35-99e9-4f45-9bee-487857caa8b8"}]: dispatch 2024-09-16T00:25:17.669 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:17 smithi122 ceph-mon[35412]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "48690f35-99e9-4f45-9bee-487857caa8b8"}]': finished 2024-09-16T00:25:17.669 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:17 smithi122 ceph-mon[35412]: osdmap e15: 3 total, 2 up, 3 in 2024-09-16T00:25:17.669 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:17 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:25:17.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:17 smithi038 ceph-mon[30945]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:25:17.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:17 smithi038 ceph-mon[30945]: from='client.? 172.21.15.122:0/631901866' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48690f35-99e9-4f45-9bee-487857caa8b8"}]: dispatch 2024-09-16T00:25:17.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:17 smithi038 ceph-mon[30945]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48690f35-99e9-4f45-9bee-487857caa8b8"}]: dispatch 2024-09-16T00:25:17.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:17 smithi038 ceph-mon[30945]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "48690f35-99e9-4f45-9bee-487857caa8b8"}]': finished 2024-09-16T00:25:17.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:17 smithi038 ceph-mon[30945]: osdmap e15: 3 total, 2 up, 3 in 2024-09-16T00:25:17.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:17 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:25:17.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:17 smithi073 ceph-mon[35254]: pgmap v61: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:25:17.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:17 smithi073 ceph-mon[35254]: from='client.? 172.21.15.122:0/631901866' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48690f35-99e9-4f45-9bee-487857caa8b8"}]: dispatch 2024-09-16T00:25:17.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:17 smithi073 ceph-mon[35254]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "48690f35-99e9-4f45-9bee-487857caa8b8"}]: dispatch 2024-09-16T00:25:17.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:17 smithi073 ceph-mon[35254]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "48690f35-99e9-4f45-9bee-487857caa8b8"}]': finished 2024-09-16T00:25:17.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:17 smithi073 ceph-mon[35254]: osdmap e15: 3 total, 2 up, 3 in 2024-09-16T00:25:17.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:17 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:25:18.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:18 smithi038 ceph-mon[30945]: from='client.? 172.21.15.122:0/3251993960' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:25:18.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:18 smithi122 ceph-mon[35412]: from='client.? 172.21.15.122:0/3251993960' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:25:18.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:18 smithi073 ceph-mon[35254]: from='client.? 172.21.15.122:0/3251993960' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:25:19.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:19 smithi038 ceph-mon[30945]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:25:19.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:19 smithi122 ceph-mon[35412]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:25:19.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:19 smithi073 ceph-mon[35254]: pgmap v63: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:25:20.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:20 smithi038 ceph-mon[30945]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:25:20.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:20 smithi122 ceph-mon[35412]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:25:20.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:20 smithi073 ceph-mon[35254]: pgmap v64: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:25:22.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:22 smithi073 ceph-mon[35254]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:25:22.825 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:22 smithi122 ceph-mon[35412]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:25:22.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:22 smithi038 ceph-mon[30945]: pgmap v65: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:25:24.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:24 smithi122 ceph-mon[35412]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:25:24.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:24 smithi073 ceph-mon[35254]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:25:24.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:24 smithi038 ceph-mon[30945]: pgmap v66: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:25:25.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:25 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T00:25:25.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:25 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:25.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:25 smithi122 ceph-mon[35412]: Deploying daemon osd.2 on smithi122 2024-09-16T00:25:25.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:25 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T00:25:25.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:25 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:25.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:25 smithi073 ceph-mon[35254]: Deploying daemon osd.2 on smithi122 2024-09-16T00:25:25.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:25 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-16T00:25:25.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:25 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:25.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:25 smithi038 ceph-mon[30945]: Deploying daemon osd.2 on smithi122 2024-09-16T00:25:26.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:26 smithi073 ceph-mon[35254]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:25:26.802 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:26 smithi122 ceph-mon[35412]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:25:26.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:26 smithi038 ceph-mon[30945]: pgmap v67: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:25:28.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:28 smithi122 ceph-mon[35412]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:25:28.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:28 smithi073 ceph-mon[35254]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:25:28.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:28 smithi038 ceph-mon[30945]: pgmap v68: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:25:30.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:30 smithi073 ceph-mon[35254]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:25:30.853 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:30 smithi122 ceph-mon[35412]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:25:30.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:30 smithi038 ceph-mon[30945]: pgmap v69: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:25:32.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:32 smithi073 ceph-mon[35254]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:25:32.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:32 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:25:32.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:32 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:32.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:32 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:32.915 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:32 smithi122 ceph-mon[35412]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:25:32.915 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:32 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:25:32.915 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:32 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:32.915 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:32 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:32.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:32 smithi038 ceph-mon[30945]: pgmap v70: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:25:32.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:32 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:25:32.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:32 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:32.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:32 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:35.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:35 smithi038 ceph-mon[30945]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:25:35.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:35 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:35.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:35 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:35.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:35 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:35.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:35 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:25:35.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:35 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:35.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:35 smithi122 ceph-mon[35412]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:25:35.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:35 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:35.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:35 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:35.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:35 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:35.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:35 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:25:35.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:35 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:35.497 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:35 smithi073 ceph-mon[35254]: pgmap v71: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:25:35.497 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:35 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:35.497 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:35 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:35.497 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:35 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:35.497 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:35 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:25:35.498 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:35 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:35.923 INFO:teuthology.orchestra.run.smithi122.stdout:Created osd(s) 2 on host 'smithi122' 2024-09-16T00:25:35.924 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:35.923+0000 7f57f97fa640 1 -- 172.21.15.122:0/1468703196 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f57fc062630 con 0x7f57e40a31e0 2024-09-16T00:25:35.925 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:35.925+0000 7f58023da640 1 -- 172.21.15.122:0/1468703196 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f57e40a31e0 msgr2=0x7f57e40a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:35.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:35.925+0000 7f58023da640 1 --2- 172.21.15.122:0/1468703196 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f57e40a31e0 0x7f57e40a56a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f57ec02aa00 tx=0x7f57ec0315b0 comp rx=0 tx=0).stop 2024-09-16T00:25:35.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:35.925+0000 7f58023da640 1 -- 172.21.15.122:0/1468703196 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57fc102e00 msgr2=0x7f57fc1983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:35.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:35.925+0000 7f58023da640 1 --2- 172.21.15.122:0/1468703196 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57fc102e00 0x7f57fc1983c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f57f000c920 tx=0x7f57f000cde0 comp rx=0 tx=0).stop 2024-09-16T00:25:35.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:35.925+0000 7f58023da640 1 -- 172.21.15.122:0/1468703196 shutdown_connections 2024-09-16T00:25:35.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:35.925+0000 7f58023da640 1 --2- 172.21.15.122:0/1468703196 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f57e40a31e0 0x7f57e40a56a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:35.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:35.925+0000 7f58023da640 1 --2- 172.21.15.122:0/1468703196 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57fc1037c0 0x7f57fc19f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:35.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:35.925+0000 7f58023da640 1 --2- 172.21.15.122:0/1468703196 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57fc102e00 0x7f57fc1983c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:35.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:35.925+0000 7f58023da640 1 --2- 172.21.15.122:0/1468703196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57fc101c00 0x7f57fc197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:35.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:35.925+0000 7f58023da640 1 -- 172.21.15.122:0/1468703196 >> 172.21.15.122:0/1468703196 conn(0x7f57fc0fd390 msgr2=0x7f57fc106100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:35.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:35.925+0000 7f58023da640 1 -- 172.21.15.122:0/1468703196 shutdown_connections 2024-09-16T00:25:35.926 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:25:35.925+0000 7f58023da640 1 -- 172.21.15.122:0/1468703196 wait complete. 2024-09-16T00:25:36.214 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:36 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:25:36.215 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:36 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:36.215 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:36 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:36.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:36 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:25:36.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:36 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:36.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:36 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:36.515 DEBUG:teuthology.orchestra.run.smithi122:osd.2> sudo journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@osd.2.service 2024-09-16T00:25:36.517 INFO:tasks.cephadm:Deploying osd.3 on smithi038 with /dev/vg_nvme/lv_3... 2024-09-16T00:25:36.517 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-16T00:25:36.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:36 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:25:36.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:36 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:36.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:36 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:36.770 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:25:37.192 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:37 smithi038 ceph-mon[30945]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:25:37.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:37 smithi122 ceph-mon[35412]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:25:37.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:37 smithi073 ceph-mon[35254]: pgmap v72: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:25:37.967 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 16 00:25:37 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-2[40385]: 2024-09-16T00:25:37.771+0000 7f2255ba5640 -1 osd.2 0 log_to_monitors true 2024-09-16T00:25:38.400 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:38 smithi038 ceph-mon[30945]: from='osd.2 [v2:172.21.15.122:6800/1754741655,v1:172.21.15.122:6801/1754741655]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T00:25:38.400 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:38 smithi038 ceph-mon[30945]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T00:25:38.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:38 smithi122 ceph-mon[35412]: from='osd.2 [v2:172.21.15.122:6800/1754741655,v1:172.21.15.122:6801/1754741655]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T00:25:38.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:38 smithi122 ceph-mon[35412]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T00:25:38.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:38 smithi073 ceph-mon[35254]: from='osd.2 [v2:172.21.15.122:6800/1754741655,v1:172.21.15.122:6801/1754741655]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T00:25:38.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:38 smithi073 ceph-mon[35254]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-16T00:25:39.297 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 16 00:25:39 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-2[40385]: 2024-09-16T00:25:39.142+0000 7f2251b28640 -1 osd.2 0 waiting for initial osdmap 2024-09-16T00:25:39.297 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 16 00:25:39 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-2[40385]: 2024-09-16T00:25:39.148+0000 7f224d950640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:25:39.297 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:39 smithi122 ceph-mon[35412]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:25:39.297 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:39 smithi122 ceph-mon[35412]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T00:25:39.297 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:39 smithi122 ceph-mon[35412]: osdmap e16: 3 total, 2 up, 3 in 2024-09-16T00:25:39.297 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:39 smithi122 ceph-mon[35412]: from='osd.2 [v2:172.21.15.122:6800/1754741655,v1:172.21.15.122:6801/1754741655]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-16T00:25:39.297 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:39 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:25:39.297 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:39 smithi122 ceph-mon[35412]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-16T00:25:39.305 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:39 smithi038 ceph-mon[30945]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:25:39.305 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:39 smithi038 ceph-mon[30945]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T00:25:39.305 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:39 smithi038 ceph-mon[30945]: osdmap e16: 3 total, 2 up, 3 in 2024-09-16T00:25:39.305 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:39 smithi038 ceph-mon[30945]: from='osd.2 [v2:172.21.15.122:6800/1754741655,v1:172.21.15.122:6801/1754741655]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-16T00:25:39.306 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:39 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:25:39.306 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:39 smithi038 ceph-mon[30945]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-16T00:25:39.307 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:25:39.338 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi038:vg_nvme/lv_3 2024-09-16T00:25:39.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:39 smithi073 ceph-mon[35254]: pgmap v73: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:25:39.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:39 smithi073 ceph-mon[35254]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-16T00:25:39.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:39 smithi073 ceph-mon[35254]: osdmap e16: 3 total, 2 up, 3 in 2024-09-16T00:25:39.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:39 smithi073 ceph-mon[35254]: from='osd.2 [v2:172.21.15.122:6800/1754741655,v1:172.21.15.122:6801/1754741655]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-16T00:25:39.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:39 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:25:39.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:39 smithi073 ceph-mon[35254]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-16T00:25:39.594 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:25:40.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:40.079+0000 7f47ca396640 1 -- 172.21.15.38:0/1116883731 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47c4101c00 msgr2=0x7f47c4102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:40.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:40.079+0000 7f47ca396640 1 --2- 172.21.15.38:0/1116883731 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47c4101c00 0x7f47c4102000 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f47b4009900 tx=0x7f47b402a410 comp rx=0 tx=0).stop 2024-09-16T00:25:40.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:40.079+0000 7f47ca396640 1 -- 172.21.15.38:0/1116883731 shutdown_connections 2024-09-16T00:25:40.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:40.079+0000 7f47ca396640 1 --2- 172.21.15.38:0/1116883731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f47c41037c0 0x7f47c410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:40.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:40.079+0000 7f47ca396640 1 --2- 172.21.15.38:0/1116883731 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47c4102e00 0x7f47c4103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:40.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:40.079+0000 7f47ca396640 1 --2- 172.21.15.38:0/1116883731 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47c4101c00 0x7f47c4102000 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:40.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:40.079+0000 7f47ca396640 1 -- 172.21.15.38:0/1116883731 >> 172.21.15.38:0/1116883731 conn(0x7f47c40fd390 msgr2=0x7f47c40ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:25:40.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:40.079+0000 7f47ca396640 1 -- 172.21.15.38:0/1116883731 shutdown_connections 2024-09-16T00:25:40.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:40.079+0000 7f47ca396640 1 -- 172.21.15.38:0/1116883731 wait complete. 2024-09-16T00:25:40.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:40.079+0000 7f47ca396640 1 Processor -- start 2024-09-16T00:25:40.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:40.079+0000 7f47ca396640 1 -- start start 2024-09-16T00:25:40.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:40.080+0000 7f47ca396640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47c4101c00 0x7f47c4197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:40.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:40.080+0000 7f47ca396640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47c4102e00 0x7f47c4198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:40.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:40.080+0000 7f47ca396640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f47c41037c0 0x7f47c419f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:40.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:40.080+0000 7f47ca396640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f47c410d890 con 0x7f47c4102e00 2024-09-16T00:25:40.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:40.080+0000 7f47ca396640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f47c410d710 con 0x7f47c4101c00 2024-09-16T00:25:40.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:40.080+0000 7f47ca396640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f47c410da10 con 0x7f47c41037c0 2024-09-16T00:25:40.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:40.080+0000 7f47c37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47c4102e00 0x7f47c4198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:40.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:40.080+0000 7f47c37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47c4102e00 0x7f47c4198410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46016/0 (socket says 172.21.15.38:46016) 2024-09-16T00:25:40.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:40.080+0000 7f47c37fe640 1 -- 172.21.15.38:0/4043741542 learned_addr learned my addr 172.21.15.38:0/4043741542 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:25:40.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:40.080+0000 7f47c8933640 1 --2- 172.21.15.38:0/4043741542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f47c41037c0 0x7f47c419f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:40.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:40.080+0000 7f47c3fff640 1 --2- 172.21.15.38:0/4043741542 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47c4101c00 0x7f47c4197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:40.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:40.080+0000 7f47c37fe640 1 -- 172.21.15.38:0/4043741542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f47c41037c0 msgr2=0x7f47c419f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:40.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:40.080+0000 7f47c37fe640 1 --2- 172.21.15.38:0/4043741542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f47c41037c0 0x7f47c419f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:40.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:40.080+0000 7f47c37fe640 1 -- 172.21.15.38:0/4043741542 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47c4101c00 msgr2=0x7f47c4197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:25:40.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:40.080+0000 7f47c37fe640 1 --2- 172.21.15.38:0/4043741542 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47c4101c00 0x7f47c4197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:25:40.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:40.080+0000 7f47c37fe640 1 -- 172.21.15.38:0/4043741542 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c419fb00 con 0x7f47c4102e00 2024-09-16T00:25:40.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:40.080+0000 7f47c37fe640 1 --2- 172.21.15.38:0/4043741542 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47c4102e00 0x7f47c4198410 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f47b800d950 tx=0x7f47b800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:40.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:40.080+0000 7f47c17fa640 1 -- 172.21.15.38:0/4043741542 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b8017070 con 0x7f47c4102e00 2024-09-16T00:25:40.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:40.080+0000 7f47ca396640 1 -- 172.21.15.38:0/4043741542 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47c419fdf0 con 0x7f47c4102e00 2024-09-16T00:25:40.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:40.080+0000 7f47c17fa640 1 -- 172.21.15.38:0/4043741542 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f47b8009c30 con 0x7f47c4102e00 2024-09-16T00:25:40.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:40.080+0000 7f47c17fa640 1 -- 172.21.15.38:0/4043741542 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47b800f6b0 con 0x7f47c4102e00 2024-09-16T00:25:40.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:40.081+0000 7f47ca396640 1 -- 172.21.15.38:0/4043741542 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47c41a02a0 con 0x7f47c4102e00 2024-09-16T00:25:40.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:40.081+0000 7f47ca396640 1 -- 172.21.15.38:0/4043741542 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47c4108fa0 con 0x7f47c4102e00 2024-09-16T00:25:40.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:40.084+0000 7f47c17fa640 1 -- 172.21.15.38:0/4043741542 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 15) ==== 138299+0+0 (secure 0 0 0) 0x7f47b80109d0 con 0x7f47c4102e00 2024-09-16T00:25:40.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:40.084+0000 7f47c17fa640 1 --2- 172.21.15.38:0/4043741542 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f47ac0a31e0 0x7f47ac0a56a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:25:40.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:40.084+0000 7f47c3fff640 1 --2- 172.21.15.38:0/4043741542 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f47ac0a31e0 0x7f47ac0a56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:25:40.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:40.084+0000 7f47c17fa640 1 -- 172.21.15.38:0/4043741542 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(17..17 src has 1..17) ==== 2571+0+0 (secure 0 0 0) 0x7f47b80cdd00 con 0x7f47c4102e00 2024-09-16T00:25:40.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:40.085+0000 7f47c3fff640 1 --2- 172.21.15.38:0/4043741542 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f47ac0a31e0 0x7f47ac0a56a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f47b4006010 tx=0x7f47b4000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:25:40.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:40.088+0000 7f47c17fa640 1 -- 172.21.15.38:0/4043741542 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f47b809c170 con 0x7f47c4102e00 2024-09-16T00:25:40.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:40.273+0000 7f47ca396640 1 -- 172.21.15.38:0/4043741542 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f47c4062630 con 0x7f47ac0a31e0 2024-09-16T00:25:40.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:40 smithi038 ceph-mon[30945]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-09-16T00:25:40.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:40 smithi038 ceph-mon[30945]: osdmap e17: 3 total, 2 up, 3 in 2024-09-16T00:25:40.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:40 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:25:40.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:40 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:25:40.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:40 smithi038 ceph-mon[30945]: Detected new or changed devices on smithi122 2024-09-16T00:25:40.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:40 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:40.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:40 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:40.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:40 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:25:40.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:40 smithi038 ceph-mon[30945]: Adjusting osd_memory_target on smithi122 to 17087M 2024-09-16T00:25:40.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:40 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:40.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:40 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:40.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:40 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:25:40.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:40 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:40.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:40 smithi122 ceph-mon[35412]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-09-16T00:25:40.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:40 smithi122 ceph-mon[35412]: osdmap e17: 3 total, 2 up, 3 in 2024-09-16T00:25:40.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:40 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:25:40.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:40 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:25:40.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:40 smithi122 ceph-mon[35412]: Detected new or changed devices on smithi122 2024-09-16T00:25:40.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:40 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:40.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:40 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:40.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:40 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:25:40.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:40 smithi122 ceph-mon[35412]: Adjusting osd_memory_target on smithi122 to 17087M 2024-09-16T00:25:40.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:40 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:40.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:40 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:40.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:40 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:25:40.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:40 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:40.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:40 smithi073 ceph-mon[35254]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-09-16T00:25:40.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:40 smithi073 ceph-mon[35254]: osdmap e17: 3 total, 2 up, 3 in 2024-09-16T00:25:40.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:40 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:25:40.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:40 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:25:40.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:40 smithi073 ceph-mon[35254]: Detected new or changed devices on smithi122 2024-09-16T00:25:40.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:40 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:40.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:40 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:40.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:40 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:25:40.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:40 smithi073 ceph-mon[35254]: Adjusting osd_memory_target on smithi122 to 17087M 2024-09-16T00:25:40.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:40 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:40.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:40 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:40.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:40 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:25:40.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:40 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:41.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:41 smithi038 ceph-mon[30945]: purged_snaps scrub starts 2024-09-16T00:25:41.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:41 smithi038 ceph-mon[30945]: purged_snaps scrub ok 2024-09-16T00:25:41.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:41 smithi038 ceph-mon[30945]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:25:41.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:41 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:25:41.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:41 smithi038 ceph-mon[30945]: osd.2 [v2:172.21.15.122:6800/1754741655,v1:172.21.15.122:6801/1754741655] boot 2024-09-16T00:25:41.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:41 smithi038 ceph-mon[30945]: osdmap e18: 3 total, 3 up, 3 in 2024-09-16T00:25:41.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:41 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:25:41.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:41 smithi038 ceph-mon[30945]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:25:41.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:41 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:25:41.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:41 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:25:41.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:41 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:41.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:41 smithi122 ceph-mon[35412]: purged_snaps scrub starts 2024-09-16T00:25:41.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:41 smithi122 ceph-mon[35412]: purged_snaps scrub ok 2024-09-16T00:25:41.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:41 smithi122 ceph-mon[35412]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:25:41.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:41 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:25:41.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:41 smithi122 ceph-mon[35412]: osd.2 [v2:172.21.15.122:6800/1754741655,v1:172.21.15.122:6801/1754741655] boot 2024-09-16T00:25:41.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:41 smithi122 ceph-mon[35412]: osdmap e18: 3 total, 3 up, 3 in 2024-09-16T00:25:41.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:41 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:25:41.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:41 smithi122 ceph-mon[35412]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:25:41.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:41 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:25:41.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:41 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:25:41.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:41 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:41.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:41 smithi073 ceph-mon[35254]: purged_snaps scrub starts 2024-09-16T00:25:41.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:41 smithi073 ceph-mon[35254]: purged_snaps scrub ok 2024-09-16T00:25:41.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:41 smithi073 ceph-mon[35254]: pgmap v76: 0 pgs: ; 0 B data, 57 MiB used, 179 GiB / 179 GiB avail 2024-09-16T00:25:41.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:41 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:25:41.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:41 smithi073 ceph-mon[35254]: osd.2 [v2:172.21.15.122:6800/1754741655,v1:172.21.15.122:6801/1754741655] boot 2024-09-16T00:25:41.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:41 smithi073 ceph-mon[35254]: osdmap e18: 3 total, 3 up, 3 in 2024-09-16T00:25:41.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:41 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-16T00:25:41.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:41 smithi073 ceph-mon[35254]: from='client.14283 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:25:41.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:41 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:25:41.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:41 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:25:41.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:41 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:42.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:42 smithi038 ceph-mon[30945]: osdmap e19: 3 total, 3 up, 3 in 2024-09-16T00:25:42.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:42 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-16T00:25:42.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:42 smithi122 ceph-mon[35412]: osdmap e19: 3 total, 3 up, 3 in 2024-09-16T00:25:42.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:42 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-16T00:25:42.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:42 smithi073 ceph-mon[35254]: osdmap e19: 3 total, 3 up, 3 in 2024-09-16T00:25:42.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:42 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-09-16T00:25:43.715 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 16 00:25:43 smithi038 sudo[49692]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:25:43.715 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 16 00:25:43 smithi038 sudo[49692]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:25:43.715 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 16 00:25:43 smithi038 sudo[49692]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:25:43.715 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 16 00:25:43 smithi038 sudo[49692]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:25:43.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:43 smithi038 ceph-mon[30945]: pgmap v79: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:25:43.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:43 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T00:25:43.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:43 smithi038 ceph-mon[30945]: osdmap e20: 3 total, 3 up, 3 in 2024-09-16T00:25:43.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:43 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T00:25:43.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:43 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/1979859050' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28ebb4a0-92e7-4f57-82a5-b9b761eca269"}]: dispatch 2024-09-16T00:25:43.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:43 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T00:25:43.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:43 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/1979859050' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "28ebb4a0-92e7-4f57-82a5-b9b761eca269"}]': finished 2024-09-16T00:25:43.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:43 smithi038 ceph-mon[30945]: osdmap e21: 4 total, 3 up, 4 in 2024-09-16T00:25:43.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:43 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:25:43.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:43 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/3908365524' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:25:43.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:43 smithi122 ceph-mon[35412]: pgmap v79: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:25:43.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:43 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T00:25:43.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:43 smithi122 ceph-mon[35412]: osdmap e20: 3 total, 3 up, 3 in 2024-09-16T00:25:43.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:43 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T00:25:43.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:43 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/1979859050' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28ebb4a0-92e7-4f57-82a5-b9b761eca269"}]: dispatch 2024-09-16T00:25:43.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:43 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T00:25:43.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:43 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/1979859050' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "28ebb4a0-92e7-4f57-82a5-b9b761eca269"}]': finished 2024-09-16T00:25:43.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:43 smithi122 ceph-mon[35412]: osdmap e21: 4 total, 3 up, 4 in 2024-09-16T00:25:43.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:43 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:25:43.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:43 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/3908365524' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:25:43.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:43 smithi073 ceph-mon[35254]: pgmap v79: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:25:43.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:43 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-16T00:25:43.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:43 smithi073 ceph-mon[35254]: osdmap e20: 3 total, 3 up, 3 in 2024-09-16T00:25:43.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:43 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T00:25:43.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:43 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/1979859050' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "28ebb4a0-92e7-4f57-82a5-b9b761eca269"}]: dispatch 2024-09-16T00:25:43.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:43 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-16T00:25:43.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:43 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/1979859050' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "28ebb4a0-92e7-4f57-82a5-b9b761eca269"}]': finished 2024-09-16T00:25:43.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:43 smithi073 ceph-mon[35254]: osdmap e21: 4 total, 3 up, 4 in 2024-09-16T00:25:43.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:43 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:25:43.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:43 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/3908365524' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:25:44.215 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 16 00:25:43 smithi038 sudo[49741]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:25:44.215 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 16 00:25:43 smithi038 sudo[49741]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:25:44.215 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 16 00:25:43 smithi038 sudo[49741]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:25:44.215 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 16 00:25:43 smithi038 sudo[49741]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:25:44.288 INFO:journalctl@ceph.osd.1.smithi073.stdout:Sep 16 00:25:44 smithi073 sudo[43835]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:25:44.288 INFO:journalctl@ceph.osd.1.smithi073.stdout:Sep 16 00:25:44 smithi073 sudo[43835]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:25:44.288 INFO:journalctl@ceph.osd.1.smithi073.stdout:Sep 16 00:25:44 smithi073 sudo[43835]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:25:44.288 INFO:journalctl@ceph.osd.1.smithi073.stdout:Sep 16 00:25:44 smithi073 sudo[43835]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:25:44.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:25:44.384+0000 7f47c17fa640 1 -- 172.21.15.38:0/4043741542 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f47b80980e0 con 0x7f47c4102e00 2024-09-16T00:25:44.544 INFO:journalctl@ceph.osd.1.smithi073.stdout:Sep 16 00:25:44 smithi073 sudo[43838]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:25:44.544 INFO:journalctl@ceph.osd.1.smithi073.stdout:Sep 16 00:25:44 smithi073 sudo[43838]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:25:44.544 INFO:journalctl@ceph.osd.1.smithi073.stdout:Sep 16 00:25:44 smithi073 sudo[43838]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:25:44.544 INFO:journalctl@ceph.osd.1.smithi073.stdout:Sep 16 00:25:44 smithi073 sudo[43838]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:25:44.816 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 16 00:25:44 smithi122 sudo[44297]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:25:44.816 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 16 00:25:44 smithi122 sudo[44297]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:25:44.816 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 16 00:25:44 smithi122 sudo[44297]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:25:44.816 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 16 00:25:44 smithi122 sudo[44297]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:25:44.816 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:44 smithi122 ceph-mon[35412]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:25:44.816 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:44 smithi122 ceph-mon[35412]: osdmap e22: 4 total, 3 up, 4 in 2024-09-16T00:25:44.816 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:44 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:25:44.816 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:44 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:44.817 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:44 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:44.817 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:44 smithi122 ceph-mon[35412]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-16T00:25:45.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:44 smithi073 ceph-mon[35254]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:25:45.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:44 smithi073 ceph-mon[35254]: osdmap e22: 4 total, 3 up, 4 in 2024-09-16T00:25:45.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:44 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:25:45.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:44 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:45.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:44 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:45.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:44 smithi073 ceph-mon[35254]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-16T00:25:45.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:44 smithi038 ceph-mon[30945]: pgmap v82: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:25:45.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:44 smithi038 ceph-mon[30945]: osdmap e22: 4 total, 3 up, 4 in 2024-09-16T00:25:45.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:44 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:25:45.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:44 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:45.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:44 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:45.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:44 smithi038 ceph-mon[30945]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-09-16T00:25:45.216 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 16 00:25:44 smithi122 sudo[44300]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:25:45.216 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 16 00:25:44 smithi122 sudo[44300]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:25:45.216 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 16 00:25:44 smithi122 sudo[44300]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:25:45.216 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 16 00:25:44 smithi122 sudo[44300]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:25:45.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:45 smithi038 sudo[50122]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:25:45.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:45 smithi038 sudo[50122]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:25:45.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:45 smithi038 sudo[50122]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:25:45.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:45 smithi038 sudo[50122]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:25:46.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:45 smithi038 sudo[50127]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:25:46.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:45 smithi038 sudo[50127]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:25:46.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:45 smithi038 sudo[50127]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:25:46.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:45 smithi038 sudo[50127]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:25:46.235 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:45 smithi122 sudo[44303]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:25:46.236 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:45 smithi122 sudo[44303]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:25:46.236 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:45 smithi122 sudo[44303]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:25:46.236 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:45 smithi122 sudo[44303]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:25:46.500 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:46 smithi073 sudo[43845]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-16T00:25:46.500 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:46 smithi073 sudo[43845]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:25:46.500 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:46 smithi073 sudo[43845]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:25:46.500 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:46 smithi073 sudo[43845]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:25:46.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:46 smithi122 sudo[44306]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:25:46.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:46 smithi122 sudo[44306]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:25:46.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:46 smithi122 sudo[44306]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:25:46.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:46 smithi122 sudo[44306]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:25:46.788 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:46 smithi073 sudo[43848]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-16T00:25:46.789 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:46 smithi073 sudo[43848]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-16T00:25:46.789 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:46 smithi073 sudo[43848]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-16T00:25:46.789 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:46 smithi073 sudo[43848]: pam_unix(sudo:session): session closed for user root 2024-09-16T00:25:46.789 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:46 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:46.789 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:46 smithi073 ceph-mon[35254]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:25:46.789 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:46 smithi073 ceph-mon[35254]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:25:46.789 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:46 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:25:46.789 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:46 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:25:46.789 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:46 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:25:46.789 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:46 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:25:46.789 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:46 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:25:46.789 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:46 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:25:47.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:46 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:47.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:46 smithi038 ceph-mon[30945]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:25:47.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:46 smithi038 ceph-mon[30945]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:25:47.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:46 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:25:47.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:46 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:25:47.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:46 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:25:47.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:46 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:25:47.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:46 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:25:47.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:46 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:25:47.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:46 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:47.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:46 smithi122 ceph-mon[35412]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:25:47.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:46 smithi122 ceph-mon[35412]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:25:47.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:46 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:25:47.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:46 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:25:47.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:46 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:25:47.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:46 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:25:47.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:46 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:25:47.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:46 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:25:47.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:47 smithi038 ceph-mon[30945]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:25:47.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:47 smithi038 ceph-mon[30945]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:25:47.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:47 smithi038 ceph-mon[30945]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:25:47.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:47 smithi038 ceph-mon[30945]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:25:47.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:47 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:25:47.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:47 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:25:47.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:47 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:25:47.966 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:47 smithi038 ceph-mon[30945]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:25:48.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:47 smithi073 ceph-mon[35254]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:25:48.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:47 smithi073 ceph-mon[35254]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:25:48.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:47 smithi073 ceph-mon[35254]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:25:48.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:47 smithi073 ceph-mon[35254]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:25:48.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:47 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:25:48.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:47 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:25:48.045 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:47 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:25:48.045 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:47 smithi073 ceph-mon[35254]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:25:48.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:47 smithi122 ceph-mon[35412]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:25:48.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:47 smithi122 ceph-mon[35412]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:25:48.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:47 smithi122 ceph-mon[35412]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:25:48.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:47 smithi122 ceph-mon[35412]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-16T00:25:48.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:47 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-16T00:25:48.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:47 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-16T00:25:48.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:47 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-16T00:25:48.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:47 smithi122 ceph-mon[35412]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-16T00:25:49.037 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:48 smithi038 ceph-mon[30945]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:25:49.043 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:48 smithi073 ceph-mon[35254]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:25:49.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:48 smithi122 ceph-mon[35412]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:25:50.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:50 smithi038 ceph-mon[30945]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:25:51.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:50 smithi073 ceph-mon[35254]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:25:51.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:50 smithi122 ceph-mon[35412]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:25:51.967 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:51 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T00:25:51.968 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:51 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:51.968 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:51 smithi038 ceph-mon[30945]: Deploying daemon osd.3 on smithi038 2024-09-16T00:25:52.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:51 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T00:25:52.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:51 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:52.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:51 smithi073 ceph-mon[35254]: Deploying daemon osd.3 on smithi038 2024-09-16T00:25:52.219 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:51 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-16T00:25:52.264 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:51 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:25:52.264 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:51 smithi122 ceph-mon[35412]: Deploying daemon osd.3 on smithi038 2024-09-16T00:25:52.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:52 smithi038 ceph-mon[30945]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-16T00:25:53.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:52 smithi073 ceph-mon[35254]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-16T00:25:53.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:52 smithi122 ceph-mon[35412]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 44 KiB/s wr, 3 op/s 2024-09-16T00:25:54.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:54 smithi038 ceph-mon[30945]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-16T00:25:55.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:54 smithi073 ceph-mon[35254]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-16T00:25:55.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:54 smithi122 ceph-mon[35412]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 1.1 KiB/s rd, 33 KiB/s wr, 2 op/s 2024-09-16T00:25:57.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:56 smithi073 ceph-mon[35254]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-16T00:25:57.098 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:56 smithi038 ceph-mon[30945]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-16T00:25:57.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:56 smithi122 ceph-mon[35412]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 921 B/s rd, 26 KiB/s wr, 2 op/s 2024-09-16T00:25:58.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:57 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:25:58.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:57 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:58.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:57 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:58.188 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:57 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:25:58.188 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:57 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:58.189 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:57 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:58.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:57 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:25:58.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:57 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:58.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:57 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:25:59.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:25:58 smithi073 ceph-mon[35254]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-16T00:25:59.151 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:25:58 smithi038 ceph-mon[30945]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-16T00:25:59.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:25:58 smithi122 ceph-mon[35412]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-16T00:26:01.049 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:00 smithi038 ceph-mon[30945]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-16T00:26:01.049 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:00 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:01.049 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:00 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:01.049 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:00 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:01.050 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:00 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:26:01.050 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:00 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:01.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:00 smithi122 ceph-mon[35412]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-16T00:26:01.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:00 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:01.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:00 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:01.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:00 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:01.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:00 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:26:01.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:00 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:01.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:00 smithi073 ceph-mon[35254]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-16T00:26:01.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:00 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:01.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:00 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:01.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:00 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:01.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:00 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:26:01.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:00 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:01.815 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:01.815+0000 7f47c17fa640 1 -- 172.21.15.38:0/4043741542 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f47c4062630 con 0x7f47ac0a31e0 2024-09-16T00:26:01.816 INFO:teuthology.orchestra.run.smithi038.stdout:Created osd(s) 3 on host 'smithi038' 2024-09-16T00:26:01.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:01.817+0000 7f47ca396640 1 -- 172.21.15.38:0/4043741542 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f47ac0a31e0 msgr2=0x7f47ac0a56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:01.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:01.817+0000 7f47ca396640 1 --2- 172.21.15.38:0/4043741542 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f47ac0a31e0 0x7f47ac0a56a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f47b4006010 tx=0x7f47b4000bc0 comp rx=0 tx=0).stop 2024-09-16T00:26:01.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:01.817+0000 7f47ca396640 1 -- 172.21.15.38:0/4043741542 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47c4102e00 msgr2=0x7f47c4198410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:01.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:01.817+0000 7f47ca396640 1 --2- 172.21.15.38:0/4043741542 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47c4102e00 0x7f47c4198410 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f47b800d950 tx=0x7f47b800de10 comp rx=0 tx=0).stop 2024-09-16T00:26:01.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:01.817+0000 7f47ca396640 1 -- 172.21.15.38:0/4043741542 shutdown_connections 2024-09-16T00:26:01.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:01.817+0000 7f47ca396640 1 --2- 172.21.15.38:0/4043741542 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f47ac0a31e0 0x7f47ac0a56a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:01.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:01.817+0000 7f47ca396640 1 --2- 172.21.15.38:0/4043741542 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f47c41037c0 0x7f47c419f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:01.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:01.817+0000 7f47ca396640 1 --2- 172.21.15.38:0/4043741542 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f47c4102e00 0x7f47c4198410 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:01.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:01.817+0000 7f47ca396640 1 --2- 172.21.15.38:0/4043741542 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47c4101c00 0x7f47c4197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:01.818 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:01.817+0000 7f47ca396640 1 -- 172.21.15.38:0/4043741542 >> 172.21.15.38:0/4043741542 conn(0x7f47c40fd390 msgr2=0x7f47c4106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:01.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:01.818+0000 7f47ca396640 1 -- 172.21.15.38:0/4043741542 shutdown_connections 2024-09-16T00:26:01.819 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:01.818+0000 7f47ca396640 1 -- 172.21.15.38:0/4043741542 wait complete. 2024-09-16T00:26:02.490 DEBUG:teuthology.orchestra.run.smithi038:osd.3> sudo journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@osd.3.service 2024-09-16T00:26:02.493 INFO:tasks.cephadm:Deploying osd.4 on smithi073 with /dev/vg_nvme/lv_3... 2024-09-16T00:26:02.493 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-16T00:26:02.744 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:26:03.083 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:02 smithi073 ceph-mon[35254]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-16T00:26:03.083 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:02 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:26:03.083 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:02 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:03.083 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:02 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:03.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:02 smithi038 ceph-mon[30945]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-16T00:26:03.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:02 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:26:03.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:02 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:03.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:02 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:03.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:02 smithi122 ceph-mon[35412]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail; 767 B/s rd, 22 KiB/s wr, 1 op/s 2024-09-16T00:26:03.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:02 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:26:03.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:02 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:03.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:02 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:03.965 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 16 00:26:03 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-3[51710]: 2024-09-16T00:26:03.821+0000 7f6ea5586640 -1 osd.3 0 log_to_monitors true 2024-09-16T00:26:05.050 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:04 smithi038 ceph-mon[30945]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:26:05.050 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:04 smithi038 ceph-mon[30945]: from='osd.3 [v2:172.21.15.38:6810/3523190079,v1:172.21.15.38:6811/3523190079]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T00:26:05.103 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:04 smithi073 ceph-mon[35254]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:26:05.104 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:04 smithi073 ceph-mon[35254]: from='osd.3 [v2:172.21.15.38:6810/3523190079,v1:172.21.15.38:6811/3523190079]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T00:26:05.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:04 smithi122 ceph-mon[35412]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 87 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:26:05.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:04 smithi122 ceph-mon[35412]: from='osd.3 [v2:172.21.15.38:6810/3523190079,v1:172.21.15.38:6811/3523190079]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-16T00:26:05.374 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-16T00:26:05.400 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi073:vg_nvme/lv_3 2024-09-16T00:26:05.652 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:26:05.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:05 smithi073 ceph-mon[35254]: from='osd.3 [v2:172.21.15.38:6810/3523190079,v1:172.21.15.38:6811/3523190079]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T00:26:05.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:05 smithi073 ceph-mon[35254]: osdmap e23: 4 total, 3 up, 4 in 2024-09-16T00:26:05.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:05 smithi073 ceph-mon[35254]: from='osd.3 [v2:172.21.15.38:6810/3523190079,v1:172.21.15.38:6811/3523190079]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-16T00:26:05.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:05 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:26:05.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:05 smithi073 ceph-mon[35254]: Detected new or changed devices on smithi038 2024-09-16T00:26:05.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:05 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:05.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:05 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:05.930 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:05 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:26:05.931 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:05 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:26:05.931 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:05 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:05.931 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:05 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:05.931 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:05 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:26:05.931 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:05 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:06.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.143+0000 7fb2bb300640 1 -- 172.21.15.73:0/1433339233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2b4100320 msgr2=0x7fb2b4107350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:06.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.143+0000 7fb2bb300640 1 --2- 172.21.15.73:0/1433339233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2b4100320 0x7fb2b4107350 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4009900 tx=0x7fb2a402a410 comp rx=0 tx=0).stop 2024-09-16T00:26:06.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.144+0000 7fb2bb300640 1 -- 172.21.15.73:0/1433339233 shutdown_connections 2024-09-16T00:26:06.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.144+0000 7fb2bb300640 1 --2- 172.21.15.73:0/1433339233 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb2b41078e0 0x7fb2b4109da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:06.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.144+0000 7fb2bb300640 1 --2- 172.21.15.73:0/1433339233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2b4100320 0x7fb2b4107350 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:06.146 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.144+0000 7fb2bb300640 1 --2- 172.21.15.73:0/1433339233 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2b40ff9e0 0x7fb2b40ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:06.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.144+0000 7fb2bb300640 1 -- 172.21.15.73:0/1433339233 >> 172.21.15.73:0/1433339233 conn(0x7fb2b40fb610 msgr2=0x7fb2b40fda50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:06.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.144+0000 7fb2bb300640 1 -- 172.21.15.73:0/1433339233 shutdown_connections 2024-09-16T00:26:06.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.144+0000 7fb2bb300640 1 -- 172.21.15.73:0/1433339233 wait complete. 2024-09-16T00:26:06.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.144+0000 7fb2bb300640 1 Processor -- start 2024-09-16T00:26:06.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.144+0000 7fb2bb300640 1 -- start start 2024-09-16T00:26:06.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.145+0000 7fb2bb300640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb2b40ff9e0 0x7fb2b4102c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:06.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.145+0000 7fb2bb300640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2b4100320 0x7fb2b41012e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:06.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.145+0000 7fb2bb300640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2b41078e0 0x7fb2b4101820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:06.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.145+0000 7fb2bb300640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b410d870 con 0x7fb2b4100320 2024-09-16T00:26:06.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.145+0000 7fb2bb300640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b410d6f0 con 0x7fb2b41078e0 2024-09-16T00:26:06.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.145+0000 7fb2bb300640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b410d9f0 con 0x7fb2b40ff9e0 2024-09-16T00:26:06.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.145+0000 7fb2b989d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2b41078e0 0x7fb2b4101820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:06.147 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.145+0000 7fb2b989d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2b41078e0 0x7fb2b4101820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56570/0 (socket says 172.21.15.73:56570) 2024-09-16T00:26:06.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.145+0000 7fb2b909c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb2b40ff9e0 0x7fb2b4102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:06.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.145+0000 7fb2b989d640 1 -- 172.21.15.73:0/3600276852 learned_addr learned my addr 172.21.15.73:0/3600276852 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-09-16T00:26:06.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.145+0000 7fb2b889b640 1 --2- 172.21.15.73:0/3600276852 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2b4100320 0x7fb2b41012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:06.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.145+0000 7fb2b989d640 1 -- 172.21.15.73:0/3600276852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb2b40ff9e0 msgr2=0x7fb2b4102c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:06.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.145+0000 7fb2b989d640 1 --2- 172.21.15.73:0/3600276852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb2b40ff9e0 0x7fb2b4102c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:06.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.145+0000 7fb2b989d640 1 -- 172.21.15.73:0/3600276852 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2b4100320 msgr2=0x7fb2b41012e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:06.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.145+0000 7fb2b989d640 1 --2- 172.21.15.73:0/3600276852 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2b4100320 0x7fb2b41012e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:06.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.145+0000 7fb2b989d640 1 -- 172.21.15.73:0/3600276852 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2b41020e0 con 0x7fb2b41078e0 2024-09-16T00:26:06.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.145+0000 7fb2b909c640 1 --2- 172.21.15.73:0/3600276852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb2b40ff9e0 0x7fb2b4102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:26:06.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.145+0000 7fb2b989d640 1 --2- 172.21.15.73:0/3600276852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2b41078e0 0x7fb2b4101820 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac00eaa0 tx=0x7fb2ac00ef60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:06.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.145+0000 7fb2aa7fc640 1 -- 172.21.15.73:0/3600276852 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac00cdb0 con 0x7fb2b41078e0 2024-09-16T00:26:06.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.145+0000 7fb2aa7fc640 1 -- 172.21.15.73:0/3600276852 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb2ac005020 con 0x7fb2b41078e0 2024-09-16T00:26:06.148 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.145+0000 7fb2bb300640 1 -- 172.21.15.73:0/3600276852 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2b41a20b0 con 0x7fb2b41078e0 2024-09-16T00:26:06.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.145+0000 7fb2aa7fc640 1 -- 172.21.15.73:0/3600276852 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac0116b0 con 0x7fb2b41078e0 2024-09-16T00:26:06.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.146+0000 7fb2bb300640 1 -- 172.21.15.73:0/3600276852 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2b41a25c0 con 0x7fb2b41078e0 2024-09-16T00:26:06.149 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.147+0000 7fb2bb300640 1 -- 172.21.15.73:0/3600276852 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2b4106160 con 0x7fb2b41078e0 2024-09-16T00:26:06.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.148+0000 7fb2aa7fc640 1 -- 172.21.15.73:0/3600276852 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fb2ac005560 con 0x7fb2b41078e0 2024-09-16T00:26:06.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.148+0000 7fb2aa7fc640 1 --2- 172.21.15.73:0/3600276852 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb2a00a3240 0x7fb2a00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:06.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.148+0000 7fb2aa7fc640 1 -- 172.21.15.73:0/3600276852 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 3389+0+0 (secure 0 0 0) 0x7fb2ac0cead0 con 0x7fb2b41078e0 2024-09-16T00:26:06.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.149+0000 7fb2b909c640 1 --2- 172.21.15.73:0/3600276852 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb2a00a3240 0x7fb2a00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:06.150 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.149+0000 7fb2b909c640 1 --2- 172.21.15.73:0/3600276852 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb2a00a3240 0x7fb2a00a5700 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0009fc0 tx=0x7fb2b0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:06.152 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.152+0000 7fb2aa7fc640 1 -- 172.21.15.73:0/3600276852 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb2ac016030 con 0x7fb2b41078e0 2024-09-16T00:26:06.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:05 smithi038 ceph-mon[30945]: from='osd.3 [v2:172.21.15.38:6810/3523190079,v1:172.21.15.38:6811/3523190079]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T00:26:06.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:05 smithi038 ceph-mon[30945]: osdmap e23: 4 total, 3 up, 4 in 2024-09-16T00:26:06.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:05 smithi038 ceph-mon[30945]: from='osd.3 [v2:172.21.15.38:6810/3523190079,v1:172.21.15.38:6811/3523190079]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-16T00:26:06.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:05 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:26:06.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:05 smithi038 ceph-mon[30945]: Detected new or changed devices on smithi038 2024-09-16T00:26:06.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:05 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:06.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:05 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:06.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:05 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:26:06.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:05 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:26:06.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:05 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:06.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:05 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:06.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:05 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:26:06.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:05 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:06.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:05 smithi122 ceph-mon[35412]: from='osd.3 [v2:172.21.15.38:6810/3523190079,v1:172.21.15.38:6811/3523190079]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-16T00:26:06.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:05 smithi122 ceph-mon[35412]: osdmap e23: 4 total, 3 up, 4 in 2024-09-16T00:26:06.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:05 smithi122 ceph-mon[35412]: from='osd.3 [v2:172.21.15.38:6810/3523190079,v1:172.21.15.38:6811/3523190079]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-16T00:26:06.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:05 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:26:06.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:05 smithi122 ceph-mon[35412]: Detected new or changed devices on smithi038 2024-09-16T00:26:06.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:05 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:06.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:05 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:06.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:05 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:26:06.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:05 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:26:06.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:05 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:06.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:05 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:06.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:05 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:26:06.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:05 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:06.333 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:06.332+0000 7fb2bb300640 1 -- 172.21.15.73:0/3600276852 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fb2b4062630 con 0x7fb2a00a3240 2024-09-16T00:26:06.742 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 16 00:26:06 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-3[51710]: 2024-09-16T00:26:06.485+0000 7f6ea1509640 -1 osd.3 0 waiting for initial osdmap 2024-09-16T00:26:06.743 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 16 00:26:06 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-3[51710]: 2024-09-16T00:26:06.490+0000 7f6e99329640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:26:07.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:06 smithi073 ceph-mon[35254]: Adjusting osd_memory_target on smithi038 to 8546M 2024-09-16T00:26:07.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:06 smithi073 ceph-mon[35254]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:26:07.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:06 smithi073 ceph-mon[35254]: from='osd.3 [v2:172.21.15.38:6810/3523190079,v1:172.21.15.38:6811/3523190079]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-16T00:26:07.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:06 smithi073 ceph-mon[35254]: osdmap e24: 4 total, 3 up, 4 in 2024-09-16T00:26:07.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:06 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:26:07.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:06 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:26:07.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:06 smithi073 ceph-mon[35254]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:26:07.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:06 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:26:07.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:06 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:26:07.045 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:06 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:07.045 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:06 smithi073 ceph-mon[35254]: from='osd.3 [v2:172.21.15.38:6810/3523190079,v1:172.21.15.38:6811/3523190079]' entity='osd.3' 2024-09-16T00:26:07.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:06 smithi038 ceph-mon[30945]: Adjusting osd_memory_target on smithi038 to 8546M 2024-09-16T00:26:07.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:06 smithi038 ceph-mon[30945]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:26:07.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:06 smithi038 ceph-mon[30945]: from='osd.3 [v2:172.21.15.38:6810/3523190079,v1:172.21.15.38:6811/3523190079]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-16T00:26:07.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:06 smithi038 ceph-mon[30945]: osdmap e24: 4 total, 3 up, 4 in 2024-09-16T00:26:07.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:06 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:26:07.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:06 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:26:07.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:06 smithi038 ceph-mon[30945]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:26:07.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:06 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:26:07.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:06 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:26:07.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:06 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:07.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:06 smithi038 ceph-mon[30945]: from='osd.3 [v2:172.21.15.38:6810/3523190079,v1:172.21.15.38:6811/3523190079]' entity='osd.3' 2024-09-16T00:26:07.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:06 smithi122 ceph-mon[35412]: Adjusting osd_memory_target on smithi038 to 8546M 2024-09-16T00:26:07.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:06 smithi122 ceph-mon[35412]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-09-16T00:26:07.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:06 smithi122 ceph-mon[35412]: from='osd.3 [v2:172.21.15.38:6810/3523190079,v1:172.21.15.38:6811/3523190079]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-16T00:26:07.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:06 smithi122 ceph-mon[35412]: osdmap e24: 4 total, 3 up, 4 in 2024-09-16T00:26:07.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:06 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:26:07.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:06 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:26:07.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:06 smithi122 ceph-mon[35412]: from='client.24209 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:26:07.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:06 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:26:07.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:06 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:26:07.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:06 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:07.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:06 smithi122 ceph-mon[35412]: from='osd.3 [v2:172.21.15.38:6810/3523190079,v1:172.21.15.38:6811/3523190079]' entity='osd.3' 2024-09-16T00:26:08.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:07 smithi073 ceph-mon[35254]: purged_snaps scrub starts 2024-09-16T00:26:08.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:07 smithi073 ceph-mon[35254]: purged_snaps scrub ok 2024-09-16T00:26:08.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:07 smithi073 ceph-mon[35254]: osd.3 [v2:172.21.15.38:6810/3523190079,v1:172.21.15.38:6811/3523190079] boot 2024-09-16T00:26:08.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:07 smithi073 ceph-mon[35254]: osdmap e25: 4 total, 4 up, 4 in 2024-09-16T00:26:08.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:07 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:26:08.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:07 smithi073 ceph-mon[35254]: osdmap e26: 4 total, 4 up, 4 in 2024-09-16T00:26:08.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:07 smithi038 ceph-mon[30945]: purged_snaps scrub starts 2024-09-16T00:26:08.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:07 smithi038 ceph-mon[30945]: purged_snaps scrub ok 2024-09-16T00:26:08.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:07 smithi038 ceph-mon[30945]: osd.3 [v2:172.21.15.38:6810/3523190079,v1:172.21.15.38:6811/3523190079] boot 2024-09-16T00:26:08.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:07 smithi038 ceph-mon[30945]: osdmap e25: 4 total, 4 up, 4 in 2024-09-16T00:26:08.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:07 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:26:08.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:07 smithi038 ceph-mon[30945]: osdmap e26: 4 total, 4 up, 4 in 2024-09-16T00:26:08.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:07 smithi122 ceph-mon[35412]: purged_snaps scrub starts 2024-09-16T00:26:08.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:07 smithi122 ceph-mon[35412]: purged_snaps scrub ok 2024-09-16T00:26:08.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:07 smithi122 ceph-mon[35412]: osd.3 [v2:172.21.15.38:6810/3523190079,v1:172.21.15.38:6811/3523190079] boot 2024-09-16T00:26:08.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:07 smithi122 ceph-mon[35412]: osdmap e25: 4 total, 4 up, 4 in 2024-09-16T00:26:08.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:07 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-16T00:26:08.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:07 smithi122 ceph-mon[35412]: osdmap e26: 4 total, 4 up, 4 in 2024-09-16T00:26:09.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:08 smithi038 ceph-mon[30945]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:26:09.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:08 smithi038 ceph-mon[30945]: from='client.? 172.21.15.73:0/2230699956' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ea363f7-7ad8-45ed-b7e4-4b010fe7f8cc"}]: dispatch 2024-09-16T00:26:09.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:08 smithi038 ceph-mon[30945]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ea363f7-7ad8-45ed-b7e4-4b010fe7f8cc"}]: dispatch 2024-09-16T00:26:09.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:08 smithi038 ceph-mon[30945]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5ea363f7-7ad8-45ed-b7e4-4b010fe7f8cc"}]': finished 2024-09-16T00:26:09.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:08 smithi038 ceph-mon[30945]: osdmap e27: 5 total, 4 up, 5 in 2024-09-16T00:26:09.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:08 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:26:09.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:08 smithi122 ceph-mon[35412]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:26:09.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:08 smithi122 ceph-mon[35412]: from='client.? 172.21.15.73:0/2230699956' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ea363f7-7ad8-45ed-b7e4-4b010fe7f8cc"}]: dispatch 2024-09-16T00:26:09.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:08 smithi122 ceph-mon[35412]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ea363f7-7ad8-45ed-b7e4-4b010fe7f8cc"}]: dispatch 2024-09-16T00:26:09.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:08 smithi122 ceph-mon[35412]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5ea363f7-7ad8-45ed-b7e4-4b010fe7f8cc"}]': finished 2024-09-16T00:26:09.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:08 smithi122 ceph-mon[35412]: osdmap e27: 5 total, 4 up, 5 in 2024-09-16T00:26:09.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:08 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:26:09.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:08 smithi073 ceph-mon[35254]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:26:09.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:08 smithi073 ceph-mon[35254]: from='client.? 172.21.15.73:0/2230699956' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ea363f7-7ad8-45ed-b7e4-4b010fe7f8cc"}]: dispatch 2024-09-16T00:26:09.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:08 smithi073 ceph-mon[35254]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5ea363f7-7ad8-45ed-b7e4-4b010fe7f8cc"}]: dispatch 2024-09-16T00:26:09.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:08 smithi073 ceph-mon[35254]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5ea363f7-7ad8-45ed-b7e4-4b010fe7f8cc"}]': finished 2024-09-16T00:26:09.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:08 smithi073 ceph-mon[35254]: osdmap e27: 5 total, 4 up, 5 in 2024-09-16T00:26:09.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:08 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:26:10.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:09 smithi038 ceph-mon[30945]: from='client.? 172.21.15.73:0/1398687116' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:26:10.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:09 smithi038 ceph-mon[30945]: osdmap e28: 5 total, 4 up, 5 in 2024-09-16T00:26:10.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:09 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:26:10.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:09 smithi122 ceph-mon[35412]: from='client.? 172.21.15.73:0/1398687116' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:26:10.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:09 smithi122 ceph-mon[35412]: osdmap e28: 5 total, 4 up, 5 in 2024-09-16T00:26:10.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:09 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:26:10.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:09 smithi073 ceph-mon[35254]: from='client.? 172.21.15.73:0/1398687116' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:26:10.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:09 smithi073 ceph-mon[35254]: osdmap e28: 5 total, 4 up, 5 in 2024-09-16T00:26:10.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:09 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:26:11.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:10 smithi073 ceph-mon[35254]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:26:11.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:10 smithi073 ceph-mon[35254]: osdmap e29: 5 total, 4 up, 5 in 2024-09-16T00:26:11.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:10 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:26:11.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:10 smithi038 ceph-mon[30945]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:26:11.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:10 smithi038 ceph-mon[30945]: osdmap e29: 5 total, 4 up, 5 in 2024-09-16T00:26:11.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:10 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:26:11.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:10 smithi122 ceph-mon[35412]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:26:11.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:10 smithi122 ceph-mon[35412]: osdmap e29: 5 total, 4 up, 5 in 2024-09-16T00:26:11.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:10 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:26:13.110 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:12 smithi073 ceph-mon[35254]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-16T00:26:13.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:12 smithi038 ceph-mon[30945]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-16T00:26:13.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:12 smithi122 ceph-mon[35412]: pgmap v104: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-16T00:26:15.203 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:14 smithi073 ceph-mon[35254]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-16T00:26:15.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:14 smithi038 ceph-mon[30945]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-16T00:26:15.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:14 smithi122 ceph-mon[35412]: pgmap v105: 1 pgs: 1 peering; 577 KiB data, 117 MiB used, 357 GiB / 358 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-16T00:26:17.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:16 smithi038 ceph-mon[30945]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-16T00:26:17.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:16 smithi122 ceph-mon[35412]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-16T00:26:17.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:16 smithi073 ceph-mon[35254]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-09-16T00:26:18.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:17 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T00:26:18.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:17 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:18.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:17 smithi038 ceph-mon[30945]: Deploying daemon osd.4 on smithi073 2024-09-16T00:26:18.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:17 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T00:26:18.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:17 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:18.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:17 smithi122 ceph-mon[35412]: Deploying daemon osd.4 on smithi073 2024-09-16T00:26:18.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:17 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-16T00:26:18.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:17 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:18.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:17 smithi073 ceph-mon[35254]: Deploying daemon osd.4 on smithi073 2024-09-16T00:26:18.961 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:18 smithi073 ceph-mon[35254]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T00:26:19.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:18 smithi038 ceph-mon[30945]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T00:26:19.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:18 smithi122 ceph-mon[35412]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-09-16T00:26:21.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:20 smithi073 ceph-mon[35254]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-16T00:26:21.214 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:20 smithi038 ceph-mon[30945]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-16T00:26:21.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:20 smithi122 ceph-mon[35412]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 59 KiB/s, 0 objects/s recovering 2024-09-16T00:26:23.191 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:22 smithi073 ceph-mon[35254]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:26:23.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:22 smithi038 ceph-mon[30945]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:26:23.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:22 smithi122 ceph-mon[35412]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:26:24.188 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:23 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:26:24.188 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:23 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:24.188 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:23 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:24.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:23 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:26:24.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:23 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:24.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:23 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:24.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:23 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:26:24.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:23 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:24.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:23 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:25.185 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:24 smithi073 ceph-mon[35254]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:26:25.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:24 smithi038 ceph-mon[30945]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:26:25.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:24 smithi122 ceph-mon[35412]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:26:27.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:27 smithi073 ceph-mon[35254]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:26:27.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:27 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:27.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:27 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:27.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:27 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:27.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:27 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:26:27.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:27 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:27.644 INFO:teuthology.orchestra.run.smithi073.stdout:Created osd(s) 4 on host 'smithi073' 2024-09-16T00:26:27.644 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:27.643+0000 7fb2aa7fc640 1 -- 172.21.15.73:0/3600276852 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fb2b4062630 con 0x7fb2a00a3240 2024-09-16T00:26:27.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:27.645+0000 7fb2bb300640 1 -- 172.21.15.73:0/3600276852 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb2a00a3240 msgr2=0x7fb2a00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:27.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:27.645+0000 7fb2bb300640 1 --2- 172.21.15.73:0/3600276852 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb2a00a3240 0x7fb2a00a5700 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0009fc0 tx=0x7fb2b0005f90 comp rx=0 tx=0).stop 2024-09-16T00:26:27.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:27.646+0000 7fb2bb300640 1 -- 172.21.15.73:0/3600276852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2b41078e0 msgr2=0x7fb2b4101820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:27.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:27.646+0000 7fb2bb300640 1 --2- 172.21.15.73:0/3600276852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2b41078e0 0x7fb2b4101820 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac00eaa0 tx=0x7fb2ac00ef60 comp rx=0 tx=0).stop 2024-09-16T00:26:27.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:27.646+0000 7fb2bb300640 1 -- 172.21.15.73:0/3600276852 shutdown_connections 2024-09-16T00:26:27.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:27.646+0000 7fb2bb300640 1 --2- 172.21.15.73:0/3600276852 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb2a00a3240 0x7fb2a00a5700 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:27.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:27.646+0000 7fb2bb300640 1 --2- 172.21.15.73:0/3600276852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2b41078e0 0x7fb2b4101820 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:27.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:27.646+0000 7fb2bb300640 1 --2- 172.21.15.73:0/3600276852 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb2b4100320 0x7fb2b41012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:27.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:27.646+0000 7fb2bb300640 1 --2- 172.21.15.73:0/3600276852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb2b40ff9e0 0x7fb2b4102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:27.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:27.646+0000 7fb2bb300640 1 -- 172.21.15.73:0/3600276852 >> 172.21.15.73:0/3600276852 conn(0x7fb2b40fb610 msgr2=0x7fb2b4103330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:27.647 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:27.646+0000 7fb2bb300640 1 -- 172.21.15.73:0/3600276852 shutdown_connections 2024-09-16T00:26:27.648 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:26:27.646+0000 7fb2bb300640 1 -- 172.21.15.73:0/3600276852 wait complete. 2024-09-16T00:26:27.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:27 smithi038 ceph-mon[30945]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:26:27.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:27 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:27.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:27 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:27.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:27 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:27.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:27 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:26:27.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:27 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:27.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:27 smithi122 ceph-mon[35412]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:26:27.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:27 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:27.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:27 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:27.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:27 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:27.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:27 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:26:27.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:27 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:28.224 DEBUG:teuthology.orchestra.run.smithi073:osd.4> sudo journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@osd.4.service 2024-09-16T00:26:28.227 INFO:tasks.cephadm:Deploying osd.5 on smithi122 with /dev/vg_nvme/lv_3... 2024-09-16T00:26:28.227 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_3 2024-09-16T00:26:28.451 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:28 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:26:28.451 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:28 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:28.451 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:28 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:28.492 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:26:28.612 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:28 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:26:28.612 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:28 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:28.612 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:28 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:28.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:28 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:26:28.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:28 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:28.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:28 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:29.512 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:29 smithi073 ceph-mon[35254]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:26:29.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:29 smithi038 ceph-mon[30945]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:26:29.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:29 smithi122 ceph-mon[35412]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:26:30.044 INFO:journalctl@ceph.osd.4.smithi073.stdout:Sep 16 00:26:29 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-4[46579]: 2024-09-16T00:26:29.777+0000 7f14deb87640 -1 osd.4 0 log_to_monitors true 2024-09-16T00:26:30.506 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:30 smithi122 ceph-mon[35412]: from='osd.4 [v2:172.21.15.73:6808/1469927628,v1:172.21.15.73:6809/1469927628]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T00:26:30.506 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:30 smithi122 ceph-mon[35412]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T00:26:30.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:30 smithi073 ceph-mon[35254]: from='osd.4 [v2:172.21.15.73:6808/1469927628,v1:172.21.15.73:6809/1469927628]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T00:26:30.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:30 smithi073 ceph-mon[35254]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T00:26:30.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:30 smithi038 ceph-mon[30945]: from='osd.4 [v2:172.21.15.73:6808/1469927628,v1:172.21.15.73:6809/1469927628]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T00:26:30.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:30 smithi038 ceph-mon[30945]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-16T00:26:31.453 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:31 smithi122 ceph-mon[35412]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:26:31.453 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:31 smithi122 ceph-mon[35412]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T00:26:31.453 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:31 smithi122 ceph-mon[35412]: osdmap e30: 5 total, 4 up, 5 in 2024-09-16T00:26:31.453 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:31 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:26:31.453 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:31 smithi122 ceph-mon[35412]: from='osd.4 [v2:172.21.15.73:6808/1469927628,v1:172.21.15.73:6809/1469927628]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-09-16T00:26:31.453 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:31 smithi122 ceph-mon[35412]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-09-16T00:26:31.456 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-16T00:26:31.482 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi122:vg_nvme/lv_3 2024-09-16T00:26:31.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:31 smithi073 ceph-mon[35254]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:26:31.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:31 smithi073 ceph-mon[35254]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T00:26:31.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:31 smithi073 ceph-mon[35254]: osdmap e30: 5 total, 4 up, 5 in 2024-09-16T00:26:31.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:31 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:26:31.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:31 smithi073 ceph-mon[35254]: from='osd.4 [v2:172.21.15.73:6808/1469927628,v1:172.21.15.73:6809/1469927628]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-09-16T00:26:31.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:31 smithi073 ceph-mon[35254]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-09-16T00:26:31.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:31 smithi038 ceph-mon[30945]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:26:31.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:31 smithi038 ceph-mon[30945]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-16T00:26:31.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:31 smithi038 ceph-mon[30945]: osdmap e30: 5 total, 4 up, 5 in 2024-09-16T00:26:31.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:31 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:26:31.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:31 smithi038 ceph-mon[30945]: from='osd.4 [v2:172.21.15.73:6808/1469927628,v1:172.21.15.73:6809/1469927628]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-09-16T00:26:31.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:31 smithi038 ceph-mon[30945]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-09-16T00:26:31.737 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:26:32.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:32.185+0000 7f49aed9e640 1 -- 172.21.15.122:0/3821708224 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49a8101cd0 msgr2=0x7f49a8102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:32.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:32.185+0000 7f49aed9e640 1 --2- 172.21.15.122:0/3821708224 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49a8101cd0 0x7f49a8102150 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4994009900 tx=0x7f499402a410 comp rx=0 tx=0).stop 2024-09-16T00:26:32.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:32.185+0000 7f49aed9e640 1 -- 172.21.15.122:0/3821708224 shutdown_connections 2024-09-16T00:26:32.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:32.185+0000 7f49aed9e640 1 --2- 172.21.15.122:0/3821708224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49a8102690 0x7f49a8108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:32.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:32.185+0000 7f49aed9e640 1 --2- 172.21.15.122:0/3821708224 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49a8101cd0 0x7f49a8102150 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:32.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:32.185+0000 7f49aed9e640 1 --2- 172.21.15.122:0/3821708224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49a8100ad0 0x7f49a8100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:32.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:32.185+0000 7f49aed9e640 1 -- 172.21.15.122:0/3821708224 >> 172.21.15.122:0/3821708224 conn(0x7f49a80fc280 msgr2=0x7f49a80fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:32.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:32.185+0000 7f49aed9e640 1 -- 172.21.15.122:0/3821708224 shutdown_connections 2024-09-16T00:26:32.207 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:32.185+0000 7f49aed9e640 1 -- 172.21.15.122:0/3821708224 wait complete. 2024-09-16T00:26:32.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:32.185+0000 7f49aed9e640 1 Processor -- start 2024-09-16T00:26:32.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:32.186+0000 7f49aed9e640 1 -- start start 2024-09-16T00:26:32.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:32.186+0000 7f49aed9e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49a8100ad0 0x7f49a806aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:32.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:32.186+0000 7f49aed9e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49a8101cd0 0x7f49a819f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:32.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:32.186+0000 7f49aed9e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49a8102690 0x7f49a81a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:32.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:32.186+0000 7f49aed9e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f49a810c730 con 0x7f49a8101cd0 2024-09-16T00:26:32.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:32.186+0000 7f49aed9e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f49a810c5b0 con 0x7f49a8102690 2024-09-16T00:26:32.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:32.186+0000 7f49aed9e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f49a810c8b0 con 0x7f49a8100ad0 2024-09-16T00:26:32.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:32.186+0000 7f49acb3a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49a8100ad0 0x7f49a806aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:32.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:32.186+0000 7f49acb3a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49a8100ad0 0x7f49a806aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:49320/0 (socket says 172.21.15.122:49320) 2024-09-16T00:26:32.208 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:32.186+0000 7f49acb3a640 1 -- 172.21.15.122:0/3954039371 learned_addr learned my addr 172.21.15.122:0/3954039371 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-16T00:26:32.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:32.186+0000 7f49a7fff640 1 --2- 172.21.15.122:0/3954039371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49a8101cd0 0x7f49a819f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:32.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:32.186+0000 7f49ad33b640 1 --2- 172.21.15.122:0/3954039371 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49a8102690 0x7f49a81a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:32.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:32.186+0000 7f49acb3a640 1 -- 172.21.15.122:0/3954039371 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49a8102690 msgr2=0x7f49a81a15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:32.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:32.186+0000 7f49acb3a640 1 --2- 172.21.15.122:0/3954039371 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49a8102690 0x7f49a81a15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:32.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:32.186+0000 7f49acb3a640 1 -- 172.21.15.122:0/3954039371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49a8101cd0 msgr2=0x7f49a819f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:32.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:32.186+0000 7f49acb3a640 1 --2- 172.21.15.122:0/3954039371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49a8101cd0 0x7f49a819f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:32.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:32.186+0000 7f49acb3a640 1 -- 172.21.15.122:0/3954039371 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49a81a1d90 con 0x7f49a8100ad0 2024-09-16T00:26:32.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:32.186+0000 7f49acb3a640 1 --2- 172.21.15.122:0/3954039371 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49a8100ad0 0x7f49a806aad0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f499c00c970 tx=0x7f499c00ce30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:32.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:32.186+0000 7f49a5ffb640 1 -- 172.21.15.122:0/3954039371 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c004220 con 0x7f49a8100ad0 2024-09-16T00:26:32.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:32.186+0000 7f49a5ffb640 1 -- 172.21.15.122:0/3954039371 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f499c0043c0 con 0x7f49a8100ad0 2024-09-16T00:26:32.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:32.186+0000 7f49aed9e640 1 -- 172.21.15.122:0/3954039371 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49a81a1fc0 con 0x7f49a8100ad0 2024-09-16T00:26:32.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:32.186+0000 7f49a5ffb640 1 -- 172.21.15.122:0/3954039371 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c00f450 con 0x7f49a8100ad0 2024-09-16T00:26:32.209 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:32.187+0000 7f49aed9e640 1 -- 172.21.15.122:0/3954039371 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49a81a2470 con 0x7f49a8100ad0 2024-09-16T00:26:32.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:32.188+0000 7f49aed9e640 1 -- 172.21.15.122:0/3954039371 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49a8100ed0 con 0x7f49a8100ad0 2024-09-16T00:26:32.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:32.189+0000 7f49a5ffb640 1 -- 172.21.15.122:0/3954039371 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f499c0079b0 con 0x7f49a8100ad0 2024-09-16T00:26:32.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:32.190+0000 7f49a5ffb640 1 --2- 172.21.15.122:0/3954039371 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f49980a3270 0x7f49980a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:32.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:32.190+0000 7f49a5ffb640 1 -- 172.21.15.122:0/3954039371 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 3821+0+0 (secure 0 0 0) 0x7f499c0ce530 con 0x7f49a8100ad0 2024-09-16T00:26:32.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:32.190+0000 7f49a7fff640 1 --2- 172.21.15.122:0/3954039371 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f49980a3270 0x7f49980a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:32.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:32.191+0000 7f49a7fff640 1 --2- 172.21.15.122:0/3954039371 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f49980a3270 0x7f49980a5730 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f49940314e0 tx=0x7f4994000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:32.210 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:32.194+0000 7f49a5ffb640 1 -- 172.21.15.122:0/3954039371 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f499c014030 con 0x7f49a8100ad0 2024-09-16T00:26:32.254 INFO:journalctl@ceph.osd.4.smithi073.stdout:Sep 16 00:26:31 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-4[46579]: 2024-09-16T00:26:31.938+0000 7f14dab0a640 -1 osd.4 0 waiting for initial osdmap 2024-09-16T00:26:32.254 INFO:journalctl@ceph.osd.4.smithi073.stdout:Sep 16 00:26:31 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-4[46579]: 2024-09-16T00:26:31.945+0000 7f14d292a640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:26:32.374 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:32 smithi122 ceph-mon[35412]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-09-16T00:26:32.375 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:32 smithi122 ceph-mon[35412]: osdmap e31: 5 total, 4 up, 5 in 2024-09-16T00:26:32.375 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:32 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:26:32.375 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:32 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:26:32.375 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:32 smithi122 ceph-mon[35412]: Detected new or changed devices on smithi073 2024-09-16T00:26:32.375 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:32 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:32.375 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:32 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:32.375 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:32 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:26:32.375 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:32 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:26:32.375 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:32 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:32.375 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:32 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:32.375 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:32 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:26:32.375 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:32 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:32.375 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:32 smithi122 ceph-mon[35412]: from='osd.4 ' entity='osd.4' 2024-09-16T00:26:32.376 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:32.374+0000 7f49aed9e640 1 -- 172.21.15.122:0/3954039371 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f49a8062630 con 0x7f49980a3270 2024-09-16T00:26:32.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:32 smithi073 ceph-mon[35254]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-09-16T00:26:32.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:32 smithi073 ceph-mon[35254]: osdmap e31: 5 total, 4 up, 5 in 2024-09-16T00:26:32.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:32 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:26:32.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:32 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:26:32.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:32 smithi073 ceph-mon[35254]: Detected new or changed devices on smithi073 2024-09-16T00:26:32.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:32 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:32.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:32 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:32.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:32 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:26:32.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:32 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:26:32.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:32 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:32.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:32 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:32.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:32 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:26:32.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:32 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:32.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:32 smithi073 ceph-mon[35254]: from='osd.4 ' entity='osd.4' 2024-09-16T00:26:32.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:32 smithi038 ceph-mon[30945]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-09-16T00:26:32.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:32 smithi038 ceph-mon[30945]: osdmap e31: 5 total, 4 up, 5 in 2024-09-16T00:26:32.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:32 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:26:32.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:32 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:26:32.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:32 smithi038 ceph-mon[30945]: Detected new or changed devices on smithi073 2024-09-16T00:26:32.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:32 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:32.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:32 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:32.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:32 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:26:32.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:32 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:26:32.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:32 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:32.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:32 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:32.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:32 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:26:32.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:32 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:32.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:32 smithi038 ceph-mon[30945]: from='osd.4 ' entity='osd.4' 2024-09-16T00:26:33.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:33 smithi073 ceph-mon[35254]: purged_snaps scrub starts 2024-09-16T00:26:33.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:33 smithi073 ceph-mon[35254]: purged_snaps scrub ok 2024-09-16T00:26:33.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:33 smithi073 ceph-mon[35254]: Adjusting osd_memory_target on smithi073 to 8544M 2024-09-16T00:26:33.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:33 smithi073 ceph-mon[35254]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:26:33.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:33 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:26:33.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:33 smithi073 ceph-mon[35254]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:26:33.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:33 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:26:33.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:33 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:26:33.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:33 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:33.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:33 smithi073 ceph-mon[35254]: osd.4 [v2:172.21.15.73:6808/1469927628,v1:172.21.15.73:6809/1469927628] boot 2024-09-16T00:26:33.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:33 smithi073 ceph-mon[35254]: osdmap e32: 5 total, 5 up, 5 in 2024-09-16T00:26:33.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:33 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:26:33.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:33 smithi038 ceph-mon[30945]: purged_snaps scrub starts 2024-09-16T00:26:33.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:33 smithi038 ceph-mon[30945]: purged_snaps scrub ok 2024-09-16T00:26:33.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:33 smithi038 ceph-mon[30945]: Adjusting osd_memory_target on smithi073 to 8544M 2024-09-16T00:26:33.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:33 smithi038 ceph-mon[30945]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:26:33.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:33 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:26:33.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:33 smithi038 ceph-mon[30945]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:26:33.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:33 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:26:33.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:33 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:26:33.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:33 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:33.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:33 smithi038 ceph-mon[30945]: osd.4 [v2:172.21.15.73:6808/1469927628,v1:172.21.15.73:6809/1469927628] boot 2024-09-16T00:26:33.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:33 smithi038 ceph-mon[30945]: osdmap e32: 5 total, 5 up, 5 in 2024-09-16T00:26:33.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:33 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:26:33.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:33 smithi122 ceph-mon[35412]: purged_snaps scrub starts 2024-09-16T00:26:33.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:33 smithi122 ceph-mon[35412]: purged_snaps scrub ok 2024-09-16T00:26:33.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:33 smithi122 ceph-mon[35412]: Adjusting osd_memory_target on smithi073 to 8544M 2024-09-16T00:26:33.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:33 smithi122 ceph-mon[35412]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-16T00:26:33.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:33 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:26:33.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:33 smithi122 ceph-mon[35412]: from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:26:33.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:33 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:26:33.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:33 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:26:33.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:33 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:33.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:33 smithi122 ceph-mon[35412]: osd.4 [v2:172.21.15.73:6808/1469927628,v1:172.21.15.73:6809/1469927628] boot 2024-09-16T00:26:33.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:33 smithi122 ceph-mon[35412]: osdmap e32: 5 total, 5 up, 5 in 2024-09-16T00:26:33.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:33 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-16T00:26:35.517 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:35 smithi038 ceph-mon[30945]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:26:35.517 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:35 smithi038 ceph-mon[30945]: osdmap e33: 5 total, 5 up, 5 in 2024-09-16T00:26:35.517 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:35 smithi038 ceph-mon[30945]: from='client.? 172.21.15.122:0/1175256458' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3c66d43-dd28-4bb1-a886-62a869be82cb"}]: dispatch 2024-09-16T00:26:35.518 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:35 smithi038 ceph-mon[30945]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3c66d43-dd28-4bb1-a886-62a869be82cb"}]: dispatch 2024-09-16T00:26:35.518 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:35 smithi038 ceph-mon[30945]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e3c66d43-dd28-4bb1-a886-62a869be82cb"}]': finished 2024-09-16T00:26:35.518 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:35 smithi038 ceph-mon[30945]: osdmap e34: 6 total, 5 up, 6 in 2024-09-16T00:26:35.518 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:35 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:26:35.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:35 smithi073 ceph-mon[35254]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:26:35.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:35 smithi073 ceph-mon[35254]: osdmap e33: 5 total, 5 up, 5 in 2024-09-16T00:26:35.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:35 smithi073 ceph-mon[35254]: from='client.? 172.21.15.122:0/1175256458' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3c66d43-dd28-4bb1-a886-62a869be82cb"}]: dispatch 2024-09-16T00:26:35.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:35 smithi073 ceph-mon[35254]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3c66d43-dd28-4bb1-a886-62a869be82cb"}]: dispatch 2024-09-16T00:26:35.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:35 smithi073 ceph-mon[35254]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e3c66d43-dd28-4bb1-a886-62a869be82cb"}]': finished 2024-09-16T00:26:35.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:35 smithi073 ceph-mon[35254]: osdmap e34: 6 total, 5 up, 6 in 2024-09-16T00:26:35.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:35 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:26:35.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:35 smithi122 ceph-mon[35412]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:26:35.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:35 smithi122 ceph-mon[35412]: osdmap e33: 5 total, 5 up, 5 in 2024-09-16T00:26:35.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:35 smithi122 ceph-mon[35412]: from='client.? 172.21.15.122:0/1175256458' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3c66d43-dd28-4bb1-a886-62a869be82cb"}]: dispatch 2024-09-16T00:26:35.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:35 smithi122 ceph-mon[35412]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e3c66d43-dd28-4bb1-a886-62a869be82cb"}]: dispatch 2024-09-16T00:26:35.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:35 smithi122 ceph-mon[35412]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e3c66d43-dd28-4bb1-a886-62a869be82cb"}]': finished 2024-09-16T00:26:35.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:35 smithi122 ceph-mon[35412]: osdmap e34: 6 total, 5 up, 6 in 2024-09-16T00:26:35.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:35 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:26:36.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:36 smithi073 ceph-mon[35254]: from='client.? 172.21.15.122:0/2668473758' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:26:36.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:36 smithi038 ceph-mon[30945]: from='client.? 172.21.15.122:0/2668473758' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:26:36.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:36 smithi122 ceph-mon[35412]: from='client.? 172.21.15.122:0/2668473758' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:26:37.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:37 smithi073 ceph-mon[35254]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:26:37.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:37 smithi038 ceph-mon[30945]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:26:37.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:37 smithi122 ceph-mon[35412]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:26:39.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:39 smithi073 ceph-mon[35254]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:26:39.691 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:39 smithi122 ceph-mon[35412]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:26:39.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:39 smithi038 ceph-mon[30945]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:26:41.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:41 smithi122 ceph-mon[35412]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:26:41.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:41 smithi038 ceph-mon[30945]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:26:41.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:41 smithi073 ceph-mon[35254]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:26:43.565 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:43 smithi122 ceph-mon[35412]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:26:43.565 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:43 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T00:26:43.565 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:43 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:43.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:43 smithi038 ceph-mon[30945]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:26:43.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:43 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T00:26:43.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:43 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:43.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:43 smithi073 ceph-mon[35254]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:26:43.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:43 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-16T00:26:43.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:43 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:44.538 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:44 smithi122 ceph-mon[35412]: Deploying daemon osd.5 on smithi122 2024-09-16T00:26:44.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:44 smithi038 ceph-mon[30945]: Deploying daemon osd.5 on smithi122 2024-09-16T00:26:44.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:44 smithi073 ceph-mon[35254]: Deploying daemon osd.5 on smithi122 2024-09-16T00:26:45.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:45 smithi122 ceph-mon[35412]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:26:45.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:45 smithi038 ceph-mon[30945]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:26:45.797 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:45 smithi073 ceph-mon[35254]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:26:47.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:47 smithi038 ceph-mon[30945]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:26:47.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:47 smithi122 ceph-mon[35412]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:26:47.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:47 smithi073 ceph-mon[35254]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:26:49.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:49 smithi038 ceph-mon[30945]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:26:49.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:49 smithi122 ceph-mon[35412]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:26:49.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:49 smithi073 ceph-mon[35254]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:26:50.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:50 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:26:50.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:50 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:50.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:50 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:50.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:50 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:26:50.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:50 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:50.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:50 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:50.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:50 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:26:50.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:50 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:50.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:50 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:51.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:51 smithi038 ceph-mon[30945]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:26:51.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:51 smithi122 ceph-mon[35412]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:26:51.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:51 smithi073 ceph-mon[35254]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:26:53.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:53 smithi038 ceph-mon[30945]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:26:53.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:53 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:53.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:53 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:53.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:53 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:53.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:53 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:26:53.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:53 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:53.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:53 smithi122 ceph-mon[35412]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:26:53.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:53 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:53.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:53 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:53.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:53 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:53.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:53 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:26:53.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:53 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:53.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:53 smithi073 ceph-mon[35254]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:26:53.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:53 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:53.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:53 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:53.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:53 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:53.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:53 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:26:53.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:53 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:53.550 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:53.550+0000 7f49a5ffb640 1 -- 172.21.15.122:0/3954039371 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f49a8062630 con 0x7f49980a3270 2024-09-16T00:26:53.550 INFO:teuthology.orchestra.run.smithi122.stdout:Created osd(s) 5 on host 'smithi122' 2024-09-16T00:26:53.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:53.552+0000 7f49aed9e640 1 -- 172.21.15.122:0/3954039371 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f49980a3270 msgr2=0x7f49980a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:53.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:53.552+0000 7f49aed9e640 1 --2- 172.21.15.122:0/3954039371 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f49980a3270 0x7f49980a5730 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f49940314e0 tx=0x7f4994000bc0 comp rx=0 tx=0).stop 2024-09-16T00:26:53.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:53.552+0000 7f49aed9e640 1 -- 172.21.15.122:0/3954039371 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49a8100ad0 msgr2=0x7f49a806aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:53.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:53.552+0000 7f49aed9e640 1 --2- 172.21.15.122:0/3954039371 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49a8100ad0 0x7f49a806aad0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f499c00c970 tx=0x7f499c00ce30 comp rx=0 tx=0).stop 2024-09-16T00:26:53.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:53.552+0000 7f49aed9e640 1 -- 172.21.15.122:0/3954039371 shutdown_connections 2024-09-16T00:26:53.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:53.552+0000 7f49aed9e640 1 --2- 172.21.15.122:0/3954039371 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f49980a3270 0x7f49980a5730 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:53.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:53.552+0000 7f49aed9e640 1 --2- 172.21.15.122:0/3954039371 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49a8102690 0x7f49a81a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:53.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:53.552+0000 7f49aed9e640 1 --2- 172.21.15.122:0/3954039371 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f49a8101cd0 0x7f49a819f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:53.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:53.552+0000 7f49aed9e640 1 --2- 172.21.15.122:0/3954039371 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f49a8100ad0 0x7f49a806aad0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:53.553 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:53.552+0000 7f49aed9e640 1 -- 172.21.15.122:0/3954039371 >> 172.21.15.122:0/3954039371 conn(0x7f49a80fc280 msgr2=0x7f49a8105110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:53.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:53.553+0000 7f49aed9e640 1 -- 172.21.15.122:0/3954039371 shutdown_connections 2024-09-16T00:26:53.554 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:26:53.553+0000 7f49aed9e640 1 -- 172.21.15.122:0/3954039371 wait complete. 2024-09-16T00:26:54.207 DEBUG:teuthology.orchestra.run.smithi122:osd.5> sudo journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@osd.5.service 2024-09-16T00:26:54.210 INFO:tasks.cephadm:Deploying osd.6 on smithi038 with /dev/vg_nvme/lv_2... 2024-09-16T00:26:54.210 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-16T00:26:54.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:54 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:26:54.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:54 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:54.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:54 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:54.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:54 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:26:54.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:54 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:54.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:54 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:54.473 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:26:54.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:54 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:26:54.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:54 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:54.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:54 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:55.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:55 smithi038 ceph-mon[30945]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:26:55.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:55 smithi122 ceph-mon[35412]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:26:55.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:55 smithi073 ceph-mon[35254]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:26:55.870 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:26:55 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5[47134]: 2024-09-16T00:26:55.690+0000 7fbb4958f640 -1 osd.5 0 log_to_monitors true 2024-09-16T00:26:56.368 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:56 smithi038 ceph-mon[30945]: from='osd.5 [v2:172.21.15.122:6808/2954413058,v1:172.21.15.122:6809/2954413058]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T00:26:56.368 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:56 smithi038 ceph-mon[30945]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T00:26:56.445 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:56 smithi122 ceph-mon[35412]: from='osd.5 [v2:172.21.15.122:6808/2954413058,v1:172.21.15.122:6809/2954413058]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T00:26:56.446 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:56 smithi122 ceph-mon[35412]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T00:26:56.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:56 smithi073 ceph-mon[35254]: from='osd.5 [v2:172.21.15.122:6808/2954413058,v1:172.21.15.122:6809/2954413058]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T00:26:56.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:56 smithi073 ceph-mon[35254]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-16T00:26:57.269 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:26:57.295 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi038:vg_nvme/lv_2 2024-09-16T00:26:57.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:57 smithi038 ceph-mon[30945]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:26:57.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:57 smithi038 ceph-mon[30945]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T00:26:57.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:57 smithi038 ceph-mon[30945]: osdmap e35: 6 total, 5 up, 6 in 2024-09-16T00:26:57.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:57 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:26:57.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:57 smithi038 ceph-mon[30945]: from='osd.5 [v2:172.21.15.122:6808/2954413058,v1:172.21.15.122:6809/2954413058]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-16T00:26:57.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:57 smithi038 ceph-mon[30945]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-16T00:26:57.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:57 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:57.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:57 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:57.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:57 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:26:57.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:57 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:26:57.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:57 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:57.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:57 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:57.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:57 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:26:57.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:57 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:57.466 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:26:57 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5[47134]: 2024-09-16T00:26:57.212+0000 7fbb45512640 -1 osd.5 0 waiting for initial osdmap 2024-09-16T00:26:57.466 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:26:57 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5[47134]: 2024-09-16T00:26:57.224+0000 7fbb4133a640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:26:57.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:57 smithi122 ceph-mon[35412]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:26:57.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:57 smithi122 ceph-mon[35412]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T00:26:57.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:57 smithi122 ceph-mon[35412]: osdmap e35: 6 total, 5 up, 6 in 2024-09-16T00:26:57.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:57 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:26:57.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:57 smithi122 ceph-mon[35412]: from='osd.5 [v2:172.21.15.122:6808/2954413058,v1:172.21.15.122:6809/2954413058]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-16T00:26:57.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:57 smithi122 ceph-mon[35412]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-16T00:26:57.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:57 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:57.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:57 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:57.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:57 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:26:57.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:57 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:26:57.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:57 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:57.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:57 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:57.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:57 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:26:57.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:57 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:57.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:57 smithi073 ceph-mon[35254]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:26:57.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:57 smithi073 ceph-mon[35254]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-16T00:26:57.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:57 smithi073 ceph-mon[35254]: osdmap e35: 6 total, 5 up, 6 in 2024-09-16T00:26:57.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:57 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:26:57.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:57 smithi073 ceph-mon[35254]: from='osd.5 [v2:172.21.15.122:6808/2954413058,v1:172.21.15.122:6809/2954413058]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-16T00:26:57.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:57 smithi073 ceph-mon[35254]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-16T00:26:57.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:57 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:57.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:57 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:57.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:57 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:26:57.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:57 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:26:57.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:57 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:57.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:57 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:57.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:57 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:26:57.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:57 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:26:57.548 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:26:58.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.038+0000 7f2dfbcc6640 1 -- 172.21.15.38:0/1626156609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2df4100490 msgr2=0x7f2df4100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:58.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.038+0000 7f2dfbcc6640 1 --2- 172.21.15.38:0/1626156609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2df4100490 0x7f2df4100890 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2df0009900 tx=0x7f2df002a410 comp rx=0 tx=0).stop 2024-09-16T00:26:58.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.038+0000 7f2dfbcc6640 1 -- 172.21.15.38:0/1626156609 shutdown_connections 2024-09-16T00:26:58.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.038+0000 7f2dfbcc6640 1 --2- 172.21.15.38:0/1626156609 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2df4101770 0x7f2df4101c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:58.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.038+0000 7f2dfbcc6640 1 --2- 172.21.15.38:0/1626156609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2df4100dd0 0x7f2df4101230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:58.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.038+0000 7f2dfbcc6640 1 --2- 172.21.15.38:0/1626156609 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2df4100490 0x7f2df4100890 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:58.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.038+0000 7f2dfbcc6640 1 -- 172.21.15.38:0/1626156609 >> 172.21.15.38:0/1626156609 conn(0x7f2df40fc2a0 msgr2=0x7f2df40fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:26:58.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.039+0000 7f2dfbcc6640 1 -- 172.21.15.38:0/1626156609 shutdown_connections 2024-09-16T00:26:58.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.039+0000 7f2dfbcc6640 1 -- 172.21.15.38:0/1626156609 wait complete. 2024-09-16T00:26:58.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.039+0000 7f2dfbcc6640 1 Processor -- start 2024-09-16T00:26:58.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.039+0000 7f2dfbcc6640 1 -- start start 2024-09-16T00:26:58.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.039+0000 7f2dfbcc6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2df4100490 0x7f2df419a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:58.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.039+0000 7f2dfbcc6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2df4100dd0 0x7f2df419a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:58.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.039+0000 7f2dfbcc6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2df4101770 0x7f2df41a1690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:58.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.039+0000 7f2dfbcc6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2df410fae0 con 0x7f2df4100490 2024-09-16T00:26:58.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.039+0000 7f2dfbcc6640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2df410f960 con 0x7f2df4101770 2024-09-16T00:26:58.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.039+0000 7f2dfbcc6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2df410fc60 con 0x7f2df4100dd0 2024-09-16T00:26:58.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.039+0000 7f2df9a62640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2df4100490 0x7f2df419a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:58.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.039+0000 7f2df9a62640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2df4100490 0x7f2df419a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50584/0 (socket says 172.21.15.38:50584) 2024-09-16T00:26:58.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.039+0000 7f2df9a62640 1 -- 172.21.15.38:0/4089996770 learned_addr learned my addr 172.21.15.38:0/4089996770 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:26:58.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.040+0000 7f2dfa263640 1 --2- 172.21.15.38:0/4089996770 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2df4101770 0x7f2df41a1690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:58.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.040+0000 7f2df9261640 1 --2- 172.21.15.38:0/4089996770 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2df4100dd0 0x7f2df419a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:58.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.040+0000 7f2df9a62640 1 -- 172.21.15.38:0/4089996770 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2df4100dd0 msgr2=0x7f2df419a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:58.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.040+0000 7f2df9a62640 1 --2- 172.21.15.38:0/4089996770 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2df4100dd0 0x7f2df419a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:58.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.040+0000 7f2df9a62640 1 -- 172.21.15.38:0/4089996770 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2df4101770 msgr2=0x7f2df41a1690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:26:58.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.040+0000 7f2df9a62640 1 --2- 172.21.15.38:0/4089996770 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2df4101770 0x7f2df41a1690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:26:58.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.040+0000 7f2df9a62640 1 -- 172.21.15.38:0/4089996770 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df41a1d00 con 0x7f2df4100490 2024-09-16T00:26:58.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.040+0000 7f2df9261640 1 --2- 172.21.15.38:0/4089996770 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2df4100dd0 0x7f2df419a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:26:58.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.040+0000 7f2dfa263640 1 --2- 172.21.15.38:0/4089996770 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2df4101770 0x7f2df41a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:26:58.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.040+0000 7f2df9a62640 1 --2- 172.21.15.38:0/4089996770 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2df4100490 0x7f2df419a0d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2df0009230 tx=0x7f2df0004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:58.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.040+0000 7f2deaffd640 1 -- 172.21.15.38:0/4089996770 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df003f030 con 0x7f2df4100490 2024-09-16T00:26:58.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.040+0000 7f2deaffd640 1 -- 172.21.15.38:0/4089996770 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2df0003ce0 con 0x7f2df4100490 2024-09-16T00:26:58.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.040+0000 7f2dfbcc6640 1 -- 172.21.15.38:0/4089996770 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2df41a1f90 con 0x7f2df4100490 2024-09-16T00:26:58.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.040+0000 7f2deaffd640 1 -- 172.21.15.38:0/4089996770 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2df0035070 con 0x7f2df4100490 2024-09-16T00:26:58.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.040+0000 7f2dfbcc6640 1 -- 172.21.15.38:0/4089996770 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2df41a2410 con 0x7f2df4100490 2024-09-16T00:26:58.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.041+0000 7f2dfbcc6640 1 -- 172.21.15.38:0/4089996770 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2df410b1e0 con 0x7f2df4100490 2024-09-16T00:26:58.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.042+0000 7f2deaffd640 1 -- 172.21.15.38:0/4089996770 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f2df0042030 con 0x7f2df4100490 2024-09-16T00:26:58.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.043+0000 7f2deaffd640 1 --2- 172.21.15.38:0/4089996770 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f2de00a32f0 0x7f2de00a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:26:58.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.043+0000 7f2df9261640 1 --2- 172.21.15.38:0/4089996770 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f2de00a32f0 0x7f2de00a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:26:58.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.043+0000 7f2deaffd640 1 -- 172.21.15.38:0/4089996770 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4253+0+0 (secure 0 0 0) 0x7f2df00ef580 con 0x7f2df4100490 2024-09-16T00:26:58.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.044+0000 7f2df9261640 1 --2- 172.21.15.38:0/4089996770 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f2de00a32f0 0x7f2de00a57b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2df419b5f0 tx=0x7f2de4005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:26:58.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.046+0000 7f2deaffd640 1 -- 172.21.15.38:0/4089996770 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2df00bd1e0 con 0x7f2df4100490 2024-09-16T00:26:58.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:26:58.233+0000 7f2dfbcc6640 1 -- 172.21.15.38:0/4089996770 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f2df4062630 con 0x7f2de00a32f0 2024-09-16T00:26:58.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:58 smithi073 ceph-mon[35254]: Detected new or changed devices on smithi122 2024-09-16T00:26:58.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:58 smithi073 ceph-mon[35254]: Adjusting osd_memory_target on smithi122 to 8543M 2024-09-16T00:26:58.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:58 smithi073 ceph-mon[35254]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-09-16T00:26:58.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:58 smithi073 ceph-mon[35254]: osdmap e36: 6 total, 5 up, 6 in 2024-09-16T00:26:58.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:58 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:26:58.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:58 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:26:58.649 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:58 smithi038 ceph-mon[30945]: Detected new or changed devices on smithi122 2024-09-16T00:26:58.649 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:58 smithi038 ceph-mon[30945]: Adjusting osd_memory_target on smithi122 to 8543M 2024-09-16T00:26:58.649 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:58 smithi038 ceph-mon[30945]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-09-16T00:26:58.649 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:58 smithi038 ceph-mon[30945]: osdmap e36: 6 total, 5 up, 6 in 2024-09-16T00:26:58.649 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:58 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:26:58.649 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:58 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:26:58.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:58 smithi122 ceph-mon[35412]: Detected new or changed devices on smithi122 2024-09-16T00:26:58.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:58 smithi122 ceph-mon[35412]: Adjusting osd_memory_target on smithi122 to 8543M 2024-09-16T00:26:58.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:58 smithi122 ceph-mon[35412]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-09-16T00:26:58.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:58 smithi122 ceph-mon[35412]: osdmap e36: 6 total, 5 up, 6 in 2024-09-16T00:26:58.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:58 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:26:58.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:58 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:26:59.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:59 smithi073 ceph-mon[35254]: purged_snaps scrub starts 2024-09-16T00:26:59.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:59 smithi073 ceph-mon[35254]: purged_snaps scrub ok 2024-09-16T00:26:59.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:59 smithi073 ceph-mon[35254]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:26:59.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:59 smithi073 ceph-mon[35254]: osd.5 [v2:172.21.15.122:6808/2954413058,v1:172.21.15.122:6809/2954413058] boot 2024-09-16T00:26:59.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:59 smithi073 ceph-mon[35254]: osdmap e37: 6 total, 6 up, 6 in 2024-09-16T00:26:59.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:59 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:26:59.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:59 smithi073 ceph-mon[35254]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:26:59.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:59 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:26:59.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:59 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:26:59.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:59 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:59.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:26:59 smithi073 ceph-mon[35254]: osdmap e38: 6 total, 6 up, 6 in 2024-09-16T00:26:59.670 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:59 smithi038 ceph-mon[30945]: purged_snaps scrub starts 2024-09-16T00:26:59.670 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:59 smithi038 ceph-mon[30945]: purged_snaps scrub ok 2024-09-16T00:26:59.670 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:59 smithi038 ceph-mon[30945]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:26:59.671 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:59 smithi038 ceph-mon[30945]: osd.5 [v2:172.21.15.122:6808/2954413058,v1:172.21.15.122:6809/2954413058] boot 2024-09-16T00:26:59.671 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:59 smithi038 ceph-mon[30945]: osdmap e37: 6 total, 6 up, 6 in 2024-09-16T00:26:59.671 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:59 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:26:59.671 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:59 smithi038 ceph-mon[30945]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:26:59.671 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:59 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:26:59.671 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:59 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:26:59.671 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:59 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:59.671 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:26:59 smithi038 ceph-mon[30945]: osdmap e38: 6 total, 6 up, 6 in 2024-09-16T00:26:59.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:59 smithi122 ceph-mon[35412]: purged_snaps scrub starts 2024-09-16T00:26:59.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:59 smithi122 ceph-mon[35412]: purged_snaps scrub ok 2024-09-16T00:26:59.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:59 smithi122 ceph-mon[35412]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 147 MiB used, 447 GiB / 447 GiB avail 2024-09-16T00:26:59.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:59 smithi122 ceph-mon[35412]: osd.5 [v2:172.21.15.122:6808/2954413058,v1:172.21.15.122:6809/2954413058] boot 2024-09-16T00:26:59.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:59 smithi122 ceph-mon[35412]: osdmap e37: 6 total, 6 up, 6 in 2024-09-16T00:26:59.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:59 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-16T00:26:59.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:59 smithi122 ceph-mon[35412]: from='client.14352 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:26:59.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:59 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:26:59.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:59 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:26:59.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:59 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:26:59.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:26:59 smithi122 ceph-mon[35412]: osdmap e38: 6 total, 6 up, 6 in 2024-09-16T00:27:01.457 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:01 smithi038 ceph-mon[30945]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:27:01.457 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:01 smithi038 ceph-mon[30945]: osdmap e39: 6 total, 6 up, 6 in 2024-09-16T00:27:01.457 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:01 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/147134839' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "211497ac-7c48-4095-bfdb-1d4df0149f57"}]: dispatch 2024-09-16T00:27:01.457 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:01 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/147134839' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "211497ac-7c48-4095-bfdb-1d4df0149f57"}]': finished 2024-09-16T00:27:01.457 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:01 smithi038 ceph-mon[30945]: osdmap e40: 7 total, 6 up, 7 in 2024-09-16T00:27:01.457 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:01 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:27:01.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:01 smithi073 ceph-mon[35254]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:27:01.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:01 smithi073 ceph-mon[35254]: osdmap e39: 6 total, 6 up, 6 in 2024-09-16T00:27:01.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:01 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/147134839' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "211497ac-7c48-4095-bfdb-1d4df0149f57"}]: dispatch 2024-09-16T00:27:01.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:01 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/147134839' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "211497ac-7c48-4095-bfdb-1d4df0149f57"}]': finished 2024-09-16T00:27:01.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:01 smithi073 ceph-mon[35254]: osdmap e40: 7 total, 6 up, 7 in 2024-09-16T00:27:01.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:01 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:27:01.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:01 smithi122 ceph-mon[35412]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:27:01.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:01 smithi122 ceph-mon[35412]: osdmap e39: 6 total, 6 up, 6 in 2024-09-16T00:27:01.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:01 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/147134839' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "211497ac-7c48-4095-bfdb-1d4df0149f57"}]: dispatch 2024-09-16T00:27:01.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:01 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/147134839' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "211497ac-7c48-4095-bfdb-1d4df0149f57"}]': finished 2024-09-16T00:27:01.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:01 smithi122 ceph-mon[35412]: osdmap e40: 7 total, 6 up, 7 in 2024-09-16T00:27:01.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:01 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:27:02.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:02 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/1822767437' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:27:02.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:02 smithi073 ceph-mon[35254]: osdmap e41: 7 total, 6 up, 7 in 2024-09-16T00:27:02.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:02 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:27:02.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:02 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/1822767437' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:27:02.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:02 smithi038 ceph-mon[30945]: osdmap e41: 7 total, 6 up, 7 in 2024-09-16T00:27:02.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:02 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:27:02.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:02 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/1822767437' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:27:02.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:02 smithi122 ceph-mon[35412]: osdmap e41: 7 total, 6 up, 7 in 2024-09-16T00:27:02.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:02 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:27:03.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:03 smithi073 ceph-mon[35254]: pgmap v140: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-16T00:27:03.618 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:03 smithi122 ceph-mon[35412]: pgmap v140: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-16T00:27:03.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:03 smithi038 ceph-mon[30945]: pgmap v140: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-09-16T00:27:05.512 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:05 smithi038 ceph-mon[30945]: pgmap v142: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-09-16T00:27:05.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:05 smithi073 ceph-mon[35254]: pgmap v142: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-09-16T00:27:05.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:05 smithi122 ceph-mon[35412]: pgmap v142: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 135 KiB/s, 0 objects/s recovering 2024-09-16T00:27:07.421 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:07 smithi038 ceph-mon[30945]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T00:27:07.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:07 smithi073 ceph-mon[35254]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T00:27:07.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:07 smithi122 ceph-mon[35412]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 175 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T00:27:09.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:09 smithi073 ceph-mon[35254]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-16T00:27:09.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:09 smithi038 ceph-mon[30945]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-16T00:27:09.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:09 smithi122 ceph-mon[35412]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-16T00:27:10.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:10 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-16T00:27:10.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:10 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:27:10.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:10 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-16T00:27:10.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:10 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:27:10.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:10 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-16T00:27:10.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:10 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:27:11.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:11 smithi073 ceph-mon[35254]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:27:11.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:11 smithi073 ceph-mon[35254]: Deploying daemon osd.6 on smithi038 2024-09-16T00:27:11.576 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:11 smithi038 ceph-mon[30945]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:27:11.577 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:11 smithi038 ceph-mon[30945]: Deploying daemon osd.6 on smithi038 2024-09-16T00:27:11.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:11 smithi122 ceph-mon[35412]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:27:11.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:11 smithi122 ceph-mon[35412]: Deploying daemon osd.6 on smithi038 2024-09-16T00:27:13.361 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:13 smithi038 ceph-mon[30945]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:27:13.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:13 smithi073 ceph-mon[35254]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:27:13.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:13 smithi122 ceph-mon[35412]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:27:15.532 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:15 smithi038 ceph-mon[30945]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:27:15.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:15 smithi073 ceph-mon[35254]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:27:15.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:15 smithi122 ceph-mon[35412]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:27:16.531 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:16 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:27:16.531 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:16 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:16.532 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:16 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:16.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:16 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:27:16.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:16 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:16.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:16 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:16.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:16 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:27:16.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:16 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:16.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:16 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:17.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:17 smithi073 ceph-mon[35254]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:27:17.635 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:17 smithi038 ceph-mon[30945]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:27:17.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:17 smithi122 ceph-mon[35412]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:27:19.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:19 smithi038 ceph-mon[30945]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:27:19.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:19 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:19.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:19 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:19.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:19 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:27:19.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:19 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:27:19.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:19 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:19.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:19 smithi073 ceph-mon[35254]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:27:19.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:19 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:19.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:19 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:19.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:19 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:27:19.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:19 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:27:19.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:19 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:19.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:19 smithi122 ceph-mon[35412]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:27:19.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:19 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:19.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:19 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:19.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:19 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:27:19.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:19 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:27:19.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:19 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:19.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:27:19.880+0000 7f2deaffd640 1 -- 172.21.15.38:0/4089996770 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f2df4062630 con 0x7f2de00a32f0 2024-09-16T00:27:19.881 INFO:teuthology.orchestra.run.smithi038.stdout:Created osd(s) 6 on host 'smithi038' 2024-09-16T00:27:19.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:27:19.883+0000 7f2dfbcc6640 1 -- 172.21.15.38:0/4089996770 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f2de00a32f0 msgr2=0x7f2de00a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:19.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:27:19.883+0000 7f2dfbcc6640 1 --2- 172.21.15.38:0/4089996770 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f2de00a32f0 0x7f2de00a57b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2df419b5f0 tx=0x7f2de4005d90 comp rx=0 tx=0).stop 2024-09-16T00:27:19.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:27:19.883+0000 7f2dfbcc6640 1 -- 172.21.15.38:0/4089996770 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2df4100490 msgr2=0x7f2df419a0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:19.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:27:19.883+0000 7f2dfbcc6640 1 --2- 172.21.15.38:0/4089996770 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2df4100490 0x7f2df419a0d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f2df0009230 tx=0x7f2df0004430 comp rx=0 tx=0).stop 2024-09-16T00:27:19.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:27:19.883+0000 7f2dfbcc6640 1 -- 172.21.15.38:0/4089996770 shutdown_connections 2024-09-16T00:27:19.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:27:19.883+0000 7f2dfbcc6640 1 --2- 172.21.15.38:0/4089996770 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f2de00a32f0 0x7f2de00a57b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:19.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:27:19.883+0000 7f2dfbcc6640 1 --2- 172.21.15.38:0/4089996770 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2df4101770 0x7f2df41a1690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:19.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:27:19.883+0000 7f2dfbcc6640 1 --2- 172.21.15.38:0/4089996770 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2df4100dd0 0x7f2df419a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:19.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:27:19.883+0000 7f2dfbcc6640 1 --2- 172.21.15.38:0/4089996770 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2df4100490 0x7f2df419a0d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:19.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:27:19.883+0000 7f2dfbcc6640 1 -- 172.21.15.38:0/4089996770 >> 172.21.15.38:0/4089996770 conn(0x7f2df40fc2a0 msgr2=0x7f2df40fdd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:19.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:27:19.883+0000 7f2dfbcc6640 1 -- 172.21.15.38:0/4089996770 shutdown_connections 2024-09-16T00:27:19.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:27:19.883+0000 7f2dfbcc6640 1 -- 172.21.15.38:0/4089996770 wait complete. 2024-09-16T00:27:20.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:20 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:27:20.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:20 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:20.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:20 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:20.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:20 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:27:20.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:20 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:20.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:20 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:20.669 DEBUG:teuthology.orchestra.run.smithi038:osd.6> sudo journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@osd.6.service 2024-09-16T00:27:20.672 INFO:tasks.cephadm:Deploying osd.7 on smithi073 with /dev/vg_nvme/lv_2... 2024-09-16T00:27:20.672 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-16T00:27:20.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:20 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:27:20.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:20 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:20.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:20 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:20.923 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:27:21.329 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:21 smithi073 ceph-mon[35254]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:27:21.567 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:21 smithi038 ceph-mon[30945]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:27:21.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:21 smithi122 ceph-mon[35412]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:27:21.965 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:27:21 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6[58162]: 2024-09-16T00:27:21.733+0000 7fdd282b8640 -1 osd.6 0 log_to_monitors true 2024-09-16T00:27:22.392 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:22 smithi073 ceph-mon[35254]: from='osd.6 [v2:172.21.15.38:6818/1415719550,v1:172.21.15.38:6819/1415719550]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T00:27:22.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:22 smithi038 ceph-mon[30945]: from='osd.6 [v2:172.21.15.38:6818/1415719550,v1:172.21.15.38:6819/1415719550]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T00:27:22.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:22 smithi122 ceph-mon[35412]: from='osd.6 [v2:172.21.15.38:6818/1415719550,v1:172.21.15.38:6819/1415719550]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-16T00:27:23.703 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:23 smithi038 ceph-mon[30945]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:27:23.703 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:23 smithi038 ceph-mon[30945]: from='osd.6 [v2:172.21.15.38:6818/1415719550,v1:172.21.15.38:6819/1415719550]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-16T00:27:23.704 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:23 smithi038 ceph-mon[30945]: osdmap e42: 7 total, 6 up, 7 in 2024-09-16T00:27:23.704 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:23 smithi038 ceph-mon[30945]: from='osd.6 [v2:172.21.15.38:6818/1415719550,v1:172.21.15.38:6819/1415719550]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-16T00:27:23.704 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:23 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:27:23.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:23 smithi122 ceph-mon[35412]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:27:23.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:23 smithi122 ceph-mon[35412]: from='osd.6 [v2:172.21.15.38:6818/1415719550,v1:172.21.15.38:6819/1415719550]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-16T00:27:23.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:23 smithi122 ceph-mon[35412]: osdmap e42: 7 total, 6 up, 7 in 2024-09-16T00:27:23.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:23 smithi122 ceph-mon[35412]: from='osd.6 [v2:172.21.15.38:6818/1415719550,v1:172.21.15.38:6819/1415719550]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-16T00:27:23.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:23 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:27:23.758 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:23 smithi073 ceph-mon[35254]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:27:23.758 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:23 smithi073 ceph-mon[35254]: from='osd.6 [v2:172.21.15.38:6818/1415719550,v1:172.21.15.38:6819/1415719550]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-16T00:27:23.758 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:23 smithi073 ceph-mon[35254]: osdmap e42: 7 total, 6 up, 7 in 2024-09-16T00:27:23.758 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:23 smithi073 ceph-mon[35254]: from='osd.6 [v2:172.21.15.38:6818/1415719550,v1:172.21.15.38:6819/1415719550]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-16T00:27:23.758 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:23 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:27:23.760 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-16T00:27:23.788 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi073:vg_nvme/lv_2 2024-09-16T00:27:23.965 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:27:23 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6[58162]: 2024-09-16T00:27:23.857+0000 7fdd2423b640 -1 osd.6 0 waiting for initial osdmap 2024-09-16T00:27:23.965 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:27:23 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6[58162]: 2024-09-16T00:27:23.867+0000 7fdd1c05b640 -1 osd.6 43 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:27:24.040 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:27:24.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:24 smithi073 ceph-mon[35254]: from='osd.6 [v2:172.21.15.38:6818/1415719550,v1:172.21.15.38:6819/1415719550]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-16T00:27:24.469 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:24 smithi073 ceph-mon[35254]: osdmap e43: 7 total, 6 up, 7 in 2024-09-16T00:27:24.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:24 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:27:24.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:24 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:27:24.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:24 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:24.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:24 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:24.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:24 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:27:24.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:24 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:27:24.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:24 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:27:24.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:24 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:24.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:24 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:27:24.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:24 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:27:24.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:24 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:24.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:24 smithi073 ceph-mon[35254]: from='osd.6 [v2:172.21.15.38:6818/1415719550,v1:172.21.15.38:6819/1415719550]' entity='osd.6' 2024-09-16T00:27:24.470 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:24 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:27:24.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:24.551+0000 7f594230c640 1 -- 172.21.15.73:0/81506475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f593c104f90 msgr2=0x7f593c107380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:24.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:24.551+0000 7f594230c640 1 --2- 172.21.15.73:0/81506475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f593c104f90 0x7f593c107380 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5930009900 tx=0x7f593002a410 comp rx=0 tx=0).stop 2024-09-16T00:27:24.578 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:24.551+0000 7f594230c640 1 -- 172.21.15.73:0/81506475 shutdown_connections 2024-09-16T00:27:24.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:24.551+0000 7f594230c640 1 --2- 172.21.15.73:0/81506475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f593c1078c0 0x7f593c109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:24.551+0000 7f594230c640 1 --2- 172.21.15.73:0/81506475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f593c104f90 0x7f593c107380 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:24.551+0000 7f594230c640 1 --2- 172.21.15.73:0/81506475 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f593c102660 0x7f593c104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:24.551+0000 7f594230c640 1 -- 172.21.15.73:0/81506475 >> 172.21.15.73:0/81506475 conn(0x7f593c0fc280 msgr2=0x7f593c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:24.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:24.551+0000 7f594230c640 1 -- 172.21.15.73:0/81506475 shutdown_connections 2024-09-16T00:27:24.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:24.551+0000 7f594230c640 1 -- 172.21.15.73:0/81506475 wait complete. 2024-09-16T00:27:24.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:24.551+0000 7f594230c640 1 Processor -- start 2024-09-16T00:27:24.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:24.551+0000 7f594230c640 1 -- start start 2024-09-16T00:27:24.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:24.552+0000 7f594230c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f593c102660 0x7f593c101fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:24.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:24.552+0000 7f594230c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f593c104f90 0x7f593c100610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:24.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:24.552+0000 7f594230c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f593c1078c0 0x7f593c100b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:24.579 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:24.552+0000 7f594230c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f593c10d860 con 0x7f593c104f90 2024-09-16T00:27:24.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:24.552+0000 7f594230c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f593c10d6e0 con 0x7f593c1078c0 2024-09-16T00:27:24.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:24.552+0000 7f594230c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f593c10d9e0 con 0x7f593c102660 2024-09-16T00:27:24.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:24.552+0000 7f59408a9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f593c1078c0 0x7f593c100b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:24.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:24.552+0000 7f59408a9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f593c1078c0 0x7f593c100b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37052/0 (socket says 172.21.15.73:37052) 2024-09-16T00:27:24.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:24.552+0000 7f59408a9640 1 -- 172.21.15.73:0/1652970753 learned_addr learned my addr 172.21.15.73:0/1652970753 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-09-16T00:27:24.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:24.552+0000 7f593bfff640 1 --2- 172.21.15.73:0/1652970753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f593c102660 0x7f593c101fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:24.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:24.552+0000 7f593b7fe640 1 --2- 172.21.15.73:0/1652970753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f593c104f90 0x7f593c100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:24.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:24.552+0000 7f59408a9640 1 -- 172.21.15.73:0/1652970753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f593c102660 msgr2=0x7f593c101fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:24.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:24.552+0000 7f59408a9640 1 --2- 172.21.15.73:0/1652970753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f593c102660 0x7f593c101fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:24.552+0000 7f59408a9640 1 -- 172.21.15.73:0/1652970753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f593c104f90 msgr2=0x7f593c100610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:24.580 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:24.552+0000 7f59408a9640 1 --2- 172.21.15.73:0/1652970753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f593c104f90 0x7f593c100610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:24.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:24.552+0000 7f59408a9640 1 -- 172.21.15.73:0/1652970753 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f593c101410 con 0x7f593c1078c0 2024-09-16T00:27:24.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:24.552+0000 7f59408a9640 1 --2- 172.21.15.73:0/1652970753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f593c1078c0 0x7f593c100b50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5934009fd0 tx=0x7f593400b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:24.581 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:24.552+0000 7f59397fa640 1 -- 172.21.15.73:0/1652970753 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5934017070 con 0x7f593c1078c0 2024-09-16T00:27:24.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:24.552+0000 7f594230c640 1 -- 172.21.15.73:0/1652970753 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f593c1a2120 con 0x7f593c1078c0 2024-09-16T00:27:24.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:24.552+0000 7f594230c640 1 -- 172.21.15.73:0/1652970753 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f593c1a2630 con 0x7f593c1078c0 2024-09-16T00:27:24.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:24.552+0000 7f59397fa640 1 -- 172.21.15.73:0/1652970753 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5934003e50 con 0x7f593c1078c0 2024-09-16T00:27:24.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:24.552+0000 7f59397fa640 1 -- 172.21.15.73:0/1652970753 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5934020060 con 0x7f593c1078c0 2024-09-16T00:27:24.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:24.553+0000 7f594230c640 1 -- 172.21.15.73:0/1652970753 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59280052c0 con 0x7f593c1078c0 2024-09-16T00:27:24.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:24.554+0000 7f59397fa640 1 -- 172.21.15.73:0/1652970753 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f5934003970 con 0x7f593c1078c0 2024-09-16T00:27:24.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:24.555+0000 7f59397fa640 1 --2- 172.21.15.73:0/1652970753 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f59240a3240 0x7f59240a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:24.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:24.556+0000 7f59397fa640 1 -- 172.21.15.73:0/1652970753 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4685+0+0 (secure 0 0 0) 0x7f59340ce6d0 con 0x7f593c1078c0 2024-09-16T00:27:24.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:24.556+0000 7f593bfff640 1 --2- 172.21.15.73:0/1652970753 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f59240a3240 0x7f59240a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:24.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:24.556+0000 7f593bfff640 1 --2- 172.21.15.73:0/1652970753 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f59240a3240 0x7f59240a5700 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f592c005fd0 tx=0x7f592c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:24.582 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:24.559+0000 7f59397fa640 1 -- 172.21.15.73:0/1652970753 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f593409bbb0 con 0x7f593c1078c0 2024-09-16T00:27:24.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:24 smithi122 ceph-mon[35412]: from='osd.6 [v2:172.21.15.38:6818/1415719550,v1:172.21.15.38:6819/1415719550]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-16T00:27:24.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:24 smithi122 ceph-mon[35412]: osdmap e43: 7 total, 6 up, 7 in 2024-09-16T00:27:24.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:24 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:27:24.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:24 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:27:24.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:24 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:24.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:24 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:24.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:24 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:27:24.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:24 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:27:24.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:24 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:27:24.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:24 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:24.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:24 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:27:24.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:24 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:27:24.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:24 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:24.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:24 smithi122 ceph-mon[35412]: from='osd.6 [v2:172.21.15.38:6818/1415719550,v1:172.21.15.38:6819/1415719550]' entity='osd.6' 2024-09-16T00:27:24.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:24 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:27:24.719 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:24 smithi038 ceph-mon[30945]: from='osd.6 [v2:172.21.15.38:6818/1415719550,v1:172.21.15.38:6819/1415719550]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-16T00:27:24.719 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:24 smithi038 ceph-mon[30945]: osdmap e43: 7 total, 6 up, 7 in 2024-09-16T00:27:24.719 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:24 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:27:24.719 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:24 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:27:24.719 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:24 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:24.719 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:24 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:24.719 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:24 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:27:24.719 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:24 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:27:24.719 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:24 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:27:24.719 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:24 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:24.719 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:24 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:27:24.719 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:24 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:27:24.719 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:24 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:24.719 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:24 smithi038 ceph-mon[30945]: from='osd.6 [v2:172.21.15.38:6818/1415719550,v1:172.21.15.38:6819/1415719550]' entity='osd.6' 2024-09-16T00:27:24.719 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:24 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:27:24.732 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:24.732+0000 7f594230c640 1 -- 172.21.15.73:0/1652970753 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f5928000bf0 con 0x7f59240a3240 2024-09-16T00:27:25.447 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:25 smithi073 ceph-mon[35254]: purged_snaps scrub starts 2024-09-16T00:27:25.448 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:25 smithi073 ceph-mon[35254]: purged_snaps scrub ok 2024-09-16T00:27:25.448 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:25 smithi073 ceph-mon[35254]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:27:25.448 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:25 smithi073 ceph-mon[35254]: Detected new or changed devices on smithi038 2024-09-16T00:27:25.448 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:25 smithi073 ceph-mon[35254]: Adjusting osd_memory_target on smithi038 to 5697M 2024-09-16T00:27:25.448 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:25 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:27:25.448 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:25 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:27:25.448 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:25 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:27:25.448 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:25 smithi073 ceph-mon[35254]: osd.6 [v2:172.21.15.38:6818/1415719550,v1:172.21.15.38:6819/1415719550] boot 2024-09-16T00:27:25.448 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:25 smithi073 ceph-mon[35254]: osdmap e44: 7 total, 7 up, 7 in 2024-09-16T00:27:25.448 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:25 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:27:25.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:25 smithi038 ceph-mon[30945]: purged_snaps scrub starts 2024-09-16T00:27:25.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:25 smithi038 ceph-mon[30945]: purged_snaps scrub ok 2024-09-16T00:27:25.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:25 smithi038 ceph-mon[30945]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:27:25.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:25 smithi038 ceph-mon[30945]: Detected new or changed devices on smithi038 2024-09-16T00:27:25.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:25 smithi038 ceph-mon[30945]: Adjusting osd_memory_target on smithi038 to 5697M 2024-09-16T00:27:25.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:25 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:27:25.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:25 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:27:25.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:25 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:27:25.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:25 smithi038 ceph-mon[30945]: osd.6 [v2:172.21.15.38:6818/1415719550,v1:172.21.15.38:6819/1415719550] boot 2024-09-16T00:27:25.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:25 smithi038 ceph-mon[30945]: osdmap e44: 7 total, 7 up, 7 in 2024-09-16T00:27:25.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:25 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:27:25.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:25 smithi122 ceph-mon[35412]: purged_snaps scrub starts 2024-09-16T00:27:25.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:25 smithi122 ceph-mon[35412]: purged_snaps scrub ok 2024-09-16T00:27:25.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:25 smithi122 ceph-mon[35412]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 174 MiB used, 536 GiB / 536 GiB avail 2024-09-16T00:27:25.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:25 smithi122 ceph-mon[35412]: Detected new or changed devices on smithi038 2024-09-16T00:27:25.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:25 smithi122 ceph-mon[35412]: Adjusting osd_memory_target on smithi038 to 5697M 2024-09-16T00:27:25.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:25 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:27:25.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:25 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:27:25.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:25 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:27:25.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:25 smithi122 ceph-mon[35412]: osd.6 [v2:172.21.15.38:6818/1415719550,v1:172.21.15.38:6819/1415719550] boot 2024-09-16T00:27:25.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:25 smithi122 ceph-mon[35412]: osdmap e44: 7 total, 7 up, 7 in 2024-09-16T00:27:25.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:25 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-16T00:27:26.451 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:26 smithi073 ceph-mon[35254]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:27:26.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:26 smithi038 ceph-mon[30945]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:27:26.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:26 smithi122 ceph-mon[35412]: from='client.24272 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:27:27.670 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:27 smithi073 ceph-mon[35254]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:27:27.670 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:27 smithi073 ceph-mon[35254]: osdmap e45: 7 total, 7 up, 7 in 2024-09-16T00:27:27.670 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:27 smithi073 ceph-mon[35254]: from='client.? 172.21.15.73:0/2530995146' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06512315-3f5e-44f8-976d-d3f5f1ba00e3"}]: dispatch 2024-09-16T00:27:27.670 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:27 smithi073 ceph-mon[35254]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06512315-3f5e-44f8-976d-d3f5f1ba00e3"}]: dispatch 2024-09-16T00:27:27.671 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:27 smithi073 ceph-mon[35254]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06512315-3f5e-44f8-976d-d3f5f1ba00e3"}]': finished 2024-09-16T00:27:27.671 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:27 smithi073 ceph-mon[35254]: osdmap e46: 8 total, 7 up, 8 in 2024-09-16T00:27:27.671 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:27 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:27:27.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:27 smithi038 ceph-mon[30945]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:27:27.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:27 smithi038 ceph-mon[30945]: osdmap e45: 7 total, 7 up, 7 in 2024-09-16T00:27:27.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:27 smithi038 ceph-mon[30945]: from='client.? 172.21.15.73:0/2530995146' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06512315-3f5e-44f8-976d-d3f5f1ba00e3"}]: dispatch 2024-09-16T00:27:27.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:27 smithi038 ceph-mon[30945]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06512315-3f5e-44f8-976d-d3f5f1ba00e3"}]: dispatch 2024-09-16T00:27:27.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:27 smithi038 ceph-mon[30945]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06512315-3f5e-44f8-976d-d3f5f1ba00e3"}]': finished 2024-09-16T00:27:27.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:27 smithi038 ceph-mon[30945]: osdmap e46: 8 total, 7 up, 8 in 2024-09-16T00:27:27.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:27 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:27:27.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:27 smithi122 ceph-mon[35412]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:27:27.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:27 smithi122 ceph-mon[35412]: osdmap e45: 7 total, 7 up, 7 in 2024-09-16T00:27:27.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:27 smithi122 ceph-mon[35412]: from='client.? 172.21.15.73:0/2530995146' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06512315-3f5e-44f8-976d-d3f5f1ba00e3"}]: dispatch 2024-09-16T00:27:27.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:27 smithi122 ceph-mon[35412]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "06512315-3f5e-44f8-976d-d3f5f1ba00e3"}]: dispatch 2024-09-16T00:27:27.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:27 smithi122 ceph-mon[35412]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "06512315-3f5e-44f8-976d-d3f5f1ba00e3"}]': finished 2024-09-16T00:27:27.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:27 smithi122 ceph-mon[35412]: osdmap e46: 8 total, 7 up, 8 in 2024-09-16T00:27:27.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:27 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:27:28.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:28 smithi038 ceph-mon[30945]: from='client.? 172.21.15.73:0/3910913189' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:27:28.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:28 smithi122 ceph-mon[35412]: from='client.? 172.21.15.73:0/3910913189' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:27:28.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:28 smithi073 ceph-mon[35254]: from='client.? 172.21.15.73:0/3910913189' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:27:29.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:29 smithi038 ceph-mon[30945]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:27:29.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:29 smithi122 ceph-mon[35412]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:27:29.793 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:29 smithi073 ceph-mon[35254]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:27:31.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:31 smithi038 ceph-mon[30945]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:27:31.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:31 smithi122 ceph-mon[35412]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:27:31.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:31 smithi073 ceph-mon[35254]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 205 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:27:33.655 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:33 smithi073 ceph-mon[35254]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:27:33.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:33 smithi038 ceph-mon[30945]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:27:33.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:33 smithi122 ceph-mon[35412]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:27:35.622 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:35 smithi073 ceph-mon[35254]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:27:35.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:35 smithi038 ceph-mon[30945]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:27:35.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:35 smithi122 ceph-mon[35412]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:27:36.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:36 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-16T00:27:36.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:36 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:27:36.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:36 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-16T00:27:36.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:36 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:27:36.765 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:36 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-16T00:27:36.765 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:36 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:27:37.660 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:37 smithi073 ceph-mon[35254]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:27:37.660 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:37 smithi073 ceph-mon[35254]: Deploying daemon osd.7 on smithi073 2024-09-16T00:27:37.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:37 smithi038 ceph-mon[30945]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:27:37.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:37 smithi038 ceph-mon[30945]: Deploying daemon osd.7 on smithi073 2024-09-16T00:27:37.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:37 smithi122 ceph-mon[35412]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:27:37.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:37 smithi122 ceph-mon[35412]: Deploying daemon osd.7 on smithi073 2024-09-16T00:27:39.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:39 smithi038 ceph-mon[30945]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:27:39.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:39 smithi122 ceph-mon[35412]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:27:39.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:39 smithi073 ceph-mon[35254]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:27:41.488 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:41 smithi073 ceph-mon[35254]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:27:41.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:41 smithi038 ceph-mon[30945]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:27:41.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:41 smithi122 ceph-mon[35412]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:27:42.455 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:42 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:27:42.455 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:42 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:42.455 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:42 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:42.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:42 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:27:42.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:42 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:42.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:42 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:42.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:42 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:27:42.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:42 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:42.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:42 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:43.687 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:43 smithi073 ceph-mon[35254]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:27:43.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:43 smithi038 ceph-mon[30945]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:27:43.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:43 smithi122 ceph-mon[35412]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:27:45.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:45 smithi073 ceph-mon[35254]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:27:45.679 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:45 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:45.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:45 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:45.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:45 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:27:45.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:45 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:27:45.680 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:45 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:45.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:45 smithi038 ceph-mon[30945]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:27:45.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:45 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:45.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:45 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:45.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:45 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:27:45.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:45 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:27:45.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:45 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:45.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:45 smithi122 ceph-mon[35412]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:27:45.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:45 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:45.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:45 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:45.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:45 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:27:45.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:45 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:27:45.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:45 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:45.725 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:45.723+0000 7f59397fa640 1 -- 172.21.15.73:0/1652970753 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f5928000bf0 con 0x7f59240a3240 2024-09-16T00:27:45.728 INFO:teuthology.orchestra.run.smithi073.stdout:Created osd(s) 7 on host 'smithi073' 2024-09-16T00:27:45.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:45.726+0000 7f594230c640 1 -- 172.21.15.73:0/1652970753 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f59240a3240 msgr2=0x7f59240a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:45.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:45.726+0000 7f594230c640 1 --2- 172.21.15.73:0/1652970753 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f59240a3240 0x7f59240a5700 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f592c005fd0 tx=0x7f592c00b390 comp rx=0 tx=0).stop 2024-09-16T00:27:45.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:45.726+0000 7f594230c640 1 -- 172.21.15.73:0/1652970753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f593c1078c0 msgr2=0x7f593c100b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:45.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:45.726+0000 7f594230c640 1 --2- 172.21.15.73:0/1652970753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f593c1078c0 0x7f593c100b50 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5934009fd0 tx=0x7f593400b260 comp rx=0 tx=0).stop 2024-09-16T00:27:45.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:45.726+0000 7f594230c640 1 -- 172.21.15.73:0/1652970753 shutdown_connections 2024-09-16T00:27:45.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:45.726+0000 7f594230c640 1 --2- 172.21.15.73:0/1652970753 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f59240a3240 0x7f59240a5700 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:45.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:45.726+0000 7f594230c640 1 --2- 172.21.15.73:0/1652970753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f593c1078c0 0x7f593c100b50 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:45.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:45.726+0000 7f594230c640 1 --2- 172.21.15.73:0/1652970753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f593c104f90 0x7f593c100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:45.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:45.726+0000 7f594230c640 1 --2- 172.21.15.73:0/1652970753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f593c102660 0x7f593c101fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:45.729 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:45.726+0000 7f594230c640 1 -- 172.21.15.73:0/1652970753 >> 172.21.15.73:0/1652970753 conn(0x7f593c0fc280 msgr2=0x7f593c102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:45.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:45.726+0000 7f594230c640 1 -- 172.21.15.73:0/1652970753 shutdown_connections 2024-09-16T00:27:45.730 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:27:45.726+0000 7f594230c640 1 -- 172.21.15.73:0/1652970753 wait complete. 2024-09-16T00:27:46.483 DEBUG:teuthology.orchestra.run.smithi073:osd.7> sudo journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@osd.7.service 2024-09-16T00:27:46.485 INFO:tasks.cephadm:Deploying osd.8 on smithi122 with /dev/vg_nvme/lv_2... 2024-09-16T00:27:46.485 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_2 2024-09-16T00:27:46.667 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:46 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:27:46.667 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:46 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:46.668 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:46 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:46.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:46 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:27:46.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:46 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:46.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:46 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:46.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:46 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:27:46.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:46 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:46.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:46 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:46.737 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:27:47.691 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:47 smithi073 ceph-mon[35254]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:27:47.691 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:27:47 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:27:47.502+0000 7f51a21b4640 -1 osd.7 0 log_to_monitors true 2024-09-16T00:27:47.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:47 smithi038 ceph-mon[30945]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:27:47.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:47 smithi122 ceph-mon[35412]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:27:48.700 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:48 smithi073 ceph-mon[35254]: from='osd.7 [v2:172.21.15.73:6816/2634361473,v1:172.21.15.73:6817/2634361473]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T00:27:48.701 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:48 smithi073 ceph-mon[35254]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T00:27:48.709 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:48 smithi122 ceph-mon[35412]: from='osd.7 [v2:172.21.15.73:6816/2634361473,v1:172.21.15.73:6817/2634361473]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T00:27:48.709 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:48 smithi122 ceph-mon[35412]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T00:27:48.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:48 smithi038 ceph-mon[30945]: from='osd.7 [v2:172.21.15.73:6816/2634361473,v1:172.21.15.73:6817/2634361473]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T00:27:48.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:48 smithi038 ceph-mon[30945]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-16T00:27:49.450 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:49 smithi122 ceph-mon[35412]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:27:49.450 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:49 smithi122 ceph-mon[35412]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-16T00:27:49.451 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:49 smithi122 ceph-mon[35412]: osdmap e47: 8 total, 7 up, 8 in 2024-09-16T00:27:49.451 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:49 smithi122 ceph-mon[35412]: from='osd.7 [v2:172.21.15.73:6816/2634361473,v1:172.21.15.73:6817/2634361473]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-09-16T00:27:49.451 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:49 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:27:49.451 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:49 smithi122 ceph-mon[35412]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-09-16T00:27:49.453 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-16T00:27:49.481 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi122:vg_nvme/lv_2 2024-09-16T00:27:49.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:49 smithi038 ceph-mon[30945]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:27:49.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:49 smithi038 ceph-mon[30945]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-16T00:27:49.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:49 smithi038 ceph-mon[30945]: osdmap e47: 8 total, 7 up, 8 in 2024-09-16T00:27:49.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:49 smithi038 ceph-mon[30945]: from='osd.7 [v2:172.21.15.73:6816/2634361473,v1:172.21.15.73:6817/2634361473]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-09-16T00:27:49.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:49 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:27:49.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:49 smithi038 ceph-mon[30945]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-09-16T00:27:49.747 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:27:49.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:49 smithi073 ceph-mon[35254]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:27:49.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:49 smithi073 ceph-mon[35254]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-16T00:27:49.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:49 smithi073 ceph-mon[35254]: osdmap e47: 8 total, 7 up, 8 in 2024-09-16T00:27:49.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:49 smithi073 ceph-mon[35254]: from='osd.7 [v2:172.21.15.73:6816/2634361473,v1:172.21.15.73:6817/2634361473]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-09-16T00:27:49.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:49 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:27:49.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:49 smithi073 ceph-mon[35254]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-09-16T00:27:49.794 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:27:49 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:27:49.434+0000 7f519e137640 -1 osd.7 0 waiting for initial osdmap 2024-09-16T00:27:49.794 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:27:49 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:27:49.442+0000 7f5199f5f640 -1 osd.7 48 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:27:50.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.196+0000 7f4c3a1ce640 1 -- 172.21.15.122:0/190448351 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c34100ad0 msgr2=0x7f4c34100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:50.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.196+0000 7f4c3a1ce640 1 --2- 172.21.15.122:0/190448351 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c34100ad0 0x7f4c34100ed0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f4c24009900 tx=0x7f4c2402a410 comp rx=0 tx=0).stop 2024-09-16T00:27:50.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.196+0000 7f4c3a1ce640 1 -- 172.21.15.122:0/190448351 shutdown_connections 2024-09-16T00:27:50.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.196+0000 7f4c3a1ce640 1 --2- 172.21.15.122:0/190448351 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c34102690 0x7f4c34108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:50.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.196+0000 7f4c3a1ce640 1 --2- 172.21.15.122:0/190448351 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c34101cd0 0x7f4c34102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:50.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.196+0000 7f4c3a1ce640 1 --2- 172.21.15.122:0/190448351 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c34100ad0 0x7f4c34100ed0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:50.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.196+0000 7f4c3a1ce640 1 -- 172.21.15.122:0/190448351 >> 172.21.15.122:0/190448351 conn(0x7f4c340fc280 msgr2=0x7f4c340fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:27:50.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.196+0000 7f4c3a1ce640 1 -- 172.21.15.122:0/190448351 shutdown_connections 2024-09-16T00:27:50.213 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.196+0000 7f4c3a1ce640 1 -- 172.21.15.122:0/190448351 wait complete. 2024-09-16T00:27:50.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.197+0000 7f4c3a1ce640 1 Processor -- start 2024-09-16T00:27:50.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.197+0000 7f4c3a1ce640 1 -- start start 2024-09-16T00:27:50.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.197+0000 7f4c3a1ce640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c34100ad0 0x7f4c34196e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:50.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.197+0000 7f4c3a1ce640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c34101cd0 0x7f4c341973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:50.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.197+0000 7f4c3a1ce640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c34102690 0x7f4c3419e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:50.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.197+0000 7f4c3a1ce640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3410c7b0 con 0x7f4c34100ad0 2024-09-16T00:27:50.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.197+0000 7f4c3a1ce640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3410c630 con 0x7f4c34102690 2024-09-16T00:27:50.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.197+0000 7f4c3a1ce640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4c3410c930 con 0x7f4c34101cd0 2024-09-16T00:27:50.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.198+0000 7f4c32ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c34101cd0 0x7f4c341973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:50.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.198+0000 7f4c32ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c34101cd0 0x7f4c341973b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:57530/0 (socket says 172.21.15.122:57530) 2024-09-16T00:27:50.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.198+0000 7f4c32ffd640 1 -- 172.21.15.122:0/86743892 learned_addr learned my addr 172.21.15.122:0/86743892 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-16T00:27:50.214 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.198+0000 7f4c33fff640 1 --2- 172.21.15.122:0/86743892 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c34102690 0x7f4c3419e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:50.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.198+0000 7f4c337fe640 1 --2- 172.21.15.122:0/86743892 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c34100ad0 0x7f4c34196e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:50.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.198+0000 7f4c32ffd640 1 -- 172.21.15.122:0/86743892 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c34102690 msgr2=0x7f4c3419e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:50.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.198+0000 7f4c32ffd640 1 --2- 172.21.15.122:0/86743892 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c34102690 0x7f4c3419e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:50.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.198+0000 7f4c32ffd640 1 -- 172.21.15.122:0/86743892 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c34100ad0 msgr2=0x7f4c34196e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:27:50.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.198+0000 7f4c32ffd640 1 --2- 172.21.15.122:0/86743892 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c34100ad0 0x7f4c34196e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:27:50.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.198+0000 7f4c32ffd640 1 -- 172.21.15.122:0/86743892 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c3419eaa0 con 0x7f4c34101cd0 2024-09-16T00:27:50.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.198+0000 7f4c337fe640 1 --2- 172.21.15.122:0/86743892 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c34100ad0 0x7f4c34196e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:27:50.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.198+0000 7f4c32ffd640 1 --2- 172.21.15.122:0/86743892 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c34101cd0 0x7f4c341973b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4c2800d950 tx=0x7f4c2800de10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:50.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.198+0000 7f4c30ff9640 1 -- 172.21.15.122:0/86743892 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c28017070 con 0x7f4c34101cd0 2024-09-16T00:27:50.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.198+0000 7f4c3a1ce640 1 -- 172.21.15.122:0/86743892 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c3419ed90 con 0x7f4c34101cd0 2024-09-16T00:27:50.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.198+0000 7f4c30ff9640 1 -- 172.21.15.122:0/86743892 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c28009c30 con 0x7f4c34101cd0 2024-09-16T00:27:50.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.198+0000 7f4c30ff9640 1 -- 172.21.15.122:0/86743892 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c2800f6b0 con 0x7f4c34101cd0 2024-09-16T00:27:50.215 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.198+0000 7f4c3a1ce640 1 -- 172.21.15.122:0/86743892 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c3419f2d0 con 0x7f4c34101cd0 2024-09-16T00:27:50.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.199+0000 7f4c3a1ce640 1 -- 172.21.15.122:0/86743892 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c34100ed0 con 0x7f4c34101cd0 2024-09-16T00:27:50.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.201+0000 7f4c30ff9640 1 -- 172.21.15.122:0/86743892 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f4c280109d0 con 0x7f4c34101cd0 2024-09-16T00:27:50.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.201+0000 7f4c30ff9640 1 --2- 172.21.15.122:0/86743892 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f4c1c0a3270 0x7f4c1c0a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:27:50.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.201+0000 7f4c30ff9640 1 -- 172.21.15.122:0/86743892 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5430+0+0 (secure 0 0 0) 0x7f4c280ce920 con 0x7f4c34101cd0 2024-09-16T00:27:50.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.202+0000 7f4c337fe640 1 --2- 172.21.15.122:0/86743892 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f4c1c0a3270 0x7f4c1c0a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:27:50.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.202+0000 7f4c337fe640 1 --2- 172.21.15.122:0/86743892 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f4c1c0a3270 0x7f4c1c0a5730 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4c24006010 tx=0x7f4c24032dd0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:27:50.216 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.205+0000 7f4c30ff9640 1 -- 172.21.15.122:0/86743892 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4c2809c180 con 0x7f4c34101cd0 2024-09-16T00:27:50.386 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:27:50.386+0000 7f4c3a1ce640 1 -- 172.21.15.122:0/86743892 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f4c34062630 con 0x7f4c1c0a3270 2024-09-16T00:27:50.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:50 smithi038 ceph-mon[30945]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-09-16T00:27:50.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:50 smithi038 ceph-mon[30945]: osdmap e48: 8 total, 7 up, 8 in 2024-09-16T00:27:50.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:50 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:27:50.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:50 smithi038 ceph-mon[30945]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:27:50.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:50 smithi038 ceph-mon[30945]: Detected new or changed devices on smithi073 2024-09-16T00:27:50.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:50 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:50.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:50 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:50.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:50 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:27:50.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:50 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:27:50.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:50 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:27:50.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:50 smithi038 ceph-mon[30945]: Adjusting osd_memory_target on smithi073 to 5696M 2024-09-16T00:27:50.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:50 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:50.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:50 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:27:50.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:50 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:27:50.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:50 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:50.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:50 smithi038 ceph-mon[30945]: osd.7 [v2:172.21.15.73:6816/2634361473,v1:172.21.15.73:6817/2634361473] boot 2024-09-16T00:27:50.717 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:50 smithi038 ceph-mon[30945]: osdmap e49: 8 total, 8 up, 8 in 2024-09-16T00:27:50.717 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:50 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:27:50.717 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:50 smithi038 ceph-mon[30945]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:27:50.717 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:50 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:27:50.717 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:50 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:27:50.717 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:50 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:27:50.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:50 smithi122 ceph-mon[35412]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-09-16T00:27:50.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:50 smithi122 ceph-mon[35412]: osdmap e48: 8 total, 7 up, 8 in 2024-09-16T00:27:50.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:50 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:27:50.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:50 smithi122 ceph-mon[35412]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:27:50.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:50 smithi122 ceph-mon[35412]: Detected new or changed devices on smithi073 2024-09-16T00:27:50.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:50 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:50.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:50 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:50.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:50 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:27:50.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:50 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:27:50.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:50 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:27:50.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:50 smithi122 ceph-mon[35412]: Adjusting osd_memory_target on smithi073 to 5696M 2024-09-16T00:27:50.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:50 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:50.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:50 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:27:50.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:50 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:27:50.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:50 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:50.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:50 smithi122 ceph-mon[35412]: osd.7 [v2:172.21.15.73:6816/2634361473,v1:172.21.15.73:6817/2634361473] boot 2024-09-16T00:27:50.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:50 smithi122 ceph-mon[35412]: osdmap e49: 8 total, 8 up, 8 in 2024-09-16T00:27:50.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:50 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:27:50.719 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:50 smithi122 ceph-mon[35412]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:27:50.719 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:50 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:27:50.719 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:50 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:27:50.719 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:50 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:27:50.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:50 smithi073 ceph-mon[35254]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-09-16T00:27:50.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:50 smithi073 ceph-mon[35254]: osdmap e48: 8 total, 7 up, 8 in 2024-09-16T00:27:50.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:50 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:27:50.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:50 smithi073 ceph-mon[35254]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 206 MiB used, 626 GiB / 626 GiB avail 2024-09-16T00:27:50.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:50 smithi073 ceph-mon[35254]: Detected new or changed devices on smithi073 2024-09-16T00:27:50.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:50 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:50.795 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:50 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:50.795 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:50 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:27:50.795 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:50 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:27:50.795 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:50 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:27:50.795 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:50 smithi073 ceph-mon[35254]: Adjusting osd_memory_target on smithi073 to 5696M 2024-09-16T00:27:50.795 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:50 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:50.795 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:50 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:27:50.795 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:50 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:27:50.795 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:50 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:27:50.795 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:50 smithi073 ceph-mon[35254]: osd.7 [v2:172.21.15.73:6816/2634361473,v1:172.21.15.73:6817/2634361473] boot 2024-09-16T00:27:50.795 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:50 smithi073 ceph-mon[35254]: osdmap e49: 8 total, 8 up, 8 in 2024-09-16T00:27:50.795 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:50 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-16T00:27:50.795 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:50 smithi073 ceph-mon[35254]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:27:50.795 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:50 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:27:50.795 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:50 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:27:50.796 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:50 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:27:51.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:51 smithi038 ceph-mon[30945]: purged_snaps scrub starts 2024-09-16T00:27:51.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:51 smithi038 ceph-mon[30945]: purged_snaps scrub ok 2024-09-16T00:27:51.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:51 smithi038 ceph-mon[30945]: osdmap e50: 8 total, 8 up, 8 in 2024-09-16T00:27:51.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:51 smithi122 ceph-mon[35412]: purged_snaps scrub starts 2024-09-16T00:27:51.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:51 smithi122 ceph-mon[35412]: purged_snaps scrub ok 2024-09-16T00:27:51.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:51 smithi122 ceph-mon[35412]: osdmap e50: 8 total, 8 up, 8 in 2024-09-16T00:27:51.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:51 smithi073 ceph-mon[35254]: purged_snaps scrub starts 2024-09-16T00:27:51.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:51 smithi073 ceph-mon[35254]: purged_snaps scrub ok 2024-09-16T00:27:51.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:51 smithi073 ceph-mon[35254]: osdmap e50: 8 total, 8 up, 8 in 2024-09-16T00:27:52.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:52 smithi122 ceph-mon[35412]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:27:52.967 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:52 smithi122 ceph-mon[35412]: osdmap e51: 8 total, 8 up, 8 in 2024-09-16T00:27:53.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:52 smithi073 ceph-mon[35254]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:27:53.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:52 smithi073 ceph-mon[35254]: osdmap e51: 8 total, 8 up, 8 in 2024-09-16T00:27:53.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:52 smithi038 ceph-mon[30945]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:27:53.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:52 smithi038 ceph-mon[30945]: osdmap e51: 8 total, 8 up, 8 in 2024-09-16T00:27:54.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:53 smithi073 ceph-mon[35254]: osdmap e52: 8 total, 8 up, 8 in 2024-09-16T00:27:54.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:53 smithi073 ceph-mon[35254]: from='client.? 172.21.15.122:0/3874383076' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db703c0a-1f87-46f0-be5f-f826f7f9daac"}]: dispatch 2024-09-16T00:27:54.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:53 smithi073 ceph-mon[35254]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db703c0a-1f87-46f0-be5f-f826f7f9daac"}]: dispatch 2024-09-16T00:27:54.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:53 smithi073 ceph-mon[35254]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db703c0a-1f87-46f0-be5f-f826f7f9daac"}]': finished 2024-09-16T00:27:54.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:53 smithi073 ceph-mon[35254]: osdmap e53: 9 total, 8 up, 9 in 2024-09-16T00:27:54.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:53 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:27:54.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:53 smithi073 ceph-mon[35254]: from='client.? 172.21.15.122:0/1264118742' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:27:54.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:53 smithi038 ceph-mon[30945]: osdmap e52: 8 total, 8 up, 8 in 2024-09-16T00:27:54.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:53 smithi038 ceph-mon[30945]: from='client.? 172.21.15.122:0/3874383076' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db703c0a-1f87-46f0-be5f-f826f7f9daac"}]: dispatch 2024-09-16T00:27:54.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:53 smithi038 ceph-mon[30945]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db703c0a-1f87-46f0-be5f-f826f7f9daac"}]: dispatch 2024-09-16T00:27:54.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:53 smithi038 ceph-mon[30945]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db703c0a-1f87-46f0-be5f-f826f7f9daac"}]': finished 2024-09-16T00:27:54.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:53 smithi038 ceph-mon[30945]: osdmap e53: 9 total, 8 up, 9 in 2024-09-16T00:27:54.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:53 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:27:54.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:53 smithi038 ceph-mon[30945]: from='client.? 172.21.15.122:0/1264118742' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:27:54.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:53 smithi122 ceph-mon[35412]: osdmap e52: 8 total, 8 up, 8 in 2024-09-16T00:27:54.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:53 smithi122 ceph-mon[35412]: from='client.? 172.21.15.122:0/3874383076' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db703c0a-1f87-46f0-be5f-f826f7f9daac"}]: dispatch 2024-09-16T00:27:54.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:53 smithi122 ceph-mon[35412]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "db703c0a-1f87-46f0-be5f-f826f7f9daac"}]: dispatch 2024-09-16T00:27:54.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:53 smithi122 ceph-mon[35412]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "db703c0a-1f87-46f0-be5f-f826f7f9daac"}]': finished 2024-09-16T00:27:54.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:53 smithi122 ceph-mon[35412]: osdmap e53: 9 total, 8 up, 9 in 2024-09-16T00:27:54.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:53 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:27:54.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:53 smithi122 ceph-mon[35412]: from='client.? 172.21.15.122:0/1264118742' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:27:55.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:54 smithi038 ceph-mon[30945]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:27:55.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:54 smithi122 ceph-mon[35412]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:27:55.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:54 smithi073 ceph-mon[35254]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:27:57.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:56 smithi038 ceph-mon[30945]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:27:57.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:56 smithi122 ceph-mon[35412]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:27:57.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:56 smithi073 ceph-mon[35254]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:27:59.076 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:27:58 smithi122 ceph-mon[35412]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:27:59.214 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:27:58 smithi038 ceph-mon[30945]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:27:59.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:27:58 smithi073 ceph-mon[35254]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:28:01.144 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:00 smithi122 ceph-mon[35412]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:28:01.214 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:00 smithi038 ceph-mon[30945]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:28:01.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:00 smithi073 ceph-mon[35254]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:28:02.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:01 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-16T00:28:02.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:01 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:02.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:01 smithi038 ceph-mon[30945]: Deploying daemon osd.8 on smithi122 2024-09-16T00:28:02.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:01 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-16T00:28:02.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:01 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:02.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:01 smithi122 ceph-mon[35412]: Deploying daemon osd.8 on smithi122 2024-09-16T00:28:02.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:01 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-09-16T00:28:02.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:01 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:02.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:01 smithi073 ceph-mon[35254]: Deploying daemon osd.8 on smithi122 2024-09-16T00:28:02.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:02 smithi122 ceph-mon[35412]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:28:03.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:02 smithi038 ceph-mon[30945]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:28:03.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:02 smithi073 ceph-mon[35254]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:28:05.068 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:04 smithi122 ceph-mon[35412]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:28:05.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:04 smithi038 ceph-mon[30945]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:28:05.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:04 smithi073 ceph-mon[35254]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:28:07.122 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:06 smithi122 ceph-mon[35412]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:28:07.214 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:06 smithi038 ceph-mon[30945]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:28:07.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:06 smithi073 ceph-mon[35254]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:28:08.100 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:07 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:28:08.100 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:07 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:08.101 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:07 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:08.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:07 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:28:08.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:07 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:08.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:07 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:08.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:07 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:28:08.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:07 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:08.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:07 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:09.621 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:09 smithi122 ceph-mon[35412]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:28:09.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:09 smithi038 ceph-mon[30945]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:28:09.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:09 smithi073 ceph-mon[35254]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:28:11.235 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:28:11.234+0000 7f4c30ff9640 1 -- 172.21.15.122:0/86743892 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f4c34062630 con 0x7f4c1c0a3270 2024-09-16T00:28:11.235 INFO:teuthology.orchestra.run.smithi122.stdout:Created osd(s) 8 on host 'smithi122' 2024-09-16T00:28:11.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:28:11.236+0000 7f4c3a1ce640 1 -- 172.21.15.122:0/86743892 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f4c1c0a3270 msgr2=0x7f4c1c0a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:11.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:28:11.236+0000 7f4c3a1ce640 1 --2- 172.21.15.122:0/86743892 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f4c1c0a3270 0x7f4c1c0a5730 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f4c24006010 tx=0x7f4c24032dd0 comp rx=0 tx=0).stop 2024-09-16T00:28:11.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:28:11.236+0000 7f4c3a1ce640 1 -- 172.21.15.122:0/86743892 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c34101cd0 msgr2=0x7f4c341973b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:11.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:28:11.236+0000 7f4c3a1ce640 1 --2- 172.21.15.122:0/86743892 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c34101cd0 0x7f4c341973b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f4c2800d950 tx=0x7f4c2800de10 comp rx=0 tx=0).stop 2024-09-16T00:28:11.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:28:11.236+0000 7f4c3a1ce640 1 -- 172.21.15.122:0/86743892 shutdown_connections 2024-09-16T00:28:11.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:28:11.236+0000 7f4c3a1ce640 1 --2- 172.21.15.122:0/86743892 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f4c1c0a3270 0x7f4c1c0a5730 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:11.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:28:11.236+0000 7f4c3a1ce640 1 --2- 172.21.15.122:0/86743892 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c34102690 0x7f4c3419e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:11.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:28:11.236+0000 7f4c3a1ce640 1 --2- 172.21.15.122:0/86743892 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c34101cd0 0x7f4c341973b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:11.237 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:28:11.236+0000 7f4c3a1ce640 1 --2- 172.21.15.122:0/86743892 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c34100ad0 0x7f4c34196e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:11.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:28:11.236+0000 7f4c3a1ce640 1 -- 172.21.15.122:0/86743892 >> 172.21.15.122:0/86743892 conn(0x7f4c340fc280 msgr2=0x7f4c34105140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:11.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:28:11.237+0000 7f4c3a1ce640 1 -- 172.21.15.122:0/86743892 shutdown_connections 2024-09-16T00:28:11.238 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:28:11.237+0000 7f4c3a1ce640 1 -- 172.21.15.122:0/86743892 wait complete. 2024-09-16T00:28:11.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:11 smithi038 ceph-mon[30945]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:28:11.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:11 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:11.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:11 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:11.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:11 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:11.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:11 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:28:11.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:11 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:11.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:11 smithi122 ceph-mon[35412]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:28:11.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:11 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:11.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:11 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:11.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:11 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:11.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:11 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:28:11.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:11 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:11.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:11 smithi073 ceph-mon[35254]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:28:11.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:11 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:11.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:11 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:11.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:11 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:11.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:11 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:28:11.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:11 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:11.942 DEBUG:teuthology.orchestra.run.smithi122:osd.8> sudo journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@osd.8.service 2024-09-16T00:28:11.945 INFO:tasks.cephadm:Deploying osd.9 on smithi038 with /dev/vg_nvme/lv_1... 2024-09-16T00:28:11.945 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-16T00:28:12.209 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:12.330 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:12 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:28:12.330 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:12 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:12.331 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:12 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:12.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:12 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:28:12.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:12 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:12.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:12 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:12.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:12 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:28:12.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:12 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:12.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:12 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:13.097 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:28:12 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:28:12.912+0000 7ff61b1ce640 -1 osd.8 0 log_to_monitors true 2024-09-16T00:28:13.416 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:13 smithi122 ceph-mon[35412]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:28:13.417 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:13 smithi122 ceph-mon[35412]: from='osd.8 [v2:172.21.15.122:6816/1800567633,v1:172.21.15.122:6817/1800567633]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T00:28:13.417 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:13 smithi122 ceph-mon[35412]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T00:28:13.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:13 smithi073 ceph-mon[35254]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:28:13.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:13 smithi073 ceph-mon[35254]: from='osd.8 [v2:172.21.15.122:6816/1800567633,v1:172.21.15.122:6817/1800567633]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T00:28:13.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:13 smithi073 ceph-mon[35254]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T00:28:13.598 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:13 smithi038 ceph-mon[30945]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:28:13.599 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:13 smithi038 ceph-mon[30945]: from='osd.8 [v2:172.21.15.122:6816/1800567633,v1:172.21.15.122:6817/1800567633]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T00:28:13.599 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:13 smithi038 ceph-mon[30945]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-09-16T00:28:14.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:14 smithi038 ceph-mon[30945]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-16T00:28:14.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:14 smithi038 ceph-mon[30945]: from='osd.8 [v2:172.21.15.122:6816/1800567633,v1:172.21.15.122:6817/1800567633]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-16T00:28:14.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:14 smithi038 ceph-mon[30945]: osdmap e54: 9 total, 8 up, 9 in 2024-09-16T00:28:14.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:14 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:28:14.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:14 smithi038 ceph-mon[30945]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-16T00:28:14.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:14 smithi038 ceph-mon[30945]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-09-16T00:28:14.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:14 smithi038 ceph-mon[30945]: osdmap e55: 9 total, 8 up, 9 in 2024-09-16T00:28:14.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:14 smithi073 ceph-mon[35254]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-16T00:28:14.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:14 smithi073 ceph-mon[35254]: from='osd.8 [v2:172.21.15.122:6816/1800567633,v1:172.21.15.122:6817/1800567633]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-16T00:28:14.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:14 smithi073 ceph-mon[35254]: osdmap e54: 9 total, 8 up, 9 in 2024-09-16T00:28:14.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:14 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:28:14.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:14 smithi073 ceph-mon[35254]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-16T00:28:14.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:14 smithi073 ceph-mon[35254]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-09-16T00:28:14.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:14 smithi073 ceph-mon[35254]: osdmap e55: 9 total, 8 up, 9 in 2024-09-16T00:28:14.620 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:14 smithi122 ceph-mon[35412]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-16T00:28:14.621 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:14 smithi122 ceph-mon[35412]: from='osd.8 [v2:172.21.15.122:6816/1800567633,v1:172.21.15.122:6817/1800567633]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-16T00:28:14.621 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:14 smithi122 ceph-mon[35412]: osdmap e54: 9 total, 8 up, 9 in 2024-09-16T00:28:14.621 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:14 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:28:14.621 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:14 smithi122 ceph-mon[35412]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-16T00:28:14.621 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:14 smithi122 ceph-mon[35412]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-09-16T00:28:14.621 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:14 smithi122 ceph-mon[35412]: osdmap e55: 9 total, 8 up, 9 in 2024-09-16T00:28:15.039 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:28:15.068 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi038:vg_nvme/lv_1 2024-09-16T00:28:15.216 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:28:14 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:28:14.931+0000 7ff617151640 -1 osd.8 0 waiting for initial osdmap 2024-09-16T00:28:15.217 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:28:14 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:28:14.940+0000 7ff60ef71640 -1 osd.8 56 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:28:15.333 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:28:15.453 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:15 smithi038 ceph-mon[30945]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:28:15.454 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:15 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:28:15.454 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:15 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:28:15.454 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:15 smithi038 ceph-mon[30945]: osdmap e56: 9 total, 8 up, 9 in 2024-09-16T00:28:15.454 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:15 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:28:15.454 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:15 smithi038 ceph-mon[30945]: from='osd.8 ' entity='osd.8' 2024-09-16T00:28:15.454 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:15 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:15.454 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:15 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:15.454 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:15 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:28:15.454 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:15 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:28:15.454 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:15 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:28:15.454 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:15 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:15.454 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:15 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:15.455 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:15 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:28:15.455 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:15 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:15.455 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:15 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:28:15.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:15 smithi073 ceph-mon[35254]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:28:15.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:15 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:28:15.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:15 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:28:15.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:15 smithi073 ceph-mon[35254]: osdmap e56: 9 total, 8 up, 9 in 2024-09-16T00:28:15.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:15 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:28:15.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:15 smithi073 ceph-mon[35254]: from='osd.8 ' entity='osd.8' 2024-09-16T00:28:15.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:15 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:15.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:15 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:15.546 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:15 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:28:15.546 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:15 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:28:15.546 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:15 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:28:15.546 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:15 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:15.546 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:15 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:15.546 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:15 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:28:15.546 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:15 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:15.546 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:15 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:28:15.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:15 smithi122 ceph-mon[35412]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:28:15.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:15 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:28:15.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:15 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:28:15.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:15 smithi122 ceph-mon[35412]: osdmap e56: 9 total, 8 up, 9 in 2024-09-16T00:28:15.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:15 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:28:15.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:15 smithi122 ceph-mon[35412]: from='osd.8 ' entity='osd.8' 2024-09-16T00:28:15.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:15 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:15.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:15 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:15.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:15 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:28:15.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:15 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:28:15.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:15 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:28:15.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:15 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:15.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:15 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:15.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:15 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:28:15.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:15 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:15.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:15 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:28:15.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:15.851+0000 7f70c99eb640 1 -- 172.21.15.38:0/2051124291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70c4102660 msgr2=0x7f70c4104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:15.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:15.851+0000 7f70c99eb640 1 --2- 172.21.15.38:0/2051124291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70c4102660 0x7f70c4104a50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f70b4009900 tx=0x7f70b402a410 comp rx=0 tx=0).stop 2024-09-16T00:28:15.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:15.851+0000 7f70c99eb640 1 -- 172.21.15.38:0/2051124291 shutdown_connections 2024-09-16T00:28:15.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:15.851+0000 7f70c99eb640 1 --2- 172.21.15.38:0/2051124291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70c41078c0 0x7f70c4109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:15.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:15.851+0000 7f70c99eb640 1 --2- 172.21.15.38:0/2051124291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70c4104f90 0x7f70c4107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:15.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:15.851+0000 7f70c99eb640 1 --2- 172.21.15.38:0/2051124291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70c4102660 0x7f70c4104a50 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:15.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:15.851+0000 7f70c99eb640 1 -- 172.21.15.38:0/2051124291 >> 172.21.15.38:0/2051124291 conn(0x7f70c40fc280 msgr2=0x7f70c40fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:15.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:15.851+0000 7f70c99eb640 1 -- 172.21.15.38:0/2051124291 shutdown_connections 2024-09-16T00:28:15.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:15.851+0000 7f70c99eb640 1 -- 172.21.15.38:0/2051124291 wait complete. 2024-09-16T00:28:15.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:15.851+0000 7f70c99eb640 1 Processor -- start 2024-09-16T00:28:15.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:15.851+0000 7f70c99eb640 1 -- start start 2024-09-16T00:28:15.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:15.852+0000 7f70c99eb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70c4102660 0x7f70c4197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:15.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:15.852+0000 7f70c99eb640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70c4104f90 0x7f70c41983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:15.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:15.852+0000 7f70c99eb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70c41078c0 0x7f70c419f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:15.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:15.852+0000 7f70c99eb640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f70c410d870 con 0x7f70c41078c0 2024-09-16T00:28:15.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:15.852+0000 7f70c99eb640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f70c410d6f0 con 0x7f70c4104f90 2024-09-16T00:28:15.867 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:15.852+0000 7f70c99eb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f70c410d9f0 con 0x7f70c4102660 2024-09-16T00:28:15.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:15.852+0000 7f70c37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70c41078c0 0x7f70c419f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:15.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:15.852+0000 7f70c37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70c41078c0 0x7f70c419f460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:56350/0 (socket says 172.21.15.38:56350) 2024-09-16T00:28:15.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:15.852+0000 7f70c37fe640 1 -- 172.21.15.38:0/2543994144 learned_addr learned my addr 172.21.15.38:0/2543994144 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:28:15.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:15.852+0000 7f70c27fc640 1 --2- 172.21.15.38:0/2543994144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70c4104f90 0x7f70c41983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:15.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:15.852+0000 7f70c2ffd640 1 --2- 172.21.15.38:0/2543994144 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70c4102660 0x7f70c4197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:15.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:15.852+0000 7f70c37fe640 1 -- 172.21.15.38:0/2543994144 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70c4102660 msgr2=0x7f70c4197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:15.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:15.852+0000 7f70c37fe640 1 --2- 172.21.15.38:0/2543994144 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70c4102660 0x7f70c4197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:15.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:15.852+0000 7f70c37fe640 1 -- 172.21.15.38:0/2543994144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70c4104f90 msgr2=0x7f70c41983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:15.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:15.852+0000 7f70c37fe640 1 --2- 172.21.15.38:0/2543994144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70c4104f90 0x7f70c41983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:15.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:15.852+0000 7f70c37fe640 1 -- 172.21.15.38:0/2543994144 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70c419fad0 con 0x7f70c41078c0 2024-09-16T00:28:15.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:15.852+0000 7f70c27fc640 1 --2- 172.21.15.38:0/2543994144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70c4104f90 0x7f70c41983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:28:15.868 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:15.852+0000 7f70c37fe640 1 --2- 172.21.15.38:0/2543994144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70c41078c0 0x7f70c419f460 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f70bc00cce0 tx=0x7f70bc00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:15.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:15.852+0000 7f70c89e9640 1 -- 172.21.15.38:0/2543994144 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70bc00f450 con 0x7f70c41078c0 2024-09-16T00:28:15.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:15.852+0000 7f70c89e9640 1 -- 172.21.15.38:0/2543994144 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f70bc00fad0 con 0x7f70c41078c0 2024-09-16T00:28:15.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:15.852+0000 7f70c99eb640 1 -- 172.21.15.38:0/2543994144 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70c419fdc0 con 0x7f70c41078c0 2024-09-16T00:28:15.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:15.853+0000 7f70c89e9640 1 -- 172.21.15.38:0/2543994144 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70bc018c10 con 0x7f70c41078c0 2024-09-16T00:28:15.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:15.853+0000 7f70c99eb640 1 -- 172.21.15.38:0/2543994144 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70c41a0270 con 0x7f70c41078c0 2024-09-16T00:28:15.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:15.854+0000 7f70c99eb640 1 -- 172.21.15.38:0/2543994144 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70c40fdd70 con 0x7f70c41078c0 2024-09-16T00:28:15.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:15.855+0000 7f70c89e9640 1 -- 172.21.15.38:0/2543994144 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f70bc00f5f0 con 0x7f70c41078c0 2024-09-16T00:28:15.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:15.855+0000 7f70c89e9640 1 --2- 172.21.15.38:0/2543994144 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f70ac0a32f0 0x7f70ac0a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:15.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:15.855+0000 7f70c2ffd640 1 --2- 172.21.15.38:0/2543994144 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f70ac0a32f0 0x7f70ac0a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:15.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:15.855+0000 7f70c89e9640 1 -- 172.21.15.38:0/2543994144 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5862+0+0 (secure 0 0 0) 0x7f70bc0cf640 con 0x7f70c41078c0 2024-09-16T00:28:15.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:15.856+0000 7f70c2ffd640 1 --2- 172.21.15.38:0/2543994144 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f70ac0a32f0 0x7f70ac0a57b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f70b4000c00 tx=0x7f70b40315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:15.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:15.860+0000 7f70c89e9640 1 -- 172.21.15.38:0/2543994144 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f70bc014030 con 0x7f70c41078c0 2024-09-16T00:28:16.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:16.046+0000 7f70c99eb640 1 -- 172.21.15.38:0/2543994144 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f70c4062630 con 0x7f70ac0a32f0 2024-09-16T00:28:16.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:16 smithi038 ceph-mon[30945]: purged_snaps scrub starts 2024-09-16T00:28:16.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:16 smithi038 ceph-mon[30945]: purged_snaps scrub ok 2024-09-16T00:28:16.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:16 smithi038 ceph-mon[30945]: Detected new or changed devices on smithi122 2024-09-16T00:28:16.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:16 smithi038 ceph-mon[30945]: Adjusting osd_memory_target on smithi122 to 5695M 2024-09-16T00:28:16.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:16 smithi038 ceph-mon[30945]: osd.8 [v2:172.21.15.122:6816/1800567633,v1:172.21.15.122:6817/1800567633] boot 2024-09-16T00:28:16.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:16 smithi038 ceph-mon[30945]: osdmap e57: 9 total, 9 up, 9 in 2024-09-16T00:28:16.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:16 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:28:16.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:16 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:28:16.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:16 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:28:16.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:16 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:16.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:16 smithi073 ceph-mon[35254]: purged_snaps scrub starts 2024-09-16T00:28:16.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:16 smithi073 ceph-mon[35254]: purged_snaps scrub ok 2024-09-16T00:28:16.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:16 smithi073 ceph-mon[35254]: Detected new or changed devices on smithi122 2024-09-16T00:28:16.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:16 smithi073 ceph-mon[35254]: Adjusting osd_memory_target on smithi122 to 5695M 2024-09-16T00:28:16.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:16 smithi073 ceph-mon[35254]: osd.8 [v2:172.21.15.122:6816/1800567633,v1:172.21.15.122:6817/1800567633] boot 2024-09-16T00:28:16.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:16 smithi073 ceph-mon[35254]: osdmap e57: 9 total, 9 up, 9 in 2024-09-16T00:28:16.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:16 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:28:16.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:16 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:28:16.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:16 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:28:16.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:16 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:16.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:16 smithi122 ceph-mon[35412]: purged_snaps scrub starts 2024-09-16T00:28:16.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:16 smithi122 ceph-mon[35412]: purged_snaps scrub ok 2024-09-16T00:28:16.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:16 smithi122 ceph-mon[35412]: Detected new or changed devices on smithi122 2024-09-16T00:28:16.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:16 smithi122 ceph-mon[35412]: Adjusting osd_memory_target on smithi122 to 5695M 2024-09-16T00:28:16.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:16 smithi122 ceph-mon[35412]: osd.8 [v2:172.21.15.122:6816/1800567633,v1:172.21.15.122:6817/1800567633] boot 2024-09-16T00:28:16.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:16 smithi122 ceph-mon[35412]: osdmap e57: 9 total, 9 up, 9 in 2024-09-16T00:28:16.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:16 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-09-16T00:28:16.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:16 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:28:16.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:16 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:28:16.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:16 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:17.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:17 smithi038 ceph-mon[30945]: pgmap v193: 1 pgs: 1 remapped+peering; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:28:17.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:17 smithi038 ceph-mon[30945]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:28:17.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:17 smithi038 ceph-mon[30945]: osdmap e58: 9 total, 9 up, 9 in 2024-09-16T00:28:17.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:17 smithi073 ceph-mon[35254]: pgmap v193: 1 pgs: 1 remapped+peering; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:28:17.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:17 smithi073 ceph-mon[35254]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:28:17.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:17 smithi073 ceph-mon[35254]: osdmap e58: 9 total, 9 up, 9 in 2024-09-16T00:28:17.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:17 smithi122 ceph-mon[35412]: pgmap v193: 1 pgs: 1 remapped+peering; 577 KiB data, 233 MiB used, 715 GiB / 715 GiB avail 2024-09-16T00:28:17.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:17 smithi122 ceph-mon[35412]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi038:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:28:17.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:17 smithi122 ceph-mon[35412]: osdmap e58: 9 total, 9 up, 9 in 2024-09-16T00:28:19.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:18 smithi073 ceph-mon[35254]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:28:19.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:18 smithi073 ceph-mon[35254]: osdmap e59: 9 total, 9 up, 9 in 2024-09-16T00:28:19.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:18 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/4098680405' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5384200-b3b4-4936-85e1-fc57a9fe2bb8"}]: dispatch 2024-09-16T00:28:19.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:18 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/4098680405' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f5384200-b3b4-4936-85e1-fc57a9fe2bb8"}]': finished 2024-09-16T00:28:19.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:18 smithi073 ceph-mon[35254]: osdmap e60: 10 total, 9 up, 10 in 2024-09-16T00:28:19.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:18 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:28:19.046 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:18 smithi038 ceph-mon[30945]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:28:19.046 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:18 smithi038 ceph-mon[30945]: osdmap e59: 9 total, 9 up, 9 in 2024-09-16T00:28:19.046 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:18 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/4098680405' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5384200-b3b4-4936-85e1-fc57a9fe2bb8"}]: dispatch 2024-09-16T00:28:19.046 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:18 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/4098680405' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f5384200-b3b4-4936-85e1-fc57a9fe2bb8"}]': finished 2024-09-16T00:28:19.046 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:18 smithi038 ceph-mon[30945]: osdmap e60: 10 total, 9 up, 10 in 2024-09-16T00:28:19.047 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:18 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:28:19.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:18 smithi122 ceph-mon[35412]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 268 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:28:19.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:18 smithi122 ceph-mon[35412]: osdmap e59: 9 total, 9 up, 9 in 2024-09-16T00:28:19.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:18 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/4098680405' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f5384200-b3b4-4936-85e1-fc57a9fe2bb8"}]: dispatch 2024-09-16T00:28:19.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:18 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/4098680405' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f5384200-b3b4-4936-85e1-fc57a9fe2bb8"}]': finished 2024-09-16T00:28:19.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:18 smithi122 ceph-mon[35412]: osdmap e60: 10 total, 9 up, 10 in 2024-09-16T00:28:19.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:18 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:28:20.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:19 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/2926729711' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:28:20.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:19 smithi073 ceph-mon[35254]: osdmap e61: 10 total, 9 up, 10 in 2024-09-16T00:28:20.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:19 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:28:20.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:19 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/2926729711' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:28:20.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:19 smithi038 ceph-mon[30945]: osdmap e61: 10 total, 9 up, 10 in 2024-09-16T00:28:20.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:19 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:28:20.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:19 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/2926729711' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:28:20.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:19 smithi122 ceph-mon[35412]: osdmap e61: 10 total, 9 up, 10 in 2024-09-16T00:28:20.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:19 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:28:21.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:20 smithi073 ceph-mon[35254]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:28:21.214 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:20 smithi038 ceph-mon[30945]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:28:21.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:20 smithi122 ceph-mon[35412]: pgmap v199: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:28:23.043 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:22 smithi073 ceph-mon[35254]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-09-16T00:28:23.064 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:22 smithi038 ceph-mon[30945]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-09-16T00:28:23.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:22 smithi122 ceph-mon[35412]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 121 KiB/s, 0 objects/s recovering 2024-09-16T00:28:24.913 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:24 smithi038 ceph-mon[30945]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T00:28:25.043 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:24 smithi073 ceph-mon[35254]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T00:28:25.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:24 smithi122 ceph-mon[35412]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T00:28:27.014 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:26 smithi038 ceph-mon[30945]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-16T00:28:27.043 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:26 smithi073 ceph-mon[35254]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-16T00:28:27.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:26 smithi122 ceph-mon[35412]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-09-16T00:28:27.997 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:27 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-16T00:28:27.997 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:27 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:27.998 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:27 smithi038 ceph-mon[30945]: Deploying daemon osd.9 on smithi038 2024-09-16T00:28:28.043 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:27 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-16T00:28:28.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:27 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:28.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:27 smithi073 ceph-mon[35254]: Deploying daemon osd.9 on smithi038 2024-09-16T00:28:28.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:27 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-09-16T00:28:28.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:27 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:28.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:27 smithi122 ceph-mon[35412]: Deploying daemon osd.9 on smithi038 2024-09-16T00:28:29.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:28 smithi073 ceph-mon[35254]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-16T00:28:29.091 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:28 smithi038 ceph-mon[30945]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-16T00:28:29.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:28 smithi122 ceph-mon[35412]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-09-16T00:28:30.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:30 smithi038 ceph-mon[30945]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T00:28:31.043 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:30 smithi073 ceph-mon[35254]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T00:28:31.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:30 smithi122 ceph-mon[35412]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-16T00:28:32.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:32 smithi038 ceph-mon[30945]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-16T00:28:33.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:32 smithi073 ceph-mon[35254]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-16T00:28:33.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:32 smithi122 ceph-mon[35412]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-16T00:28:34.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:33 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:28:34.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:33 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:34.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:33 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:34.150 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:33 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:28:34.150 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:33 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:34.151 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:33 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:34.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:33 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:28:34.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:33 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:34.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:33 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:35.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:34 smithi073 ceph-mon[35254]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:28:35.169 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:34 smithi038 ceph-mon[30945]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:28:35.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:34 smithi122 ceph-mon[35412]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:28:36.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:36 smithi038 ceph-mon[30945]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:28:37.043 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:36 smithi073 ceph-mon[35254]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:28:37.134 INFO:teuthology.orchestra.run.smithi038.stdout:Created osd(s) 9 on host 'smithi038' 2024-09-16T00:28:37.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:37.133+0000 7f70c89e9640 1 -- 172.21.15.38:0/2543994144 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f70c4062630 con 0x7f70ac0a32f0 2024-09-16T00:28:37.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:37.137+0000 7f70c99eb640 1 -- 172.21.15.38:0/2543994144 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f70ac0a32f0 msgr2=0x7f70ac0a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:37.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:37.137+0000 7f70c99eb640 1 --2- 172.21.15.38:0/2543994144 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f70ac0a32f0 0x7f70ac0a57b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f70b4000c00 tx=0x7f70b40315d0 comp rx=0 tx=0).stop 2024-09-16T00:28:37.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:37.137+0000 7f70c99eb640 1 -- 172.21.15.38:0/2543994144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70c41078c0 msgr2=0x7f70c419f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:37.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:37.137+0000 7f70c99eb640 1 --2- 172.21.15.38:0/2543994144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70c41078c0 0x7f70c419f460 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f70bc00cce0 tx=0x7f70bc00a620 comp rx=0 tx=0).stop 2024-09-16T00:28:37.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:37.137+0000 7f70c99eb640 1 -- 172.21.15.38:0/2543994144 shutdown_connections 2024-09-16T00:28:37.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:37.137+0000 7f70c99eb640 1 --2- 172.21.15.38:0/2543994144 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f70ac0a32f0 0x7f70ac0a57b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:37.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:37.137+0000 7f70c99eb640 1 --2- 172.21.15.38:0/2543994144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f70c41078c0 0x7f70c419f460 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:37.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:37.137+0000 7f70c99eb640 1 --2- 172.21.15.38:0/2543994144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70c4104f90 0x7f70c41983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:37.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:37.137+0000 7f70c99eb640 1 --2- 172.21.15.38:0/2543994144 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f70c4102660 0x7f70c4197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:37.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:37.137+0000 7f70c99eb640 1 -- 172.21.15.38:0/2543994144 >> 172.21.15.38:0/2543994144 conn(0x7f70c40fc280 msgr2=0x7f70c4102f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:37.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:37.137+0000 7f70c99eb640 1 -- 172.21.15.38:0/2543994144 shutdown_connections 2024-09-16T00:28:37.138 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:28:37.137+0000 7f70c99eb640 1 -- 172.21.15.38:0/2543994144 wait complete. 2024-09-16T00:28:37.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:36 smithi122 ceph-mon[35412]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:28:37.830 DEBUG:teuthology.orchestra.run.smithi038:osd.9> sudo journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@osd.9.service 2024-09-16T00:28:37.833 INFO:tasks.cephadm:Deploying osd.10 on smithi073 with /dev/vg_nvme/lv_1... 2024-09-16T00:28:37.833 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-16T00:28:38.086 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:28:38.380 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:38 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:38.380 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:38 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:38.380 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:38 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:38.380 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:38 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:38.388 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:38 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:38.388 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:38 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:38.388 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:38 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:38.388 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:38 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:38.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:38 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:38.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:38 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:38.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:38 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:38.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:38 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:39.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:39 smithi073 ceph-mon[35254]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:28:39.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:39 smithi038 ceph-mon[30945]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:28:39.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:39 smithi122 ceph-mon[35412]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:28:39.841 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:28:39 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:28:39.488+0000 7f3070f80640 -1 osd.9 0 log_to_monitors true 2024-09-16T00:28:40.284 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:40 smithi073 ceph-mon[35254]: from='osd.9 [v2:172.21.15.38:6826/445727833,v1:172.21.15.38:6827/445727833]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-16T00:28:40.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:40 smithi038 ceph-mon[30945]: from='osd.9 [v2:172.21.15.38:6826/445727833,v1:172.21.15.38:6827/445727833]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-16T00:28:40.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:40 smithi122 ceph-mon[35412]: from='osd.9 [v2:172.21.15.38:6826/445727833,v1:172.21.15.38:6827/445727833]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-09-16T00:28:40.817 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-09-16T00:28:40.842 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi073:vg_nvme/lv_1 2024-09-16T00:28:41.088 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:28:41.415 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:41 smithi073 ceph-mon[35254]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:28:41.416 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:41 smithi073 ceph-mon[35254]: from='osd.9 [v2:172.21.15.38:6826/445727833,v1:172.21.15.38:6827/445727833]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-16T00:28:41.416 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:41 smithi073 ceph-mon[35254]: osdmap e62: 10 total, 9 up, 10 in 2024-09-16T00:28:41.416 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:41 smithi073 ceph-mon[35254]: from='osd.9 [v2:172.21.15.38:6826/445727833,v1:172.21.15.38:6827/445727833]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-16T00:28:41.416 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:41 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:28:41.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:28:41 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:28:41.155+0000 7f306cf03640 -1 osd.9 0 waiting for initial osdmap 2024-09-16T00:28:41.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:28:41 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:28:41.168+0000 7f3068d2b640 -1 osd.9 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:28:41.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:41 smithi038 ceph-mon[30945]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:28:41.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:41 smithi038 ceph-mon[30945]: from='osd.9 [v2:172.21.15.38:6826/445727833,v1:172.21.15.38:6827/445727833]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-16T00:28:41.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:41 smithi038 ceph-mon[30945]: osdmap e62: 10 total, 9 up, 10 in 2024-09-16T00:28:41.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:41 smithi038 ceph-mon[30945]: from='osd.9 [v2:172.21.15.38:6826/445727833,v1:172.21.15.38:6827/445727833]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-16T00:28:41.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:41 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:28:41.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:41 smithi122 ceph-mon[35412]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:28:41.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:41 smithi122 ceph-mon[35412]: from='osd.9 [v2:172.21.15.38:6826/445727833,v1:172.21.15.38:6827/445727833]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-16T00:28:41.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:41 smithi122 ceph-mon[35412]: osdmap e62: 10 total, 9 up, 10 in 2024-09-16T00:28:41.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:41 smithi122 ceph-mon[35412]: from='osd.9 [v2:172.21.15.38:6826/445727833,v1:172.21.15.38:6827/445727833]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]: dispatch 2024-09-16T00:28:41.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:41 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:28:41.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.611+0000 7ff8536fe640 1 -- 172.21.15.73:0/4079689956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff84c105030 msgr2=0x7ff84c107420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:41.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.611+0000 7ff8536fe640 1 --2- 172.21.15.73:0/4079689956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff84c105030 0x7ff84c107420 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7ff83c009900 tx=0x7ff83c02a410 comp rx=0 tx=0).stop 2024-09-16T00:28:41.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.611+0000 7ff8536fe640 1 -- 172.21.15.73:0/4079689956 shutdown_connections 2024-09-16T00:28:41.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.611+0000 7ff8536fe640 1 --2- 172.21.15.73:0/4079689956 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff84c107960 0x7ff84c109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:41.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.611+0000 7ff8536fe640 1 --2- 172.21.15.73:0/4079689956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff84c105030 0x7ff84c107420 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:41.627 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.611+0000 7ff8536fe640 1 --2- 172.21.15.73:0/4079689956 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff84c102700 0x7ff84c104af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:41.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.611+0000 7ff8536fe640 1 -- 172.21.15.73:0/4079689956 >> 172.21.15.73:0/4079689956 conn(0x7ff84c0fc280 msgr2=0x7ff84c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:28:41.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.611+0000 7ff8536fe640 1 -- 172.21.15.73:0/4079689956 shutdown_connections 2024-09-16T00:28:41.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.611+0000 7ff8536fe640 1 -- 172.21.15.73:0/4079689956 wait complete. 2024-09-16T00:28:41.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.611+0000 7ff8536fe640 1 Processor -- start 2024-09-16T00:28:41.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.611+0000 7ff8536fe640 1 -- start start 2024-09-16T00:28:41.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.612+0000 7ff8536fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff84c102700 0x7ff84c102060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:41.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.612+0000 7ff8536fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff84c105030 0x7ff84c1006b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:41.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.612+0000 7ff8536fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff84c107960 0x7ff84c100bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:41.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.612+0000 7ff8536fe640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff84c10d830 con 0x7ff84c102700 2024-09-16T00:28:41.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.612+0000 7ff8536fe640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff84c10d6b0 con 0x7ff84c107960 2024-09-16T00:28:41.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.612+0000 7ff8536fe640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff84c10d9b0 con 0x7ff84c105030 2024-09-16T00:28:41.628 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.612+0000 7ff851c9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff84c107960 0x7ff84c100bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:41.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.612+0000 7ff85149a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff84c102700 0x7ff84c102060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:41.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.612+0000 7ff851c9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff84c107960 0x7ff84c100bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50250/0 (socket says 172.21.15.73:50250) 2024-09-16T00:28:41.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.612+0000 7ff851c9b640 1 -- 172.21.15.73:0/2845718851 learned_addr learned my addr 172.21.15.73:0/2845718851 (peer_addr_for_me v2:172.21.15.73:0/0) 2024-09-16T00:28:41.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.612+0000 7ff850c99640 1 --2- 172.21.15.73:0/2845718851 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff84c105030 0x7ff84c1006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:41.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.612+0000 7ff851c9b640 1 -- 172.21.15.73:0/2845718851 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff84c105030 msgr2=0x7ff84c1006b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:41.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.612+0000 7ff851c9b640 1 --2- 172.21.15.73:0/2845718851 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff84c105030 0x7ff84c1006b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:41.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.612+0000 7ff851c9b640 1 -- 172.21.15.73:0/2845718851 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff84c102700 msgr2=0x7ff84c102060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:28:41.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.612+0000 7ff851c9b640 1 --2- 172.21.15.73:0/2845718851 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff84c102700 0x7ff84c102060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:28:41.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.612+0000 7ff851c9b640 1 -- 172.21.15.73:0/2845718851 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff84c1014b0 con 0x7ff84c107960 2024-09-16T00:28:41.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.612+0000 7ff85149a640 1 --2- 172.21.15.73:0/2845718851 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff84c102700 0x7ff84c102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:28:41.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.612+0000 7ff851c9b640 1 --2- 172.21.15.73:0/2845718851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff84c107960 0x7ff84c100bf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff84400eb10 tx=0x7ff84400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:41.629 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.612+0000 7ff8427fc640 1 -- 172.21.15.73:0/2845718851 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff844004b20 con 0x7ff84c107960 2024-09-16T00:28:41.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.613+0000 7ff8427fc640 1 -- 172.21.15.73:0/2845718851 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff844004cc0 con 0x7ff84c107960 2024-09-16T00:28:41.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.613+0000 7ff8536fe640 1 -- 172.21.15.73:0/2845718851 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff84c1a20a0 con 0x7ff84c107960 2024-09-16T00:28:41.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.613+0000 7ff8427fc640 1 -- 172.21.15.73:0/2845718851 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff844010760 con 0x7ff84c107960 2024-09-16T00:28:41.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.613+0000 7ff8536fe640 1 -- 172.21.15.73:0/2845718851 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff84c1a25b0 con 0x7ff84c107960 2024-09-16T00:28:41.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.614+0000 7ff8536fe640 1 -- 172.21.15.73:0/2845718851 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff84c0fdd70 con 0x7ff84c107960 2024-09-16T00:28:41.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.615+0000 7ff8427fc640 1 -- 172.21.15.73:0/2845718851 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7ff844005020 con 0x7ff84c107960 2024-09-16T00:28:41.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.615+0000 7ff8427fc640 1 --2- 172.21.15.73:0/2845718851 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ff8380a32c0 0x7ff8380a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:28:41.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.615+0000 7ff8427fc640 1 -- 172.21.15.73:0/2845718851 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5981+0+0 (secure 0 0 0) 0x7ff8440cf520 con 0x7ff84c107960 2024-09-16T00:28:41.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.616+0000 7ff85149a640 1 --2- 172.21.15.73:0/2845718851 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ff8380a32c0 0x7ff8380a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:28:41.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.616+0000 7ff85149a640 1 --2- 172.21.15.73:0/2845718851 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ff8380a32c0 0x7ff8380a5780 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff84800bfd0 tx=0x7ff84800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:28:41.630 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.620+0000 7ff8427fc640 1 -- 172.21.15.73:0/2845718851 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff84409cb60 con 0x7ff84c107960 2024-09-16T00:28:41.797 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:28:41.797+0000 7ff8536fe640 1 -- 172.21.15.73:0/2845718851 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7ff84c062630 con 0x7ff8380a32c0 2024-09-16T00:28:42.455 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:42 smithi073 ceph-mon[35254]: from='osd.9 [v2:172.21.15.38:6826/445727833,v1:172.21.15.38:6827/445727833]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-16T00:28:42.455 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:42 smithi073 ceph-mon[35254]: osdmap e63: 10 total, 9 up, 10 in 2024-09-16T00:28:42.455 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:42 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:28:42.455 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:42 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:28:42.455 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:42 smithi073 ceph-mon[35254]: Detected new or changed devices on smithi038 2024-09-16T00:28:42.456 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:42 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:42.456 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:42 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:42.456 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:42 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:28:42.456 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:42 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:28:42.456 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:42 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:28:42.456 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:42 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:28:42.456 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:42 smithi073 ceph-mon[35254]: Adjusting osd_memory_target on smithi038 to 4273M 2024-09-16T00:28:42.456 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:42 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:42.456 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:42 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:42.456 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:42 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:28:42.456 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:42 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:42.456 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:42 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:28:42.456 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:42 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:42.457 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:42 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:28:42.457 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:42 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:42.457 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:42 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:28:42.457 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:42 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:28:42.457 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:42 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:42.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:42 smithi038 ceph-mon[30945]: from='osd.9 [v2:172.21.15.38:6826/445727833,v1:172.21.15.38:6827/445727833]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-16T00:28:42.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:42 smithi038 ceph-mon[30945]: osdmap e63: 10 total, 9 up, 10 in 2024-09-16T00:28:42.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:42 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:28:42.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:42 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:28:42.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:42 smithi038 ceph-mon[30945]: Detected new or changed devices on smithi038 2024-09-16T00:28:42.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:42 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:42.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:42 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:42.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:42 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:28:42.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:42 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:28:42.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:42 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:28:42.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:42 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:28:42.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:42 smithi038 ceph-mon[30945]: Adjusting osd_memory_target on smithi038 to 4273M 2024-09-16T00:28:42.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:42 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:42.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:42 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:42.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:42 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:28:42.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:42 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:42.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:42 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:28:42.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:42 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:42.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:42 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:28:42.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:42 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:42.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:42 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:28:42.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:42 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:28:42.467 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:42 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:42.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:42 smithi122 ceph-mon[35412]: from='osd.9 [v2:172.21.15.38:6826/445727833,v1:172.21.15.38:6827/445727833]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi038", "root=default"]}]': finished 2024-09-16T00:28:42.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:42 smithi122 ceph-mon[35412]: osdmap e63: 10 total, 9 up, 10 in 2024-09-16T00:28:42.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:42 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:28:42.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:42 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:28:42.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:42 smithi122 ceph-mon[35412]: Detected new or changed devices on smithi038 2024-09-16T00:28:42.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:42 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:42.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:42 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:42.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:42 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:28:42.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:42 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:28:42.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:42 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:28:42.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:42 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:28:42.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:42 smithi122 ceph-mon[35412]: Adjusting osd_memory_target on smithi038 to 4273M 2024-09-16T00:28:42.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:42 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:42.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:42 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:42.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:42 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:28:42.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:42 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:42.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:42 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:28:42.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:42 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:42.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:42 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:28:42.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:42 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:42.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:42 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:28:42.469 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:42 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:28:42.469 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:42 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:43.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:43 smithi073 ceph-mon[35254]: purged_snaps scrub starts 2024-09-16T00:28:43.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:43 smithi073 ceph-mon[35254]: purged_snaps scrub ok 2024-09-16T00:28:43.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:43 smithi073 ceph-mon[35254]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:28:43.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:43 smithi073 ceph-mon[35254]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:28:43.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:43 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:28:43.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:43 smithi073 ceph-mon[35254]: osd.9 [v2:172.21.15.38:6826/445727833,v1:172.21.15.38:6827/445727833] boot 2024-09-16T00:28:43.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:43 smithi073 ceph-mon[35254]: osdmap e64: 10 total, 10 up, 10 in 2024-09-16T00:28:43.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:43 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:28:43.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:43 smithi038 ceph-mon[30945]: purged_snaps scrub starts 2024-09-16T00:28:43.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:43 smithi038 ceph-mon[30945]: purged_snaps scrub ok 2024-09-16T00:28:43.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:43 smithi038 ceph-mon[30945]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:28:43.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:43 smithi038 ceph-mon[30945]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:28:43.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:43 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:28:43.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:43 smithi038 ceph-mon[30945]: osd.9 [v2:172.21.15.38:6826/445727833,v1:172.21.15.38:6827/445727833] boot 2024-09-16T00:28:43.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:43 smithi038 ceph-mon[30945]: osdmap e64: 10 total, 10 up, 10 in 2024-09-16T00:28:43.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:43 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:28:43.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:43 smithi122 ceph-mon[35412]: purged_snaps scrub starts 2024-09-16T00:28:43.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:43 smithi122 ceph-mon[35412]: purged_snaps scrub ok 2024-09-16T00:28:43.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:43 smithi122 ceph-mon[35412]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 273 MiB used, 804 GiB / 805 GiB avail 2024-09-16T00:28:43.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:43 smithi122 ceph-mon[35412]: from='client.24344 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi073:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:28:43.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:43 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:28:43.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:43 smithi122 ceph-mon[35412]: osd.9 [v2:172.21.15.38:6826/445727833,v1:172.21.15.38:6827/445727833] boot 2024-09-16T00:28:43.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:43 smithi122 ceph-mon[35412]: osdmap e64: 10 total, 10 up, 10 in 2024-09-16T00:28:43.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:43 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-09-16T00:28:44.710 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:44 smithi073 ceph-mon[35254]: osdmap e65: 10 total, 10 up, 10 in 2024-09-16T00:28:44.710 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:44 smithi073 ceph-mon[35254]: from='client.? 172.21.15.73:0/2831047406' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c22883d9-6672-4796-a961-963d21eb37fc"}]: dispatch 2024-09-16T00:28:44.711 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:44 smithi073 ceph-mon[35254]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c22883d9-6672-4796-a961-963d21eb37fc"}]: dispatch 2024-09-16T00:28:44.711 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:44 smithi073 ceph-mon[35254]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c22883d9-6672-4796-a961-963d21eb37fc"}]': finished 2024-09-16T00:28:44.711 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:44 smithi073 ceph-mon[35254]: osdmap e66: 11 total, 10 up, 11 in 2024-09-16T00:28:44.711 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:44 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:28:44.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:44 smithi038 ceph-mon[30945]: osdmap e65: 10 total, 10 up, 10 in 2024-09-16T00:28:44.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:44 smithi038 ceph-mon[30945]: from='client.? 172.21.15.73:0/2831047406' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c22883d9-6672-4796-a961-963d21eb37fc"}]: dispatch 2024-09-16T00:28:44.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:44 smithi038 ceph-mon[30945]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c22883d9-6672-4796-a961-963d21eb37fc"}]: dispatch 2024-09-16T00:28:44.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:44 smithi038 ceph-mon[30945]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c22883d9-6672-4796-a961-963d21eb37fc"}]': finished 2024-09-16T00:28:44.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:44 smithi038 ceph-mon[30945]: osdmap e66: 11 total, 10 up, 11 in 2024-09-16T00:28:44.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:44 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:28:44.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:44 smithi122 ceph-mon[35412]: osdmap e65: 10 total, 10 up, 10 in 2024-09-16T00:28:44.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:44 smithi122 ceph-mon[35412]: from='client.? 172.21.15.73:0/2831047406' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c22883d9-6672-4796-a961-963d21eb37fc"}]: dispatch 2024-09-16T00:28:44.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:44 smithi122 ceph-mon[35412]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c22883d9-6672-4796-a961-963d21eb37fc"}]: dispatch 2024-09-16T00:28:44.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:44 smithi122 ceph-mon[35412]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c22883d9-6672-4796-a961-963d21eb37fc"}]': finished 2024-09-16T00:28:44.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:44 smithi122 ceph-mon[35412]: osdmap e66: 11 total, 10 up, 11 in 2024-09-16T00:28:44.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:44 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:28:45.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:45 smithi038 ceph-mon[30945]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:28:45.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:45 smithi038 ceph-mon[30945]: from='client.? 172.21.15.73:0/1784072557' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:28:45.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:45 smithi122 ceph-mon[35412]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:28:45.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:45 smithi122 ceph-mon[35412]: from='client.? 172.21.15.73:0/1784072557' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:28:45.793 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:45 smithi073 ceph-mon[35254]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:28:45.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:45 smithi073 ceph-mon[35254]: from='client.? 172.21.15.73:0/1784072557' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:28:47.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:47 smithi038 ceph-mon[30945]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:28:47.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:47 smithi122 ceph-mon[35412]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:28:47.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:47 smithi073 ceph-mon[35254]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:28:49.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:49 smithi073 ceph-mon[35254]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:28:49.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:49 smithi038 ceph-mon[30945]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:28:49.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:49 smithi122 ceph-mon[35412]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:28:50.744 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:50 smithi073 ceph-mon[35254]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:28:50.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:50 smithi038 ceph-mon[30945]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:28:50.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:50 smithi122 ceph-mon[35412]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:28:52.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:52 smithi073 ceph-mon[35254]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:28:52.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:52 smithi038 ceph-mon[30945]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:28:52.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:52 smithi122 ceph-mon[35412]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:28:53.616 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:53 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-16T00:28:53.616 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:53 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:53.616 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:53 smithi073 ceph-mon[35254]: Deploying daemon osd.10 on smithi073 2024-09-16T00:28:53.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:53 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-16T00:28:53.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:53 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:53.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:53 smithi038 ceph-mon[30945]: Deploying daemon osd.10 on smithi073 2024-09-16T00:28:53.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:53 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-09-16T00:28:53.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:53 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:28:53.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:53 smithi122 ceph-mon[35412]: Deploying daemon osd.10 on smithi073 2024-09-16T00:28:54.771 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:54 smithi073 ceph-mon[35254]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:28:54.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:54 smithi038 ceph-mon[30945]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:28:54.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:54 smithi122 ceph-mon[35412]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:28:56.674 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:56 smithi073 ceph-mon[35254]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:28:56.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:56 smithi038 ceph-mon[30945]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:28:56.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:56 smithi122 ceph-mon[35412]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:28:58.777 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:58 smithi073 ceph-mon[35254]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:28:58.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:58 smithi038 ceph-mon[30945]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:28:58.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:58 smithi122 ceph-mon[35412]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:28:59.793 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:59 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:28:59.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:59 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:59.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:28:59 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:59.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:59 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:28:59.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:59 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:59.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:28:59 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:59.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:59 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:28:59.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:59 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:28:59.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:28:59 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:01.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:01 smithi073 ceph-mon[35254]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:29:01.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:01 smithi038 ceph-mon[30945]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:29:01.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:01 smithi122 ceph-mon[35412]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:29:02.569 INFO:teuthology.orchestra.run.smithi073.stdout:Created osd(s) 10 on host 'smithi073' 2024-09-16T00:29:02.569 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:29:02.568+0000 7ff8427fc640 1 -- 172.21.15.73:0/2845718851 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7ff84c062630 con 0x7ff8380a32c0 2024-09-16T00:29:02.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:29:02.570+0000 7ff8536fe640 1 -- 172.21.15.73:0/2845718851 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ff8380a32c0 msgr2=0x7ff8380a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:02.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:29:02.570+0000 7ff8536fe640 1 --2- 172.21.15.73:0/2845718851 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ff8380a32c0 0x7ff8380a5780 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff84800bfd0 tx=0x7ff84800b390 comp rx=0 tx=0).stop 2024-09-16T00:29:02.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:29:02.570+0000 7ff8536fe640 1 -- 172.21.15.73:0/2845718851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff84c107960 msgr2=0x7ff84c100bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:02.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:29:02.570+0000 7ff8536fe640 1 --2- 172.21.15.73:0/2845718851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff84c107960 0x7ff84c100bf0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff84400eb10 tx=0x7ff84400c2d0 comp rx=0 tx=0).stop 2024-09-16T00:29:02.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:29:02.571+0000 7ff8536fe640 1 -- 172.21.15.73:0/2845718851 shutdown_connections 2024-09-16T00:29:02.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:29:02.571+0000 7ff8536fe640 1 --2- 172.21.15.73:0/2845718851 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ff8380a32c0 0x7ff8380a5780 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:02.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:29:02.571+0000 7ff8536fe640 1 --2- 172.21.15.73:0/2845718851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff84c107960 0x7ff84c100bf0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:02.571 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:29:02.571+0000 7ff8536fe640 1 --2- 172.21.15.73:0/2845718851 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff84c105030 0x7ff84c1006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:02.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:29:02.571+0000 7ff8536fe640 1 --2- 172.21.15.73:0/2845718851 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff84c102700 0x7ff84c102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:02.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:29:02.571+0000 7ff8536fe640 1 -- 172.21.15.73:0/2845718851 >> 172.21.15.73:0/2845718851 conn(0x7ff84c0fc280 msgr2=0x7ff84c103080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:02.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:29:02.571+0000 7ff8536fe640 1 -- 172.21.15.73:0/2845718851 shutdown_connections 2024-09-16T00:29:02.572 INFO:teuthology.orchestra.run.smithi073.stderr:2024-09-16T00:29:02.571+0000 7ff8536fe640 1 -- 172.21.15.73:0/2845718851 wait complete. 2024-09-16T00:29:03.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:03 smithi073 ceph-mon[35254]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:29:03.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:03 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:03.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:03 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:03.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:03 smithi038 ceph-mon[30945]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:29:03.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:03 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:03.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:03 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:03.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:03 smithi122 ceph-mon[35412]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:29:03.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:03 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:03.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:03 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:03.495 DEBUG:teuthology.orchestra.run.smithi073:osd.10> sudo journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@osd.10.service 2024-09-16T00:29:03.497 INFO:tasks.cephadm:Deploying osd.11 on smithi122 with /dev/vg_nvme/lv_1... 2024-09-16T00:29:03.497 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- lvm zap /dev/vg_nvme/lv_1 2024-09-16T00:29:03.751 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:29:04.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:04 smithi038 ceph-mon[30945]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:29:04.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:04 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:04.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:04 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:04.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:04 smithi122 ceph-mon[35412]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:29:04.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:04 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:04.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:04 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:05.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:04 smithi073 ceph-mon[35254]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:29:05.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:04 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:05.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:04 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:05.311 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:29:05 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:29:05.044+0000 7fb12d90a640 -1 osd.10 0 log_to_monitors true 2024-09-16T00:29:05.866 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:05 smithi038 ceph-mon[30945]: from='osd.10 [v2:172.21.15.73:6824/43297139,v1:172.21.15.73:6825/43297139]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T00:29:05.867 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:05 smithi038 ceph-mon[30945]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T00:29:05.967 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:05 smithi122 ceph-mon[35412]: from='osd.10 [v2:172.21.15.73:6824/43297139,v1:172.21.15.73:6825/43297139]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T00:29:05.967 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:05 smithi122 ceph-mon[35412]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T00:29:06.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:05 smithi073 ceph-mon[35254]: from='osd.10 [v2:172.21.15.73:6824/43297139,v1:172.21.15.73:6825/43297139]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T00:29:06.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:05 smithi073 ceph-mon[35254]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-09-16T00:29:06.569 INFO:teuthology.orchestra.run.smithi122.stdout: 2024-09-16T00:29:06.598 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph orch daemon add osd smithi122:vg_nvme/lv_1 2024-09-16T00:29:06.853 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:29:06.878 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:06 smithi122 ceph-mon[35412]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:29:06.878 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:06 smithi122 ceph-mon[35412]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-16T00:29:06.878 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:06 smithi122 ceph-mon[35412]: from='osd.10 [v2:172.21.15.73:6824/43297139,v1:172.21.15.73:6825/43297139]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-09-16T00:29:06.878 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:06 smithi122 ceph-mon[35412]: osdmap e67: 11 total, 10 up, 11 in 2024-09-16T00:29:06.878 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:06 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:29:06.878 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:06 smithi122 ceph-mon[35412]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-09-16T00:29:06.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:06 smithi038 ceph-mon[30945]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:29:06.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:06 smithi038 ceph-mon[30945]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-16T00:29:06.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:06 smithi038 ceph-mon[30945]: from='osd.10 [v2:172.21.15.73:6824/43297139,v1:172.21.15.73:6825/43297139]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-09-16T00:29:06.966 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:06 smithi038 ceph-mon[30945]: osdmap e67: 11 total, 10 up, 11 in 2024-09-16T00:29:06.966 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:06 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:29:06.966 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:06 smithi038 ceph-mon[30945]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-09-16T00:29:06.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:06 smithi073 ceph-mon[35254]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:29:06.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:06 smithi073 ceph-mon[35254]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-16T00:29:06.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:06 smithi073 ceph-mon[35254]: from='osd.10 [v2:172.21.15.73:6824/43297139,v1:172.21.15.73:6825/43297139]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-09-16T00:29:06.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:06 smithi073 ceph-mon[35254]: osdmap e67: 11 total, 10 up, 11 in 2024-09-16T00:29:06.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:06 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:29:06.969 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:06 smithi073 ceph-mon[35254]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]: dispatch 2024-09-16T00:29:07.378 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:07.364+0000 7fb3832d6640 1 -- 172.21.15.122:0/1360034882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb37c102660 msgr2=0x7fb37c104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:07.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:07.364+0000 7fb3832d6640 1 --2- 172.21.15.122:0/1360034882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb37c102660 0x7fb37c104a50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb378009900 tx=0x7fb37802a410 comp rx=0 tx=0).stop 2024-09-16T00:29:07.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:07.365+0000 7fb3832d6640 1 -- 172.21.15.122:0/1360034882 shutdown_connections 2024-09-16T00:29:07.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:07.365+0000 7fb3832d6640 1 --2- 172.21.15.122:0/1360034882 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb37c1078c0 0x7fb37c109d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:07.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:07.365+0000 7fb3832d6640 1 --2- 172.21.15.122:0/1360034882 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb37c104f90 0x7fb37c107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:07.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:07.365+0000 7fb3832d6640 1 --2- 172.21.15.122:0/1360034882 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb37c102660 0x7fb37c104a50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:07.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:07.365+0000 7fb3832d6640 1 -- 172.21.15.122:0/1360034882 >> 172.21.15.122:0/1360034882 conn(0x7fb37c0fc280 msgr2=0x7fb37c0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:07.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:07.365+0000 7fb3832d6640 1 -- 172.21.15.122:0/1360034882 shutdown_connections 2024-09-16T00:29:07.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:07.365+0000 7fb3832d6640 1 -- 172.21.15.122:0/1360034882 wait complete. 2024-09-16T00:29:07.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:07.365+0000 7fb3832d6640 1 Processor -- start 2024-09-16T00:29:07.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:07.365+0000 7fb3832d6640 1 -- start start 2024-09-16T00:29:07.379 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:07.366+0000 7fb3832d6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb37c102660 0x7fb37c197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:07.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:07.366+0000 7fb3832d6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb37c104f90 0x7fb37c1984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:07.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:07.366+0000 7fb3832d6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb37c1078c0 0x7fb37c19f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:07.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:07.366+0000 7fb3832d6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c10d900 con 0x7fb37c104f90 2024-09-16T00:29:07.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:07.366+0000 7fb3832d6640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c10d780 con 0x7fb37c1078c0 2024-09-16T00:29:07.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:07.366+0000 7fb3832d6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c10da80 con 0x7fb37c102660 2024-09-16T00:29:07.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:07.366+0000 7fb381072640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb37c102660 0x7fb37c197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:07.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:07.366+0000 7fb381072640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb37c102660 0x7fb37c197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:52982/0 (socket says 172.21.15.122:52982) 2024-09-16T00:29:07.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:07.366+0000 7fb381072640 1 -- 172.21.15.122:0/3600174881 learned_addr learned my addr 172.21.15.122:0/3600174881 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-16T00:29:07.380 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:07.366+0000 7fb381873640 1 --2- 172.21.15.122:0/3600174881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb37c1078c0 0x7fb37c19f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:07.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:07.366+0000 7fb380871640 1 --2- 172.21.15.122:0/3600174881 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb37c104f90 0x7fb37c1984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:07.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:07.366+0000 7fb381072640 1 -- 172.21.15.122:0/3600174881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb37c1078c0 msgr2=0x7fb37c19f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:07.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:07.366+0000 7fb381072640 1 --2- 172.21.15.122:0/3600174881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb37c1078c0 0x7fb37c19f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:07.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:07.366+0000 7fb381072640 1 -- 172.21.15.122:0/3600174881 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb37c104f90 msgr2=0x7fb37c1984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:07.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:07.366+0000 7fb381072640 1 --2- 172.21.15.122:0/3600174881 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb37c104f90 0x7fb37c1984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:07.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:07.366+0000 7fb381072640 1 -- 172.21.15.122:0/3600174881 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb37c19fbb0 con 0x7fb37c102660 2024-09-16T00:29:07.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:07.366+0000 7fb381072640 1 --2- 172.21.15.122:0/3600174881 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb37c102660 0x7fb37c197f80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb378009230 tx=0x7fb378004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:07.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:07.366+0000 7fb3727fc640 1 -- 172.21.15.122:0/3600174881 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37803f030 con 0x7fb37c102660 2024-09-16T00:29:07.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:07.366+0000 7fb3832d6640 1 -- 172.21.15.122:0/3600174881 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb37c19fe40 con 0x7fb37c102660 2024-09-16T00:29:07.381 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:07.366+0000 7fb3727fc640 1 -- 172.21.15.122:0/3600174881 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb378003ce0 con 0x7fb37c102660 2024-09-16T00:29:07.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:07.367+0000 7fb3727fc640 1 -- 172.21.15.122:0/3600174881 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb378035070 con 0x7fb37c102660 2024-09-16T00:29:07.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:07.367+0000 7fb3832d6640 1 -- 172.21.15.122:0/3600174881 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb37c1a02b0 con 0x7fb37c102660 2024-09-16T00:29:07.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:07.368+0000 7fb3832d6640 1 -- 172.21.15.122:0/3600174881 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb37c0fdd70 con 0x7fb37c102660 2024-09-16T00:29:07.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:07.369+0000 7fb3727fc640 1 -- 172.21.15.122:0/3600174881 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fb378042030 con 0x7fb37c102660 2024-09-16T00:29:07.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:07.369+0000 7fb3727fc640 1 --2- 172.21.15.122:0/3600174881 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb3680a3270 0x7fb3680a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:07.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:07.369+0000 7fb3727fc640 1 -- 172.21.15.122:0/3600174881 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6414+0+0 (secure 0 0 0) 0x7fb3780ef6f0 con 0x7fb37c102660 2024-09-16T00:29:07.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:07.369+0000 7fb380871640 1 --2- 172.21.15.122:0/3600174881 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb3680a3270 0x7fb3680a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:07.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:07.370+0000 7fb380871640 1 --2- 172.21.15.122:0/3600174881 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb3680a3270 0x7fb3680a5730 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb37c1994a0 tx=0x7fb36c009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:07.382 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:07.373+0000 7fb3727fc640 1 -- 172.21.15.122:0/3600174881 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb3780bcb80 con 0x7fb37c102660 2024-09-16T00:29:07.544 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:29:07 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:29:07.285+0000 7fb12988d640 -1 osd.10 0 waiting for initial osdmap 2024-09-16T00:29:07.544 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:29:07 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:29:07.301+0000 7fb1216ad640 -1 osd.10 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:29:07.560 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:07.559+0000 7fb3832d6640 1 -- 172.21.15.122:0/3600174881 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fb37c062630 con 0x7fb3680a3270 2024-09-16T00:29:07.892 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:07 smithi122 ceph-mon[35412]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-09-16T00:29:07.892 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:07 smithi122 ceph-mon[35412]: osdmap e68: 11 total, 10 up, 11 in 2024-09-16T00:29:07.892 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:07 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:29:07.892 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:07 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:29:07.892 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:07 smithi122 ceph-mon[35412]: Detected new or changed devices on smithi073 2024-09-16T00:29:07.893 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:07 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:07.893 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:07 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:07.893 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:07 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:29:07.893 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:07 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:29:07.893 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:07 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:29:07.893 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:07 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:29:07.893 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:07 smithi122 ceph-mon[35412]: Adjusting osd_memory_target on smithi073 to 4272M 2024-09-16T00:29:07.893 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:07 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:07.893 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:07 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:07.893 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:07 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:29:07.893 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:07 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:07.893 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:07 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:29:07.893 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:07 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:07.894 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:07 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:29:07.894 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:07 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:07.894 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:07 smithi122 ceph-mon[35412]: from='osd.10 ' entity='osd.10' 2024-09-16T00:29:07.894 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:07 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:29:07.894 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:07 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:29:07.894 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:07 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:07.894 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:07 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:29:07.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:07 smithi038 ceph-mon[30945]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-09-16T00:29:07.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:07 smithi038 ceph-mon[30945]: osdmap e68: 11 total, 10 up, 11 in 2024-09-16T00:29:07.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:07 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:29:07.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:07 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:29:07.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:07 smithi038 ceph-mon[30945]: Detected new or changed devices on smithi073 2024-09-16T00:29:07.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:07 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:07.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:07 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:07.966 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:07 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:29:07.966 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:07 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:29:07.966 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:07 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:29:07.966 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:07 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:29:07.966 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:07 smithi038 ceph-mon[30945]: Adjusting osd_memory_target on smithi073 to 4272M 2024-09-16T00:29:07.966 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:07 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:07.966 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:07 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:07.966 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:07 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:29:07.966 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:07 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:07.966 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:07 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:29:07.966 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:07 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:07.966 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:07 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:29:07.966 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:07 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:07.966 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:07 smithi038 ceph-mon[30945]: from='osd.10 ' entity='osd.10' 2024-09-16T00:29:07.967 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:07 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:29:07.967 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:07 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:29:07.967 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:07 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:07.967 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:07 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:29:08.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:07 smithi073 ceph-mon[35254]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi073", "root=default"]}]': finished 2024-09-16T00:29:08.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:07 smithi073 ceph-mon[35254]: osdmap e68: 11 total, 10 up, 11 in 2024-09-16T00:29:08.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:07 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:29:08.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:07 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:29:08.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:07 smithi073 ceph-mon[35254]: Detected new or changed devices on smithi073 2024-09-16T00:29:08.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:07 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:08.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:07 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:08.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:07 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:29:08.045 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:07 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:29:08.045 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:07 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:29:08.045 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:07 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:29:08.045 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:07 smithi073 ceph-mon[35254]: Adjusting osd_memory_target on smithi073 to 4272M 2024-09-16T00:29:08.045 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:07 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:08.045 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:07 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:08.045 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:07 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:29:08.045 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:07 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:08.045 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:07 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:29:08.045 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:07 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:08.045 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:07 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:29:08.045 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:07 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:08.046 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:07 smithi073 ceph-mon[35254]: from='osd.10 ' entity='osd.10' 2024-09-16T00:29:08.046 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:07 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-16T00:29:08.046 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:07 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-16T00:29:08.046 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:07 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:08.046 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:07 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:29:08.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:08 smithi038 ceph-mon[30945]: purged_snaps scrub starts 2024-09-16T00:29:08.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:08 smithi038 ceph-mon[30945]: purged_snaps scrub ok 2024-09-16T00:29:08.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:08 smithi038 ceph-mon[30945]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:29:08.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:08 smithi038 ceph-mon[30945]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:29:08.966 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:08 smithi038 ceph-mon[30945]: osd.10 [v2:172.21.15.73:6824/43297139,v1:172.21.15.73:6825/43297139] boot 2024-09-16T00:29:08.966 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:08 smithi038 ceph-mon[30945]: osdmap e69: 11 total, 11 up, 11 in 2024-09-16T00:29:08.966 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:08 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:29:08.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:08 smithi122 ceph-mon[35412]: purged_snaps scrub starts 2024-09-16T00:29:08.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:08 smithi122 ceph-mon[35412]: purged_snaps scrub ok 2024-09-16T00:29:08.967 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:08 smithi122 ceph-mon[35412]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:29:08.967 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:08 smithi122 ceph-mon[35412]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:29:08.967 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:08 smithi122 ceph-mon[35412]: osd.10 [v2:172.21.15.73:6824/43297139,v1:172.21.15.73:6825/43297139] boot 2024-09-16T00:29:08.967 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:08 smithi122 ceph-mon[35412]: osdmap e69: 11 total, 11 up, 11 in 2024-09-16T00:29:08.967 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:08 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:29:09.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:08 smithi073 ceph-mon[35254]: purged_snaps scrub starts 2024-09-16T00:29:09.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:08 smithi073 ceph-mon[35254]: purged_snaps scrub ok 2024-09-16T00:29:09.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:08 smithi073 ceph-mon[35254]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-09-16T00:29:09.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:08 smithi073 ceph-mon[35254]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi122:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:29:09.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:08 smithi073 ceph-mon[35254]: osd.10 [v2:172.21.15.73:6824/43297139,v1:172.21.15.73:6825/43297139] boot 2024-09-16T00:29:09.045 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:08 smithi073 ceph-mon[35254]: osdmap e69: 11 total, 11 up, 11 in 2024-09-16T00:29:09.045 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:08 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-09-16T00:29:09.820 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:09 smithi122 ceph-mon[35412]: osdmap e70: 11 total, 11 up, 11 in 2024-09-16T00:29:09.964 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:09 smithi038 ceph-mon[30945]: osdmap e70: 11 total, 11 up, 11 in 2024-09-16T00:29:10.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:09 smithi073 ceph-mon[35254]: osdmap e70: 11 total, 11 up, 11 in 2024-09-16T00:29:10.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:10 smithi122 ceph-mon[35412]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:29:10.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:10 smithi122 ceph-mon[35412]: osdmap e71: 11 total, 11 up, 11 in 2024-09-16T00:29:10.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:10 smithi122 ceph-mon[35412]: from='client.? 172.21.15.122:0/2944825712' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f98e8cb-1bff-4b8b-aa19-56b691387383"}]: dispatch 2024-09-16T00:29:10.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:10 smithi122 ceph-mon[35412]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f98e8cb-1bff-4b8b-aa19-56b691387383"}]: dispatch 2024-09-16T00:29:10.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:10 smithi122 ceph-mon[35412]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f98e8cb-1bff-4b8b-aa19-56b691387383"}]': finished 2024-09-16T00:29:10.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:10 smithi122 ceph-mon[35412]: osdmap e72: 12 total, 11 up, 12 in 2024-09-16T00:29:10.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:10 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:29:10.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:10 smithi038 ceph-mon[30945]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:29:10.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:10 smithi038 ceph-mon[30945]: osdmap e71: 11 total, 11 up, 11 in 2024-09-16T00:29:10.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:10 smithi038 ceph-mon[30945]: from='client.? 172.21.15.122:0/2944825712' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f98e8cb-1bff-4b8b-aa19-56b691387383"}]: dispatch 2024-09-16T00:29:10.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:10 smithi038 ceph-mon[30945]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f98e8cb-1bff-4b8b-aa19-56b691387383"}]: dispatch 2024-09-16T00:29:10.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:10 smithi038 ceph-mon[30945]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f98e8cb-1bff-4b8b-aa19-56b691387383"}]': finished 2024-09-16T00:29:10.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:10 smithi038 ceph-mon[30945]: osdmap e72: 12 total, 11 up, 12 in 2024-09-16T00:29:10.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:10 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:29:11.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:10 smithi073 ceph-mon[35254]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:29:11.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:10 smithi073 ceph-mon[35254]: osdmap e71: 11 total, 11 up, 11 in 2024-09-16T00:29:11.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:10 smithi073 ceph-mon[35254]: from='client.? 172.21.15.122:0/2944825712' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f98e8cb-1bff-4b8b-aa19-56b691387383"}]: dispatch 2024-09-16T00:29:11.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:10 smithi073 ceph-mon[35254]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9f98e8cb-1bff-4b8b-aa19-56b691387383"}]: dispatch 2024-09-16T00:29:11.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:10 smithi073 ceph-mon[35254]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9f98e8cb-1bff-4b8b-aa19-56b691387383"}]': finished 2024-09-16T00:29:11.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:10 smithi073 ceph-mon[35254]: osdmap e72: 12 total, 11 up, 12 in 2024-09-16T00:29:11.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:10 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:29:11.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:11 smithi038 ceph-mon[30945]: from='client.? 172.21.15.122:0/4199726978' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:29:11.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:11 smithi038 ceph-mon[30945]: osdmap e73: 12 total, 11 up, 12 in 2024-09-16T00:29:11.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:11 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:29:11.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:11 smithi122 ceph-mon[35412]: from='client.? 172.21.15.122:0/4199726978' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:29:11.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:11 smithi122 ceph-mon[35412]: osdmap e73: 12 total, 11 up, 12 in 2024-09-16T00:29:11.967 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:11 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:29:12.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:11 smithi073 ceph-mon[35254]: from='client.? 172.21.15.122:0/4199726978' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-16T00:29:12.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:11 smithi073 ceph-mon[35254]: osdmap e73: 12 total, 11 up, 12 in 2024-09-16T00:29:12.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:11 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:29:13.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:13 smithi038 ceph-mon[30945]: pgmap v238: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 149 KiB/s, 0 objects/s recovering 2024-09-16T00:29:13.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:13 smithi122 ceph-mon[35412]: pgmap v238: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 149 KiB/s, 0 objects/s recovering 2024-09-16T00:29:13.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:13 smithi073 ceph-mon[35254]: pgmap v238: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 149 KiB/s, 0 objects/s recovering 2024-09-16T00:29:15.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:15 smithi122 ceph-mon[35412]: pgmap v239: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-09-16T00:29:15.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:15 smithi038 ceph-mon[30945]: pgmap v239: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-09-16T00:29:15.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:15 smithi073 ceph-mon[35254]: pgmap v239: 1 pgs: 1 peering; 577 KiB data, 332 MiB used, 983 GiB / 983 GiB avail; 118 KiB/s, 0 objects/s recovering 2024-09-16T00:29:17.463 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:17 smithi073 ceph-mon[35254]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T00:29:17.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:17 smithi038 ceph-mon[30945]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T00:29:17.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:17 smithi122 ceph-mon[35412]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-16T00:29:19.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:19 smithi038 ceph-mon[30945]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-16T00:29:19.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:19 smithi122 ceph-mon[35412]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-16T00:29:19.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:19 smithi073 ceph-mon[35254]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-16T00:29:20.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:20 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-16T00:29:20.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:20 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:20.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:20 smithi122 ceph-mon[35412]: Deploying daemon osd.11 on smithi122 2024-09-16T00:29:20.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:20 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-16T00:29:20.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:20 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:20.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:20 smithi038 ceph-mon[30945]: Deploying daemon osd.11 on smithi122 2024-09-16T00:29:20.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:20 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-09-16T00:29:20.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:20 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:20.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:20 smithi073 ceph-mon[35254]: Deploying daemon osd.11 on smithi122 2024-09-16T00:29:21.270 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:21 smithi122 ceph-mon[35412]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-16T00:29:21.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:21 smithi038 ceph-mon[30945]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-16T00:29:21.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:21 smithi073 ceph-mon[35254]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail; 62 KiB/s, 0 objects/s recovering 2024-09-16T00:29:23.451 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:23 smithi122 ceph-mon[35412]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:29:23.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:23 smithi038 ceph-mon[30945]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:29:23.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:23 smithi073 ceph-mon[35254]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:29:25.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:25 smithi038 ceph-mon[30945]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:29:25.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:25 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:29:25.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:25 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:25.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:25 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:25.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:25 smithi122 ceph-mon[35412]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:29:25.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:25 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:29:25.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:25 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:25.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:25 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:25.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:25 smithi073 ceph-mon[35254]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:29:25.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:25 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:29:25.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:25 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:25.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:25 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:27.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:27 smithi038 ceph-mon[30945]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:29:27.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:27 smithi122 ceph-mon[35412]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:29:27.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:27 smithi073 ceph-mon[35254]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:29:28.495 INFO:teuthology.orchestra.run.smithi122.stdout:Created osd(s) 11 on host 'smithi122' 2024-09-16T00:29:28.495 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:28.495+0000 7fb3727fc640 1 -- 172.21.15.122:0/3600174881 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fb37c062630 con 0x7fb3680a3270 2024-09-16T00:29:28.497 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:28.497+0000 7fb3832d6640 1 -- 172.21.15.122:0/3600174881 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb3680a3270 msgr2=0x7fb3680a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:28.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:28.497+0000 7fb3832d6640 1 --2- 172.21.15.122:0/3600174881 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb3680a3270 0x7fb3680a5730 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb37c1994a0 tx=0x7fb36c009110 comp rx=0 tx=0).stop 2024-09-16T00:29:28.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:28.497+0000 7fb3832d6640 1 -- 172.21.15.122:0/3600174881 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb37c102660 msgr2=0x7fb37c197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:28.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:28.497+0000 7fb3832d6640 1 --2- 172.21.15.122:0/3600174881 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb37c102660 0x7fb37c197f80 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fb378009230 tx=0x7fb378004360 comp rx=0 tx=0).stop 2024-09-16T00:29:28.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:28.497+0000 7fb3832d6640 1 -- 172.21.15.122:0/3600174881 shutdown_connections 2024-09-16T00:29:28.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:28.497+0000 7fb3832d6640 1 --2- 172.21.15.122:0/3600174881 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb3680a3270 0x7fb3680a5730 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:28.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:28.497+0000 7fb3832d6640 1 --2- 172.21.15.122:0/3600174881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb37c1078c0 0x7fb37c19f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:28.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:28.497+0000 7fb3832d6640 1 --2- 172.21.15.122:0/3600174881 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb37c104f90 0x7fb37c1984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:28.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:28.497+0000 7fb3832d6640 1 --2- 172.21.15.122:0/3600174881 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb37c102660 0x7fb37c197f80 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:28.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:28.497+0000 7fb3832d6640 1 -- 172.21.15.122:0/3600174881 >> 172.21.15.122:0/3600174881 conn(0x7fb37c0fc280 msgr2=0x7fb37c1030a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:28.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:28.497+0000 7fb3832d6640 1 -- 172.21.15.122:0/3600174881 shutdown_connections 2024-09-16T00:29:28.498 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:28.497+0000 7fb3832d6640 1 -- 172.21.15.122:0/3600174881 wait complete. 2024-09-16T00:29:29.146 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:28 smithi122 ceph-mon[35412]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:29:29.147 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:28 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:29.147 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:28 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:29.147 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:28 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:29.147 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:28 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:29:29.147 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:28 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:29.147 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:28 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:29:29.147 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:28 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:29.147 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:28 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:29.174 DEBUG:teuthology.orchestra.run.smithi122:osd.11> sudo journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@osd.11.service 2024-09-16T00:29:29.177 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-16T00:29:29.177 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-16T00:29:29.206 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:28 smithi038 ceph-mon[30945]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:29:29.206 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:28 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:29.206 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:28 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:29.206 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:28 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:29.206 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:28 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:29:29.206 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:28 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:29.207 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:28 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:29:29.207 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:28 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:29.207 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:28 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:29.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:28 smithi073 ceph-mon[35254]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:29:29.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:28 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:29.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:28 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:29.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:28 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:29.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:28 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:29:29.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:28 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:29.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:28 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:29:29.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:28 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:29.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:28 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:29.445 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:29:29.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:29.935+0000 7f0fb3677640 1 -- 172.21.15.38:0/1190545442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0fac1078c0 msgr2=0x7f0fac109d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:29.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:29.935+0000 7f0fb3677640 1 --2- 172.21.15.38:0/1190545442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0fac1078c0 0x7f0fac109d10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f0fa4009ca0 tx=0x7f0fa402a8c0 comp rx=0 tx=0).stop 2024-09-16T00:29:29.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:29.935+0000 7f0fb3677640 1 -- 172.21.15.38:0/1190545442 shutdown_connections 2024-09-16T00:29:29.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:29.935+0000 7f0fb3677640 1 --2- 172.21.15.38:0/1190545442 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0fac1078c0 0x7f0fac109d10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:29.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:29.935+0000 7f0fb3677640 1 --2- 172.21.15.38:0/1190545442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fac104f90 0x7f0fac107380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:29.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:29.935+0000 7f0fb3677640 1 --2- 172.21.15.38:0/1190545442 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0fac102660 0x7f0fac104a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:29.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:29.935+0000 7f0fb3677640 1 -- 172.21.15.38:0/1190545442 >> 172.21.15.38:0/1190545442 conn(0x7f0fac0fc280 msgr2=0x7f0fac0fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:29.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:29.936+0000 7f0fb3677640 1 -- 172.21.15.38:0/1190545442 shutdown_connections 2024-09-16T00:29:29.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:29.936+0000 7f0fb3677640 1 -- 172.21.15.38:0/1190545442 wait complete. 2024-09-16T00:29:29.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:29.936+0000 7f0fb3677640 1 Processor -- start 2024-09-16T00:29:29.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:29.936+0000 7f0fb3677640 1 -- start start 2024-09-16T00:29:29.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:29.936+0000 7f0fb3677640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0fac102660 0x7f0fac197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:29.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:29.936+0000 7f0fb3677640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0fac104f90 0x7f0fac198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:29.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:29.936+0000 7f0fb3677640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fac1078c0 0x7f0fac19f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:29.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:29.936+0000 7f0fb3677640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0fac10d880 con 0x7f0fac104f90 2024-09-16T00:29:29.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:29.936+0000 7f0fb3677640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0fac10d700 con 0x7f0fac1078c0 2024-09-16T00:29:29.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:29.936+0000 7f0fb3677640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0fac10da00 con 0x7f0fac102660 2024-09-16T00:29:29.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:29.937+0000 7f0fb0c12640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0fac104f90 0x7f0fac198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:29.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:29.937+0000 7f0fb1413640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0fac102660 0x7f0fac197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:29.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:29.937+0000 7f0fb0c12640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0fac104f90 0x7f0fac198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40096/0 (socket says 172.21.15.38:40096) 2024-09-16T00:29:29.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:29.937+0000 7f0fb0c12640 1 -- 172.21.15.38:0/2149633268 learned_addr learned my addr 172.21.15.38:0/2149633268 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:29:29.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:29.937+0000 7f0fb1c14640 1 --2- 172.21.15.38:0/2149633268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fac1078c0 0x7f0fac19f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:29.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:29.937+0000 7f0fb1413640 1 -- 172.21.15.38:0/2149633268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fac1078c0 msgr2=0x7f0fac19f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:29.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:29.937+0000 7f0fb1413640 1 --2- 172.21.15.38:0/2149633268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fac1078c0 0x7f0fac19f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:29.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:29.937+0000 7f0fb1413640 1 -- 172.21.15.38:0/2149633268 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0fac104f90 msgr2=0x7f0fac198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:29.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:29.937+0000 7f0fb1413640 1 --2- 172.21.15.38:0/2149633268 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0fac104f90 0x7f0fac198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:29.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:29.937+0000 7f0fb1413640 1 -- 172.21.15.38:0/2149633268 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fac19fb10 con 0x7f0fac102660 2024-09-16T00:29:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:29.937+0000 7f0fb0c12640 1 --2- 172.21.15.38:0/2149633268 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0fac104f90 0x7f0fac198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:29:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:29.937+0000 7f0fb1413640 1 --2- 172.21.15.38:0/2149633268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0fac102660 0x7f0fac197ee0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0fa800e9f0 tx=0x7f0fa800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:29.937+0000 7f0fa27fc640 1 -- 172.21.15.38:0/2149633268 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa800ccc0 con 0x7f0fac102660 2024-09-16T00:29:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:29.937+0000 7f0fb3677640 1 -- 172.21.15.38:0/2149633268 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fac19fe00 con 0x7f0fac102660 2024-09-16T00:29:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:29.937+0000 7f0fa27fc640 1 -- 172.21.15.38:0/2149633268 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fa8005020 con 0x7f0fac102660 2024-09-16T00:29:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:29.937+0000 7f0fa27fc640 1 -- 172.21.15.38:0/2149633268 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fa80049e0 con 0x7f0fac102660 2024-09-16T00:29:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:29.937+0000 7f0fb3677640 1 -- 172.21.15.38:0/2149633268 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fac1a0250 con 0x7f0fac102660 2024-09-16T00:29:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:29.938+0000 7f0fb3677640 1 -- 172.21.15.38:0/2149633268 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fac0fdd70 con 0x7f0fac102660 2024-09-16T00:29:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:29.940+0000 7f0fa27fc640 1 -- 172.21.15.38:0/2149633268 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f0fa8005560 con 0x7f0fac102660 2024-09-16T00:29:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:29.941+0000 7f0fa27fc640 1 --2- 172.21.15.38:0/2149633268 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f0f980a3270 0x7f0f980a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:29.941+0000 7f0fa27fc640 1 -- 172.21.15.38:0/2149633268 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 6809+0+0 (secure 0 0 0) 0x7f0fa80cf450 con 0x7f0fac102660 2024-09-16T00:29:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:29.941+0000 7f0fb0c12640 1 --2- 172.21.15.38:0/2149633268 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f0f980a3270 0x7f0f980a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:29.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:29.941+0000 7f0fb0c12640 1 --2- 172.21.15.38:0/2149633268 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f0f980a3270 0x7f0f980a5730 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0fac199400 tx=0x7f0f9c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:29.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:29.943+0000 7f0fa27fc640 1 -- 172.21.15.38:0/2149633268 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0fa80051d0 con 0x7f0fac102660 2024-09-16T00:29:30.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:30.120+0000 7f0fb3677640 1 -- 172.21.15.38:0/2149633268 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f0fac02da30 con 0x7f0fac102660 2024-09-16T00:29:30.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:30.121+0000 7f0fa27fc640 1 -- 172.21.15.38:0/2149633268 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v73) ==== 74+0+133 (secure 0 0 0) 0x7f0fa809c740 con 0x7f0fac102660 2024-09-16T00:29:30.122 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:29:30.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:30.123+0000 7f0fb3677640 1 -- 172.21.15.38:0/2149633268 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f0f980a3270 msgr2=0x7f0f980a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:30.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:30.123+0000 7f0fb3677640 1 --2- 172.21.15.38:0/2149633268 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f0f980a3270 0x7f0f980a5730 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0fac199400 tx=0x7f0f9c005f90 comp rx=0 tx=0).stop 2024-09-16T00:29:30.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:30.123+0000 7f0fb3677640 1 -- 172.21.15.38:0/2149633268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0fac102660 msgr2=0x7f0fac197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:30.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:30.123+0000 7f0fb3677640 1 --2- 172.21.15.38:0/2149633268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0fac102660 0x7f0fac197ee0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0fa800e9f0 tx=0x7f0fa800eeb0 comp rx=0 tx=0).stop 2024-09-16T00:29:30.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:30.123+0000 7f0fb3677640 1 -- 172.21.15.38:0/2149633268 shutdown_connections 2024-09-16T00:29:30.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:30.123+0000 7f0fb3677640 1 --2- 172.21.15.38:0/2149633268 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f0f980a3270 0x7f0f980a5730 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:30.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:30.123+0000 7f0fb3677640 1 --2- 172.21.15.38:0/2149633268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fac1078c0 0x7f0fac19f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:30.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:30.123+0000 7f0fb3677640 1 --2- 172.21.15.38:0/2149633268 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0fac104f90 0x7f0fac198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:30.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:30.123+0000 7f0fb3677640 1 --2- 172.21.15.38:0/2149633268 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0fac102660 0x7f0fac197ee0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:30.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:30.123+0000 7f0fb3677640 1 -- 172.21.15.38:0/2149633268 >> 172.21.15.38:0/2149633268 conn(0x7f0fac0fc280 msgr2=0x7f0fac102f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:30.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:30.123+0000 7f0fb3677640 1 -- 172.21.15.38:0/2149633268 shutdown_connections 2024-09-16T00:29:30.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:30.123+0000 7f0fb3677640 1 -- 172.21.15.38:0/2149633268 wait complete. 2024-09-16T00:29:30.922 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:30 smithi038 ceph-mon[30945]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:29:30.923 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:30 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/2149633268' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T00:29:30.942 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:30 smithi122 ceph-mon[35412]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:29:30.942 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:30 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/2149633268' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T00:29:30.942 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:29:30 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:29:30.875+0000 7ffb3d192640 -1 osd.11 0 log_to_monitors true 2024-09-16T00:29:30.951 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":73,"num_osds":12,"num_up_osds":11,"osd_up_since":1726446547,"num_in_osds":12,"osd_in_since":1726446550,"num_remapped_pgs":0} 2024-09-16T00:29:31.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:30 smithi073 ceph-mon[35254]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:29:31.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:30 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/2149633268' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T00:29:31.952 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-16T00:29:31.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:31 smithi122 ceph-mon[35412]: from='osd.11 [v2:172.21.15.122:6824/1886446061,v1:172.21.15.122:6825/1886446061]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T00:29:31.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:31 smithi122 ceph-mon[35412]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T00:29:32.214 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:31 smithi038 ceph-mon[30945]: from='osd.11 [v2:172.21.15.122:6824/1886446061,v1:172.21.15.122:6825/1886446061]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T00:29:32.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:31 smithi038 ceph-mon[30945]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T00:29:32.227 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:29:32.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:31 smithi073 ceph-mon[35254]: from='osd.11 [v2:172.21.15.122:6824/1886446061,v1:172.21.15.122:6825/1886446061]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T00:29:32.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:31 smithi073 ceph-mon[35254]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-09-16T00:29:32.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.759+0000 7f695e748640 1 -- 172.21.15.38:0/84056702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69581026d0 msgr2=0x7f6958108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:32.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.759+0000 7f695e748640 1 --2- 172.21.15.38:0/84056702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69581026d0 0x7f6958108f60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6950009ca0 tx=0x7f695002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.759+0000 7f695e748640 1 -- 172.21.15.38:0/84056702 shutdown_connections 2024-09-16T00:29:32.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.759+0000 7f695e748640 1 --2- 172.21.15.38:0/84056702 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f69581026d0 0x7f6958108f60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.759+0000 7f695e748640 1 --2- 172.21.15.38:0/84056702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6958101d10 0x7f6958102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.759+0000 7f695e748640 1 --2- 172.21.15.38:0/84056702 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6958100b10 0x7f6958100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.759+0000 7f695e748640 1 -- 172.21.15.38:0/84056702 >> 172.21.15.38:0/84056702 conn(0x7f69580fc2a0 msgr2=0x7f69580fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:32.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.759+0000 7f695e748640 1 -- 172.21.15.38:0/84056702 shutdown_connections 2024-09-16T00:29:32.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.759+0000 7f695e748640 1 -- 172.21.15.38:0/84056702 wait complete. 2024-09-16T00:29:32.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.760+0000 7f695e748640 1 Processor -- start 2024-09-16T00:29:32.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.760+0000 7f695e748640 1 -- start start 2024-09-16T00:29:32.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.760+0000 7f695e748640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6958100b10 0x7f6958197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:32.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.760+0000 7f695e748640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6958101d10 0x7f69581983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:32.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.760+0000 7f695e748640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69581026d0 0x7f695819f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:32.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.760+0000 7f695e748640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f695810c790 con 0x7f6958100b10 2024-09-16T00:29:32.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.760+0000 7f695e748640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f695810c610 con 0x7f6958101d10 2024-09-16T00:29:32.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.760+0000 7f695e748640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f695810c910 con 0x7f69581026d0 2024-09-16T00:29:32.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.760+0000 7f6957fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6958100b10 0x7f6958197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:32.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.760+0000 7f6957fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6958100b10 0x7f6958197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40106/0 (socket says 172.21.15.38:40106) 2024-09-16T00:29:32.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.760+0000 7f6957fff640 1 -- 172.21.15.38:0/1378161753 learned_addr learned my addr 172.21.15.38:0/1378161753 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:29:32.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.760+0000 7f695cce5640 1 --2- 172.21.15.38:0/1378161753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69581026d0 0x7f695819f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:32.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.760+0000 7f694ffff640 1 --2- 172.21.15.38:0/1378161753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6958101d10 0x7f69581983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:32.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.760+0000 7f6957fff640 1 -- 172.21.15.38:0/1378161753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69581026d0 msgr2=0x7f695819f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:32.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.760+0000 7f6957fff640 1 --2- 172.21.15.38:0/1378161753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69581026d0 0x7f695819f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.760+0000 7f6957fff640 1 -- 172.21.15.38:0/1378161753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6958101d10 msgr2=0x7f69581983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:32.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.760+0000 7f6957fff640 1 --2- 172.21.15.38:0/1378161753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6958101d10 0x7f69581983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.760+0000 7f6957fff640 1 -- 172.21.15.38:0/1378161753 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f695819fae0 con 0x7f6958100b10 2024-09-16T00:29:32.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.760+0000 7f694ffff640 1 --2- 172.21.15.38:0/1378161753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6958101d10 0x7f69581983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:29:32.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.760+0000 7f695cce5640 1 --2- 172.21.15.38:0/1378161753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69581026d0 0x7f695819f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:29:32.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.760+0000 7f6957fff640 1 --2- 172.21.15.38:0/1378161753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6958100b10 0x7f6958197eb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f694800c970 tx=0x7f694800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:32.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.761+0000 7f6955ffb640 1 -- 172.21.15.38:0/1378161753 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6948004220 con 0x7f6958100b10 2024-09-16T00:29:32.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.761+0000 7f695e748640 1 -- 172.21.15.38:0/1378161753 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f695819fdd0 con 0x7f6958100b10 2024-09-16T00:29:32.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.761+0000 7f6955ffb640 1 -- 172.21.15.38:0/1378161753 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f69480043c0 con 0x7f6958100b10 2024-09-16T00:29:32.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.761+0000 7f6955ffb640 1 -- 172.21.15.38:0/1378161753 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6948004a00 con 0x7f6958100b10 2024-09-16T00:29:32.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.761+0000 7f695e748640 1 -- 172.21.15.38:0/1378161753 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69581a0220 con 0x7f6958100b10 2024-09-16T00:29:32.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.762+0000 7f695e748640 1 -- 172.21.15.38:0/1378161753 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69581a04e0 con 0x7f6958100b10 2024-09-16T00:29:32.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.763+0000 7f6955ffb640 1 -- 172.21.15.38:0/1378161753 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f69480079b0 con 0x7f6958100b10 2024-09-16T00:29:32.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.764+0000 7f6955ffb640 1 --2- 172.21.15.38:0/1378161753 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f69440a3270 0x7f69440a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:32.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.764+0000 7f694ffff640 1 --2- 172.21.15.38:0/1378161753 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f69440a3270 0x7f69440a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:32.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.764+0000 7f6955ffb640 1 -- 172.21.15.38:0/1378161753 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6831+0+0 (secure 0 0 0) 0x7f69480cf540 con 0x7f6958100b10 2024-09-16T00:29:32.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.764+0000 7f694ffff640 1 --2- 172.21.15.38:0/1378161753 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f69440a3270 0x7f69440a5730 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f69581993d0 tx=0x7f694000b5f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:32.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.767+0000 7f6955ffb640 1 -- 172.21.15.38:0/1378161753 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6948014030 con 0x7f6958100b10 2024-09-16T00:29:32.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.944+0000 7f695e748640 1 -- 172.21.15.38:0/1378161753 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f69581a0820 con 0x7f6958100b10 2024-09-16T00:29:32.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.945+0000 7f6955ffb640 1 -- 172.21.15.38:0/1378161753 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) ==== 74+0+133 (secure 0 0 0) 0x7f694809c830 con 0x7f6958100b10 2024-09-16T00:29:32.945 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:32 smithi038 ceph-mon[30945]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:29:32.945 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:32 smithi038 ceph-mon[30945]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-16T00:29:32.946 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:32 smithi038 ceph-mon[30945]: from='osd.11 [v2:172.21.15.122:6824/1886446061,v1:172.21.15.122:6825/1886446061]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-16T00:29:32.946 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:32 smithi038 ceph-mon[30945]: osdmap e74: 12 total, 11 up, 12 in 2024-09-16T00:29:32.946 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:32 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:29:32.946 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:32 smithi038 ceph-mon[30945]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-16T00:29:32.946 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:32 smithi038 ceph-mon[30945]: Detected new or changed devices on smithi122 2024-09-16T00:29:32.946 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:32 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:32.946 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:32 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:32.946 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:32 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:29:32.946 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:32 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:29:32.946 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:32 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:29:32.946 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:32 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:29:32.946 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:32 smithi038 ceph-mon[30945]: Adjusting osd_memory_target on smithi122 to 4271M 2024-09-16T00:29:32.946 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:32 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:32.946 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:32 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:32.946 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:32 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:29:32.947 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:32 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:32.947 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:29:32.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.947+0000 7f695e748640 1 -- 172.21.15.38:0/1378161753 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f69440a3270 msgr2=0x7f69440a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:32.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.947+0000 7f695e748640 1 --2- 172.21.15.38:0/1378161753 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f69440a3270 0x7f69440a5730 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f69581993d0 tx=0x7f694000b5f0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.947+0000 7f695e748640 1 -- 172.21.15.38:0/1378161753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6958100b10 msgr2=0x7f6958197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:32.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.947+0000 7f695e748640 1 --2- 172.21.15.38:0/1378161753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6958100b10 0x7f6958197eb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f694800c970 tx=0x7f694800ce30 comp rx=0 tx=0).stop 2024-09-16T00:29:32.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.947+0000 7f695e748640 1 -- 172.21.15.38:0/1378161753 shutdown_connections 2024-09-16T00:29:32.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.947+0000 7f695e748640 1 --2- 172.21.15.38:0/1378161753 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f69440a3270 0x7f69440a5730 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.947+0000 7f695e748640 1 --2- 172.21.15.38:0/1378161753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f69581026d0 0x7f695819f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.947+0000 7f695e748640 1 --2- 172.21.15.38:0/1378161753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6958101d10 0x7f69581983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.947+0000 7f695e748640 1 --2- 172.21.15.38:0/1378161753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6958100b10 0x7f6958197eb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:32.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.947+0000 7f695e748640 1 -- 172.21.15.38:0/1378161753 >> 172.21.15.38:0/1378161753 conn(0x7f69580fc2a0 msgr2=0x7f69580fddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:32.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.947+0000 7f695e748640 1 -- 172.21.15.38:0/1378161753 shutdown_connections 2024-09-16T00:29:32.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:32.947+0000 7f695e748640 1 -- 172.21.15.38:0/1378161753 wait complete. 2024-09-16T00:29:33.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:32 smithi122 ceph-mon[35412]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:29:33.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:32 smithi122 ceph-mon[35412]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-16T00:29:33.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:32 smithi122 ceph-mon[35412]: from='osd.11 [v2:172.21.15.122:6824/1886446061,v1:172.21.15.122:6825/1886446061]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-16T00:29:33.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:32 smithi122 ceph-mon[35412]: osdmap e74: 12 total, 11 up, 12 in 2024-09-16T00:29:33.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:32 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:29:33.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:32 smithi122 ceph-mon[35412]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-16T00:29:33.218 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:32 smithi122 ceph-mon[35412]: Detected new or changed devices on smithi122 2024-09-16T00:29:33.218 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:32 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:33.218 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:32 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:33.218 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:32 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:29:33.218 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:32 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:29:33.218 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:32 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:29:33.218 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:32 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:29:33.218 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:32 smithi122 ceph-mon[35412]: Adjusting osd_memory_target on smithi122 to 4271M 2024-09-16T00:29:33.218 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:32 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:33.218 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:32 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:33.218 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:32 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:29:33.218 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:32 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:33.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:32 smithi073 ceph-mon[35254]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:29:33.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:32 smithi073 ceph-mon[35254]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-16T00:29:33.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:32 smithi073 ceph-mon[35254]: from='osd.11 [v2:172.21.15.122:6824/1886446061,v1:172.21.15.122:6825/1886446061]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-16T00:29:33.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:32 smithi073 ceph-mon[35254]: osdmap e74: 12 total, 11 up, 12 in 2024-09-16T00:29:33.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:32 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:29:33.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:32 smithi073 ceph-mon[35254]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]: dispatch 2024-09-16T00:29:33.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:32 smithi073 ceph-mon[35254]: Detected new or changed devices on smithi122 2024-09-16T00:29:33.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:32 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:33.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:32 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:33.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:32 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:29:33.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:32 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:29:33.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:32 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:29:33.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:32 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-09-16T00:29:33.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:32 smithi073 ceph-mon[35254]: Adjusting osd_memory_target on smithi122 to 4271M 2024-09-16T00:29:33.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:32 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:33.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:32 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:33.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:32 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:29:33.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:32 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:33.733 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":75,"num_osds":12,"num_up_osds":11,"osd_up_since":1726446547,"num_in_osds":12,"osd_in_since":1726446550,"num_remapped_pgs":0} 2024-09-16T00:29:33.830 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:29:33 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:29:33.500+0000 7ffb39929640 -1 osd.11 0 waiting for initial osdmap 2024-09-16T00:29:33.831 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:29:33 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:29:33.513+0000 7ffb30f35640 -1 osd.11 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-16T00:29:33.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:33 smithi038 ceph-mon[30945]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-09-16T00:29:33.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:33 smithi038 ceph-mon[30945]: osdmap e75: 12 total, 11 up, 12 in 2024-09-16T00:29:33.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:33 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:29:33.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:33 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:29:33.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:33 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/1378161753' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T00:29:33.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:33 smithi038 ceph-mon[30945]: from='osd.11 ' entity='osd.11' 2024-09-16T00:29:34.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:33 smithi122 ceph-mon[35412]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-09-16T00:29:34.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:33 smithi122 ceph-mon[35412]: osdmap e75: 12 total, 11 up, 12 in 2024-09-16T00:29:34.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:33 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:29:34.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:33 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:29:34.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:33 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/1378161753' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T00:29:34.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:33 smithi122 ceph-mon[35412]: from='osd.11 ' entity='osd.11' 2024-09-16T00:29:34.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:33 smithi073 ceph-mon[35254]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi122", "root=default"]}]': finished 2024-09-16T00:29:34.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:33 smithi073 ceph-mon[35254]: osdmap e75: 12 total, 11 up, 12 in 2024-09-16T00:29:34.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:33 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:29:34.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:33 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:29:34.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:33 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/1378161753' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T00:29:34.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:33 smithi073 ceph-mon[35254]: from='osd.11 ' entity='osd.11' 2024-09-16T00:29:34.734 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd stat -f json 2024-09-16T00:29:34.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:34 smithi122 ceph-mon[35412]: purged_snaps scrub starts 2024-09-16T00:29:34.967 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:34 smithi122 ceph-mon[35412]: purged_snaps scrub ok 2024-09-16T00:29:34.967 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:34 smithi122 ceph-mon[35412]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:29:34.967 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:34 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:29:34.967 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:34 smithi122 ceph-mon[35412]: osd.11 [v2:172.21.15.122:6824/1886446061,v1:172.21.15.122:6825/1886446061] boot 2024-09-16T00:29:34.967 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:34 smithi122 ceph-mon[35412]: osdmap e76: 12 total, 12 up, 12 in 2024-09-16T00:29:34.967 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:34 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:29:35.006 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:29:35.126 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:34 smithi038 ceph-mon[30945]: purged_snaps scrub starts 2024-09-16T00:29:35.126 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:34 smithi038 ceph-mon[30945]: purged_snaps scrub ok 2024-09-16T00:29:35.126 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:34 smithi038 ceph-mon[30945]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:29:35.127 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:34 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:29:35.127 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:34 smithi038 ceph-mon[30945]: osd.11 [v2:172.21.15.122:6824/1886446061,v1:172.21.15.122:6825/1886446061] boot 2024-09-16T00:29:35.127 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:34 smithi038 ceph-mon[30945]: osdmap e76: 12 total, 12 up, 12 in 2024-09-16T00:29:35.127 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:34 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:29:35.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:34 smithi073 ceph-mon[35254]: purged_snaps scrub starts 2024-09-16T00:29:35.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:34 smithi073 ceph-mon[35254]: purged_snaps scrub ok 2024-09-16T00:29:35.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:34 smithi073 ceph-mon[35254]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 333 MiB used, 983 GiB / 983 GiB avail 2024-09-16T00:29:35.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:34 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:29:35.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:34 smithi073 ceph-mon[35254]: osd.11 [v2:172.21.15.122:6824/1886446061,v1:172.21.15.122:6825/1886446061] boot 2024-09-16T00:29:35.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:34 smithi073 ceph-mon[35254]: osdmap e76: 12 total, 12 up, 12 in 2024-09-16T00:29:35.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:34 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-09-16T00:29:35.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.494+0000 7f7c8135b640 1 -- 172.21.15.38:0/4155949645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c7c1037c0 msgr2=0x7f7c7c10a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:35.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.494+0000 7f7c8135b640 1 --2- 172.21.15.38:0/4155949645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c7c1037c0 0x7f7c7c10a050 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f7c74009ca0 tx=0x7f7c7402a8c0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.494+0000 7f7c8135b640 1 -- 172.21.15.38:0/4155949645 shutdown_connections 2024-09-16T00:29:35.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.494+0000 7f7c8135b640 1 --2- 172.21.15.38:0/4155949645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c7c1037c0 0x7f7c7c10a050 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.494+0000 7f7c8135b640 1 --2- 172.21.15.38:0/4155949645 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c7c102e00 0x7f7c7c103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.494+0000 7f7c8135b640 1 --2- 172.21.15.38:0/4155949645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c7c101c00 0x7f7c7c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.494+0000 7f7c8135b640 1 -- 172.21.15.38:0/4155949645 >> 172.21.15.38:0/4155949645 conn(0x7f7c7c0fd390 msgr2=0x7f7c7c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:35.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.494+0000 7f7c8135b640 1 -- 172.21.15.38:0/4155949645 shutdown_connections 2024-09-16T00:29:35.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.494+0000 7f7c8135b640 1 -- 172.21.15.38:0/4155949645 wait complete. 2024-09-16T00:29:35.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.495+0000 7f7c8135b640 1 Processor -- start 2024-09-16T00:29:35.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.495+0000 7f7c8135b640 1 -- start start 2024-09-16T00:29:35.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.495+0000 7f7c8135b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c7c101c00 0x7f7c7c197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:35.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.496+0000 7f7c8135b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c7c102e00 0x7f7c7c198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:35.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.496+0000 7f7c8135b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c7c1037c0 0x7f7c7c19f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:35.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.496+0000 7f7c8135b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7c10d930 con 0x7f7c7c1037c0 2024-09-16T00:29:35.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.496+0000 7f7c8135b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7c10d7b0 con 0x7f7c7c102e00 2024-09-16T00:29:35.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.496+0000 7f7c8135b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7c7c10dab0 con 0x7f7c7c101c00 2024-09-16T00:29:35.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.496+0000 7f7c7affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c7c101c00 0x7f7c7c197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:35.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.496+0000 7f7c7b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c7c1037c0 0x7f7c7c19f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:35.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.496+0000 7f7c7b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c7c1037c0 0x7f7c7c19f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40124/0 (socket says 172.21.15.38:40124) 2024-09-16T00:29:35.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.496+0000 7f7c7affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c7c101c00 0x7f7c7c197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.38:33486/0 (socket says 172.21.15.38:33486) 2024-09-16T00:29:35.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.496+0000 7f7c7b7fe640 1 -- 172.21.15.38:0/3155287205 learned_addr learned my addr 172.21.15.38:0/3155287205 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:29:35.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.496+0000 7f7c7a7fc640 1 --2- 172.21.15.38:0/3155287205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c7c102e00 0x7f7c7c198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:35.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.496+0000 7f7c7b7fe640 1 -- 172.21.15.38:0/3155287205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c7c101c00 msgr2=0x7f7c7c197e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:35.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.496+0000 7f7c7b7fe640 1 --2- 172.21.15.38:0/3155287205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c7c101c00 0x7f7c7c197e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.496+0000 7f7c7b7fe640 1 -- 172.21.15.38:0/3155287205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c7c102e00 msgr2=0x7f7c7c198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:35.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.496+0000 7f7c7b7fe640 1 --2- 172.21.15.38:0/3155287205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c7c102e00 0x7f7c7c198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.496+0000 7f7c7b7fe640 1 -- 172.21.15.38:0/3155287205 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c7c19fa70 con 0x7f7c7c1037c0 2024-09-16T00:29:35.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.496+0000 7f7c7affd640 1 --2- 172.21.15.38:0/3155287205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c7c101c00 0x7f7c7c197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:29:35.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.496+0000 7f7c7b7fe640 1 --2- 172.21.15.38:0/3155287205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c7c1037c0 0x7f7c7c19f400 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7c74003ba0 tx=0x7f7c74003bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:35.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.496+0000 7f7c63fff640 1 -- 172.21.15.38:0/3155287205 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c74004040 con 0x7f7c7c1037c0 2024-09-16T00:29:35.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.496+0000 7f7c63fff640 1 -- 172.21.15.38:0/3155287205 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7c740046c0 con 0x7f7c7c1037c0 2024-09-16T00:29:35.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.496+0000 7f7c8135b640 1 -- 172.21.15.38:0/3155287205 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c7c19fd00 con 0x7f7c7c1037c0 2024-09-16T00:29:35.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.496+0000 7f7c63fff640 1 -- 172.21.15.38:0/3155287205 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c7402fda0 con 0x7f7c7c1037c0 2024-09-16T00:29:35.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.497+0000 7f7c8135b640 1 -- 172.21.15.38:0/3155287205 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c7c1a0260 con 0x7f7c7c1037c0 2024-09-16T00:29:35.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.498+0000 7f7c8135b640 1 -- 172.21.15.38:0/3155287205 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c7c108ed0 con 0x7f7c7c1037c0 2024-09-16T00:29:35.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.499+0000 7f7c63fff640 1 -- 172.21.15.38:0/3155287205 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f7c74004b40 con 0x7f7c7c1037c0 2024-09-16T00:29:35.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.499+0000 7f7c63fff640 1 --2- 172.21.15.38:0/3155287205 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f7c640a3270 0x7f7c640a5730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:35.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.499+0000 7f7c7affd640 1 --2- 172.21.15.38:0/3155287205 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f7c640a3270 0x7f7c640a5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:35.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.500+0000 7f7c63fff640 1 -- 172.21.15.38:0/3155287205 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 7127+0+0 (secure 0 0 0) 0x7f7c740f0c80 con 0x7f7c7c1037c0 2024-09-16T00:29:35.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.500+0000 7f7c7affd640 1 --2- 172.21.15.38:0/3155287205 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f7c640a3270 0x7f7c640a5730 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c102c60 tx=0x7f7c6c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:35.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.503+0000 7f7c63fff640 1 -- 172.21.15.38:0/3155287205 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7c740bde40 con 0x7f7c7c1037c0 2024-09-16T00:29:35.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.679+0000 7f7c8135b640 1 -- 172.21.15.38:0/3155287205 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f7c7c062e90 con 0x7f7c7c1037c0 2024-09-16T00:29:35.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.679+0000 7f7c63fff640 1 -- 172.21.15.38:0/3155287205 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v76) ==== 74+0+133 (secure 0 0 0) 0x7f7c740bd740 con 0x7f7c7c1037c0 2024-09-16T00:29:35.680 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:29:35.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.681+0000 7f7c8135b640 1 -- 172.21.15.38:0/3155287205 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f7c640a3270 msgr2=0x7f7c640a5730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:35.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.681+0000 7f7c8135b640 1 --2- 172.21.15.38:0/3155287205 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f7c640a3270 0x7f7c640a5730 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c102c60 tx=0x7f7c6c005f90 comp rx=0 tx=0).stop 2024-09-16T00:29:35.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.681+0000 7f7c8135b640 1 -- 172.21.15.38:0/3155287205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c7c1037c0 msgr2=0x7f7c7c19f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:35.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.681+0000 7f7c8135b640 1 --2- 172.21.15.38:0/3155287205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c7c1037c0 0x7f7c7c19f400 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7c74003ba0 tx=0x7f7c74003bd0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.681+0000 7f7c8135b640 1 -- 172.21.15.38:0/3155287205 shutdown_connections 2024-09-16T00:29:35.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.681+0000 7f7c8135b640 1 --2- 172.21.15.38:0/3155287205 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f7c640a3270 0x7f7c640a5730 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.681+0000 7f7c8135b640 1 --2- 172.21.15.38:0/3155287205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7c7c1037c0 0x7f7c7c19f400 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.681+0000 7f7c8135b640 1 --2- 172.21.15.38:0/3155287205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7c7c102e00 0x7f7c7c198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.681+0000 7f7c8135b640 1 --2- 172.21.15.38:0/3155287205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7c7c101c00 0x7f7c7c197e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:35.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.681+0000 7f7c8135b640 1 -- 172.21.15.38:0/3155287205 >> 172.21.15.38:0/3155287205 conn(0x7f7c7c0fd390 msgr2=0x7f7c7c1060a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:35.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.681+0000 7f7c8135b640 1 -- 172.21.15.38:0/3155287205 shutdown_connections 2024-09-16T00:29:35.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:35.681+0000 7f7c8135b640 1 -- 172.21.15.38:0/3155287205 wait complete. 2024-09-16T00:29:35.964 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:35 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/3155287205' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T00:29:36.195 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:35 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/3155287205' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T00:29:36.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:35 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/3155287205' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-16T00:29:36.503 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":76,"num_osds":12,"num_up_osds":12,"osd_up_since":1726446574,"num_in_osds":12,"osd_in_since":1726446550,"num_remapped_pgs":0} 2024-09-16T00:29:36.503 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:29:36.764 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:29:37.163 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:36 smithi038 ceph-mon[30945]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:29:37.163 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:36 smithi038 ceph-mon[30945]: osdmap e77: 12 total, 12 up, 12 in 2024-09-16T00:29:37.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:36 smithi122 ceph-mon[35412]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:29:37.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:36 smithi122 ceph-mon[35412]: osdmap e77: 12 total, 12 up, 12 in 2024-09-16T00:29:37.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.245+0000 7f13a5bca640 1 -- 172.21.15.38:0/3044059467 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13a01008d0 msgr2=0x7f13a0100d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:37.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.245+0000 7f13a5bca640 1 --2- 172.21.15.38:0/3044059467 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13a01008d0 0x7f13a0100d30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1388009930 tx=0x7f138802a420 comp rx=0 tx=0).stop 2024-09-16T00:29:37.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.245+0000 7f13a5bca640 1 -- 172.21.15.38:0/3044059467 shutdown_connections 2024-09-16T00:29:37.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.245+0000 7f13a5bca640 1 --2- 172.21.15.38:0/3044059467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13a0101270 0x7f13a010d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.245+0000 7f13a5bca640 1 --2- 172.21.15.38:0/3044059467 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13a01008d0 0x7f13a0100d30 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.245+0000 7f13a5bca640 1 --2- 172.21.15.38:0/3044059467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13a01068d0 0x7f13a0106cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.245+0000 7f13a5bca640 1 -- 172.21.15.38:0/3044059467 >> 172.21.15.38:0/3044059467 conn(0x7f13a00fc610 msgr2=0x7f13a00fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:37.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.245+0000 7f13a5bca640 1 -- 172.21.15.38:0/3044059467 shutdown_connections 2024-09-16T00:29:37.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.245+0000 7f13a5bca640 1 -- 172.21.15.38:0/3044059467 wait complete. 2024-09-16T00:29:37.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.245+0000 7f13a5bca640 1 Processor -- start 2024-09-16T00:29:37.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.245+0000 7f13a5bca640 1 -- start start 2024-09-16T00:29:37.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.245+0000 7f13a5bca640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13a01008d0 0x7f13a0197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.245+0000 7f13a5bca640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13a0101270 0x7f13a0197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.245+0000 7f13a5bca640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13a01068d0 0x7f13a019bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.246+0000 7f13a5bca640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f13a0110d70 con 0x7f13a0101270 2024-09-16T00:29:37.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.246+0000 7f13a5bca640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f13a0110bf0 con 0x7f13a01068d0 2024-09-16T00:29:37.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.246+0000 7f13a5bca640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f13a0110ef0 con 0x7f13a01008d0 2024-09-16T00:29:37.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.246+0000 7f139ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13a0101270 0x7f13a0197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.246+0000 7f139ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13a0101270 0x7f13a0197870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45866/0 (socket says 172.21.15.38:45866) 2024-09-16T00:29:37.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.246+0000 7f139ffff640 1 -- 172.21.15.38:0/2195973054 learned_addr learned my addr 172.21.15.38:0/2195973054 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:29:37.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.246+0000 7f13a4bc8640 1 --2- 172.21.15.38:0/2195973054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13a01008d0 0x7f13a0197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.246+0000 7f13a53c9640 1 --2- 172.21.15.38:0/2195973054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13a01068d0 0x7f13a019bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.246+0000 7f139ffff640 1 -- 172.21.15.38:0/2195973054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13a01008d0 msgr2=0x7f13a0197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:37.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.246+0000 7f139ffff640 1 --2- 172.21.15.38:0/2195973054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13a01008d0 0x7f13a0197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.246+0000 7f139ffff640 1 -- 172.21.15.38:0/2195973054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13a01068d0 msgr2=0x7f13a019bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:37.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.246+0000 7f139ffff640 1 --2- 172.21.15.38:0/2195973054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13a01068d0 0x7f13a019bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.246+0000 7f139ffff640 1 -- 172.21.15.38:0/2195973054 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13a019c2e0 con 0x7f13a0101270 2024-09-16T00:29:37.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.246+0000 7f139ffff640 1 --2- 172.21.15.38:0/2195973054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13a0101270 0x7f13a0197870 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1388009a50 tx=0x7f1388004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:37.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.246+0000 7f139dffb640 1 -- 172.21.15.38:0/2195973054 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f138803f030 con 0x7f13a0101270 2024-09-16T00:29:37.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.246+0000 7f139dffb640 1 -- 172.21.15.38:0/2195973054 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1388003ce0 con 0x7f13a0101270 2024-09-16T00:29:37.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.246+0000 7f13a5bca640 1 -- 172.21.15.38:0/2195973054 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13a019c570 con 0x7f13a0101270 2024-09-16T00:29:37.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.246+0000 7f139dffb640 1 -- 172.21.15.38:0/2195973054 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1388035070 con 0x7f13a0101270 2024-09-16T00:29:37.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.247+0000 7f13a5bca640 1 -- 172.21.15.38:0/2195973054 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13a0069f70 con 0x7f13a0101270 2024-09-16T00:29:37.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.248+0000 7f13a5bca640 1 -- 172.21.15.38:0/2195973054 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13a0102010 con 0x7f13a0101270 2024-09-16T00:29:37.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.249+0000 7f139dffb640 1 -- 172.21.15.38:0/2195973054 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f1388042030 con 0x7f13a0101270 2024-09-16T00:29:37.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.250+0000 7f139dffb640 1 --2- 172.21.15.38:0/2195973054 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f138c0a32c0 0x7f138c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:37.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.250+0000 7f13a4bc8640 1 --2- 172.21.15.38:0/2195973054 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f138c0a32c0 0x7f138c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:37.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.250+0000 7f139dffb640 1 -- 172.21.15.38:0/2195973054 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f13880ef8e0 con 0x7f13a0101270 2024-09-16T00:29:37.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.250+0000 7f13a4bc8640 1 --2- 172.21.15.38:0/2195973054 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f138c0a32c0 0x7f138c0a5780 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f139000c400 tx=0x7f139000f040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:37.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.254+0000 7f139dffb640 1 -- 172.21.15.38:0/2195973054 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f13880bc450 con 0x7f13a0101270 2024-09-16T00:29:37.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:36 smithi073 ceph-mon[35254]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:29:37.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:36 smithi073 ceph-mon[35254]: osdmap e77: 12 total, 12 up, 12 in 2024-09-16T00:29:37.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.430+0000 7f13a5bca640 1 -- 172.21.15.38:0/2195973054 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f13a006a990 con 0x7f13a0101270 2024-09-16T00:29:37.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.431+0000 7f139dffb640 1 -- 172.21.15.38:0/2195973054 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+18430 (secure 0 0 0) 0x7f13880c0750 con 0x7f13a0101270 2024-09-16T00:29:37.432 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:29:37.432 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":77,"fsid":"983b2072-73c1-11ef-bceb-c7b262605968","created":"2024-09-16T00:22:34.897053+0000","modified":"2024-09-16T00:29:35.833763+0000","last_up_change":"2024-09-16T00:29:34.503134+0000","last_in_change":"2024-09-16T00:29:10.135656+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:25:41.931977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3afc20e3-869f-43a9-a911-b0e8d07de86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6803","nonce":1622028921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6805","nonce":1622028921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6809","nonce":1622028921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6807","nonce":1622028921}]},"public_addr":"172.21.15.38:6803/1622028921","cluster_addr":"172.21.15.38:6805/1622028921","heartbeat_back_addr":"172.21.15.38:6809/1622028921","heartbeat_front_addr":"172.21.15.38:6807/1622028921","state":["exists","up"]},{"osd":1,"uuid":"6d9d7acb-45ed-4543-8afc-6433fc7f096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6801","nonce":3811690336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6803","nonce":3811690336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6807","nonce":3811690336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6805","nonce":3811690336}]},"public_addr":"172.21.15.73:6801/3811690336","cluster_addr":"172.21.15.73:6803/3811690336","heartbeat_back_addr":"172.21.15.73:6807/3811690336","heartbeat_front_addr":"172.21.15.73:6805/3811690336","state":["exists","up"]},{"osd":2,"uuid":"48690f35-99e9-4f45-9bee-487857caa8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6801","nonce":1754741655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6803","nonce":1754741655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6807","nonce":1754741655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6805","nonce":1754741655}]},"public_addr":"172.21.15.122:6801/1754741655","cluster_addr":"172.21.15.122:6803/1754741655","heartbeat_back_addr":"172.21.15.122:6807/1754741655","heartbeat_front_addr":"172.21.15.122:6805/1754741655","state":["exists","up"]},{"osd":3,"uuid":"28ebb4a0-92e7-4f57-82a5-b9b761eca269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6811","nonce":3523190079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6813","nonce":3523190079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6817","nonce":3523190079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6815","nonce":3523190079}]},"public_addr":"172.21.15.38:6811/3523190079","cluster_addr":"172.21.15.38:6813/3523190079","heartbeat_back_addr":"172.21.15.38:6817/3523190079","heartbeat_front_addr":"172.21.15.38:6815/3523190079","state":["exists","up"]},{"osd":4,"uuid":"5ea363f7-7ad8-45ed-b7e4-4b010fe7f8cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6809","nonce":1469927628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6811","nonce":1469927628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6815","nonce":1469927628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6813","nonce":1469927628}]},"public_addr":"172.21.15.73:6809/1469927628","cluster_addr":"172.21.15.73:6811/1469927628","heartbeat_back_addr":"172.21.15.73:6815/1469927628","heartbeat_front_addr":"172.21.15.73:6813/1469927628","state":["exists","up"]},{"osd":5,"uuid":"e3c66d43-dd28-4bb1-a886-62a869be82cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":51,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6809","nonce":2954413058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6811","nonce":2954413058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6815","nonce":2954413058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6813","nonce":2954413058}]},"public_addr":"172.21.15.122:6809/2954413058","cluster_addr":"172.21.15.122:6811/2954413058","heartbeat_back_addr":"172.21.15.122:6815/2954413058","heartbeat_front_addr":"172.21.15.122:6813/2954413058","state":["exists","up"]},{"osd":6,"uuid":"211497ac-7c48-4095-bfdb-1d4df0149f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6819","nonce":1415719550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6821","nonce":1415719550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6825","nonce":1415719550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6823","nonce":1415719550}]},"public_addr":"172.21.15.38:6819/1415719550","cluster_addr":"172.21.15.38:6821/1415719550","heartbeat_back_addr":"172.21.15.38:6825/1415719550","heartbeat_front_addr":"172.21.15.38:6823/1415719550","state":["exists","up"]},{"osd":7,"uuid":"06512315-3f5e-44f8-976d-d3f5f1ba00e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6817","nonce":2634361473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6819","nonce":2634361473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6823","nonce":2634361473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6821","nonce":2634361473}]},"public_addr":"172.21.15.73:6817/2634361473","cluster_addr":"172.21.15.73:6819/2634361473","heartbeat_back_addr":"172.21.15.73:6823/2634361473","heartbeat_front_addr":"172.21.15.73:6821/2634361473","state":["exists","up"]},{"osd":8,"uuid":"db703c0a-1f87-46f0-be5f-f826f7f9daac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6817","nonce":1800567633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6819","nonce":1800567633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6823","nonce":1800567633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6821","nonce":1800567633}]},"public_addr":"172.21.15.122:6817/1800567633","cluster_addr":"172.21.15.122:6819/1800567633","heartbeat_back_addr":"172.21.15.122:6823/1800567633","heartbeat_front_addr":"172.21.15.122:6821/1800567633","state":["exists","up"]},{"osd":9,"uuid":"f5384200-b3b4-4936-85e1-fc57a9fe2bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6827","nonce":445727833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6829","nonce":445727833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6833","nonce":445727833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6831","nonce":445727833}]},"public_addr":"172.21.15.38:6827/445727833","cluster_addr":"172.21.15.38:6829/445727833","heartbeat_back_addr":"172.21.15.38:6833/445727833","heartbeat_front_addr":"172.21.15.38:6831/445727833","state":["exists","up"]},{"osd":10,"uuid":"c22883d9-6672-4796-a961-963d21eb37fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6825","nonce":43297139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6827","nonce":43297139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6831","nonce":43297139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6829","nonce":43297139}]},"public_addr":"172.21.15.73:6825/43297139","cluster_addr":"172.21.15.73:6827/43297139","heartbeat_back_addr":"172.21.15.73:6831/43297139","heartbeat_front_addr":"172.21.15.73:6829/43297139","state":["exists","up"]},{"osd":11,"uuid":"9f98e8cb-1bff-4b8b-aa19-56b691387383","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6825","nonce":1886446061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6827","nonce":1886446061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6831","nonce":1886446061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6829","nonce":1886446061}]},"public_addr":"172.21.15.122:6825/1886446061","cluster_addr":"172.21.15.122:6827/1886446061","heartbeat_back_addr":"172.21.15.122:6831/1886446061","heartbeat_front_addr":"172.21.15.122:6829/1886446061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:47.384012+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:12.834488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:38.770055+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:04.800466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:30.808076+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:56.646967+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:27:22.721270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:27:48.510011+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:28:13.922944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:28:40.514947+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:29:06.031052+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:29:31.882062+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6801/1779324916":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/924150447":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/141237849":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/2302081750":"2024-09-17T00:22:56.844212+0000","172.21.15.38:0/2245468594":"2024-09-17T00:22:56.844212+0000","172.21.15.38:6801/583006373":"2024-09-17T00:22:56.844212+0000","172.21.15.38:0/2435492667":"2024-09-17T00:23:21.417483+0000","172.21.15.38:6800/583006373":"2024-09-17T00:22:56.844212+0000","172.21.15.38:6800/1779324916":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/1103516837":"2024-09-17T00:22:56.844212+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:29:37.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.433+0000 7f13a5bca640 1 -- 172.21.15.38:0/2195973054 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f138c0a32c0 msgr2=0x7f138c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:37.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.433+0000 7f13a5bca640 1 --2- 172.21.15.38:0/2195973054 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f138c0a32c0 0x7f138c0a5780 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f139000c400 tx=0x7f139000f040 comp rx=0 tx=0).stop 2024-09-16T00:29:37.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.433+0000 7f13a5bca640 1 -- 172.21.15.38:0/2195973054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13a0101270 msgr2=0x7f13a0197870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:37.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.433+0000 7f13a5bca640 1 --2- 172.21.15.38:0/2195973054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13a0101270 0x7f13a0197870 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1388009a50 tx=0x7f1388004270 comp rx=0 tx=0).stop 2024-09-16T00:29:37.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.433+0000 7f13a5bca640 1 -- 172.21.15.38:0/2195973054 shutdown_connections 2024-09-16T00:29:37.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.433+0000 7f13a5bca640 1 --2- 172.21.15.38:0/2195973054 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f138c0a32c0 0x7f138c0a5780 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.433+0000 7f13a5bca640 1 --2- 172.21.15.38:0/2195973054 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13a01068d0 0x7f13a019bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.433+0000 7f13a5bca640 1 --2- 172.21.15.38:0/2195973054 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f13a0101270 0x7f13a0197870 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.433+0000 7f13a5bca640 1 --2- 172.21.15.38:0/2195973054 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f13a01008d0 0x7f13a0197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:37.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.433+0000 7f13a5bca640 1 -- 172.21.15.38:0/2195973054 >> 172.21.15.38:0/2195973054 conn(0x7f13a00fc610 msgr2=0x7f13a00fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:37.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.434+0000 7f13a5bca640 1 -- 172.21.15.38:0/2195973054 shutdown_connections 2024-09-16T00:29:37.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:37.434+0000 7f13a5bca640 1 -- 172.21.15.38:0/2195973054 wait complete. 2024-09-16T00:29:38.214 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:37 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/2195973054' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:29:38.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:37 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/2195973054' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:29:38.262 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T00:25:41.931977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-16T00:29:38.262 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-16T00:29:38.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:37 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/2195973054' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:29:38.522 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:29:38.907 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:38 smithi038 ceph-mon[30945]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:29:39.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:38.992+0000 7f892a8e8640 1 -- 172.21.15.38:0/3300436423 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8924102790 msgr2=0x7f89241067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:39.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:38.992+0000 7f892a8e8640 1 --2- 172.21.15.38:0/3300436423 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8924102790 0x7f89241067e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8918009900 tx=0x7f891802a410 comp rx=0 tx=0).stop 2024-09-16T00:29:39.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:38.992+0000 7f892a8e8640 1 -- 172.21.15.38:0/3300436423 shutdown_connections 2024-09-16T00:29:39.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:38.992+0000 7f892a8e8640 1 --2- 172.21.15.38:0/3300436423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8924106f10 0x7f892410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:39.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:38.992+0000 7f892a8e8640 1 --2- 172.21.15.38:0/3300436423 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8924102790 0x7f89241067e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:39.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:38.992+0000 7f892a8e8640 1 --2- 172.21.15.38:0/3300436423 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8924101de0 0x7f89241021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:39.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:38.992+0000 7f892a8e8640 1 -- 172.21.15.38:0/3300436423 >> 172.21.15.38:0/3300436423 conn(0x7f89240fd8c0 msgr2=0x7f89240ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:39.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:38.992+0000 7f892a8e8640 1 -- 172.21.15.38:0/3300436423 shutdown_connections 2024-09-16T00:29:39.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:38.992+0000 7f892a8e8640 1 -- 172.21.15.38:0/3300436423 wait complete. 2024-09-16T00:29:39.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:38.993+0000 7f892a8e8640 1 Processor -- start 2024-09-16T00:29:39.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:38.993+0000 7f892a8e8640 1 -- start start 2024-09-16T00:29:39.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:38.993+0000 7f892a8e8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8924101de0 0x7f89240744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:39.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:38.993+0000 7f892a8e8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8924102790 0x7f8924074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:39.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:38.993+0000 7f892a8e8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8924106f10 0x7f8924074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:39.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:38.993+0000 7f892a8e8640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8924111f70 con 0x7f8924101de0 2024-09-16T00:29:39.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:38.993+0000 7f892a8e8640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8924111df0 con 0x7f8924106f10 2024-09-16T00:29:39.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:38.993+0000 7f892a8e8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f89241120f0 con 0x7f8924102790 2024-09-16T00:29:39.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:38.993+0000 7f8923fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8924101de0 0x7f89240744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:39.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:38.993+0000 7f8923fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8924101de0 0x7f89240744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45890/0 (socket says 172.21.15.38:45890) 2024-09-16T00:29:39.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:38.993+0000 7f8923fff640 1 -- 172.21.15.38:0/1016789340 learned_addr learned my addr 172.21.15.38:0/1016789340 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:29:39.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:38.993+0000 7f8928e85640 1 --2- 172.21.15.38:0/1016789340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8924106f10 0x7f8924074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:39.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:38.994+0000 7f8923fff640 1 -- 172.21.15.38:0/1016789340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8924102790 msgr2=0x7f8924074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:39.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:38.994+0000 7f8923fff640 1 --2- 172.21.15.38:0/1016789340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8924102790 0x7f8924074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:39.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:38.994+0000 7f8923fff640 1 -- 172.21.15.38:0/1016789340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8924106f10 msgr2=0x7f8924074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:39.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:38.994+0000 7f8923fff640 1 --2- 172.21.15.38:0/1016789340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8924106f10 0x7f8924074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:39.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:38.994+0000 7f8923fff640 1 -- 172.21.15.38:0/1016789340 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89241a6890 con 0x7f8924101de0 2024-09-16T00:29:39.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:38.994+0000 7f8928e85640 1 --2- 172.21.15.38:0/1016789340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8924106f10 0x7f8924074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:29:39.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:38.994+0000 7f8923fff640 1 --2- 172.21.15.38:0/1016789340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8924101de0 0x7f89240744f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f891400c970 tx=0x7f891400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:39.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:38.994+0000 7f89217fa640 1 -- 172.21.15.38:0/1016789340 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8914007bb0 con 0x7f8924101de0 2024-09-16T00:29:39.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:38.994+0000 7f892a8e8640 1 -- 172.21.15.38:0/1016789340 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89241a6b80 con 0x7f8924101de0 2024-09-16T00:29:39.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:38.994+0000 7f89217fa640 1 -- 172.21.15.38:0/1016789340 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8914007d50 con 0x7f8924101de0 2024-09-16T00:29:39.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:38.994+0000 7f89217fa640 1 -- 172.21.15.38:0/1016789340 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891400f450 con 0x7f8924101de0 2024-09-16T00:29:39.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:38.994+0000 7f892a8e8640 1 -- 172.21.15.38:0/1016789340 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89241a7070 con 0x7f8924101de0 2024-09-16T00:29:39.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:38.995+0000 7f892a8e8640 1 -- 172.21.15.38:0/1016789340 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89241021c0 con 0x7f8924101de0 2024-09-16T00:29:39.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:38.996+0000 7f89217fa640 1 -- 172.21.15.38:0/1016789340 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f8914005020 con 0x7f8924101de0 2024-09-16T00:29:39.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:38.997+0000 7f89217fa640 1 --2- 172.21.15.38:0/1016789340 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f890c0a3240 0x7f890c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:39.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:38.997+0000 7f89217fa640 1 -- 172.21.15.38:0/1016789340 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f89140cf310 con 0x7f8924101de0 2024-09-16T00:29:39.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:38.997+0000 7f89237fe640 1 --2- 172.21.15.38:0/1016789340 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f890c0a3240 0x7f890c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:39.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:38.997+0000 7f89237fe640 1 --2- 172.21.15.38:0/1016789340 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f890c0a3240 0x7f890c0a5700 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f891802a930 tx=0x7f8918000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:39.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:39.001+0000 7f89217fa640 1 -- 172.21.15.38:0/1016789340 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8914014030 con 0x7f8924101de0 2024-09-16T00:29:39.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:39.176+0000 7f892a8e8640 1 -- 172.21.15.38:0/1016789340 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f8924075a20 con 0x7f8924101de0 2024-09-16T00:29:39.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:39.176+0000 7f89217fa640 1 -- 172.21.15.38:0/1016789340 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7f8924075a20 con 0x7f8924101de0 2024-09-16T00:29:39.177 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-09-16T00:29:39.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:39.178+0000 7f892a8e8640 1 -- 172.21.15.38:0/1016789340 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f890c0a3240 msgr2=0x7f890c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:39.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:39.179+0000 7f892a8e8640 1 --2- 172.21.15.38:0/1016789340 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f890c0a3240 0x7f890c0a5700 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f891802a930 tx=0x7f8918000bc0 comp rx=0 tx=0).stop 2024-09-16T00:29:39.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:39.179+0000 7f892a8e8640 1 -- 172.21.15.38:0/1016789340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8924101de0 msgr2=0x7f89240744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:39.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:39.179+0000 7f892a8e8640 1 --2- 172.21.15.38:0/1016789340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8924101de0 0x7f89240744f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f891400c970 tx=0x7f891400ce30 comp rx=0 tx=0).stop 2024-09-16T00:29:39.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:39.179+0000 7f892a8e8640 1 -- 172.21.15.38:0/1016789340 shutdown_connections 2024-09-16T00:29:39.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:39.179+0000 7f892a8e8640 1 --2- 172.21.15.38:0/1016789340 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f890c0a3240 0x7f890c0a5700 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:39.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:39.179+0000 7f892a8e8640 1 --2- 172.21.15.38:0/1016789340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8924106f10 0x7f8924074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:39.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:39.179+0000 7f892a8e8640 1 --2- 172.21.15.38:0/1016789340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8924102790 0x7f8924074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:39.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:39.179+0000 7f892a8e8640 1 --2- 172.21.15.38:0/1016789340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8924101de0 0x7f89240744f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:39.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:39.179+0000 7f892a8e8640 1 -- 172.21.15.38:0/1016789340 >> 172.21.15.38:0/1016789340 conn(0x7f89240fd8c0 msgr2=0x7f89240fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:39.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:39.179+0000 7f892a8e8640 1 -- 172.21.15.38:0/1016789340 shutdown_connections 2024-09-16T00:29:39.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:39.179+0000 7f892a8e8640 1 -- 172.21.15.38:0/1016789340 wait complete. 2024-09-16T00:29:39.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:38 smithi122 ceph-mon[35412]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:29:39.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:38 smithi073 ceph-mon[35254]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:29:39.955 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:39 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/1016789340' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T00:29:39.991 INFO:tasks.cephadm:Adding mds.a on smithi038 2024-09-16T00:29:39.991 INFO:tasks.cephadm:Adding mds.d on smithi038 2024-09-16T00:29:39.992 INFO:tasks.cephadm:Adding mds.g on smithi038 2024-09-16T00:29:39.992 INFO:tasks.cephadm:Adding mds.j on smithi038 2024-09-16T00:29:39.992 INFO:tasks.cephadm:Adding mds.b on smithi073 2024-09-16T00:29:39.992 INFO:tasks.cephadm:Adding mds.e on smithi073 2024-09-16T00:29:39.992 INFO:tasks.cephadm:Adding mds.h on smithi073 2024-09-16T00:29:39.992 INFO:tasks.cephadm:Adding mds.k on smithi073 2024-09-16T00:29:39.992 INFO:tasks.cephadm:Adding mds.c on smithi122 2024-09-16T00:29:39.992 INFO:tasks.cephadm:Adding mds.f on smithi122 2024-09-16T00:29:39.992 INFO:tasks.cephadm:Adding mds.i on smithi122 2024-09-16T00:29:39.992 INFO:tasks.cephadm:Adding mds.l on smithi122 2024-09-16T00:29:39.992 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph orch apply mds all '12;smithi038=a;smithi038=d;smithi038=g;smithi038=j;smithi073=b;smithi073=e;smithi073=h;smithi073=k;smithi122=c;smithi122=f;smithi122=i;smithi122=l' 2024-09-16T00:29:40.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:39 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/1016789340' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T00:29:40.252 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:29:40.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:39 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/1016789340' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T00:29:40.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.792+0000 7f101cb39640 1 -- 172.21.15.122:0/902343205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1018101c00 msgr2=0x7f1018102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:40.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.792+0000 7f101cb39640 1 --2- 172.21.15.122:0/902343205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1018101c00 0x7f1018102000 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1000009900 tx=0x7f100002a410 comp rx=0 tx=0).stop 2024-09-16T00:29:40.818 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.792+0000 7f101cb39640 1 -- 172.21.15.122:0/902343205 shutdown_connections 2024-09-16T00:29:40.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.792+0000 7f101cb39640 1 --2- 172.21.15.122:0/902343205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f10181037c0 0x7f101810a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.792+0000 7f101cb39640 1 --2- 172.21.15.122:0/902343205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1018102e00 0x7f1018103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.792+0000 7f101cb39640 1 --2- 172.21.15.122:0/902343205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1018101c00 0x7f1018102000 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.792+0000 7f101cb39640 1 -- 172.21.15.122:0/902343205 >> 172.21.15.122:0/902343205 conn(0x7f10180fd390 msgr2=0x7f10180ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:40.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.792+0000 7f101cb39640 1 -- 172.21.15.122:0/902343205 shutdown_connections 2024-09-16T00:29:40.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.792+0000 7f101cb39640 1 -- 172.21.15.122:0/902343205 wait complete. 2024-09-16T00:29:40.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.793+0000 7f101cb39640 1 Processor -- start 2024-09-16T00:29:40.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.793+0000 7f101cb39640 1 -- start start 2024-09-16T00:29:40.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.793+0000 7f101cb39640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1018101c00 0x7f1018197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:40.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.793+0000 7f101cb39640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1018102e00 0x7f1018198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:40.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.793+0000 7f101cb39640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10181037c0 0x7f101819f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:40.819 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.793+0000 7f101cb39640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f101810d880 con 0x7f1018102e00 2024-09-16T00:29:40.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.793+0000 7f101cb39640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f101810d700 con 0x7f10181037c0 2024-09-16T00:29:40.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.793+0000 7f101cb39640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f101810da00 con 0x7f1018101c00 2024-09-16T00:29:40.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.793+0000 7f101659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1018101c00 0x7f1018197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:40.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.793+0000 7f101659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1018101c00 0x7f1018197f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.122:37866/0 (socket says 172.21.15.122:37866) 2024-09-16T00:29:40.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.793+0000 7f101659c640 1 -- 172.21.15.122:0/2838231920 learned_addr learned my addr 172.21.15.122:0/2838231920 (peer_addr_for_me v2:172.21.15.122:0/0) 2024-09-16T00:29:40.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.793+0000 7f1015d9b640 1 --2- 172.21.15.122:0/2838231920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1018102e00 0x7f1018198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:40.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.793+0000 7f1016d9d640 1 --2- 172.21.15.122:0/2838231920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10181037c0 0x7f101819f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:40.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.793+0000 7f101659c640 1 -- 172.21.15.122:0/2838231920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10181037c0 msgr2=0x7f101819f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:40.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.793+0000 7f101659c640 1 --2- 172.21.15.122:0/2838231920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10181037c0 0x7f101819f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.793+0000 7f101659c640 1 -- 172.21.15.122:0/2838231920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1018102e00 msgr2=0x7f1018198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:40.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.793+0000 7f101659c640 1 --2- 172.21.15.122:0/2838231920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1018102e00 0x7f1018198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.820 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.794+0000 7f101659c640 1 -- 172.21.15.122:0/2838231920 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f101819fb30 con 0x7f1018101c00 2024-09-16T00:29:40.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.794+0000 7f1016d9d640 1 --2- 172.21.15.122:0/2838231920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10181037c0 0x7f101819f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:29:40.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.794+0000 7f101659c640 1 --2- 172.21.15.122:0/2838231920 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1018101c00 0x7f1018197f00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1000031610 tx=0x7f10000043b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:40.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.794+0000 7f100f7fe640 1 -- 172.21.15.122:0/2838231920 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100003f030 con 0x7f1018101c00 2024-09-16T00:29:40.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.794+0000 7f100f7fe640 1 -- 172.21.15.122:0/2838231920 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1000003ce0 con 0x7f1018101c00 2024-09-16T00:29:40.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.794+0000 7f101cb39640 1 -- 172.21.15.122:0/2838231920 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f101819fdc0 con 0x7f1018101c00 2024-09-16T00:29:40.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.794+0000 7f100f7fe640 1 -- 172.21.15.122:0/2838231920 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1000035070 con 0x7f1018101c00 2024-09-16T00:29:40.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.794+0000 7f101cb39640 1 -- 172.21.15.122:0/2838231920 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10181a0210 con 0x7f1018101c00 2024-09-16T00:29:40.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.795+0000 7f101cb39640 1 -- 172.21.15.122:0/2838231920 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1018109050 con 0x7f1018101c00 2024-09-16T00:29:40.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.796+0000 7f100f7fe640 1 -- 172.21.15.122:0/2838231920 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f1000042030 con 0x7f1018101c00 2024-09-16T00:29:40.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.797+0000 7f100f7fe640 1 --2- 172.21.15.122:0/2838231920 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f10040a32f0 0x7f10040a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:40.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.797+0000 7f100f7fe640 1 -- 172.21.15.122:0/2838231920 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f10000efb60 con 0x7f1018101c00 2024-09-16T00:29:40.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.798+0000 7f1015d9b640 1 --2- 172.21.15.122:0/2838231920 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f10040a32f0 0x7f10040a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:40.821 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.798+0000 7f1015d9b640 1 --2- 172.21.15.122:0/2838231920 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f10040a32f0 0x7f10040a57b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1018199420 tx=0x7f100800a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:40.822 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.800+0000 7f100f7fe640 1 -- 172.21.15.122:0/2838231920 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f10000bcd20 con 0x7f1018101c00 2024-09-16T00:29:40.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:40 smithi122 ceph-mon[35412]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:29:40.980 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.980+0000 7f101cb39640 1 -- 172.21.15.122:0/2838231920 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi038=a;smithi038=d;smithi038=g;smithi038=j;smithi073=b;smithi073=e;smithi073=h;smithi073=k;smithi122=c;smithi122=f;smithi122=i;smithi122=l", "target": ["mon-mgr", ""]}) -- 0x7f10180627f0 con 0x7f10040a32f0 2024-09-16T00:29:40.988 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.988+0000 7f100f7fe640 1 -- 172.21.15.122:0/2838231920 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f10180627f0 con 0x7f10040a32f0 2024-09-16T00:29:40.988 INFO:teuthology.orchestra.run.smithi122.stdout:Scheduled mds.all update... 2024-09-16T00:29:40.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.990+0000 7f101cb39640 1 -- 172.21.15.122:0/2838231920 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f10040a32f0 msgr2=0x7f10040a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:40.990 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.990+0000 7f101cb39640 1 --2- 172.21.15.122:0/2838231920 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f10040a32f0 0x7f10040a57b0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1018199420 tx=0x7f100800a250 comp rx=0 tx=0).stop 2024-09-16T00:29:40.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.990+0000 7f101cb39640 1 -- 172.21.15.122:0/2838231920 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1018101c00 msgr2=0x7f1018197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:40.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.990+0000 7f101cb39640 1 --2- 172.21.15.122:0/2838231920 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1018101c00 0x7f1018197f00 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1000031610 tx=0x7f10000043b0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.990+0000 7f101cb39640 1 -- 172.21.15.122:0/2838231920 shutdown_connections 2024-09-16T00:29:40.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.990+0000 7f101cb39640 1 --2- 172.21.15.122:0/2838231920 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f10040a32f0 0x7f10040a57b0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.990+0000 7f101cb39640 1 --2- 172.21.15.122:0/2838231920 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10181037c0 0x7f101819f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.990+0000 7f101cb39640 1 --2- 172.21.15.122:0/2838231920 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1018102e00 0x7f1018198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.990+0000 7f101cb39640 1 --2- 172.21.15.122:0/2838231920 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1018101c00 0x7f1018197f00 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:40.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.990+0000 7f101cb39640 1 -- 172.21.15.122:0/2838231920 >> 172.21.15.122:0/2838231920 conn(0x7f10180fd390 msgr2=0x7f1018106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:40.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.990+0000 7f101cb39640 1 -- 172.21.15.122:0/2838231920 shutdown_connections 2024-09-16T00:29:40.991 INFO:teuthology.orchestra.run.smithi122.stderr:2024-09-16T00:29:40.990+0000 7f101cb39640 1 -- 172.21.15.122:0/2838231920 wait complete. 2024-09-16T00:29:41.214 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:40 smithi038 ceph-mon[30945]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:29:41.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:40 smithi073 ceph-mon[35254]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:29:41.925 DEBUG:teuthology.orchestra.run.smithi038:mds.a> sudo journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@mds.a.service 2024-09-16T00:29:41.928 DEBUG:teuthology.orchestra.run.smithi038:mds.d> sudo journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@mds.d.service 2024-09-16T00:29:41.971 DEBUG:teuthology.orchestra.run.smithi038:mds.g> sudo journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@mds.g.service 2024-09-16T00:29:42.014 DEBUG:teuthology.orchestra.run.smithi038:mds.j> sudo journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@mds.j.service 2024-09-16T00:29:42.057 DEBUG:teuthology.orchestra.run.smithi073:mds.b> sudo journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@mds.b.service 2024-09-16T00:29:42.060 DEBUG:teuthology.orchestra.run.smithi073:mds.e> sudo journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@mds.e.service 2024-09-16T00:29:42.106 DEBUG:teuthology.orchestra.run.smithi073:mds.h> sudo journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@mds.h.service 2024-09-16T00:29:42.149 DEBUG:teuthology.orchestra.run.smithi073:mds.k> sudo journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@mds.k.service 2024-09-16T00:29:42.150 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:41 smithi122 ceph-mon[35412]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi038=a;smithi038=d;smithi038=g;smithi038=j;smithi073=b;smithi073=e;smithi073=h;smithi073=k;smithi122=c;smithi122=f;smithi122=i;smithi122=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:29:42.150 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:41 smithi122 ceph-mon[35412]: Saving service mds.all spec with placement smithi038=a;smithi038=d;smithi038=g;smithi038=j;smithi073=b;smithi073=e;smithi073=h;smithi073=k;smithi122=c;smithi122=f;smithi122=i;smithi122=l;count:12 2024-09-16T00:29:42.150 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:41 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:42.150 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:41 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:29:42.150 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:41 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:42.151 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:41 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:29:42.151 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:41 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:42.151 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:41 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:29:42.151 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:41 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:29:42.151 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:41 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:42.151 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:41 smithi122 ceph-mon[35412]: Deploying daemon mds.l on smithi122 2024-09-16T00:29:42.192 DEBUG:teuthology.orchestra.run.smithi122:mds.c> sudo journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@mds.c.service 2024-09-16T00:29:42.194 DEBUG:teuthology.orchestra.run.smithi122:mds.f> sudo journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@mds.f.service 2024-09-16T00:29:42.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:41 smithi038 ceph-mon[30945]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi038=a;smithi038=d;smithi038=g;smithi038=j;smithi073=b;smithi073=e;smithi073=h;smithi073=k;smithi122=c;smithi122=f;smithi122=i;smithi122=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:29:42.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:41 smithi038 ceph-mon[30945]: Saving service mds.all spec with placement smithi038=a;smithi038=d;smithi038=g;smithi038=j;smithi073=b;smithi073=e;smithi073=h;smithi073=k;smithi122=c;smithi122=f;smithi122=i;smithi122=l;count:12 2024-09-16T00:29:42.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:41 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:42.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:41 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:29:42.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:41 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:42.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:41 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:29:42.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:41 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:42.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:41 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:29:42.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:41 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:29:42.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:41 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:42.216 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:41 smithi038 ceph-mon[30945]: Deploying daemon mds.l on smithi122 2024-09-16T00:29:42.237 DEBUG:teuthology.orchestra.run.smithi122:mds.i> sudo journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@mds.i.service 2024-09-16T00:29:42.280 DEBUG:teuthology.orchestra.run.smithi122:mds.l> sudo journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@mds.l.service 2024-09-16T00:29:42.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:41 smithi073 ceph-mon[35254]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi038=a;smithi038=d;smithi038=g;smithi038=j;smithi073=b;smithi073=e;smithi073=h;smithi073=k;smithi122=c;smithi122=f;smithi122=i;smithi122=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:29:42.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:41 smithi073 ceph-mon[35254]: Saving service mds.all spec with placement smithi038=a;smithi038=d;smithi038=g;smithi038=j;smithi073=b;smithi073=e;smithi073=h;smithi073=k;smithi122=c;smithi122=f;smithi122=i;smithi122=l;count:12 2024-09-16T00:29:42.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:41 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:42.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:41 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:29:42.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:41 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:42.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:41 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:29:42.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:41 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:42.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:41 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:29:42.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:41 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:29:42.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:41 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:42.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:41 smithi073 ceph-mon[35254]: Deploying daemon mds.l on smithi122 2024-09-16T00:29:42.323 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-16T00:29:42.323 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:29:42.579 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:29:43.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.089+0000 7f32a7e3b640 1 -- 172.21.15.38:0/1682319781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32a01036e0 msgr2=0x7f32a0107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:43.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.089+0000 7f32a7e3b640 1 --2- 172.21.15.38:0/1682319781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32a01036e0 0x7f32a0107b90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3294009930 tx=0x7f329402a420 comp rx=0 tx=0).stop 2024-09-16T00:29:43.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.089+0000 7f32a7e3b640 1 -- 172.21.15.38:0/1682319781 shutdown_connections 2024-09-16T00:29:43.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.089+0000 7f32a7e3b640 1 --2- 172.21.15.38:0/1682319781 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32a01082a0 0x7f32a010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:43.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.089+0000 7f32a7e3b640 1 --2- 172.21.15.38:0/1682319781 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32a01036e0 0x7f32a0107b90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:43.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.089+0000 7f32a7e3b640 1 --2- 172.21.15.38:0/1682319781 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32a0102d30 0x7f32a0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:43.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.089+0000 7f32a7e3b640 1 -- 172.21.15.38:0/1682319781 >> 172.21.15.38:0/1682319781 conn(0x7f32a00fc7d0 msgr2=0x7f32a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:43.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.089+0000 7f32a7e3b640 1 -- 172.21.15.38:0/1682319781 shutdown_connections 2024-09-16T00:29:43.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.089+0000 7f32a7e3b640 1 -- 172.21.15.38:0/1682319781 wait complete. 2024-09-16T00:29:43.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.089+0000 7f32a7e3b640 1 Processor -- start 2024-09-16T00:29:43.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.089+0000 7f32a7e3b640 1 -- start start 2024-09-16T00:29:43.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.090+0000 7f32a7e3b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32a0102d30 0x7f32a0100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:43.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.090+0000 7f32a7e3b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32a01036e0 0x7f32a0100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:43.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.090+0000 7f32a7e3b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32a01082a0 0x7f32a0101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:43.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.090+0000 7f32a7e3b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f32a01120c0 con 0x7f32a0102d30 2024-09-16T00:29:43.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.090+0000 7f32a7e3b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f32a0111f40 con 0x7f32a01082a0 2024-09-16T00:29:43.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.090+0000 7f32a7e3b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f32a0112240 con 0x7f32a01036e0 2024-09-16T00:29:43.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.090+0000 7f32a5bd7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32a0102d30 0x7f32a0100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:43.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.090+0000 7f32a5bd7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32a0102d30 0x7f32a0100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45914/0 (socket says 172.21.15.38:45914) 2024-09-16T00:29:43.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.090+0000 7f32a5bd7640 1 -- 172.21.15.38:0/3010723581 learned_addr learned my addr 172.21.15.38:0/3010723581 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:29:43.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.090+0000 7f32a63d8640 1 --2- 172.21.15.38:0/3010723581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32a01082a0 0x7f32a0101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:43.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.090+0000 7f32a53d6640 1 --2- 172.21.15.38:0/3010723581 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32a01036e0 0x7f32a0100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:43.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.090+0000 7f32a5bd7640 1 -- 172.21.15.38:0/3010723581 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32a01036e0 msgr2=0x7f32a0100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:43.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.090+0000 7f32a5bd7640 1 --2- 172.21.15.38:0/3010723581 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32a01036e0 0x7f32a0100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:43.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.090+0000 7f32a5bd7640 1 -- 172.21.15.38:0/3010723581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32a01082a0 msgr2=0x7f32a0101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:43.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.090+0000 7f32a5bd7640 1 --2- 172.21.15.38:0/3010723581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32a01082a0 0x7f32a0101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:43.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.090+0000 7f32a5bd7640 1 -- 172.21.15.38:0/3010723581 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32a01a6ae0 con 0x7f32a0102d30 2024-09-16T00:29:43.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.090+0000 7f32a5bd7640 1 --2- 172.21.15.38:0/3010723581 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32a0102d30 0x7f32a0100990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f329800d950 tx=0x7f329800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:43.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.090+0000 7f329effd640 1 -- 172.21.15.38:0/3010723581 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3298017030 con 0x7f32a0102d30 2024-09-16T00:29:43.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.090+0000 7f32a7e3b640 1 -- 172.21.15.38:0/3010723581 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32a01a6dd0 con 0x7f32a0102d30 2024-09-16T00:29:43.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.090+0000 7f329effd640 1 -- 172.21.15.38:0/3010723581 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3298004020 con 0x7f32a0102d30 2024-09-16T00:29:43.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.090+0000 7f329effd640 1 -- 172.21.15.38:0/3010723581 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32980047a0 con 0x7f32a0102d30 2024-09-16T00:29:43.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.090+0000 7f32a7e3b640 1 -- 172.21.15.38:0/3010723581 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32a01a72d0 con 0x7f32a0102d30 2024-09-16T00:29:43.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.091+0000 7f32a7e3b640 1 -- 172.21.15.38:0/3010723581 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32a0103110 con 0x7f32a0102d30 2024-09-16T00:29:43.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.093+0000 7f329effd640 1 -- 172.21.15.38:0/3010723581 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f32980041d0 con 0x7f32a0102d30 2024-09-16T00:29:43.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.093+0000 7f329effd640 1 --2- 172.21.15.38:0/3010723581 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f32880a3240 0x7f32880a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:43.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.093+0000 7f32a53d6640 1 --2- 172.21.15.38:0/3010723581 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f32880a3240 0x7f32880a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:43.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.093+0000 7f329effd640 1 -- 172.21.15.38:0/3010723581 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f32980cef10 con 0x7f32a0102d30 2024-09-16T00:29:43.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.094+0000 7f32a53d6640 1 --2- 172.21.15.38:0/3010723581 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f32880a3240 0x7f32880a5700 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3294009230 tx=0x7f3294000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:43.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.098+0000 7f329effd640 1 -- 172.21.15.38:0/3010723581 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f329809c0b0 con 0x7f32a0102d30 2024-09-16T00:29:43.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.308+0000 7f32a7e3b640 1 -- 172.21.15.38:0/3010723581 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f32a010a2b0 con 0x7f32a0102d30 2024-09-16T00:29:43.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.308+0000 7f329effd640 1 -- 172.21.15.38:0/3010723581 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7f329809b9b0 con 0x7f32a0102d30 2024-09-16T00:29:43.310 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:29:43.310 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":1,"btime":"2024-09-16T00:22:34:896635+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-16T00:29:43.310 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 1 2024-09-16T00:29:43.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.311+0000 7f32a7e3b640 1 -- 172.21.15.38:0/3010723581 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f32880a3240 msgr2=0x7f32880a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:43.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.311+0000 7f32a7e3b640 1 --2- 172.21.15.38:0/3010723581 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f32880a3240 0x7f32880a5700 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f3294009230 tx=0x7f3294000bc0 comp rx=0 tx=0).stop 2024-09-16T00:29:43.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.311+0000 7f32a7e3b640 1 -- 172.21.15.38:0/3010723581 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32a0102d30 msgr2=0x7f32a0100990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:43.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.311+0000 7f32a7e3b640 1 --2- 172.21.15.38:0/3010723581 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32a0102d30 0x7f32a0100990 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f329800d950 tx=0x7f329800de10 comp rx=0 tx=0).stop 2024-09-16T00:29:43.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.311+0000 7f32a7e3b640 1 -- 172.21.15.38:0/3010723581 shutdown_connections 2024-09-16T00:29:43.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.311+0000 7f32a7e3b640 1 --2- 172.21.15.38:0/3010723581 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f32880a3240 0x7f32880a5700 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:43.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.311+0000 7f32a7e3b640 1 --2- 172.21.15.38:0/3010723581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32a01082a0 0x7f32a0101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:43.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.311+0000 7f32a7e3b640 1 --2- 172.21.15.38:0/3010723581 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32a01036e0 0x7f32a0100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:43.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.311+0000 7f32a7e3b640 1 --2- 172.21.15.38:0/3010723581 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32a0102d30 0x7f32a0100990 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:43.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.311+0000 7f32a7e3b640 1 -- 172.21.15.38:0/3010723581 >> 172.21.15.38:0/3010723581 conn(0x7f32a00fc7d0 msgr2=0x7f32a0105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:43.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.311+0000 7f32a7e3b640 1 -- 172.21.15.38:0/3010723581 shutdown_connections 2024-09-16T00:29:43.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:43.311+0000 7f32a7e3b640 1 -- 172.21.15.38:0/3010723581 wait complete. 2024-09-16T00:29:43.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:43 smithi122 ceph-mon[35412]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:29:43.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:43 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/3010723581' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:29:43.678 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:43 smithi038 ceph-mon[30945]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:29:43.678 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:43 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/3010723581' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:29:43.793 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:43 smithi073 ceph-mon[35254]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:29:43.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:43 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/3010723581' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:29:43.886 INFO:journalctl@ceph.mds.l.smithi122.stdout:Sep 16 00:29:43 smithi122 systemd[1]: Starting Ceph mds.l for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:29:44.146 INFO:journalctl@ceph.mds.l.smithi122.stdout:Sep 16 00:29:43 smithi122 podman[64513]: 2024-09-16 00:29:43.788427069 +0000 UTC m=+0.015705215 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:29:44.146 INFO:journalctl@ceph.mds.l.smithi122.stdout:Sep 16 00:29:43 smithi122 podman[64513]: 2024-09-16 00:29:43.902384553 +0000 UTC m=+0.129662730 container create 4a9bfdbf120dafb8779524c6a70d47fb37fba3e155678431853b619e7a655a99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-l, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2) 2024-09-16T00:29:44.146 INFO:journalctl@ceph.mds.l.smithi122.stdout:Sep 16 00:29:44 smithi122 podman[64513]: 2024-09-16 00:29:44.146716834 +0000 UTC m=+0.373994977 container init 4a9bfdbf120dafb8779524c6a70d47fb37fba3e155678431853b619e7a655a99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-l, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux ) 2024-09-16T00:29:44.466 INFO:journalctl@ceph.mds.l.smithi122.stdout:Sep 16 00:29:44 smithi122 podman[64513]: 2024-09-16 00:29:44.149583034 +0000 UTC m=+0.376861182 container start 4a9bfdbf120dafb8779524c6a70d47fb37fba3e155678431853b619e7a655a99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-l, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-16T00:29:44.467 INFO:journalctl@ceph.mds.l.smithi122.stdout:Sep 16 00:29:44 smithi122 ceph-mds[64532]: -- 172.21.15.122:0/2194682447 <== mon.1 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556d3003f860 con 0x556d3001c800 2024-09-16T00:29:44.467 INFO:journalctl@ceph.mds.l.smithi122.stdout:Sep 16 00:29:44 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-l[64528]: starting mds.l at 2024-09-16T00:29:44.467 INFO:journalctl@ceph.mds.l.smithi122.stdout:Sep 16 00:29:44 smithi122 bash[64513]: 4a9bfdbf120dafb8779524c6a70d47fb37fba3e155678431853b619e7a655a99 2024-09-16T00:29:44.467 INFO:journalctl@ceph.mds.l.smithi122.stdout:Sep 16 00:29:44 smithi122 systemd[1]: Started Ceph mds.l for 983b2072-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:29:45.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:45 smithi073 ceph-mon[35254]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:29:45.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:45 smithi073 ceph-mon[35254]: mds.? [v2:172.21.15.122:6832/412277519,v1:172.21.15.122:6833/412277519] up:boot 2024-09-16T00:29:45.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:45 smithi073 ceph-mon[35254]: fsmap 1 up:standby 2024-09-16T00:29:45.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:45 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-16T00:29:45.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:45 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:45.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:45 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:45.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:45 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:45.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:45 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:29:45.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:45 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:29:45.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:45 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:45.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:45 smithi073 ceph-mon[35254]: Deploying daemon mds.k on smithi073 2024-09-16T00:29:45.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:45 smithi038 ceph-mon[30945]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:29:45.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:45 smithi038 ceph-mon[30945]: mds.? [v2:172.21.15.122:6832/412277519,v1:172.21.15.122:6833/412277519] up:boot 2024-09-16T00:29:45.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:45 smithi038 ceph-mon[30945]: fsmap 1 up:standby 2024-09-16T00:29:45.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:45 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-16T00:29:45.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:45 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:45.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:45 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:45.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:45 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:45.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:45 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:29:45.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:45 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:29:45.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:45 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:45.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:45 smithi038 ceph-mon[30945]: Deploying daemon mds.k on smithi073 2024-09-16T00:29:45.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:45 smithi122 ceph-mon[35412]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:29:45.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:45 smithi122 ceph-mon[35412]: mds.? [v2:172.21.15.122:6832/412277519,v1:172.21.15.122:6833/412277519] up:boot 2024-09-16T00:29:45.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:45 smithi122 ceph-mon[35412]: fsmap 1 up:standby 2024-09-16T00:29:45.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:45 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-09-16T00:29:45.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:45 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:45.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:45 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:45.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:45 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:45.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:45 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:29:45.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:45 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:29:45.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:45 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:45.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:45 smithi122 ceph-mon[35412]: Deploying daemon mds.k on smithi073 2024-09-16T00:29:46.126 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:29:46.392 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:29:46.544 INFO:journalctl@ceph.mds.k.smithi073.stdout:Sep 16 00:29:46 smithi073 systemd[1]: Starting Ceph mds.k for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:29:46.873 INFO:journalctl@ceph.mds.k.smithi073.stdout:Sep 16 00:29:46 smithi073 podman[64461]: 2024-09-16 00:29:46.483450776 +0000 UTC m=+0.015857609 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:29:46.873 INFO:journalctl@ceph.mds.k.smithi073.stdout:Sep 16 00:29:46 smithi073 podman[64461]: 2024-09-16 00:29:46.637570734 +0000 UTC m=+0.169977562 container create 9d7f1d729e1f44e39ca303e70ac14989d8c45059f975b5279d21e78f1db663c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-k, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-16T00:29:46.874 INFO:journalctl@ceph.mds.k.smithi073.stdout:Sep 16 00:29:46 smithi073 podman[64461]: 2024-09-16 00:29:46.873504662 +0000 UTC m=+0.405911489 container init 9d7f1d729e1f44e39ca303e70ac14989d8c45059f975b5279d21e78f1db663c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-k, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-16T00:29:46.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:46.884+0000 7f79e7a22640 1 -- 172.21.15.38:0/940359265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79e0101de0 msgr2=0x7f79e01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:46.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:46.884+0000 7f79e7a22640 1 --2- 172.21.15.38:0/940359265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79e0101de0 0x7f79e01021c0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f79dc009900 tx=0x7f79dc02a410 comp rx=0 tx=0).stop 2024-09-16T00:29:46.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:46.884+0000 7f79e7a22640 1 -- 172.21.15.38:0/940359265 shutdown_connections 2024-09-16T00:29:46.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:46.884+0000 7f79e7a22640 1 --2- 172.21.15.38:0/940359265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79e0106f10 0x7f79e010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:46.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:46.884+0000 7f79e7a22640 1 --2- 172.21.15.38:0/940359265 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79e0102790 0x7f79e01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:46.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:46.884+0000 7f79e7a22640 1 --2- 172.21.15.38:0/940359265 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79e0101de0 0x7f79e01021c0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:46.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:46.884+0000 7f79e7a22640 1 -- 172.21.15.38:0/940359265 >> 172.21.15.38:0/940359265 conn(0x7f79e00fd8c0 msgr2=0x7f79e00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:46.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:46.884+0000 7f79e7a22640 1 -- 172.21.15.38:0/940359265 shutdown_connections 2024-09-16T00:29:46.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:46.884+0000 7f79e7a22640 1 -- 172.21.15.38:0/940359265 wait complete. 2024-09-16T00:29:46.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:46.885+0000 7f79e7a22640 1 Processor -- start 2024-09-16T00:29:46.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:46.885+0000 7f79e7a22640 1 -- start start 2024-09-16T00:29:46.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:46.885+0000 7f79e7a22640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79e0101de0 0x7f79e01984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:46.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:46.885+0000 7f79e7a22640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79e0102790 0x7f79e0198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:46.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:46.885+0000 7f79e7a22640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79e0106f10 0x7f79e019cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:46.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:46.885+0000 7f79e7a22640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f79e0111f70 con 0x7f79e0106f10 2024-09-16T00:29:46.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:46.885+0000 7f79e7a22640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f79e0111df0 con 0x7f79e0101de0 2024-09-16T00:29:46.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:46.885+0000 7f79e7a22640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f79e01120f0 con 0x7f79e0102790 2024-09-16T00:29:46.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:46.885+0000 7f79e5fbf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79e0106f10 0x7f79e019cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:46.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:46.885+0000 7f79e5fbf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79e0106f10 0x7f79e019cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34520/0 (socket says 172.21.15.38:34520) 2024-09-16T00:29:46.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:46.885+0000 7f79e57be640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79e0101de0 0x7f79e01984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:46.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:46.885+0000 7f79e5fbf640 1 -- 172.21.15.38:0/4076351296 learned_addr learned my addr 172.21.15.38:0/4076351296 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:29:46.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:46.885+0000 7f79e4fbd640 1 --2- 172.21.15.38:0/4076351296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79e0102790 0x7f79e0198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:46.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:46.885+0000 7f79e5fbf640 1 -- 172.21.15.38:0/4076351296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79e0102790 msgr2=0x7f79e0198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:46.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:46.885+0000 7f79e5fbf640 1 --2- 172.21.15.38:0/4076351296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79e0102790 0x7f79e0198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:46.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:46.885+0000 7f79e5fbf640 1 -- 172.21.15.38:0/4076351296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79e0101de0 msgr2=0x7f79e01984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:46.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:46.885+0000 7f79e5fbf640 1 --2- 172.21.15.38:0/4076351296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79e0101de0 0x7f79e01984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:46.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:46.885+0000 7f79e5fbf640 1 -- 172.21.15.38:0/4076351296 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79e019d480 con 0x7f79e0106f10 2024-09-16T00:29:46.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:46.885+0000 7f79e57be640 1 --2- 172.21.15.38:0/4076351296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79e0101de0 0x7f79e01984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:29:46.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:46.886+0000 7f79e5fbf640 1 --2- 172.21.15.38:0/4076351296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79e0106f10 0x7f79e019cda0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f79d8009fd0 tx=0x7f79d800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:46.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:46.886+0000 7f79d67fc640 1 -- 172.21.15.38:0/4076351296 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d8017070 con 0x7f79e0106f10 2024-09-16T00:29:46.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:46.886+0000 7f79d67fc640 1 -- 172.21.15.38:0/4076351296 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79d8003e50 con 0x7f79e0106f10 2024-09-16T00:29:46.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:46.886+0000 7f79e7a22640 1 -- 172.21.15.38:0/4076351296 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79e019d770 con 0x7f79e0106f10 2024-09-16T00:29:46.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:46.886+0000 7f79d67fc640 1 -- 172.21.15.38:0/4076351296 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d80106c0 con 0x7f79e0106f10 2024-09-16T00:29:46.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:46.886+0000 7f79e7a22640 1 -- 172.21.15.38:0/4076351296 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79e01a5050 con 0x7f79e0106f10 2024-09-16T00:29:46.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:46.887+0000 7f79e7a22640 1 -- 172.21.15.38:0/4076351296 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79e01021c0 con 0x7f79e0106f10 2024-09-16T00:29:46.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:46.888+0000 7f79d67fc640 1 -- 172.21.15.38:0/4076351296 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f79d8003970 con 0x7f79e0106f10 2024-09-16T00:29:46.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:46.889+0000 7f79d67fc640 1 --2- 172.21.15.38:0/4076351296 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f79cc0a32c0 0x7f79cc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:46.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:46.889+0000 7f79e57be640 1 --2- 172.21.15.38:0/4076351296 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f79cc0a32c0 0x7f79cc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:46.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:46.889+0000 7f79d67fc640 1 -- 172.21.15.38:0/4076351296 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f79d80cf1b0 con 0x7f79e0106f10 2024-09-16T00:29:46.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:46.890+0000 7f79e57be640 1 --2- 172.21.15.38:0/4076351296 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f79cc0a32c0 0x7f79cc0a5780 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f79dc009230 tx=0x7f79dc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:46.912 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:46.894+0000 7f79d67fc640 1 -- 172.21.15.38:0/4076351296 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f79d809c2d0 con 0x7f79e0106f10 2024-09-16T00:29:47.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:47.107+0000 7f79e7a22640 1 -- 172.21.15.38:0/4076351296 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f79e0109b60 con 0x7f79e0106f10 2024-09-16T00:29:47.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:47.107+0000 7f79d67fc640 1 -- 172.21.15.38:0/4076351296 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1435 (secure 0 0 0) 0x7f79d809bbd0 con 0x7f79e0106f10 2024-09-16T00:29:47.108 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:29:47.108 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":2,"btime":"2024-09-16T00:29:44:177480+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/412277519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":412277519},{"type":"v1","addr":"172.21.15.122:6833","nonce":412277519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-16T00:29:47.108 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 2 2024-09-16T00:29:47.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:47.109+0000 7f79e7a22640 1 -- 172.21.15.38:0/4076351296 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f79cc0a32c0 msgr2=0x7f79cc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:47.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:47.109+0000 7f79e7a22640 1 --2- 172.21.15.38:0/4076351296 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f79cc0a32c0 0x7f79cc0a5780 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f79dc009230 tx=0x7f79dc0058d0 comp rx=0 tx=0).stop 2024-09-16T00:29:47.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:47.109+0000 7f79e7a22640 1 -- 172.21.15.38:0/4076351296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79e0106f10 msgr2=0x7f79e019cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:47.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:47.109+0000 7f79e7a22640 1 --2- 172.21.15.38:0/4076351296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79e0106f10 0x7f79e019cda0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f79d8009fd0 tx=0x7f79d800b260 comp rx=0 tx=0).stop 2024-09-16T00:29:47.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:47.109+0000 7f79e7a22640 1 -- 172.21.15.38:0/4076351296 shutdown_connections 2024-09-16T00:29:47.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:47.109+0000 7f79e7a22640 1 --2- 172.21.15.38:0/4076351296 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f79cc0a32c0 0x7f79cc0a5780 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:47.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:47.109+0000 7f79e7a22640 1 --2- 172.21.15.38:0/4076351296 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79e0106f10 0x7f79e019cda0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:47.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:47.109+0000 7f79e7a22640 1 --2- 172.21.15.38:0/4076351296 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79e0102790 0x7f79e0198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:47.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:47.109+0000 7f79e7a22640 1 --2- 172.21.15.38:0/4076351296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79e0101de0 0x7f79e01984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:47.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:47.109+0000 7f79e7a22640 1 -- 172.21.15.38:0/4076351296 >> 172.21.15.38:0/4076351296 conn(0x7f79e00fd8c0 msgr2=0x7f79e00fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:47.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:47.110+0000 7f79e7a22640 1 -- 172.21.15.38:0/4076351296 shutdown_connections 2024-09-16T00:29:47.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:47.110+0000 7f79e7a22640 1 -- 172.21.15.38:0/4076351296 wait complete. 2024-09-16T00:29:47.193 INFO:journalctl@ceph.mds.k.smithi073.stdout:Sep 16 00:29:46 smithi073 podman[64461]: 2024-09-16 00:29:46.876754104 +0000 UTC m=+0.409160931 container start 9d7f1d729e1f44e39ca303e70ac14989d8c45059f975b5279d21e78f1db663c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-16T00:29:47.193 INFO:journalctl@ceph.mds.k.smithi073.stdout:Sep 16 00:29:46 smithi073 ceph-mds[64480]: -- 172.21.15.73:0/1709539838 <== mon.2 v2:172.21.15.73:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dfc72bb860 con 0x55dfc7298800 2024-09-16T00:29:47.194 INFO:journalctl@ceph.mds.k.smithi073.stdout:Sep 16 00:29:46 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-k[64476]: starting mds.k at 2024-09-16T00:29:47.194 INFO:journalctl@ceph.mds.k.smithi073.stdout:Sep 16 00:29:46 smithi073 bash[64461]: 9d7f1d729e1f44e39ca303e70ac14989d8c45059f975b5279d21e78f1db663c8 2024-09-16T00:29:47.194 INFO:journalctl@ceph.mds.k.smithi073.stdout:Sep 16 00:29:46 smithi073 systemd[1]: Started Ceph mds.k for 983b2072-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:29:47.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:47 smithi038 ceph-mon[30945]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:29:47.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:47 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:47.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:47 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:47.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:47 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:47.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:47 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:29:47.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:47 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:29:47.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:47 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:47.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:47 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/4076351296' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:29:47.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:47 smithi122 ceph-mon[35412]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:29:47.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:47 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:47.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:47 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:47.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:47 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:47.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:47 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:29:47.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:47 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:29:47.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:47 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:47.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:47 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/4076351296' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:29:47.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:47 smithi073 ceph-mon[35254]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:29:47.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:47 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:47.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:47 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:47.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:47 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:47.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:47 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:29:47.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:47 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:29:47.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:47 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:47.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:47 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/4076351296' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:29:48.291 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:48 smithi073 ceph-mon[35254]: Deploying daemon mds.j on smithi038 2024-09-16T00:29:48.291 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:48 smithi073 ceph-mon[35254]: mds.? [v2:172.21.15.73:6832/1210328177,v1:172.21.15.73:6833/1210328177] up:boot 2024-09-16T00:29:48.291 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:48 smithi073 ceph-mon[35254]: fsmap 2 up:standby 2024-09-16T00:29:48.291 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:48 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T00:29:48.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:48 smithi038 ceph-mon[30945]: Deploying daemon mds.j on smithi038 2024-09-16T00:29:48.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:48 smithi038 ceph-mon[30945]: mds.? [v2:172.21.15.73:6832/1210328177,v1:172.21.15.73:6833/1210328177] up:boot 2024-09-16T00:29:48.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:48 smithi038 ceph-mon[30945]: fsmap 2 up:standby 2024-09-16T00:29:48.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:48 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T00:29:48.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:48 smithi122 ceph-mon[35412]: Deploying daemon mds.j on smithi038 2024-09-16T00:29:48.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:48 smithi122 ceph-mon[35412]: mds.? [v2:172.21.15.73:6832/1210328177,v1:172.21.15.73:6833/1210328177] up:boot 2024-09-16T00:29:48.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:48 smithi122 ceph-mon[35412]: fsmap 2 up:standby 2024-09-16T00:29:48.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:48 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T00:29:49.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:49 smithi038 ceph-mon[30945]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:29:49.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:49 smithi122 ceph-mon[35412]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:29:49.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:49 smithi073 ceph-mon[35254]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:29:49.940 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:29:49.964 INFO:journalctl@ceph.mds.j.smithi038.stdout:Sep 16 00:29:49 smithi038 systemd[1]: Starting Ceph mds.j for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:29:50.220 INFO:journalctl@ceph.mds.j.smithi038.stdout:Sep 16 00:29:50 smithi038 podman[69966]: 2024-09-16 00:29:49.925034229 +0000 UTC m=+0.016471291 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:29:50.221 INFO:journalctl@ceph.mds.j.smithi038.stdout:Sep 16 00:29:50 smithi038 podman[69966]: 2024-09-16 00:29:50.076567544 +0000 UTC m=+0.168004592 container create 7b4923415ebd94682272d866fd5e96ce210ddb0fb29942f00919ac817d0f4a14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-j, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-16T00:29:50.458 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:29:50.581 INFO:journalctl@ceph.mds.j.smithi038.stdout:Sep 16 00:29:50 smithi038 podman[69966]: 2024-09-16 00:29:50.321087953 +0000 UTC m=+0.412525002 container init 7b4923415ebd94682272d866fd5e96ce210ddb0fb29942f00919ac817d0f4a14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-j, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, ceph=True) 2024-09-16T00:29:50.582 INFO:journalctl@ceph.mds.j.smithi038.stdout:Sep 16 00:29:50 smithi038 podman[69966]: 2024-09-16 00:29:50.324147556 +0000 UTC m=+0.415584611 container start 7b4923415ebd94682272d866fd5e96ce210ddb0fb29942f00919ac817d0f4a14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-j, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-16T00:29:50.582 INFO:journalctl@ceph.mds.j.smithi038.stdout:Sep 16 00:29:50 smithi038 ceph-mds[70026]: -- 172.21.15.38:0/3122220825 <== mon.0 v2:172.21.15.38:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555ba266b860 con 0x555ba2649400 2024-09-16T00:29:50.582 INFO:journalctl@ceph.mds.j.smithi038.stdout:Sep 16 00:29:50 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-j[70011]: starting mds.j at 2024-09-16T00:29:50.582 INFO:journalctl@ceph.mds.j.smithi038.stdout:Sep 16 00:29:50 smithi038 bash[69966]: 7b4923415ebd94682272d866fd5e96ce210ddb0fb29942f00919ac817d0f4a14 2024-09-16T00:29:50.582 INFO:journalctl@ceph.mds.j.smithi038.stdout:Sep 16 00:29:50 smithi038 systemd[1]: Started Ceph mds.j for 983b2072-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:29:50.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:50.966+0000 7fd5d0ee8640 1 -- 172.21.15.38:0/2689330040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5cc101de0 msgr2=0x7fd5cc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:50.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:50.966+0000 7fd5d0ee8640 1 --2- 172.21.15.38:0/2689330040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5cc101de0 0x7fd5cc1021c0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc009900 tx=0x7fd5bc02a410 comp rx=0 tx=0).stop 2024-09-16T00:29:50.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:50.966+0000 7fd5d0ee8640 1 -- 172.21.15.38:0/2689330040 shutdown_connections 2024-09-16T00:29:50.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:50.966+0000 7fd5d0ee8640 1 --2- 172.21.15.38:0/2689330040 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5cc106f10 0x7fd5cc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:50.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:50.966+0000 7fd5d0ee8640 1 --2- 172.21.15.38:0/2689330040 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5cc102790 0x7fd5cc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:50.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:50.966+0000 7fd5d0ee8640 1 --2- 172.21.15.38:0/2689330040 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5cc101de0 0x7fd5cc1021c0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:50.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:50.966+0000 7fd5d0ee8640 1 -- 172.21.15.38:0/2689330040 >> 172.21.15.38:0/2689330040 conn(0x7fd5cc0fd8c0 msgr2=0x7fd5cc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:50.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:50.966+0000 7fd5d0ee8640 1 -- 172.21.15.38:0/2689330040 shutdown_connections 2024-09-16T00:29:50.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:50.966+0000 7fd5d0ee8640 1 -- 172.21.15.38:0/2689330040 wait complete. 2024-09-16T00:29:50.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:50.966+0000 7fd5d0ee8640 1 Processor -- start 2024-09-16T00:29:50.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:50.967+0000 7fd5d0ee8640 1 -- start start 2024-09-16T00:29:50.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:50.967+0000 7fd5d0ee8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5cc101de0 0x7fd5cc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:50.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:50.967+0000 7fd5d0ee8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5cc102790 0x7fd5cc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:50.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:50.967+0000 7fd5d0ee8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5cc106f10 0x7fd5cc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:50.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:50.967+0000 7fd5d0ee8640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd5cc111f70 con 0x7fd5cc102790 2024-09-16T00:29:50.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:50.967+0000 7fd5d0ee8640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd5cc111df0 con 0x7fd5cc101de0 2024-09-16T00:29:50.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:50.967+0000 7fd5d0ee8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd5cc1120f0 con 0x7fd5cc106f10 2024-09-16T00:29:50.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:50.967+0000 7fd5c9d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5cc102790 0x7fd5cc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:50.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:50.967+0000 7fd5c9d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5cc102790 0x7fd5cc074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34560/0 (socket says 172.21.15.38:34560) 2024-09-16T00:29:50.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:50.967+0000 7fd5c9d9b640 1 -- 172.21.15.38:0/3145536936 learned_addr learned my addr 172.21.15.38:0/3145536936 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:29:50.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:50.967+0000 7fd5ca59c640 1 --2- 172.21.15.38:0/3145536936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5cc101de0 0x7fd5cc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:50.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:50.967+0000 7fd5cad9d640 1 --2- 172.21.15.38:0/3145536936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5cc106f10 0x7fd5cc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:50.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:50.967+0000 7fd5c9d9b640 1 -- 172.21.15.38:0/3145536936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5cc106f10 msgr2=0x7fd5cc074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:50.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:50.967+0000 7fd5c9d9b640 1 --2- 172.21.15.38:0/3145536936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5cc106f10 0x7fd5cc074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:50.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:50.967+0000 7fd5c9d9b640 1 -- 172.21.15.38:0/3145536936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5cc101de0 msgr2=0x7fd5cc0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:50.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:50.967+0000 7fd5c9d9b640 1 --2- 172.21.15.38:0/3145536936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5cc101de0 0x7fd5cc0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:50.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:50.967+0000 7fd5c9d9b640 1 -- 172.21.15.38:0/3145536936 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5cc1a6910 con 0x7fd5cc102790 2024-09-16T00:29:50.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:50.967+0000 7fd5ca59c640 1 --2- 172.21.15.38:0/3145536936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5cc101de0 0x7fd5cc0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:29:50.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:50.967+0000 7fd5c9d9b640 1 --2- 172.21.15.38:0/3145536936 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5cc102790 0x7fd5cc074a30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd5c000e9f0 tx=0x7fd5c000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:50.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:50.968+0000 7fd5bb7fe640 1 -- 172.21.15.38:0/3145536936 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c000cd80 con 0x7fd5cc102790 2024-09-16T00:29:50.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:50.968+0000 7fd5bb7fe640 1 -- 172.21.15.38:0/3145536936 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd5c0010910 con 0x7fd5cc102790 2024-09-16T00:29:50.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:50.968+0000 7fd5d0ee8640 1 -- 172.21.15.38:0/3145536936 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5cc1a6c00 con 0x7fd5cc102790 2024-09-16T00:29:50.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:50.968+0000 7fd5bb7fe640 1 -- 172.21.15.38:0/3145536936 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5c0018990 con 0x7fd5cc102790 2024-09-16T00:29:50.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:50.968+0000 7fd5d0ee8640 1 -- 172.21.15.38:0/3145536936 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5cc1a7140 con 0x7fd5cc102790 2024-09-16T00:29:50.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:50.969+0000 7fd5d0ee8640 1 -- 172.21.15.38:0/3145536936 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5cc1021c0 con 0x7fd5cc102790 2024-09-16T00:29:50.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:50.970+0000 7fd5bb7fe640 1 -- 172.21.15.38:0/3145536936 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fd5c0010ac0 con 0x7fd5cc102790 2024-09-16T00:29:50.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:50.971+0000 7fd5bb7fe640 1 --2- 172.21.15.38:0/3145536936 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd5b40a3240 0x7fd5b40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:50.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:50.971+0000 7fd5ca59c640 1 --2- 172.21.15.38:0/3145536936 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd5b40a3240 0x7fd5b40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:50.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:50.971+0000 7fd5bb7fe640 1 -- 172.21.15.38:0/3145536936 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fd5c00cfcb0 con 0x7fd5cc102790 2024-09-16T00:29:50.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:50.971+0000 7fd5ca59c640 1 --2- 172.21.15.38:0/3145536936 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd5b40a3240 0x7fd5b40a5700 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc005ab0 tx=0x7fd5bc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:50.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:50.974+0000 7fd5bb7fe640 1 -- 172.21.15.38:0/3145536936 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd5c009ce70 con 0x7fd5cc102790 2024-09-16T00:29:51.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:51.216+0000 7fd5d0ee8640 1 -- 172.21.15.38:0/3145536936 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5cc075a20 con 0x7fd5cc102790 2024-09-16T00:29:51.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:51.216+0000 7fd5bb7fe640 1 -- 172.21.15.38:0/3145536936 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3082 (secure 0 0 0) 0x7fd5cc075a20 con 0x7fd5cc102790 2024-09-16T00:29:51.217 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:29:51.217 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":4,"btime":"2024-09-16T00:29:51:204760+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/367974527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":367974527},{"type":"v1","addr":"172.21.15.38:6835","nonce":367974527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/412277519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":412277519},{"type":"v1","addr":"172.21.15.122:6833","nonce":412277519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/1210328177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1210328177},{"type":"v1","addr":"172.21.15.73:6833","nonce":1210328177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-16T00:29:51.217 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 4 2024-09-16T00:29:51.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:51.218+0000 7fd5d0ee8640 1 -- 172.21.15.38:0/3145536936 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd5b40a3240 msgr2=0x7fd5b40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:51.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:51.218+0000 7fd5d0ee8640 1 --2- 172.21.15.38:0/3145536936 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd5b40a3240 0x7fd5b40a5700 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc005ab0 tx=0x7fd5bc005a20 comp rx=0 tx=0).stop 2024-09-16T00:29:51.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:51.218+0000 7fd5d0ee8640 1 -- 172.21.15.38:0/3145536936 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5cc102790 msgr2=0x7fd5cc074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:51.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:51.218+0000 7fd5d0ee8640 1 --2- 172.21.15.38:0/3145536936 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5cc102790 0x7fd5cc074a30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fd5c000e9f0 tx=0x7fd5c000eeb0 comp rx=0 tx=0).stop 2024-09-16T00:29:51.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:51.219+0000 7fd5d0ee8640 1 -- 172.21.15.38:0/3145536936 shutdown_connections 2024-09-16T00:29:51.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:51.219+0000 7fd5d0ee8640 1 --2- 172.21.15.38:0/3145536936 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd5b40a3240 0x7fd5b40a5700 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:51.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:51.219+0000 7fd5d0ee8640 1 --2- 172.21.15.38:0/3145536936 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd5cc106f10 0x7fd5cc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:51.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:51.219+0000 7fd5d0ee8640 1 --2- 172.21.15.38:0/3145536936 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd5cc102790 0x7fd5cc074a30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:51.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:51.219+0000 7fd5d0ee8640 1 --2- 172.21.15.38:0/3145536936 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd5cc101de0 0x7fd5cc0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:51.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:51.219+0000 7fd5d0ee8640 1 -- 172.21.15.38:0/3145536936 >> 172.21.15.38:0/3145536936 conn(0x7fd5cc0fd8c0 msgr2=0x7fd5cc0fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:51.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:51.219+0000 7fd5d0ee8640 1 -- 172.21.15.38:0/3145536936 shutdown_connections 2024-09-16T00:29:51.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:51.219+0000 7fd5d0ee8640 1 -- 172.21.15.38:0/3145536936 wait complete. 2024-09-16T00:29:51.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:51 smithi038 ceph-mon[30945]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:29:51.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:51 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:51.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:51 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:51.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:51 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:51.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:51 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:29:51.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:51 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:29:51.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:51 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:51.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:51 smithi122 ceph-mon[35412]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:29:51.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:51 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:51.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:51 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:51.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:51 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:51.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:51 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:29:51.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:51 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:29:51.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:51 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:51.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:51 smithi073 ceph-mon[35254]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:29:51.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:51 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:51.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:51 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:51.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:51 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:51.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:51 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:29:51.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:51 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:29:51.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:51 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:52.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:52 smithi038 ceph-mon[30945]: Deploying daemon mds.d on smithi038 2024-09-16T00:29:52.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:52 smithi038 ceph-mon[30945]: mds.? [v2:172.21.15.38:6834/367974527,v1:172.21.15.38:6835/367974527] up:boot 2024-09-16T00:29:52.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:52 smithi038 ceph-mon[30945]: fsmap 3 up:standby 2024-09-16T00:29:52.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:52 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-16T00:29:52.276 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:52 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/3145536936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:29:52.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:52 smithi073 ceph-mon[35254]: Deploying daemon mds.d on smithi038 2024-09-16T00:29:52.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:52 smithi073 ceph-mon[35254]: mds.? [v2:172.21.15.38:6834/367974527,v1:172.21.15.38:6835/367974527] up:boot 2024-09-16T00:29:52.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:52 smithi073 ceph-mon[35254]: fsmap 3 up:standby 2024-09-16T00:29:52.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:52 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-16T00:29:52.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:52 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/3145536936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:29:52.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:52 smithi122 ceph-mon[35412]: Deploying daemon mds.d on smithi038 2024-09-16T00:29:52.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:52 smithi122 ceph-mon[35412]: mds.? [v2:172.21.15.38:6834/367974527,v1:172.21.15.38:6835/367974527] up:boot 2024-09-16T00:29:52.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:52 smithi122 ceph-mon[35412]: fsmap 3 up:standby 2024-09-16T00:29:52.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:52 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-09-16T00:29:52.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:52 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/3145536936' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:29:53.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:53 smithi073 ceph-mon[35254]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:29:53.678 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:53 smithi038 ceph-mon[30945]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:29:53.678 INFO:journalctl@ceph.mds.d.smithi038.stdout:Sep 16 00:29:53 smithi038 systemd[1]: Starting Ceph mds.d for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:29:53.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:53 smithi122 ceph-mon[35412]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:29:53.965 INFO:journalctl@ceph.mds.d.smithi038.stdout:Sep 16 00:29:53 smithi038 podman[70378]: 2024-09-16 00:29:53.580846096 +0000 UTC m=+0.016622088 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:29:53.965 INFO:journalctl@ceph.mds.d.smithi038.stdout:Sep 16 00:29:53 smithi038 podman[70378]: 2024-09-16 00:29:53.715840295 +0000 UTC m=+0.151616271 container create 250b747ee3944c6339781f21ed89d9a1b8a919c643c5b12883c639cbb64b7c1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-d, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-16T00:29:54.009 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:29:54.279 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:29:54.401 INFO:journalctl@ceph.mds.d.smithi038.stdout:Sep 16 00:29:53 smithi038 podman[70378]: 2024-09-16 00:29:53.968449883 +0000 UTC m=+0.404225860 container init 250b747ee3944c6339781f21ed89d9a1b8a919c643c5b12883c639cbb64b7c1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-d, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-16T00:29:54.401 INFO:journalctl@ceph.mds.d.smithi038.stdout:Sep 16 00:29:53 smithi038 podman[70378]: 2024-09-16 00:29:53.971450454 +0000 UTC m=+0.407226442 container start 250b747ee3944c6339781f21ed89d9a1b8a919c643c5b12883c639cbb64b7c1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-d, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True) 2024-09-16T00:29:54.401 INFO:journalctl@ceph.mds.d.smithi038.stdout:Sep 16 00:29:53 smithi038 ceph-mds[70396]: -- 172.21.15.38:0/2519424040 <== mon.0 v2:172.21.15.38:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557747945860 con 0x557747923000 2024-09-16T00:29:54.401 INFO:journalctl@ceph.mds.d.smithi038.stdout:Sep 16 00:29:53 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-d[70392]: starting mds.d at 2024-09-16T00:29:54.401 INFO:journalctl@ceph.mds.d.smithi038.stdout:Sep 16 00:29:54 smithi038 bash[70378]: 250b747ee3944c6339781f21ed89d9a1b8a919c643c5b12883c639cbb64b7c1c 2024-09-16T00:29:54.401 INFO:journalctl@ceph.mds.d.smithi038.stdout:Sep 16 00:29:54 smithi038 systemd[1]: Started Ceph mds.d for 983b2072-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:29:54.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.772+0000 7fd7f70e5640 1 -- 172.21.15.38:0/1933551144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7f0106f10 msgr2=0x7fd7f010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:54.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.772+0000 7fd7f70e5640 1 --2- 172.21.15.38:0/1933551144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7f0106f10 0x7fd7f010ea90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8009ca0 tx=0x7fd7e802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:29:54.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.772+0000 7fd7f70e5640 1 -- 172.21.15.38:0/1933551144 shutdown_connections 2024-09-16T00:29:54.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.772+0000 7fd7f70e5640 1 --2- 172.21.15.38:0/1933551144 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7f0106f10 0x7fd7f010ea90 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:54.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.772+0000 7fd7f70e5640 1 --2- 172.21.15.38:0/1933551144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7f0102790 0x7fd7f01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:54.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.772+0000 7fd7f70e5640 1 --2- 172.21.15.38:0/1933551144 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7f0101de0 0x7fd7f01021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:54.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.772+0000 7fd7f70e5640 1 -- 172.21.15.38:0/1933551144 >> 172.21.15.38:0/1933551144 conn(0x7fd7f00fd8c0 msgr2=0x7fd7f00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:54.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.772+0000 7fd7f70e5640 1 -- 172.21.15.38:0/1933551144 shutdown_connections 2024-09-16T00:29:54.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.772+0000 7fd7f70e5640 1 -- 172.21.15.38:0/1933551144 wait complete. 2024-09-16T00:29:54.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.773+0000 7fd7f70e5640 1 Processor -- start 2024-09-16T00:29:54.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.773+0000 7fd7f70e5640 1 -- start start 2024-09-16T00:29:54.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.773+0000 7fd7f70e5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7f0101de0 0x7fd7f00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:54.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.773+0000 7fd7f70e5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7f0102790 0x7fd7f0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:54.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.773+0000 7fd7f70e5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7f0106f10 0x7fd7f0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:54.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.773+0000 7fd7f70e5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f0111f70 con 0x7fd7f0106f10 2024-09-16T00:29:54.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.773+0000 7fd7f70e5640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f0111df0 con 0x7fd7f0101de0 2024-09-16T00:29:54.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.773+0000 7fd7f70e5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd7f01120f0 con 0x7fd7f0102790 2024-09-16T00:29:54.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.773+0000 7fd7f5682640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7f0106f10 0x7fd7f0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:54.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.773+0000 7fd7f5682640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7f0106f10 0x7fd7f0074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34598/0 (socket says 172.21.15.38:34598) 2024-09-16T00:29:54.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.773+0000 7fd7f5682640 1 -- 172.21.15.38:0/3980270392 learned_addr learned my addr 172.21.15.38:0/3980270392 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:29:54.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.774+0000 7fd7f4e81640 1 --2- 172.21.15.38:0/3980270392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7f0101de0 0x7fd7f00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:54.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.774+0000 7fd7e7fff640 1 --2- 172.21.15.38:0/3980270392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7f0102790 0x7fd7f0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:54.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.774+0000 7fd7f5682640 1 -- 172.21.15.38:0/3980270392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7f0102790 msgr2=0x7fd7f0074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:54.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.774+0000 7fd7f5682640 1 --2- 172.21.15.38:0/3980270392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7f0102790 0x7fd7f0074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:54.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.774+0000 7fd7f5682640 1 -- 172.21.15.38:0/3980270392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7f0101de0 msgr2=0x7fd7f00744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:54.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.774+0000 7fd7f5682640 1 --2- 172.21.15.38:0/3980270392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7f0101de0 0x7fd7f00744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:54.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.774+0000 7fd7f5682640 1 -- 172.21.15.38:0/3980270392 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7f01a6890 con 0x7fd7f0106f10 2024-09-16T00:29:54.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.774+0000 7fd7e7fff640 1 --2- 172.21.15.38:0/3980270392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7f0102790 0x7fd7f0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:29:54.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.774+0000 7fd7f4e81640 1 --2- 172.21.15.38:0/3980270392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7f0101de0 0x7fd7f00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:29:54.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.774+0000 7fd7f5682640 1 --2- 172.21.15.38:0/3980270392 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7f0106f10 0x7fd7f0074f70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8031a20 tx=0x7fd7e8031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:54.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.774+0000 7fd7e5ffb640 1 -- 172.21.15.38:0/3980270392 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e8003b70 con 0x7fd7f0106f10 2024-09-16T00:29:54.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.774+0000 7fd7e5ffb640 1 -- 172.21.15.38:0/3980270392 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd7e8003d10 con 0x7fd7f0106f10 2024-09-16T00:29:54.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.774+0000 7fd7f70e5640 1 -- 172.21.15.38:0/3980270392 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7f01a6b20 con 0x7fd7f0106f10 2024-09-16T00:29:54.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.774+0000 7fd7e5ffb640 1 -- 172.21.15.38:0/3980270392 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7e802fc40 con 0x7fd7f0106f10 2024-09-16T00:29:54.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.774+0000 7fd7f70e5640 1 -- 172.21.15.38:0/3980270392 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7f01a70b0 con 0x7fd7f0106f10 2024-09-16T00:29:54.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.775+0000 7fd7f70e5640 1 -- 172.21.15.38:0/3980270392 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7f01021c0 con 0x7fd7f0106f10 2024-09-16T00:29:54.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.777+0000 7fd7e5ffb640 1 -- 172.21.15.38:0/3980270392 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fd7e8004050 con 0x7fd7f0106f10 2024-09-16T00:29:54.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.777+0000 7fd7e5ffb640 1 --2- 172.21.15.38:0/3980270392 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd7d80a32c0 0x7fd7d80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:54.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.777+0000 7fd7f4e81640 1 --2- 172.21.15.38:0/3980270392 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd7d80a32c0 0x7fd7d80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:54.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.777+0000 7fd7e5ffb640 1 -- 172.21.15.38:0/3980270392 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fd7e80f0630 con 0x7fd7f0106f10 2024-09-16T00:29:54.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.778+0000 7fd7f4e81640 1 --2- 172.21.15.38:0/3980270392 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd7d80a32c0 0x7fd7d80a5780 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec009c20 tx=0x7fd7ec005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:54.782 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.782+0000 7fd7e5ffb640 1 -- 172.21.15.38:0/3980270392 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd7e80bd880 con 0x7fd7f0106f10 2024-09-16T00:29:54.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.992+0000 7fd7f70e5640 1 -- 172.21.15.38:0/3980270392 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd7f010a150 con 0x7fd7f0106f10 2024-09-16T00:29:54.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.993+0000 7fd7e5ffb640 1 -- 172.21.15.38:0/3980270392 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3904 (secure 0 0 0) 0x7fd7e80bd180 con 0x7fd7f0106f10 2024-09-16T00:29:54.993 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:29:54.993 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":5,"btime":"2024-09-16T00:29:54:229081+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/367974527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":367974527},{"type":"v1","addr":"172.21.15.38:6835","nonce":367974527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/335077136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":335077136},{"type":"v1","addr":"172.21.15.38:6837","nonce":335077136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/412277519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":412277519},{"type":"v1","addr":"172.21.15.122:6833","nonce":412277519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/1210328177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1210328177},{"type":"v1","addr":"172.21.15.73:6833","nonce":1210328177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-16T00:29:54.994 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 5 2024-09-16T00:29:54.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.995+0000 7fd7f70e5640 1 -- 172.21.15.38:0/3980270392 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd7d80a32c0 msgr2=0x7fd7d80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:54.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.995+0000 7fd7f70e5640 1 --2- 172.21.15.38:0/3980270392 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd7d80a32c0 0x7fd7d80a5780 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec009c20 tx=0x7fd7ec005f90 comp rx=0 tx=0).stop 2024-09-16T00:29:54.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.995+0000 7fd7f70e5640 1 -- 172.21.15.38:0/3980270392 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7f0106f10 msgr2=0x7fd7f0074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:54.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.995+0000 7fd7f70e5640 1 --2- 172.21.15.38:0/3980270392 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7f0106f10 0x7fd7f0074f70 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd7e8031a20 tx=0x7fd7e8031a50 comp rx=0 tx=0).stop 2024-09-16T00:29:54.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.995+0000 7fd7f70e5640 1 -- 172.21.15.38:0/3980270392 shutdown_connections 2024-09-16T00:29:54.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.995+0000 7fd7f70e5640 1 --2- 172.21.15.38:0/3980270392 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd7d80a32c0 0x7fd7d80a5780 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:54.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.995+0000 7fd7f70e5640 1 --2- 172.21.15.38:0/3980270392 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7f0106f10 0x7fd7f0074f70 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:54.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.995+0000 7fd7f70e5640 1 --2- 172.21.15.38:0/3980270392 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7f0102790 0x7fd7f0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:54.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.995+0000 7fd7f70e5640 1 --2- 172.21.15.38:0/3980270392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7f0101de0 0x7fd7f00744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:54.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.995+0000 7fd7f70e5640 1 -- 172.21.15.38:0/3980270392 >> 172.21.15.38:0/3980270392 conn(0x7fd7f00fd8c0 msgr2=0x7fd7f00feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:54.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.995+0000 7fd7f70e5640 1 -- 172.21.15.38:0/3980270392 shutdown_connections 2024-09-16T00:29:54.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:54.995+0000 7fd7f70e5640 1 -- 172.21.15.38:0/3980270392 wait complete. 2024-09-16T00:29:55.309 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:55 smithi038 ceph-mon[30945]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:29:55.310 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:55 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:55.310 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:55 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:55.310 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:55 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:55.310 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:55 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:29:55.310 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:55 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:29:55.310 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:55 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:55.310 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:55 smithi038 ceph-mon[30945]: Deploying daemon mds.e on smithi073 2024-09-16T00:29:55.310 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:55 smithi038 ceph-mon[30945]: mds.? [v2:172.21.15.38:6836/335077136,v1:172.21.15.38:6837/335077136] up:boot 2024-09-16T00:29:55.310 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:55 smithi038 ceph-mon[30945]: fsmap 4 up:standby 2024-09-16T00:29:55.310 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:55 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-16T00:29:55.310 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:55 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/3980270392' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:29:55.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:55 smithi122 ceph-mon[35412]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:29:55.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:55 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:55.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:55 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:55.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:55 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:55.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:55 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:29:55.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:55 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:29:55.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:55 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:55.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:55 smithi122 ceph-mon[35412]: Deploying daemon mds.e on smithi073 2024-09-16T00:29:55.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:55 smithi122 ceph-mon[35412]: mds.? [v2:172.21.15.38:6836/335077136,v1:172.21.15.38:6837/335077136] up:boot 2024-09-16T00:29:55.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:55 smithi122 ceph-mon[35412]: fsmap 4 up:standby 2024-09-16T00:29:55.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:55 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-16T00:29:55.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:55 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/3980270392' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:29:55.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:55 smithi073 ceph-mon[35254]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:29:55.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:55 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:55.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:55 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:55.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:55 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:55.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:55 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:29:55.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:55 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:29:55.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:55 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:55.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:55 smithi073 ceph-mon[35254]: Deploying daemon mds.e on smithi073 2024-09-16T00:29:55.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:55 smithi073 ceph-mon[35254]: mds.? [v2:172.21.15.38:6836/335077136,v1:172.21.15.38:6837/335077136] up:boot 2024-09-16T00:29:55.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:55 smithi073 ceph-mon[35254]: fsmap 4 up:standby 2024-09-16T00:29:55.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:55 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-09-16T00:29:55.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:55 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/3980270392' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:29:56.294 INFO:journalctl@ceph.mds.e.smithi073.stdout:Sep 16 00:29:56 smithi073 systemd[1]: Starting Ceph mds.e for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:29:56.696 INFO:journalctl@ceph.mds.e.smithi073.stdout:Sep 16 00:29:56 smithi073 podman[64774]: 2024-09-16 00:29:56.296717884 +0000 UTC m=+0.016031844 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:29:56.696 INFO:journalctl@ceph.mds.e.smithi073.stdout:Sep 16 00:29:56 smithi073 podman[64774]: 2024-09-16 00:29:56.426753036 +0000 UTC m=+0.146066990 container create 2355052d8723af358dcef72db200312946eb484e326c3a1c8d43d3d223f5d6c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-e, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-16T00:29:56.696 INFO:journalctl@ceph.mds.e.smithi073.stdout:Sep 16 00:29:56 smithi073 podman[64774]: 2024-09-16 00:29:56.696369456 +0000 UTC m=+0.415683422 container init 2355052d8723af358dcef72db200312946eb484e326c3a1c8d43d3d223f5d6c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-e, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-16T00:29:57.044 INFO:journalctl@ceph.mds.e.smithi073.stdout:Sep 16 00:29:56 smithi073 podman[64774]: 2024-09-16 00:29:56.699305182 +0000 UTC m=+0.418619143 container start 2355052d8723af358dcef72db200312946eb484e326c3a1c8d43d3d223f5d6c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-e, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-16T00:29:57.044 INFO:journalctl@ceph.mds.e.smithi073.stdout:Sep 16 00:29:56 smithi073 ceph-mds[64793]: -- 172.21.15.73:0/3612434690 <== mon.2 v2:172.21.15.73:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55914fb13860 con 0x55914faf1000 2024-09-16T00:29:57.044 INFO:journalctl@ceph.mds.e.smithi073.stdout:Sep 16 00:29:56 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-e[64789]: starting mds.e at 2024-09-16T00:29:57.044 INFO:journalctl@ceph.mds.e.smithi073.stdout:Sep 16 00:29:56 smithi073 bash[64774]: 2355052d8723af358dcef72db200312946eb484e326c3a1c8d43d3d223f5d6c0 2024-09-16T00:29:57.044 INFO:journalctl@ceph.mds.e.smithi073.stdout:Sep 16 00:29:56 smithi073 systemd[1]: Started Ceph mds.e for 983b2072-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:29:57.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:57 smithi038 ceph-mon[30945]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:29:57.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:57 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:57.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:57 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:57.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:57 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:57.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:57 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:29:57.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:57 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:29:57.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:57 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:57.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:57 smithi122 ceph-mon[35412]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:29:57.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:57 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:57.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:57 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:57.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:57 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:57.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:57 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:29:57.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:57 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:29:57.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:57 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:57.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:57 smithi073 ceph-mon[35254]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:29:57.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:57 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:57.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:57 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:57.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:57 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:29:57.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:57 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:29:57.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:57 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:29:57.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:57 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:29:57.676 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:29:57.946 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:29:58.281 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:58 smithi122 ceph-mon[35412]: Deploying daemon mds.f on smithi122 2024-09-16T00:29:58.281 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:58 smithi122 ceph-mon[35412]: mds.? [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] up:boot 2024-09-16T00:29:58.282 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:58 smithi122 ceph-mon[35412]: fsmap 5 up:standby 2024-09-16T00:29:58.282 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:58 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T00:29:58.330 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:58 smithi038 ceph-mon[30945]: Deploying daemon mds.f on smithi122 2024-09-16T00:29:58.331 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:58 smithi038 ceph-mon[30945]: mds.? [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] up:boot 2024-09-16T00:29:58.331 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:58 smithi038 ceph-mon[30945]: fsmap 5 up:standby 2024-09-16T00:29:58.331 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:58 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T00:29:58.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.412+0000 7efe7ecfd640 1 -- 172.21.15.38:0/960138906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe78106050 msgr2=0x7efe7810dbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:58.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.412+0000 7efe7ecfd640 1 --2- 172.21.15.38:0/960138906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe78106050 0x7efe7810dbd0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7efe70009ca0 tx=0x7efe7002c790 comp rx=0 tx=0).stop 2024-09-16T00:29:58.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.412+0000 7efe7ecfd640 1 -- 172.21.15.38:0/960138906 shutdown_connections 2024-09-16T00:29:58.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.412+0000 7efe7ecfd640 1 --2- 172.21.15.38:0/960138906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe78106050 0x7efe7810dbd0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:58.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.412+0000 7efe7ecfd640 1 --2- 172.21.15.38:0/960138906 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe78101df0 0x7efe78105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:58.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.412+0000 7efe7ecfd640 1 --2- 172.21.15.38:0/960138906 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe78101440 0x7efe78101820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:58.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.412+0000 7efe7ecfd640 1 -- 172.21.15.38:0/960138906 >> 172.21.15.38:0/960138906 conn(0x7efe78077210 msgr2=0x7efe780fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:58.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.413+0000 7efe7ecfd640 1 -- 172.21.15.38:0/960138906 shutdown_connections 2024-09-16T00:29:58.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.413+0000 7efe7ecfd640 1 -- 172.21.15.38:0/960138906 wait complete. 2024-09-16T00:29:58.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.413+0000 7efe7ecfd640 1 Processor -- start 2024-09-16T00:29:58.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.413+0000 7efe7ecfd640 1 -- start start 2024-09-16T00:29:58.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.413+0000 7efe7ecfd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe78101440 0x7efe781a0580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:58.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.413+0000 7efe7ecfd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe78101df0 0x7efe781a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:58.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.413+0000 7efe7ecfd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe78106050 0x7efe7819a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:58.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.413+0000 7efe7ecfd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efe7810ffb0 con 0x7efe78106050 2024-09-16T00:29:58.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.413+0000 7efe7ecfd640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efe7810fe30 con 0x7efe78101440 2024-09-16T00:29:58.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.413+0000 7efe7ecfd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efe78110130 con 0x7efe78101df0 2024-09-16T00:29:58.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.413+0000 7efe7d29a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe78106050 0x7efe7819a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:58.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.414+0000 7efe7d29a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe78106050 0x7efe7819a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32872/0 (socket says 172.21.15.38:32872) 2024-09-16T00:29:58.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.414+0000 7efe7d29a640 1 -- 172.21.15.38:0/1330728670 learned_addr learned my addr 172.21.15.38:0/1330728670 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:29:58.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.414+0000 7efe7ca99640 1 --2- 172.21.15.38:0/1330728670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe78101440 0x7efe781a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:58.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.414+0000 7efe7d29a640 1 -- 172.21.15.38:0/1330728670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe78101df0 msgr2=0x7efe781a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:58.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.414+0000 7efe7d29a640 1 --2- 172.21.15.38:0/1330728670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe78101df0 0x7efe781a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:58.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.414+0000 7efe7d29a640 1 -- 172.21.15.38:0/1330728670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe78101440 msgr2=0x7efe781a0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:58.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.414+0000 7efe7d29a640 1 --2- 172.21.15.38:0/1330728670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe78101440 0x7efe781a0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:58.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.414+0000 7efe7d29a640 1 -- 172.21.15.38:0/1330728670 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe7819aec0 con 0x7efe78106050 2024-09-16T00:29:58.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.414+0000 7efe7d29a640 1 --2- 172.21.15.38:0/1330728670 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe78106050 0x7efe7819a650 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efe7002d010 tx=0x7efe70031740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:58.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.414+0000 7efe75ffb640 1 -- 172.21.15.38:0/1330728670 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe70041070 con 0x7efe78106050 2024-09-16T00:29:58.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.414+0000 7efe75ffb640 1 -- 172.21.15.38:0/1330728670 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe7003d920 con 0x7efe78106050 2024-09-16T00:29:58.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.414+0000 7efe7ecfd640 1 -- 172.21.15.38:0/1330728670 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe7819b150 con 0x7efe78106050 2024-09-16T00:29:58.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.414+0000 7efe75ffb640 1 -- 172.21.15.38:0/1330728670 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe7000c690 con 0x7efe78106050 2024-09-16T00:29:58.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.414+0000 7efe7ecfd640 1 -- 172.21.15.38:0/1330728670 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe781a7330 con 0x7efe78106050 2024-09-16T00:29:58.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.415+0000 7efe7ecfd640 1 -- 172.21.15.38:0/1330728670 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe78102490 con 0x7efe78106050 2024-09-16T00:29:58.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.417+0000 7efe75ffb640 1 -- 172.21.15.38:0/1330728670 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7efe70044070 con 0x7efe78106050 2024-09-16T00:29:58.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.418+0000 7efe75ffb640 1 --2- 172.21.15.38:0/1330728670 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efe600a3240 0x7efe600a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:29:58.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.418+0000 7efe7ca99640 1 --2- 172.21.15.38:0/1330728670 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efe600a3240 0x7efe600a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:29:58.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.418+0000 7efe75ffb640 1 -- 172.21.15.38:0/1330728670 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7efe700f0f30 con 0x7efe78106050 2024-09-16T00:29:58.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.419+0000 7efe7ca99640 1 --2- 172.21.15.38:0/1330728670 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efe600a3240 0x7efe600a5700 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efe68009bd0 tx=0x7efe68005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:29:58.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.421+0000 7efe75ffb640 1 -- 172.21.15.38:0/1330728670 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efe700be050 con 0x7efe78106050 2024-09-16T00:29:58.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:58 smithi073 ceph-mon[35254]: Deploying daemon mds.f on smithi122 2024-09-16T00:29:58.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:58 smithi073 ceph-mon[35254]: mds.? [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] up:boot 2024-09-16T00:29:58.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:58 smithi073 ceph-mon[35254]: fsmap 5 up:standby 2024-09-16T00:29:58.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:58 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T00:29:58.632 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.632+0000 7efe7ecfd640 1 -- 172.21.15.38:0/1330728670 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efe78101820 con 0x7efe78106050 2024-09-16T00:29:58.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.632+0000 7efe75ffb640 1 -- 172.21.15.38:0/1330728670 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4729 (secure 0 0 0) 0x7efe78101820 con 0x7efe78106050 2024-09-16T00:29:58.633 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:29:58.633 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":6,"btime":"2024-09-16T00:29:57:131405+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/367974527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":367974527},{"type":"v1","addr":"172.21.15.38:6835","nonce":367974527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/335077136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":335077136},{"type":"v1","addr":"172.21.15.38:6837","nonce":335077136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/412277519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":412277519},{"type":"v1","addr":"172.21.15.122:6833","nonce":412277519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/1210328177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1210328177},{"type":"v1","addr":"172.21.15.73:6833","nonce":1210328177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2866363113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2866363113},{"type":"v1","addr":"172.21.15.73:6835","nonce":2866363113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-16T00:29:58.637 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 6 2024-09-16T00:29:58.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.634+0000 7efe7ecfd640 1 -- 172.21.15.38:0/1330728670 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efe600a3240 msgr2=0x7efe600a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:58.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.634+0000 7efe7ecfd640 1 --2- 172.21.15.38:0/1330728670 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efe600a3240 0x7efe600a5700 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7efe68009bd0 tx=0x7efe68005f90 comp rx=0 tx=0).stop 2024-09-16T00:29:58.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.634+0000 7efe7ecfd640 1 -- 172.21.15.38:0/1330728670 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe78106050 msgr2=0x7efe7819a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:29:58.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.634+0000 7efe7ecfd640 1 --2- 172.21.15.38:0/1330728670 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe78106050 0x7efe7819a650 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7efe7002d010 tx=0x7efe70031740 comp rx=0 tx=0).stop 2024-09-16T00:29:58.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.635+0000 7efe7ecfd640 1 -- 172.21.15.38:0/1330728670 shutdown_connections 2024-09-16T00:29:58.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.635+0000 7efe7ecfd640 1 --2- 172.21.15.38:0/1330728670 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efe600a3240 0x7efe600a5700 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:58.637 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.635+0000 7efe7ecfd640 1 --2- 172.21.15.38:0/1330728670 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe78106050 0x7efe7819a650 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:58.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.635+0000 7efe7ecfd640 1 --2- 172.21.15.38:0/1330728670 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe78101df0 0x7efe781a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:58.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.635+0000 7efe7ecfd640 1 --2- 172.21.15.38:0/1330728670 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe78101440 0x7efe781a0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:29:58.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.635+0000 7efe7ecfd640 1 -- 172.21.15.38:0/1330728670 >> 172.21.15.38:0/1330728670 conn(0x7efe78077210 msgr2=0x7efe78104840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:29:58.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.635+0000 7efe7ecfd640 1 -- 172.21.15.38:0/1330728670 shutdown_connections 2024-09-16T00:29:58.638 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:29:58.635+0000 7efe7ecfd640 1 -- 172.21.15.38:0/1330728670 wait complete. 2024-09-16T00:29:58.967 INFO:journalctl@ceph.mds.f.smithi122.stdout:Sep 16 00:29:58 smithi122 systemd[1]: Starting Ceph mds.f for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:29:59.248 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:59 smithi122 ceph-mon[35412]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:29:59.248 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:29:59 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/1330728670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:29:59.249 INFO:journalctl@ceph.mds.f.smithi122.stdout:Sep 16 00:29:58 smithi122 podman[64817]: 2024-09-16 00:29:58.874998979 +0000 UTC m=+0.015991629 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:29:59.249 INFO:journalctl@ceph.mds.f.smithi122.stdout:Sep 16 00:29:59 smithi122 podman[64817]: 2024-09-16 00:29:59.001442908 +0000 UTC m=+0.142435544 container create f3f7a5fa1fd3ffa4272ffedb935f0b5ffc4371cdd1bde927eb6795cbfdce2787 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-f, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-16T00:29:59.249 INFO:journalctl@ceph.mds.f.smithi122.stdout:Sep 16 00:29:59 smithi122 podman[64817]: 2024-09-16 00:29:59.246030385 +0000 UTC m=+0.387023021 container init f3f7a5fa1fd3ffa4272ffedb935f0b5ffc4371cdd1bde927eb6795cbfdce2787 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-f, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:29:59.364 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:59 smithi038 ceph-mon[30945]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:29:59.364 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:29:59 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/1330728670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:29:59.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:59 smithi073 ceph-mon[35254]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:29:59.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:29:59 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/1330728670' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:29:59.716 INFO:journalctl@ceph.mds.f.smithi122.stdout:Sep 16 00:29:59 smithi122 podman[64817]: 2024-09-16 00:29:59.249238949 +0000 UTC m=+0.390231585 container start f3f7a5fa1fd3ffa4272ffedb935f0b5ffc4371cdd1bde927eb6795cbfdce2787 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-f, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-16T00:29:59.717 INFO:journalctl@ceph.mds.f.smithi122.stdout:Sep 16 00:29:59 smithi122 ceph-mds[64836]: -- 172.21.15.122:0/1954788104 <== mon.1 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562843679860 con 0x562843657000 2024-09-16T00:29:59.717 INFO:journalctl@ceph.mds.f.smithi122.stdout:Sep 16 00:29:59 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-f[64832]: starting mds.f at 2024-09-16T00:29:59.717 INFO:journalctl@ceph.mds.f.smithi122.stdout:Sep 16 00:29:59 smithi122 bash[64817]: f3f7a5fa1fd3ffa4272ffedb935f0b5ffc4371cdd1bde927eb6795cbfdce2787 2024-09-16T00:29:59.717 INFO:journalctl@ceph.mds.f.smithi122.stdout:Sep 16 00:29:59 smithi122 systemd[1]: Started Ceph mds.f for 983b2072-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:30:00.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:00 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:00.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:00 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:00.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:00 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:00.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:00 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:30:00.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:00 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:30:00.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:00 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:00.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:00 smithi038 ceph-mon[30945]: Deploying daemon mds.c on smithi122 2024-09-16T00:30:00.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:00 smithi038 ceph-mon[30945]: overall HEALTH_OK 2024-09-16T00:30:00.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:00 smithi038 ceph-mon[30945]: mds.? [v2:172.21.15.122:6834/3987234757,v1:172.21.15.122:6835/3987234757] up:boot 2024-09-16T00:30:00.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:00 smithi038 ceph-mon[30945]: fsmap 6 up:standby 2024-09-16T00:30:00.716 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:00 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T00:30:00.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:00 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:00.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:00 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:00.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:00 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:00.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:00 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:30:00.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:00 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:30:00.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:00 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:00.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:00 smithi122 ceph-mon[35412]: Deploying daemon mds.c on smithi122 2024-09-16T00:30:00.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:00 smithi122 ceph-mon[35412]: overall HEALTH_OK 2024-09-16T00:30:00.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:00 smithi122 ceph-mon[35412]: mds.? [v2:172.21.15.122:6834/3987234757,v1:172.21.15.122:6835/3987234757] up:boot 2024-09-16T00:30:00.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:00 smithi122 ceph-mon[35412]: fsmap 6 up:standby 2024-09-16T00:30:00.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:00 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T00:30:00.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:00 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:00.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:00 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:00.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:00 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:00.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:00 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:30:00.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:00 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:30:00.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:00 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:00.795 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:00 smithi073 ceph-mon[35254]: Deploying daemon mds.c on smithi122 2024-09-16T00:30:00.795 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:00 smithi073 ceph-mon[35254]: overall HEALTH_OK 2024-09-16T00:30:00.795 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:00 smithi073 ceph-mon[35254]: mds.? [v2:172.21.15.122:6834/3987234757,v1:172.21.15.122:6835/3987234757] up:boot 2024-09-16T00:30:00.795 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:00 smithi073 ceph-mon[35254]: fsmap 6 up:standby 2024-09-16T00:30:00.795 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:00 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-09-16T00:30:01.367 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:30:01.425 INFO:journalctl@ceph.mds.c.smithi122.stdout:Sep 16 00:30:01 smithi122 systemd[1]: Starting Ceph mds.c for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:30:01.638 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:01.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:01 smithi038 ceph-mon[30945]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:01.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:01 smithi122 ceph-mon[35412]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:01.716 INFO:journalctl@ceph.mds.c.smithi122.stdout:Sep 16 00:30:01 smithi122 podman[65118]: 2024-09-16 00:30:01.469988144 +0000 UTC m=+0.015850832 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:30:01.716 INFO:journalctl@ceph.mds.c.smithi122.stdout:Sep 16 00:30:01 smithi122 podman[65118]: 2024-09-16 00:30:01.607215458 +0000 UTC m=+0.153078139 container create 6b38590c30494560b06fd03d28dd3aa06a1ddaca02ccb02fea17cf3b9ccabffb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-c, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:30:01.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:01 smithi073 ceph-mon[35254]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:02.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.131+0000 7f862af99640 1 -- 172.21.15.38:0/17910293 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86241027a0 msgr2=0x7f86241067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:02.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.131+0000 7f862af99640 1 --2- 172.21.15.38:0/17910293 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86241027a0 0x7f86241067f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f8610009900 tx=0x7f861002a410 comp rx=0 tx=0).stop 2024-09-16T00:30:02.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.132+0000 7f862af99640 1 -- 172.21.15.38:0/17910293 shutdown_connections 2024-09-16T00:30:02.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.132+0000 7f862af99640 1 --2- 172.21.15.38:0/17910293 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8624106f20 0x7f862410eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:02.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.132+0000 7f862af99640 1 --2- 172.21.15.38:0/17910293 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86241027a0 0x7f86241067f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:02.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.132+0000 7f862af99640 1 --2- 172.21.15.38:0/17910293 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8624101df0 0x7f86241021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:02.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.132+0000 7f862af99640 1 -- 172.21.15.38:0/17910293 >> 172.21.15.38:0/17910293 conn(0x7f86240fd8f0 msgr2=0x7f86240ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:02.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.132+0000 7f862af99640 1 -- 172.21.15.38:0/17910293 shutdown_connections 2024-09-16T00:30:02.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.132+0000 7f862af99640 1 -- 172.21.15.38:0/17910293 wait complete. 2024-09-16T00:30:02.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.132+0000 7f862af99640 1 Processor -- start 2024-09-16T00:30:02.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.132+0000 7f862af99640 1 -- start start 2024-09-16T00:30:02.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.133+0000 7f862af99640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8624101df0 0x7f86240744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:02.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.133+0000 7f862af99640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86241027a0 0x7f8624074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:02.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.133+0000 7f862af99640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8624106f20 0x7f8624074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:02.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.133+0000 7f862af99640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8624111f80 con 0x7f8624106f20 2024-09-16T00:30:02.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.133+0000 7f862af99640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8624111e00 con 0x7f86241027a0 2024-09-16T00:30:02.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.133+0000 7f862af99640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8624112100 con 0x7f8624101df0 2024-09-16T00:30:02.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.133+0000 7f8629536640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8624106f20 0x7f8624074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:02.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.133+0000 7f8629536640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8624106f20 0x7f8624074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32894/0 (socket says 172.21.15.38:32894) 2024-09-16T00:30:02.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.133+0000 7f8629536640 1 -- 172.21.15.38:0/4203199154 learned_addr learned my addr 172.21.15.38:0/4203199154 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:30:02.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.133+0000 7f8623fff640 1 --2- 172.21.15.38:0/4203199154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86241027a0 0x7f8624074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:02.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.133+0000 7f8628d35640 1 --2- 172.21.15.38:0/4203199154 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8624101df0 0x7f86240744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:02.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.133+0000 7f8629536640 1 -- 172.21.15.38:0/4203199154 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8624101df0 msgr2=0x7f86240744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:02.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.133+0000 7f8629536640 1 --2- 172.21.15.38:0/4203199154 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8624101df0 0x7f86240744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:02.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.133+0000 7f8629536640 1 -- 172.21.15.38:0/4203199154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86241027a0 msgr2=0x7f8624074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:02.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.133+0000 7f8629536640 1 --2- 172.21.15.38:0/4203199154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86241027a0 0x7f8624074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:02.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.133+0000 7f8629536640 1 -- 172.21.15.38:0/4203199154 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86241a69b0 con 0x7f8624106f20 2024-09-16T00:30:02.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.133+0000 7f8628d35640 1 --2- 172.21.15.38:0/4203199154 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8624101df0 0x7f86240744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:30:02.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.133+0000 7f8629536640 1 --2- 172.21.15.38:0/4203199154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8624106f20 0x7f8624074f70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f861c00eaa0 tx=0x7f861c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:02.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.133+0000 7f8621ffb640 1 -- 172.21.15.38:0/4203199154 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861c00cdb0 con 0x7f8624106f20 2024-09-16T00:30:02.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.134+0000 7f8621ffb640 1 -- 172.21.15.38:0/4203199154 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f861c005020 con 0x7f8624106f20 2024-09-16T00:30:02.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.134+0000 7f862af99640 1 -- 172.21.15.38:0/4203199154 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86241a6ca0 con 0x7f8624106f20 2024-09-16T00:30:02.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.134+0000 7f8621ffb640 1 -- 172.21.15.38:0/4203199154 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f861c0116b0 con 0x7f8624106f20 2024-09-16T00:30:02.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.134+0000 7f862af99640 1 -- 172.21.15.38:0/4203199154 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86241a7200 con 0x7f8624106f20 2024-09-16T00:30:02.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.135+0000 7f862af99640 1 -- 172.21.15.38:0/4203199154 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86241021d0 con 0x7f8624106f20 2024-09-16T00:30:02.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.136+0000 7f8621ffb640 1 -- 172.21.15.38:0/4203199154 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f861c005560 con 0x7f8624106f20 2024-09-16T00:30:02.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.137+0000 7f8621ffb640 1 --2- 172.21.15.38:0/4203199154 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f86140a32c0 0x7f86140a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:02.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.137+0000 7f8628d35640 1 --2- 172.21.15.38:0/4203199154 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f86140a32c0 0x7f86140a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:02.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.137+0000 7f8621ffb640 1 -- 172.21.15.38:0/4203199154 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f861c0cfa00 con 0x7f8624106f20 2024-09-16T00:30:02.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.138+0000 7f8628d35640 1 --2- 172.21.15.38:0/4203199154 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f86140a32c0 0x7f86140a5780 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8618009c20 tx=0x7f8618005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:02.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.140+0000 7f8621ffb640 1 -- 172.21.15.38:0/4203199154 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f861c016030 con 0x7f8624106f20 2024-09-16T00:30:02.217 INFO:journalctl@ceph.mds.c.smithi122.stdout:Sep 16 00:30:01 smithi122 podman[65118]: 2024-09-16 00:30:01.877426513 +0000 UTC m=+0.423289189 container init 6b38590c30494560b06fd03d28dd3aa06a1ddaca02ccb02fea17cf3b9ccabffb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-c, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-09-16T00:30:02.217 INFO:journalctl@ceph.mds.c.smithi122.stdout:Sep 16 00:30:01 smithi122 podman[65118]: 2024-09-16 00:30:01.880342091 +0000 UTC m=+0.426204769 container start 6b38590c30494560b06fd03d28dd3aa06a1ddaca02ccb02fea17cf3b9ccabffb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-c, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:30:02.217 INFO:journalctl@ceph.mds.c.smithi122.stdout:Sep 16 00:30:01 smithi122 ceph-mds[65137]: -- 172.21.15.122:0/2240752266 <== mon.1 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5611bcf1f860 con 0x5611bcefd000 2024-09-16T00:30:02.217 INFO:journalctl@ceph.mds.c.smithi122.stdout:Sep 16 00:30:01 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-c[65133]: starting mds.c at 2024-09-16T00:30:02.217 INFO:journalctl@ceph.mds.c.smithi122.stdout:Sep 16 00:30:01 smithi122 bash[65118]: 6b38590c30494560b06fd03d28dd3aa06a1ddaca02ccb02fea17cf3b9ccabffb 2024-09-16T00:30:02.217 INFO:journalctl@ceph.mds.c.smithi122.stdout:Sep 16 00:30:01 smithi122 systemd[1]: Started Ceph mds.c for 983b2072-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:30:02.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.363+0000 7f862af99640 1 -- 172.21.15.38:0/4203199154 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8624075a20 con 0x7f8624106f20 2024-09-16T00:30:02.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.364+0000 7f8621ffb640 1 -- 172.21.15.38:0/4203199154 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5557 (secure 0 0 0) 0x7f861c09cba0 con 0x7f8624106f20 2024-09-16T00:30:02.364 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:30:02.364 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":7,"btime":"2024-09-16T00:30:00:150227+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/367974527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":367974527},{"type":"v1","addr":"172.21.15.38:6835","nonce":367974527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/335077136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":335077136},{"type":"v1","addr":"172.21.15.38:6837","nonce":335077136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/412277519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":412277519},{"type":"v1","addr":"172.21.15.122:6833","nonce":412277519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3987234757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3987234757},{"type":"v1","addr":"172.21.15.122:6835","nonce":3987234757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/1210328177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1210328177},{"type":"v1","addr":"172.21.15.73:6833","nonce":1210328177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2866363113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2866363113},{"type":"v1","addr":"172.21.15.73:6835","nonce":2866363113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-09-16T00:30:02.365 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 7 2024-09-16T00:30:02.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.366+0000 7f862af99640 1 -- 172.21.15.38:0/4203199154 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f86140a32c0 msgr2=0x7f86140a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:02.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.366+0000 7f862af99640 1 --2- 172.21.15.38:0/4203199154 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f86140a32c0 0x7f86140a5780 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8618009c20 tx=0x7f8618005f90 comp rx=0 tx=0).stop 2024-09-16T00:30:02.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.366+0000 7f862af99640 1 -- 172.21.15.38:0/4203199154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8624106f20 msgr2=0x7f8624074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:02.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.366+0000 7f862af99640 1 --2- 172.21.15.38:0/4203199154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8624106f20 0x7f8624074f70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f861c00eaa0 tx=0x7f861c00ef60 comp rx=0 tx=0).stop 2024-09-16T00:30:02.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.366+0000 7f862af99640 1 -- 172.21.15.38:0/4203199154 shutdown_connections 2024-09-16T00:30:02.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.366+0000 7f862af99640 1 --2- 172.21.15.38:0/4203199154 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f86140a32c0 0x7f86140a5780 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:02.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.366+0000 7f862af99640 1 --2- 172.21.15.38:0/4203199154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8624106f20 0x7f8624074f70 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:02.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.366+0000 7f862af99640 1 --2- 172.21.15.38:0/4203199154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86241027a0 0x7f8624074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:02.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.366+0000 7f862af99640 1 --2- 172.21.15.38:0/4203199154 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8624101df0 0x7f86240744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:02.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.366+0000 7f862af99640 1 -- 172.21.15.38:0/4203199154 >> 172.21.15.38:0/4203199154 conn(0x7f86240fd8f0 msgr2=0x7f86240fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:02.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.366+0000 7f862af99640 1 -- 172.21.15.38:0/4203199154 shutdown_connections 2024-09-16T00:30:02.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:02.366+0000 7f862af99640 1 -- 172.21.15.38:0/4203199154 wait complete. 2024-09-16T00:30:03.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:03 smithi073 ceph-mon[35254]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:03.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:03 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:03.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:03 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:03.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:03 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:03.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:03 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:30:03.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:03 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:30:03.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:03 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:03.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:03 smithi073 ceph-mon[35254]: Deploying daemon mds.h on smithi073 2024-09-16T00:30:03.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:03 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/4203199154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:30:03.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:03 smithi073 ceph-mon[35254]: mds.? [v2:172.21.15.122:6836/1858299680,v1:172.21.15.122:6837/1858299680] up:boot 2024-09-16T00:30:03.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:03 smithi073 ceph-mon[35254]: fsmap 7 up:standby 2024-09-16T00:30:03.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:03 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T00:30:03.329 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:03 smithi038 ceph-mon[30945]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:03.329 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:03 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:03.329 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:03 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:03.329 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:03 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:03.329 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:03 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:30:03.329 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:03 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:30:03.329 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:03 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:03.330 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:03 smithi038 ceph-mon[30945]: Deploying daemon mds.h on smithi073 2024-09-16T00:30:03.330 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:03 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/4203199154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:30:03.330 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:03 smithi038 ceph-mon[30945]: mds.? [v2:172.21.15.122:6836/1858299680,v1:172.21.15.122:6837/1858299680] up:boot 2024-09-16T00:30:03.330 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:03 smithi038 ceph-mon[30945]: fsmap 7 up:standby 2024-09-16T00:30:03.330 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:03 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T00:30:03.389 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:03 smithi122 ceph-mon[35412]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:03.389 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:03 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:03.389 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:03 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:03.390 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:03 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:03.390 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:03 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:30:03.390 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:03 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:30:03.390 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:03 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:03.390 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:03 smithi122 ceph-mon[35412]: Deploying daemon mds.h on smithi073 2024-09-16T00:30:03.390 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:03 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/4203199154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:30:03.390 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:03 smithi122 ceph-mon[35412]: mds.? [v2:172.21.15.122:6836/1858299680,v1:172.21.15.122:6837/1858299680] up:boot 2024-09-16T00:30:03.390 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:03 smithi122 ceph-mon[35412]: fsmap 7 up:standby 2024-09-16T00:30:03.390 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:03 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T00:30:04.294 INFO:journalctl@ceph.mds.h.smithi073.stdout:Sep 16 00:30:04 smithi073 systemd[1]: Starting Ceph mds.h for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:30:04.627 INFO:journalctl@ceph.mds.h.smithi073.stdout:Sep 16 00:30:04 smithi073 podman[65074]: 2024-09-16 00:30:04.214157759 +0000 UTC m=+0.015713027 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:30:04.627 INFO:journalctl@ceph.mds.h.smithi073.stdout:Sep 16 00:30:04 smithi073 podman[65074]: 2024-09-16 00:30:04.340925726 +0000 UTC m=+0.142480978 container create 90fff75c7a54a1d4b3b0583f874b3928fc90642bf4139efa0ba28c9a08024160 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-h, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, ceph=True, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-16T00:30:04.627 INFO:journalctl@ceph.mds.h.smithi073.stdout:Sep 16 00:30:04 smithi073 podman[65074]: 2024-09-16 00:30:04.627308966 +0000 UTC m=+0.428864233 container init 90fff75c7a54a1d4b3b0583f874b3928fc90642bf4139efa0ba28c9a08024160 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-h, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-16T00:30:05.026 INFO:journalctl@ceph.mds.h.smithi073.stdout:Sep 16 00:30:04 smithi073 podman[65074]: 2024-09-16 00:30:04.630364365 +0000 UTC m=+0.431919636 container start 90fff75c7a54a1d4b3b0583f874b3928fc90642bf4139efa0ba28c9a08024160 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-h, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-16T00:30:05.026 INFO:journalctl@ceph.mds.h.smithi073.stdout:Sep 16 00:30:04 smithi073 ceph-mds[65094]: -- 172.21.15.73:0/2374934186 <== mon.2 v2:172.21.15.73:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5569c4691860 con 0x5569c466f000 2024-09-16T00:30:05.026 INFO:journalctl@ceph.mds.h.smithi073.stdout:Sep 16 00:30:04 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-h[65090]: starting mds.h at 2024-09-16T00:30:05.026 INFO:journalctl@ceph.mds.h.smithi073.stdout:Sep 16 00:30:04 smithi073 bash[65074]: 90fff75c7a54a1d4b3b0583f874b3928fc90642bf4139efa0ba28c9a08024160 2024-09-16T00:30:05.026 INFO:journalctl@ceph.mds.h.smithi073.stdout:Sep 16 00:30:04 smithi073 systemd[1]: Started Ceph mds.h for 983b2072-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:30:05.082 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:30:05.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:05 smithi073 ceph-mon[35254]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:05.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:05 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:05.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:05 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:05.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:05 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:05.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:05 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:30:05.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:05 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:30:05.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:05 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:05.353 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:05 smithi122 ceph-mon[35412]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:05.354 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:05 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:05.354 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:05 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:05.354 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:05 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:05.354 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:05 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:30:05.354 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:05 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:30:05.354 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:05 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:05.354 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:05.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:05 smithi038 ceph-mon[30945]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:05.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:05 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:05.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:05 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:05.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:05 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:05.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:05 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:30:05.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:05 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:30:05.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:05 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:05.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:05.890+0000 7f456f8ce640 1 -- 172.21.15.38:0/3033182332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4568101de0 msgr2=0x7f45681021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:05.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:05.890+0000 7f456f8ce640 1 --2- 172.21.15.38:0/3033182332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4568101de0 0x7f45681021c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f4564009900 tx=0x7f456402a410 comp rx=0 tx=0).stop 2024-09-16T00:30:05.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:05.890+0000 7f456f8ce640 1 -- 172.21.15.38:0/3033182332 shutdown_connections 2024-09-16T00:30:05.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:05.890+0000 7f456f8ce640 1 --2- 172.21.15.38:0/3033182332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4568106f10 0x7f456810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:05.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:05.890+0000 7f456f8ce640 1 --2- 172.21.15.38:0/3033182332 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4568102790 0x7f45681067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:05.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:05.890+0000 7f456f8ce640 1 --2- 172.21.15.38:0/3033182332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4568101de0 0x7f45681021c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:05.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:05.890+0000 7f456f8ce640 1 -- 172.21.15.38:0/3033182332 >> 172.21.15.38:0/3033182332 conn(0x7f45680fd8c0 msgr2=0x7f45680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:05.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:05.890+0000 7f456f8ce640 1 -- 172.21.15.38:0/3033182332 shutdown_connections 2024-09-16T00:30:05.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:05.890+0000 7f456f8ce640 1 -- 172.21.15.38:0/3033182332 wait complete. 2024-09-16T00:30:05.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:05.891+0000 7f456f8ce640 1 Processor -- start 2024-09-16T00:30:05.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:05.891+0000 7f456f8ce640 1 -- start start 2024-09-16T00:30:05.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:05.891+0000 7f456f8ce640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4568101de0 0x7f45681983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:05.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:05.891+0000 7f456f8ce640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4568102790 0x7f4568198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:05.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:05.891+0000 7f456f8ce640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4568106f10 0x7f456819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:05.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:05.891+0000 7f456f8ce640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4568111f70 con 0x7f4568102790 2024-09-16T00:30:05.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:05.891+0000 7f456f8ce640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4568111df0 con 0x7f4568106f10 2024-09-16T00:30:05.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:05.891+0000 7f456f8ce640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f45681120f0 con 0x7f4568101de0 2024-09-16T00:30:05.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:05.891+0000 7f456ce69640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4568102790 0x7f4568198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:05.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:05.891+0000 7f456ce69640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4568102790 0x7f4568198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32920/0 (socket says 172.21.15.38:32920) 2024-09-16T00:30:05.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:05.891+0000 7f456ce69640 1 -- 172.21.15.38:0/4108270249 learned_addr learned my addr 172.21.15.38:0/4108270249 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:30:05.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:05.891+0000 7f456de6b640 1 --2- 172.21.15.38:0/4108270249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4568106f10 0x7f456819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:05.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:05.892+0000 7f456d66a640 1 --2- 172.21.15.38:0/4108270249 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4568101de0 0x7f45681983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:05.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:05.892+0000 7f456ce69640 1 -- 172.21.15.38:0/4108270249 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4568101de0 msgr2=0x7f45681983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:05.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:05.892+0000 7f456ce69640 1 --2- 172.21.15.38:0/4108270249 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4568101de0 0x7f45681983d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:05.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:05.892+0000 7f456ce69640 1 -- 172.21.15.38:0/4108270249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4568106f10 msgr2=0x7f456819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:05.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:05.892+0000 7f456ce69640 1 --2- 172.21.15.38:0/4108270249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4568106f10 0x7f456819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:05.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:05.892+0000 7f456ce69640 1 -- 172.21.15.38:0/4108270249 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f456819d380 con 0x7f4568102790 2024-09-16T00:30:05.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:05.892+0000 7f456ce69640 1 --2- 172.21.15.38:0/4108270249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4568102790 0x7f4568198910 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f455800c970 tx=0x7f455800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:05.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:05.892+0000 7f455e7fc640 1 -- 172.21.15.38:0/4108270249 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4558007bb0 con 0x7f4568102790 2024-09-16T00:30:05.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:05.892+0000 7f455e7fc640 1 -- 172.21.15.38:0/4108270249 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4558007d50 con 0x7f4568102790 2024-09-16T00:30:05.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:05.892+0000 7f456f8ce640 1 -- 172.21.15.38:0/4108270249 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f456819d670 con 0x7f4568102790 2024-09-16T00:30:05.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:05.892+0000 7f455e7fc640 1 -- 172.21.15.38:0/4108270249 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f455800f450 con 0x7f4568102790 2024-09-16T00:30:05.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:05.892+0000 7f456f8ce640 1 -- 172.21.15.38:0/4108270249 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45681a4f50 con 0x7f4568102790 2024-09-16T00:30:05.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:05.893+0000 7f456f8ce640 1 -- 172.21.15.38:0/4108270249 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45681021c0 con 0x7f4568102790 2024-09-16T00:30:05.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:05.894+0000 7f455e7fc640 1 -- 172.21.15.38:0/4108270249 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f4558005020 con 0x7f4568102790 2024-09-16T00:30:05.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:05.895+0000 7f455e7fc640 1 --2- 172.21.15.38:0/4108270249 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f45540a32c0 0x7f45540a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:05.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:05.895+0000 7f456d66a640 1 --2- 172.21.15.38:0/4108270249 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f45540a32c0 0x7f45540a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:05.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:05.895+0000 7f455e7fc640 1 -- 172.21.15.38:0/4108270249 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f45580cf320 con 0x7f4568102790 2024-09-16T00:30:05.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:05.895+0000 7f456d66a640 1 --2- 172.21.15.38:0/4108270249 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f45540a32c0 0x7f45540a5780 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f45640096c0 tx=0x7f4564000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:05.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:05.899+0000 7f455e7fc640 1 -- 172.21.15.38:0/4108270249 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4558014030 con 0x7f4568102790 2024-09-16T00:30:06.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:06.110+0000 7f456f8ce640 1 -- 172.21.15.38:0/4108270249 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4568199730 con 0x7f4568102790 2024-09-16T00:30:06.110 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:06 smithi038 ceph-mon[30945]: Deploying daemon mds.i on smithi122 2024-09-16T00:30:06.110 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:06 smithi038 ceph-mon[30945]: mds.? [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] up:boot 2024-09-16T00:30:06.111 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:06 smithi038 ceph-mon[30945]: fsmap 8 up:standby 2024-09-16T00:30:06.111 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:06 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T00:30:06.111 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:06.110+0000 7f455e7fc640 1 -- 172.21.15.38:0/4108270249 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7210 (secure 0 0 0) 0x7f4568199730 con 0x7f4568102790 2024-09-16T00:30:06.111 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:30:06.111 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":9,"btime":"2024-09-16T00:30:05:021981+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/367974527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":367974527},{"type":"v1","addr":"172.21.15.38:6835","nonce":367974527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/335077136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":335077136},{"type":"v1","addr":"172.21.15.38:6837","nonce":335077136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/412277519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":412277519},{"type":"v1","addr":"172.21.15.122:6833","nonce":412277519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3987234757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3987234757},{"type":"v1","addr":"172.21.15.122:6835","nonce":3987234757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/1210328177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1210328177},{"type":"v1","addr":"172.21.15.73:6833","nonce":1210328177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1858299680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1858299680},{"type":"v1","addr":"172.21.15.122:6837","nonce":1858299680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2866363113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2866363113},{"type":"v1","addr":"172.21.15.73:6835","nonce":2866363113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2681081265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2681081265},{"type":"v1","addr":"172.21.15.73:6837","nonce":2681081265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-16T00:30:06.112 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 9 2024-09-16T00:30:06.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:06.113+0000 7f456f8ce640 1 -- 172.21.15.38:0/4108270249 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f45540a32c0 msgr2=0x7f45540a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:06.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:06.113+0000 7f456f8ce640 1 --2- 172.21.15.38:0/4108270249 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f45540a32c0 0x7f45540a5780 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f45640096c0 tx=0x7f4564000bc0 comp rx=0 tx=0).stop 2024-09-16T00:30:06.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:06.113+0000 7f456f8ce640 1 -- 172.21.15.38:0/4108270249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4568102790 msgr2=0x7f4568198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:06.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:06.113+0000 7f456f8ce640 1 --2- 172.21.15.38:0/4108270249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4568102790 0x7f4568198910 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f455800c970 tx=0x7f455800ce30 comp rx=0 tx=0).stop 2024-09-16T00:30:06.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:06.114+0000 7f456f8ce640 1 -- 172.21.15.38:0/4108270249 shutdown_connections 2024-09-16T00:30:06.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:06.114+0000 7f456f8ce640 1 --2- 172.21.15.38:0/4108270249 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f45540a32c0 0x7f45540a5780 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:06.114 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:06.114+0000 7f456f8ce640 1 --2- 172.21.15.38:0/4108270249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4568106f10 0x7f456819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:06.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:06.114+0000 7f456f8ce640 1 --2- 172.21.15.38:0/4108270249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4568102790 0x7f4568198910 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:06.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:06.114+0000 7f456f8ce640 1 --2- 172.21.15.38:0/4108270249 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4568101de0 0x7f45681983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:06.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:06.114+0000 7f456f8ce640 1 -- 172.21.15.38:0/4108270249 >> 172.21.15.38:0/4108270249 conn(0x7f45680fd8c0 msgr2=0x7f45680febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:06.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:06.114+0000 7f456f8ce640 1 -- 172.21.15.38:0/4108270249 shutdown_connections 2024-09-16T00:30:06.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:06.114+0000 7f456f8ce640 1 -- 172.21.15.38:0/4108270249 wait complete. 2024-09-16T00:30:06.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:06 smithi073 ceph-mon[35254]: Deploying daemon mds.i on smithi122 2024-09-16T00:30:06.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:06 smithi073 ceph-mon[35254]: mds.? [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] up:boot 2024-09-16T00:30:06.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:06 smithi073 ceph-mon[35254]: fsmap 8 up:standby 2024-09-16T00:30:06.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:06 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T00:30:06.327 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:06 smithi122 ceph-mon[35412]: Deploying daemon mds.i on smithi122 2024-09-16T00:30:06.327 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:06 smithi122 ceph-mon[35412]: mds.? [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] up:boot 2024-09-16T00:30:06.328 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:06 smithi122 ceph-mon[35412]: fsmap 8 up:standby 2024-09-16T00:30:06.328 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:06 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T00:30:06.967 INFO:journalctl@ceph.mds.i.smithi122.stdout:Sep 16 00:30:06 smithi122 podman[65429]: 2024-09-16 00:30:06.947155607 +0000 UTC m=+0.134582935 container create 0fa3ecb62b42108c88b40164811d26f0409088f6f44e3999605bca13452e2581 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-i, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-16T00:30:07.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:07 smithi038 ceph-mon[30945]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:07.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:07 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/4108270249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:30:07.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:07 smithi073 ceph-mon[35254]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:07.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:07 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/4108270249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:30:07.300 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:07 smithi122 ceph-mon[35412]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:07.300 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:07 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/4108270249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:30:07.300 INFO:journalctl@ceph.mds.i.smithi122.stdout:Sep 16 00:30:07 smithi122 podman[65429]: 2024-09-16 00:30:07.22570658 +0000 UTC m=+0.413133906 container init 0fa3ecb62b42108c88b40164811d26f0409088f6f44e3999605bca13452e2581 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-i, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:30:07.300 INFO:journalctl@ceph.mds.i.smithi122.stdout:Sep 16 00:30:07 smithi122 podman[65429]: 2024-09-16 00:30:07.228650757 +0000 UTC m=+0.416078092 container start 0fa3ecb62b42108c88b40164811d26f0409088f6f44e3999605bca13452e2581 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-i, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-16T00:30:07.301 INFO:journalctl@ceph.mds.i.smithi122.stdout:Sep 16 00:30:07 smithi122 ceph-mds[65448]: -- 172.21.15.122:0/860891342 <== mon.1 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564fa6733860 con 0x564fa6711400 2024-09-16T00:30:07.301 INFO:journalctl@ceph.mds.i.smithi122.stdout:Sep 16 00:30:07 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-i[65444]: starting mds.i at 2024-09-16T00:30:07.716 INFO:journalctl@ceph.mds.i.smithi122.stdout:Sep 16 00:30:07 smithi122 bash[65429]: 0fa3ecb62b42108c88b40164811d26f0409088f6f44e3999605bca13452e2581 2024-09-16T00:30:07.716 INFO:journalctl@ceph.mds.i.smithi122.stdout:Sep 16 00:30:07 smithi122 systemd[1]: Started Ceph mds.i for 983b2072-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:30:08.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:08 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:08.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:08 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:08.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:08 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:08.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:08 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:30:08.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:08 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:30:08.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:08 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:08.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:08 smithi038 ceph-mon[30945]: Deploying daemon mds.g on smithi038 2024-09-16T00:30:08.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:08 smithi038 ceph-mon[30945]: mds.? [v2:172.21.15.122:6838/1983975311,v1:172.21.15.122:6839/1983975311] up:boot 2024-09-16T00:30:08.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:08 smithi038 ceph-mon[30945]: fsmap 9 up:standby 2024-09-16T00:30:08.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:08 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T00:30:08.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:08 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:08.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:08 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:08.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:08 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:08.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:08 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:30:08.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:08 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:30:08.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:08 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:08.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:08 smithi122 ceph-mon[35412]: Deploying daemon mds.g on smithi038 2024-09-16T00:30:08.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:08 smithi122 ceph-mon[35412]: mds.? [v2:172.21.15.122:6838/1983975311,v1:172.21.15.122:6839/1983975311] up:boot 2024-09-16T00:30:08.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:08 smithi122 ceph-mon[35412]: fsmap 9 up:standby 2024-09-16T00:30:08.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:08 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T00:30:08.793 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:08 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:08.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:08 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:08.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:08 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:08.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:08 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:30:08.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:08 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:30:08.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:08 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:08.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:08 smithi073 ceph-mon[35254]: Deploying daemon mds.g on smithi038 2024-09-16T00:30:08.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:08 smithi073 ceph-mon[35254]: mds.? [v2:172.21.15.122:6838/1983975311,v1:172.21.15.122:6839/1983975311] up:boot 2024-09-16T00:30:08.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:08 smithi073 ceph-mon[35254]: fsmap 9 up:standby 2024-09-16T00:30:08.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:08 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T00:30:08.878 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:30:09.166 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:09.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:09 smithi038 ceph-mon[30945]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:09.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:09 smithi122 ceph-mon[35412]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:09.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.692+0000 7f313cc9c640 1 -- 172.21.15.38:0/3758527289 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f313810c170 msgr2=0x7f313810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:09.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.692+0000 7f313cc9c640 1 --2- 172.21.15.38:0/3758527289 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f313810c170 0x7f313810e610 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3130009ca0 tx=0x7f313002a950 comp rx=0 tx=0).stop 2024-09-16T00:30:09.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.692+0000 7f313cc9c640 1 -- 172.21.15.38:0/3758527289 shutdown_connections 2024-09-16T00:30:09.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.692+0000 7f313cc9c640 1 --2- 172.21.15.38:0/3758527289 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f313810c170 0x7f313810e610 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:09.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.692+0000 7f313cc9c640 1 --2- 172.21.15.38:0/3758527289 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3138103660 0x7f313810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:09.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.692+0000 7f313cc9c640 1 --2- 172.21.15.38:0/3758527289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3138102d40 0x7f3138103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:09.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.692+0000 7f313cc9c640 1 -- 172.21.15.38:0/3758527289 >> 172.21.15.38:0/3758527289 conn(0x7f31380fc7a0 msgr2=0x7f31380febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:09.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.692+0000 7f313cc9c640 1 -- 172.21.15.38:0/3758527289 shutdown_connections 2024-09-16T00:30:09.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.692+0000 7f313cc9c640 1 -- 172.21.15.38:0/3758527289 wait complete. 2024-09-16T00:30:09.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.693+0000 7f313cc9c640 1 Processor -- start 2024-09-16T00:30:09.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.693+0000 7f313cc9c640 1 -- start start 2024-09-16T00:30:09.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.693+0000 7f313cc9c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3138102d40 0x7f31381983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:09.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.693+0000 7f313cc9c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3138103660 0x7f3138198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:09.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.693+0000 7f313cc9c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f313810c170 0x7f313819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:09.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.693+0000 7f313cc9c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3138111f70 con 0x7f3138103660 2024-09-16T00:30:09.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.693+0000 7f313cc9c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3138111df0 con 0x7f313810c170 2024-09-16T00:30:09.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.693+0000 7f313cc9c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f31381120f0 con 0x7f3138102d40 2024-09-16T00:30:09.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.693+0000 7f312ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3138103660 0x7f3138198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:09.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.693+0000 7f312ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3138103660 0x7f3138198930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54052/0 (socket says 172.21.15.38:54052) 2024-09-16T00:30:09.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.693+0000 7f312ffff640 1 -- 172.21.15.38:0/3495279917 learned_addr learned my addr 172.21.15.38:0/3495279917 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:30:09.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.693+0000 7f313659c640 1 --2- 172.21.15.38:0/3495279917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3138102d40 0x7f31381983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:09.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.693+0000 7f3136d9d640 1 --2- 172.21.15.38:0/3495279917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f313810c170 0x7f313819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:09.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.693+0000 7f312ffff640 1 -- 172.21.15.38:0/3495279917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3138102d40 msgr2=0x7f31381983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:09.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.693+0000 7f312ffff640 1 --2- 172.21.15.38:0/3495279917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3138102d40 0x7f31381983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:09.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.693+0000 7f312ffff640 1 -- 172.21.15.38:0/3495279917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f313810c170 msgr2=0x7f313819ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:09.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.693+0000 7f312ffff640 1 --2- 172.21.15.38:0/3495279917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f313810c170 0x7f313819ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:09.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.693+0000 7f312ffff640 1 -- 172.21.15.38:0/3495279917 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f313819d3a0 con 0x7f3138103660 2024-09-16T00:30:09.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.693+0000 7f313659c640 1 --2- 172.21.15.38:0/3495279917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3138102d40 0x7f31381983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:09.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.693+0000 7f3136d9d640 1 --2- 172.21.15.38:0/3495279917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f313810c170 0x7f313819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:09.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.694+0000 7f312ffff640 1 --2- 172.21.15.38:0/3495279917 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3138103660 0x7f3138198930 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f312000d750 tx=0x7f312000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:09.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.694+0000 7f312f7fe640 1 -- 172.21.15.38:0/3495279917 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3120017070 con 0x7f3138103660 2024-09-16T00:30:09.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.694+0000 7f312f7fe640 1 -- 172.21.15.38:0/3495279917 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3120005020 con 0x7f3138103660 2024-09-16T00:30:09.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.694+0000 7f313cc9c640 1 -- 172.21.15.38:0/3495279917 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f313819d690 con 0x7f3138103660 2024-09-16T00:30:09.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.694+0000 7f312f7fe640 1 -- 172.21.15.38:0/3495279917 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3120004d00 con 0x7f3138103660 2024-09-16T00:30:09.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.694+0000 7f313cc9c640 1 -- 172.21.15.38:0/3495279917 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31381a4f70 con 0x7f3138103660 2024-09-16T00:30:09.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.695+0000 7f313cc9c640 1 -- 172.21.15.38:0/3495279917 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3138103120 con 0x7f3138103660 2024-09-16T00:30:09.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.696+0000 7f312f7fe640 1 -- 172.21.15.38:0/3495279917 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f3120009a30 con 0x7f3138103660 2024-09-16T00:30:09.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.696+0000 7f312f7fe640 1 --2- 172.21.15.38:0/3495279917 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f31240a3240 0x7f31240a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:09.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.697+0000 7f313659c640 1 --2- 172.21.15.38:0/3495279917 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f31240a3240 0x7f31240a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:09.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.697+0000 7f312f7fe640 1 -- 172.21.15.38:0/3495279917 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f31200ceee0 con 0x7f3138103660 2024-09-16T00:30:09.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.697+0000 7f313659c640 1 --2- 172.21.15.38:0/3495279917 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f31240a3240 0x7f31240a5700 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3128009b40 tx=0x7f3128005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:09.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.701+0000 7f312f7fe640 1 -- 172.21.15.38:0/3495279917 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f312009c0a0 con 0x7f3138103660 2024-09-16T00:30:09.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:09 smithi073 ceph-mon[35254]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:09.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.916+0000 7f313cc9c640 1 -- 172.21.15.38:0/3495279917 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31381071e0 con 0x7f3138103660 2024-09-16T00:30:09.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.916+0000 7f312f7fe640 1 -- 172.21.15.38:0/3495279917 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8040 (secure 0 0 0) 0x7f312009b9a0 con 0x7f3138103660 2024-09-16T00:30:09.917 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:30:09.918 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":10,"btime":"2024-09-16T00:30:08:043611+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/367974527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":367974527},{"type":"v1","addr":"172.21.15.38:6835","nonce":367974527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/335077136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":335077136},{"type":"v1","addr":"172.21.15.38:6837","nonce":335077136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/412277519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":412277519},{"type":"v1","addr":"172.21.15.122:6833","nonce":412277519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3987234757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3987234757},{"type":"v1","addr":"172.21.15.122:6835","nonce":3987234757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/1210328177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1210328177},{"type":"v1","addr":"172.21.15.73:6833","nonce":1210328177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1858299680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1858299680},{"type":"v1","addr":"172.21.15.122:6837","nonce":1858299680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24436,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/1983975311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1983975311},{"type":"v1","addr":"172.21.15.122:6839","nonce":1983975311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2866363113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2866363113},{"type":"v1","addr":"172.21.15.73:6835","nonce":2866363113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2681081265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2681081265},{"type":"v1","addr":"172.21.15.73:6837","nonce":2681081265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-16T00:30:09.918 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 10 2024-09-16T00:30:09.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.918+0000 7f313cc9c640 1 -- 172.21.15.38:0/3495279917 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f31240a3240 msgr2=0x7f31240a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:09.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.918+0000 7f313cc9c640 1 --2- 172.21.15.38:0/3495279917 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f31240a3240 0x7f31240a5700 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3128009b40 tx=0x7f3128005e50 comp rx=0 tx=0).stop 2024-09-16T00:30:09.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.919+0000 7f313cc9c640 1 -- 172.21.15.38:0/3495279917 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3138103660 msgr2=0x7f3138198930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:09.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.919+0000 7f313cc9c640 1 --2- 172.21.15.38:0/3495279917 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3138103660 0x7f3138198930 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f312000d750 tx=0x7f312000dc10 comp rx=0 tx=0).stop 2024-09-16T00:30:09.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.919+0000 7f313cc9c640 1 -- 172.21.15.38:0/3495279917 shutdown_connections 2024-09-16T00:30:09.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.919+0000 7f313cc9c640 1 --2- 172.21.15.38:0/3495279917 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f31240a3240 0x7f31240a5700 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:09.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.919+0000 7f313cc9c640 1 --2- 172.21.15.38:0/3495279917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f313810c170 0x7f313819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:09.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.919+0000 7f313cc9c640 1 --2- 172.21.15.38:0/3495279917 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3138103660 0x7f3138198930 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:09.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.919+0000 7f313cc9c640 1 --2- 172.21.15.38:0/3495279917 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3138102d40 0x7f31381983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:09.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.919+0000 7f313cc9c640 1 -- 172.21.15.38:0/3495279917 >> 172.21.15.38:0/3495279917 conn(0x7f31380fc7a0 msgr2=0x7f31380fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:09.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.919+0000 7f313cc9c640 1 -- 172.21.15.38:0/3495279917 shutdown_connections 2024-09-16T00:30:09.920 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:09.919+0000 7f313cc9c640 1 -- 172.21.15.38:0/3495279917 wait complete. 2024-09-16T00:30:10.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:10 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/3495279917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:30:10.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:10 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/3495279917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:30:10.793 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:10 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/3495279917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:30:11.051 INFO:journalctl@ceph.mds.g.smithi038.stdout:Sep 16 00:30:10 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-g[71267]: starting mds.g at 2024-09-16T00:30:11.051 INFO:journalctl@ceph.mds.g.smithi038.stdout:Sep 16 00:30:11 smithi038 bash[71239]: f715123d24f86a1a5a2c158c78f63895e741b6600aae82c2260603d6af7ef4f7 2024-09-16T00:30:11.380 INFO:journalctl@ceph.mds.g.smithi038.stdout:Sep 16 00:30:11 smithi038 systemd[1]: Started Ceph mds.g for 983b2072-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:30:11.676 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:11 smithi038 ceph-mon[30945]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:11.676 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:11 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:11.676 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:11 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:11.676 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:11 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:11.676 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:11 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:30:11.676 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:11 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:30:11.676 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:11 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:11.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:11 smithi122 ceph-mon[35412]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:11.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:11 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:11.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:11 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:11.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:11 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:11.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:11 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:30:11.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:11 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:30:11.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:11 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:11.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:11 smithi073 ceph-mon[35254]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:11.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:11 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:11.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:11 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:11.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:11 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:11.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:11 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:30:11.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:11 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:30:11.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:11 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:12.642 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:12 smithi038 ceph-mon[30945]: Deploying daemon mds.a on smithi038 2024-09-16T00:30:12.642 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:12 smithi038 ceph-mon[30945]: mds.? [v2:172.21.15.38:6838/3501211160,v1:172.21.15.38:6839/3501211160] up:boot 2024-09-16T00:30:12.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:12 smithi038 ceph-mon[30945]: fsmap 10 up:standby 2024-09-16T00:30:12.643 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:12 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-16T00:30:12.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:12 smithi122 ceph-mon[35412]: Deploying daemon mds.a on smithi038 2024-09-16T00:30:12.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:12 smithi122 ceph-mon[35412]: mds.? [v2:172.21.15.38:6838/3501211160,v1:172.21.15.38:6839/3501211160] up:boot 2024-09-16T00:30:12.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:12 smithi122 ceph-mon[35412]: fsmap 10 up:standby 2024-09-16T00:30:12.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:12 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-16T00:30:12.792 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:30:12.793 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:12 smithi073 ceph-mon[35254]: Deploying daemon mds.a on smithi038 2024-09-16T00:30:12.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:12 smithi073 ceph-mon[35254]: mds.? [v2:172.21.15.38:6838/3501211160,v1:172.21.15.38:6839/3501211160] up:boot 2024-09-16T00:30:12.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:12 smithi073 ceph-mon[35254]: fsmap 10 up:standby 2024-09-16T00:30:12.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:12 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-09-16T00:30:13.161 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:13.215 INFO:journalctl@ceph.mds.a.smithi038.stdout:Sep 16 00:30:13 smithi038 systemd[1]: Starting Ceph mds.a for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:30:13.589 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:13 smithi038 ceph-mon[30945]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:13.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.671+0000 7fd3c8bc5640 1 -- 172.21.15.38:0/3009582456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3c4108300 msgr2=0x7fd3c410ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:13.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.671+0000 7fd3c8bc5640 1 --2- 172.21.15.38:0/3009582456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3c4108300 0x7fd3c410ec70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc009cd0 tx=0x7fd3bc02a8d0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.671+0000 7fd3c8bc5640 1 -- 172.21.15.38:0/3009582456 shutdown_connections 2024-09-16T00:30:13.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.671+0000 7fd3c8bc5640 1 --2- 172.21.15.38:0/3009582456 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3c4108300 0x7fd3c410ec70 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.671+0000 7fd3c8bc5640 1 --2- 172.21.15.38:0/3009582456 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3c4103780 0x7fd3c4107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.671+0000 7fd3c8bc5640 1 --2- 172.21.15.38:0/3009582456 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3c4102dd0 0x7fd3c41031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.671+0000 7fd3c8bc5640 1 -- 172.21.15.38:0/3009582456 >> 172.21.15.38:0/3009582456 conn(0x7fd3c40fc7d0 msgr2=0x7fd3c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:13.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.671+0000 7fd3c8bc5640 1 -- 172.21.15.38:0/3009582456 shutdown_connections 2024-09-16T00:30:13.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.671+0000 7fd3c8bc5640 1 -- 172.21.15.38:0/3009582456 wait complete. 2024-09-16T00:30:13.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.672+0000 7fd3c8bc5640 1 Processor -- start 2024-09-16T00:30:13.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.672+0000 7fd3c8bc5640 1 -- start start 2024-09-16T00:30:13.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.672+0000 7fd3c8bc5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3c4102dd0 0x7fd3c4198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:13.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.672+0000 7fd3c8bc5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3c4103780 0x7fd3c4198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:13.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.672+0000 7fd3c8bc5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3c4108300 0x7fd3c419ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:13.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.672+0000 7fd3c8bc5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c41120d0 con 0x7fd3c4108300 2024-09-16T00:30:13.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.672+0000 7fd3c8bc5640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c4111f50 con 0x7fd3c4103780 2024-09-16T00:30:13.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.672+0000 7fd3c8bc5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd3c4112250 con 0x7fd3c4102dd0 2024-09-16T00:30:13.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.672+0000 7fd3c2d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3c4108300 0x7fd3c419ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:13.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.672+0000 7fd3c2d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3c4108300 0x7fd3c419ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54076/0 (socket says 172.21.15.38:54076) 2024-09-16T00:30:13.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.672+0000 7fd3c2d9d640 1 -- 172.21.15.38:0/2253058578 learned_addr learned my addr 172.21.15.38:0/2253058578 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:30:13.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.672+0000 7fd3c259c640 1 --2- 172.21.15.38:0/2253058578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3c4102dd0 0x7fd3c4198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:13.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.672+0000 7fd3c1d9b640 1 --2- 172.21.15.38:0/2253058578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3c4103780 0x7fd3c4198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:13.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.672+0000 7fd3c2d9d640 1 -- 172.21.15.38:0/2253058578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3c4102dd0 msgr2=0x7fd3c4198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:13.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.672+0000 7fd3c2d9d640 1 --2- 172.21.15.38:0/2253058578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3c4102dd0 0x7fd3c4198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.672+0000 7fd3c2d9d640 1 -- 172.21.15.38:0/2253058578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3c4103780 msgr2=0x7fd3c4198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:13.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.672+0000 7fd3c2d9d640 1 --2- 172.21.15.38:0/2253058578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3c4103780 0x7fd3c4198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.672+0000 7fd3c2d9d640 1 -- 172.21.15.38:0/2253058578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3c419d540 con 0x7fd3c4108300 2024-09-16T00:30:13.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.672+0000 7fd3c1d9b640 1 --2- 172.21.15.38:0/2253058578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3c4103780 0x7fd3c4198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:13.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.673+0000 7fd3c2d9d640 1 --2- 172.21.15.38:0/2253058578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3c4108300 0x7fd3c419ce60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc009ca0 tx=0x7fd3bc02f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:13.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.673+0000 7fd3b37fe640 1 -- 172.21.15.38:0/2253058578 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc031860 con 0x7fd3c4108300 2024-09-16T00:30:13.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.673+0000 7fd3b37fe640 1 -- 172.21.15.38:0/2253058578 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd3bc003e50 con 0x7fd3c4108300 2024-09-16T00:30:13.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.673+0000 7fd3c8bc5640 1 -- 172.21.15.38:0/2253058578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3c419d7d0 con 0x7fd3c4108300 2024-09-16T00:30:13.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.673+0000 7fd3b37fe640 1 -- 172.21.15.38:0/2253058578 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3bc0307f0 con 0x7fd3c4108300 2024-09-16T00:30:13.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.673+0000 7fd3c8bc5640 1 -- 172.21.15.38:0/2253058578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3c41a5110 con 0x7fd3c4108300 2024-09-16T00:30:13.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.674+0000 7fd3c8bc5640 1 -- 172.21.15.38:0/2253058578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3c41031b0 con 0x7fd3c4108300 2024-09-16T00:30:13.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.675+0000 7fd3b37fe640 1 -- 172.21.15.38:0/2253058578 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fd3bc0041d0 con 0x7fd3c4108300 2024-09-16T00:30:13.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.676+0000 7fd3b37fe640 1 --2- 172.21.15.38:0/2253058578 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd3ac0a3240 0x7fd3ac0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:13.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.676+0000 7fd3b37fe640 1 -- 172.21.15.38:0/2253058578 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fd3bc0f06d0 con 0x7fd3c4108300 2024-09-16T00:30:13.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.676+0000 7fd3c259c640 1 --2- 172.21.15.38:0/2253058578 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd3ac0a3240 0x7fd3ac0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:13.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.676+0000 7fd3c259c640 1 --2- 172.21.15.38:0/2253058578 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd3ac0a3240 0x7fd3ac0a5700 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4005fd0 tx=0x7fd3b4005e20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:13.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.679+0000 7fd3b37fe640 1 -- 172.21.15.38:0/2253058578 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd3bc0bd890 con 0x7fd3c4108300 2024-09-16T00:30:13.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:13 smithi122 ceph-mon[35412]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:13.793 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:13 smithi073 ceph-mon[35254]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:13.865 INFO:journalctl@ceph.mds.a.smithi038.stdout:Sep 16 00:30:13 smithi038 podman[71632]: 2024-09-16 00:30:13.735978192 +0000 UTC m=+0.246084009 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:30:13.865 INFO:journalctl@ceph.mds.a.smithi038.stdout:Sep 16 00:30:13 smithi038 podman[71632]: 2024-09-16 00:30:13.865937877 +0000 UTC m=+0.376043690 container create 12df256a6ebe95dc07b2ce65d194765d0282f0b365e32732cd01a9b4e29df6ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-a, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-16T00:30:13.897 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.896+0000 7fd3c8bc5640 1 -- 172.21.15.38:0/2253058578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd3c410a290 con 0x7fd3c4108300 2024-09-16T00:30:13.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.897+0000 7fd3b37fe640 1 -- 172.21.15.38:0/2253058578 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8866 (secure 0 0 0) 0x7fd3bc0bd190 con 0x7fd3c4108300 2024-09-16T00:30:13.898 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:30:13.899 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":11,"btime":"2024-09-16T00:30:11:379073+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/367974527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":367974527},{"type":"v1","addr":"172.21.15.38:6835","nonce":367974527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/335077136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":335077136},{"type":"v1","addr":"172.21.15.38:6837","nonce":335077136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/3501211160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3501211160},{"type":"v1","addr":"172.21.15.38:6839","nonce":3501211160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/412277519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":412277519},{"type":"v1","addr":"172.21.15.122:6833","nonce":412277519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3987234757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3987234757},{"type":"v1","addr":"172.21.15.122:6835","nonce":3987234757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/1210328177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1210328177},{"type":"v1","addr":"172.21.15.73:6833","nonce":1210328177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1858299680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1858299680},{"type":"v1","addr":"172.21.15.122:6837","nonce":1858299680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24436,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/1983975311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1983975311},{"type":"v1","addr":"172.21.15.122:6839","nonce":1983975311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2866363113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2866363113},{"type":"v1","addr":"172.21.15.73:6835","nonce":2866363113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2681081265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2681081265},{"type":"v1","addr":"172.21.15.73:6837","nonce":2681081265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-16T00:30:13.899 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 11 2024-09-16T00:30:13.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.899+0000 7fd3c8bc5640 1 -- 172.21.15.38:0/2253058578 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd3ac0a3240 msgr2=0x7fd3ac0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:13.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.899+0000 7fd3c8bc5640 1 --2- 172.21.15.38:0/2253058578 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd3ac0a3240 0x7fd3ac0a5700 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd3b4005fd0 tx=0x7fd3b4005e20 comp rx=0 tx=0).stop 2024-09-16T00:30:13.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.899+0000 7fd3c8bc5640 1 -- 172.21.15.38:0/2253058578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3c4108300 msgr2=0x7fd3c419ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:13.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.899+0000 7fd3c8bc5640 1 --2- 172.21.15.38:0/2253058578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3c4108300 0x7fd3c419ce60 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd3bc009ca0 tx=0x7fd3bc02f420 comp rx=0 tx=0).stop 2024-09-16T00:30:13.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.899+0000 7fd3c8bc5640 1 -- 172.21.15.38:0/2253058578 shutdown_connections 2024-09-16T00:30:13.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.899+0000 7fd3c8bc5640 1 --2- 172.21.15.38:0/2253058578 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd3ac0a3240 0x7fd3ac0a5700 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.899+0000 7fd3c8bc5640 1 --2- 172.21.15.38:0/2253058578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd3c4108300 0x7fd3c419ce60 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.899+0000 7fd3c8bc5640 1 --2- 172.21.15.38:0/2253058578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3c4103780 0x7fd3c4198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.899+0000 7fd3c8bc5640 1 --2- 172.21.15.38:0/2253058578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd3c4102dd0 0x7fd3c4198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:13.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.899+0000 7fd3c8bc5640 1 -- 172.21.15.38:0/2253058578 >> 172.21.15.38:0/2253058578 conn(0x7fd3c40fc7d0 msgr2=0x7fd3c4105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:13.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.899+0000 7fd3c8bc5640 1 -- 172.21.15.38:0/2253058578 shutdown_connections 2024-09-16T00:30:13.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:13.899+0000 7fd3c8bc5640 1 -- 172.21.15.38:0/2253058578 wait complete. 2024-09-16T00:30:14.532 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:14 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/2253058578' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:30:14.532 INFO:journalctl@ceph.mds.a.smithi038.stdout:Sep 16 00:30:14 smithi038 podman[71632]: 2024-09-16 00:30:14.312475814 +0000 UTC m=+0.822581629 container init 12df256a6ebe95dc07b2ce65d194765d0282f0b365e32732cd01a9b4e29df6ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-a, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-16T00:30:14.532 INFO:journalctl@ceph.mds.a.smithi038.stdout:Sep 16 00:30:14 smithi038 podman[71632]: 2024-09-16 00:30:14.315623913 +0000 UTC m=+0.825729735 container start 12df256a6ebe95dc07b2ce65d194765d0282f0b365e32732cd01a9b4e29df6ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-16T00:30:14.532 INFO:journalctl@ceph.mds.a.smithi038.stdout:Sep 16 00:30:14 smithi038 ceph-mds[71686]: -- 172.21.15.38:0/3690664218 <== mon.0 v2:172.21.15.38:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56529156bc20 con 0x565291549000 2024-09-16T00:30:14.533 INFO:journalctl@ceph.mds.a.smithi038.stdout:Sep 16 00:30:14 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-a[71681]: starting mds.a at 2024-09-16T00:30:14.533 INFO:journalctl@ceph.mds.a.smithi038.stdout:Sep 16 00:30:14 smithi038 bash[71632]: 12df256a6ebe95dc07b2ce65d194765d0282f0b365e32732cd01a9b4e29df6ec 2024-09-16T00:30:14.533 INFO:journalctl@ceph.mds.a.smithi038.stdout:Sep 16 00:30:14 smithi038 systemd[1]: Started Ceph mds.a for 983b2072-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:30:14.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:14 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/2253058578' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:30:14.793 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:14 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/2253058578' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:30:15.700 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:15 smithi038 ceph-mon[30945]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:15.701 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:15 smithi038 ceph-mon[30945]: mds.? [v2:172.21.15.38:6840/2623130915,v1:172.21.15.38:6841/2623130915] up:boot 2024-09-16T00:30:15.701 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:15 smithi038 ceph-mon[30945]: fsmap 11 up:standby 2024-09-16T00:30:15.701 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:15 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-16T00:30:15.701 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:15 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:15.701 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:15 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:15.701 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:15 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:15.701 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:15 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:30:15.701 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:15 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:30:15.701 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:15 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:15.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:15 smithi122 ceph-mon[35412]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:15.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:15 smithi122 ceph-mon[35412]: mds.? [v2:172.21.15.38:6840/2623130915,v1:172.21.15.38:6841/2623130915] up:boot 2024-09-16T00:30:15.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:15 smithi122 ceph-mon[35412]: fsmap 11 up:standby 2024-09-16T00:30:15.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:15 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-16T00:30:15.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:15 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:15.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:15 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:15.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:15 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:15.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:15 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:30:15.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:15 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:30:15.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:15 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:15.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:15 smithi073 ceph-mon[35254]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:15.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:15 smithi073 ceph-mon[35254]: mds.? [v2:172.21.15.38:6840/2623130915,v1:172.21.15.38:6841/2623130915] up:boot 2024-09-16T00:30:15.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:15 smithi073 ceph-mon[35254]: fsmap 11 up:standby 2024-09-16T00:30:15.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:15 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-09-16T00:30:15.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:15 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:15.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:15 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:15.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:15 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:15.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:15 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-09-16T00:30:15.795 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:15 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-16T00:30:15.795 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:15 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:16.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:16 smithi038 ceph-mon[30945]: Deploying daemon mds.b on smithi073 2024-09-16T00:30:16.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:16 smithi122 ceph-mon[35412]: Deploying daemon mds.b on smithi073 2024-09-16T00:30:16.757 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:16 smithi073 ceph-mon[35254]: Deploying daemon mds.b on smithi073 2024-09-16T00:30:16.757 INFO:journalctl@ceph.mds.b.smithi073.stdout:Sep 16 00:30:16 smithi073 systemd[1]: Starting Ceph mds.b for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:30:16.877 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:30:17.044 INFO:journalctl@ceph.mds.b.smithi073.stdout:Sep 16 00:30:16 smithi073 podman[65390]: 2024-09-16 00:30:16.659720516 +0000 UTC m=+0.015506749 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:30:17.044 INFO:journalctl@ceph.mds.b.smithi073.stdout:Sep 16 00:30:16 smithi073 podman[65390]: 2024-09-16 00:30:16.854451246 +0000 UTC m=+0.210237478 container create 47c89ec1eb8f9059ea6d20628eb4a2e3151d2d628100947b440b903a74fbb6f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-16T00:30:17.181 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:17.419 INFO:journalctl@ceph.mds.b.smithi073.stdout:Sep 16 00:30:17 smithi073 podman[65390]: 2024-09-16 00:30:17.115636718 +0000 UTC m=+0.471422963 container init 47c89ec1eb8f9059ea6d20628eb4a2e3151d2d628100947b440b903a74fbb6f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-b, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True) 2024-09-16T00:30:17.420 INFO:journalctl@ceph.mds.b.smithi073.stdout:Sep 16 00:30:17 smithi073 podman[65390]: 2024-09-16 00:30:17.118655202 +0000 UTC m=+0.474441442 container start 47c89ec1eb8f9059ea6d20628eb4a2e3151d2d628100947b440b903a74fbb6f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-b, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-16T00:30:17.420 INFO:journalctl@ceph.mds.b.smithi073.stdout:Sep 16 00:30:17 smithi073 ceph-mds[65410]: -- 172.21.15.73:0/4039704930 <== mon.2 v2:172.21.15.73:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55de1f617860 con 0x55de1f5f4800 2024-09-16T00:30:17.420 INFO:journalctl@ceph.mds.b.smithi073.stdout:Sep 16 00:30:17 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-b[65406]: starting mds.b at 2024-09-16T00:30:17.420 INFO:journalctl@ceph.mds.b.smithi073.stdout:Sep 16 00:30:17 smithi073 bash[65390]: 47c89ec1eb8f9059ea6d20628eb4a2e3151d2d628100947b440b903a74fbb6f5 2024-09-16T00:30:17.420 INFO:journalctl@ceph.mds.b.smithi073.stdout:Sep 16 00:30:17 smithi073 systemd[1]: Started Ceph mds.b for 983b2072-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:30:17.611 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:17 smithi038 ceph-mon[30945]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:17.611 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:17 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:17.612 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:17 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:17.612 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:17 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:17.612 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:17 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:17.612 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:17 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:17.612 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:17 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:30:17.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.703+0000 7f4fb6257640 1 -- 172.21.15.38:0/2783346324 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fb0102ef0 msgr2=0x7f4fb010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:17.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.703+0000 7f4fb6257640 1 --2- 172.21.15.38:0/2783346324 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fb0102ef0 0x7f4fb010da40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f4fa8009ca0 tx=0x7f4fa802a950 comp rx=0 tx=0).stop 2024-09-16T00:30:17.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.704+0000 7f4fb6257640 1 -- 172.21.15.38:0/2783346324 shutdown_connections 2024-09-16T00:30:17.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.704+0000 7f4fb6257640 1 --2- 172.21.15.38:0/2783346324 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fb0102ef0 0x7f4fb010da40 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:17.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.704+0000 7f4fb6257640 1 --2- 172.21.15.38:0/2783346324 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4fb0102570 0x7f4fb01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:17.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.704+0000 7f4fb6257640 1 --2- 172.21.15.38:0/2783346324 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fb0100e20 0x7f4fb0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:17.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.704+0000 7f4fb6257640 1 -- 172.21.15.38:0/2783346324 >> 172.21.15.38:0/2783346324 conn(0x7f4fb00fc7d0 msgr2=0x7f4fb00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:17.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.704+0000 7f4fb6257640 1 -- 172.21.15.38:0/2783346324 shutdown_connections 2024-09-16T00:30:17.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.704+0000 7f4fb6257640 1 -- 172.21.15.38:0/2783346324 wait complete. 2024-09-16T00:30:17.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.704+0000 7f4fb6257640 1 Processor -- start 2024-09-16T00:30:17.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.704+0000 7f4fb6257640 1 -- start start 2024-09-16T00:30:17.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.705+0000 7f4fb6257640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fb0100e20 0x7f4fb01992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:17.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.705+0000 7f4fb6257640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4fb0102570 0x7f4fb0199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:17.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.705+0000 7f4fb6257640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fb0102ef0 0x7f4fb01a4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:17.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.705+0000 7f4fb6257640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb0110f80 con 0x7f4fb0100e20 2024-09-16T00:30:17.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.705+0000 7f4fb6257640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb0110e00 con 0x7f4fb0102ef0 2024-09-16T00:30:17.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.705+0000 7f4fb6257640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4fb0111100 con 0x7f4fb0102570 2024-09-16T00:30:17.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.705+0000 7f4faf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fb0100e20 0x7f4fb01992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:17.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.705+0000 7f4faf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fb0100e20 0x7f4fb01992d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46050/0 (socket says 172.21.15.38:46050) 2024-09-16T00:30:17.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.705+0000 7f4faf7fe640 1 -- 172.21.15.38:0/747124249 learned_addr learned my addr 172.21.15.38:0/747124249 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:30:17.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.705+0000 7f4faeffd640 1 --2- 172.21.15.38:0/747124249 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4fb0102570 0x7f4fb0199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:17.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.705+0000 7f4faffff640 1 --2- 172.21.15.38:0/747124249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fb0102ef0 0x7f4fb01a4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:17.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.705+0000 7f4faf7fe640 1 -- 172.21.15.38:0/747124249 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4fb0102570 msgr2=0x7f4fb0199810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:17.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.705+0000 7f4faf7fe640 1 --2- 172.21.15.38:0/747124249 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4fb0102570 0x7f4fb0199810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:17.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.705+0000 7f4faf7fe640 1 -- 172.21.15.38:0/747124249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fb0102ef0 msgr2=0x7f4fb01a4350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:17.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.705+0000 7f4faf7fe640 1 --2- 172.21.15.38:0/747124249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fb0102ef0 0x7f4fb01a4350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:17.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.705+0000 7f4faf7fe640 1 -- 172.21.15.38:0/747124249 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fb01a4890 con 0x7f4fb0100e20 2024-09-16T00:30:17.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.705+0000 7f4faf7fe640 1 --2- 172.21.15.38:0/747124249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fb0100e20 0x7f4fb01992d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4fa000e9f0 tx=0x7f4fa000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:17.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.705+0000 7f4facff9640 1 -- 172.21.15.38:0/747124249 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa000cd80 con 0x7f4fb0100e20 2024-09-16T00:30:17.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.705+0000 7f4fb6257640 1 -- 172.21.15.38:0/747124249 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fb01a4a60 con 0x7f4fb0100e20 2024-09-16T00:30:17.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.706+0000 7f4facff9640 1 -- 172.21.15.38:0/747124249 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4fa0010910 con 0x7f4fb0100e20 2024-09-16T00:30:17.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.706+0000 7f4facff9640 1 -- 172.21.15.38:0/747124249 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fa00188f0 con 0x7f4fb0100e20 2024-09-16T00:30:17.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.706+0000 7f4fb6257640 1 -- 172.21.15.38:0/747124249 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fb01a4fa0 con 0x7f4fb0100e20 2024-09-16T00:30:17.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.707+0000 7f4fb6257640 1 -- 172.21.15.38:0/747124249 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fb0101200 con 0x7f4fb0100e20 2024-09-16T00:30:17.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.708+0000 7f4facff9640 1 -- 172.21.15.38:0/747124249 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f4fa0010ac0 con 0x7f4fb0100e20 2024-09-16T00:30:17.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.708+0000 7f4facff9640 1 --2- 172.21.15.38:0/747124249 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f4f980a3240 0x7f4f980a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:17.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.709+0000 7f4faeffd640 1 --2- 172.21.15.38:0/747124249 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f4f980a3240 0x7f4f980a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:17.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.709+0000 7f4facff9640 1 -- 172.21.15.38:0/747124249 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7f4fa00cf8a0 con 0x7f4fb0100e20 2024-09-16T00:30:17.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.709+0000 7f4faeffd640 1 --2- 172.21.15.38:0/747124249 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f4f980a3240 0x7f4f980a5700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4fb006aac0 tx=0x7f4fa400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:17.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.713+0000 7f4facff9640 1 -- 172.21.15.38:0/747124249 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4fa009c370 con 0x7f4fb0100e20 2024-09-16T00:30:17.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:17 smithi122 ceph-mon[35412]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:17.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:17 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:17.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:17 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:17.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:17 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:17.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:17 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:17.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:17 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:17.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:17 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:30:17.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:17 smithi073 ceph-mon[35254]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:17.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:17 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:17.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:17 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:17.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:17 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:17.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:17 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:17.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:17 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:17.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:17 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:30:17.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.936+0000 7f4fb6257640 1 -- 172.21.15.38:0/747124249 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4fb01a5200 con 0x7f4fb0100e20 2024-09-16T00:30:17.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.937+0000 7f4facff9640 1 -- 172.21.15.38:0/747124249 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10518 (secure 0 0 0) 0x7f4fa00a0670 con 0x7f4fb0100e20 2024-09-16T00:30:17.937 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:30:17.938 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":13,"btime":"2024-09-16T00:30:17:414689+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/367974527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":367974527},{"type":"v1","addr":"172.21.15.38:6835","nonce":367974527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/335077136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":335077136},{"type":"v1","addr":"172.21.15.38:6837","nonce":335077136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/3501211160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3501211160},{"type":"v1","addr":"172.21.15.38:6839","nonce":3501211160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6841/2623130915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":2623130915},{"type":"v1","addr":"172.21.15.38:6841","nonce":2623130915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/412277519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":412277519},{"type":"v1","addr":"172.21.15.122:6833","nonce":412277519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3987234757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3987234757},{"type":"v1","addr":"172.21.15.122:6835","nonce":3987234757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/1210328177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1210328177},{"type":"v1","addr":"172.21.15.73:6833","nonce":1210328177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1858299680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1858299680},{"type":"v1","addr":"172.21.15.122:6837","nonce":1858299680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24436,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/1983975311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1983975311},{"type":"v1","addr":"172.21.15.122:6839","nonce":1983975311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2866363113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2866363113},{"type":"v1","addr":"172.21.15.73:6835","nonce":2866363113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2681081265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2681081265},{"type":"v1","addr":"172.21.15.73:6837","nonce":2681081265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24494,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6839/2396224525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2396224525},{"type":"v1","addr":"172.21.15.73:6839","nonce":2396224525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-16T00:30:17.938 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 13 2024-09-16T00:30:17.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.939+0000 7f4fb6257640 1 -- 172.21.15.38:0/747124249 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f4f980a3240 msgr2=0x7f4f980a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:17.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.939+0000 7f4fb6257640 1 --2- 172.21.15.38:0/747124249 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f4f980a3240 0x7f4f980a5700 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f4fb006aac0 tx=0x7f4fa400b410 comp rx=0 tx=0).stop 2024-09-16T00:30:17.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.939+0000 7f4fb6257640 1 -- 172.21.15.38:0/747124249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fb0100e20 msgr2=0x7f4fb01992d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:17.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.939+0000 7f4fb6257640 1 --2- 172.21.15.38:0/747124249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fb0100e20 0x7f4fb01992d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4fa000e9f0 tx=0x7f4fa000eeb0 comp rx=0 tx=0).stop 2024-09-16T00:30:17.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.939+0000 7f4fb6257640 1 -- 172.21.15.38:0/747124249 shutdown_connections 2024-09-16T00:30:17.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.939+0000 7f4fb6257640 1 --2- 172.21.15.38:0/747124249 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f4f980a3240 0x7f4f980a5700 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:17.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.939+0000 7f4fb6257640 1 --2- 172.21.15.38:0/747124249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4fb0102ef0 0x7f4fb01a4350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:17.940 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.939+0000 7f4fb6257640 1 --2- 172.21.15.38:0/747124249 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4fb0102570 0x7f4fb0199810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:17.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.939+0000 7f4fb6257640 1 --2- 172.21.15.38:0/747124249 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4fb0100e20 0x7f4fb01992d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:17.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.939+0000 7f4fb6257640 1 -- 172.21.15.38:0/747124249 >> 172.21.15.38:0/747124249 conn(0x7f4fb00fc7d0 msgr2=0x7f4fb0106840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:17.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.940+0000 7f4fb6257640 1 -- 172.21.15.38:0/747124249 shutdown_connections 2024-09-16T00:30:17.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:17.940+0000 7f4fb6257640 1 -- 172.21.15.38:0/747124249 wait complete. 2024-09-16T00:30:18.594 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:18 smithi038 ceph-mon[30945]: mds.? [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] up:boot 2024-09-16T00:30:18.594 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:18 smithi038 ceph-mon[30945]: fsmap 12 up:standby 2024-09-16T00:30:18.594 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:18 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T00:30:18.595 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:18 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/747124249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:30:18.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:18 smithi122 ceph-mon[35412]: mds.? [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] up:boot 2024-09-16T00:30:18.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:18 smithi122 ceph-mon[35412]: fsmap 12 up:standby 2024-09-16T00:30:18.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:18 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T00:30:18.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:18 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/747124249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:30:18.786 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:18 smithi073 ceph-mon[35254]: mds.? [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] up:boot 2024-09-16T00:30:18.787 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:18 smithi073 ceph-mon[35254]: fsmap 12 up:standby 2024-09-16T00:30:18.787 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:18 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T00:30:18.787 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:18 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/747124249' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:30:18.943 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-09-16T00:30:19.312 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:19.700 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:19 smithi038 ceph-mon[30945]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:19.701 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:19 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:19.701 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:19 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:19.701 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:19 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:19.701 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:19 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:19.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:19 smithi122 ceph-mon[35412]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:19.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:19 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:19.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:19 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:19.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:19 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:19.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:19 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:19.754 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:19 smithi073 ceph-mon[35254]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:19.755 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:19 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:19.755 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:19 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:19.755 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:19 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:19.755 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:19 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:19.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.782+0000 7effcd308640 1 -- 172.21.15.38:0/2648303279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effc8102d90 msgr2=0x7effc8103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:19.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.782+0000 7effcd308640 1 --2- 172.21.15.38:0/2648303279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effc8102d90 0x7effc8103170 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7effb8009930 tx=0x7effb802a420 comp rx=0 tx=0).stop 2024-09-16T00:30:19.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.782+0000 7effcd308640 1 -- 172.21.15.38:0/2648303279 shutdown_connections 2024-09-16T00:30:19.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.782+0000 7effcd308640 1 --2- 172.21.15.38:0/2648303279 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effc8108290 0x7effc810ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.782+0000 7effcd308640 1 --2- 172.21.15.38:0/2648303279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effc8103740 0x7effc8107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.782+0000 7effcd308640 1 --2- 172.21.15.38:0/2648303279 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effc8102d90 0x7effc8103170 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.782+0000 7effcd308640 1 -- 172.21.15.38:0/2648303279 >> 172.21.15.38:0/2648303279 conn(0x7effc80fc7d0 msgr2=0x7effc80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:19.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.782+0000 7effcd308640 1 -- 172.21.15.38:0/2648303279 shutdown_connections 2024-09-16T00:30:19.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.782+0000 7effcd308640 1 -- 172.21.15.38:0/2648303279 wait complete. 2024-09-16T00:30:19.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.783+0000 7effcd308640 1 Processor -- start 2024-09-16T00:30:19.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.783+0000 7effcd308640 1 -- start start 2024-09-16T00:30:19.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.783+0000 7effcd308640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effc8102d90 0x7effc8198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:19.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.783+0000 7effcd308640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effc8103740 0x7effc8198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:19.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.783+0000 7effcd308640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effc8108290 0x7effc819ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:19.810 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.784+0000 7effcd308640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7effc8112090 con 0x7effc8102d90 2024-09-16T00:30:19.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.784+0000 7effcd308640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7effc8111f10 con 0x7effc8108290 2024-09-16T00:30:19.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.784+0000 7effcd308640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7effc8112210 con 0x7effc8103740 2024-09-16T00:30:19.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.784+0000 7effc6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effc8102d90 0x7effc8198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:19.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.784+0000 7effc6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effc8102d90 0x7effc8198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46068/0 (socket says 172.21.15.38:46068) 2024-09-16T00:30:19.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.784+0000 7effc6ffd640 1 -- 172.21.15.38:0/4005757175 learned_addr learned my addr 172.21.15.38:0/4005757175 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:30:19.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.784+0000 7effc77fe640 1 --2- 172.21.15.38:0/4005757175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effc8108290 0x7effc819ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:19.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.784+0000 7effc67fc640 1 --2- 172.21.15.38:0/4005757175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effc8103740 0x7effc8198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:19.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.784+0000 7effc6ffd640 1 -- 172.21.15.38:0/4005757175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effc8103740 msgr2=0x7effc8198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:19.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.784+0000 7effc6ffd640 1 --2- 172.21.15.38:0/4005757175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effc8103740 0x7effc8198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.784+0000 7effc6ffd640 1 -- 172.21.15.38:0/4005757175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effc8108290 msgr2=0x7effc819ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:19.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.784+0000 7effc6ffd640 1 --2- 172.21.15.38:0/4005757175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effc8108290 0x7effc819ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.784+0000 7effc6ffd640 1 -- 172.21.15.38:0/4005757175 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effc819d540 con 0x7effc8102d90 2024-09-16T00:30:19.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.784+0000 7effc6ffd640 1 --2- 172.21.15.38:0/4005757175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effc8102d90 0x7effc8198590 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7effb8009230 tx=0x7effb8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:19.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.784+0000 7effaffff640 1 -- 172.21.15.38:0/4005757175 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb803f030 con 0x7effc8102d90 2024-09-16T00:30:19.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.784+0000 7effaffff640 1 -- 172.21.15.38:0/4005757175 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7effb8003ad0 con 0x7effc8102d90 2024-09-16T00:30:19.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.784+0000 7effcd308640 1 -- 172.21.15.38:0/4005757175 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effc819d7d0 con 0x7effc8102d90 2024-09-16T00:30:19.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.784+0000 7effaffff640 1 -- 172.21.15.38:0/4005757175 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb8035070 con 0x7effc8102d90 2024-09-16T00:30:19.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.784+0000 7effcd308640 1 -- 172.21.15.38:0/4005757175 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effc81a5110 con 0x7effc8102d90 2024-09-16T00:30:19.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.785+0000 7effcd308640 1 -- 172.21.15.38:0/4005757175 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effc806a7f0 con 0x7effc8102d90 2024-09-16T00:30:19.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.787+0000 7effaffff640 1 -- 172.21.15.38:0/4005757175 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7effb8042030 con 0x7effc8102d90 2024-09-16T00:30:19.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.787+0000 7effaffff640 1 --2- 172.21.15.38:0/4005757175 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7effb00a3240 0x7effb00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:19.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.788+0000 7effc67fc640 1 --2- 172.21.15.38:0/4005757175 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7effb00a3240 0x7effb00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:19.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.788+0000 7effaffff640 1 -- 172.21.15.38:0/4005757175 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7effb80efda0 con 0x7effc8102d90 2024-09-16T00:30:19.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.788+0000 7effc67fc640 1 --2- 172.21.15.38:0/4005757175 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7effb00a3240 0x7effb00a5700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7effbc005fd0 tx=0x7effbc005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:19.812 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.791+0000 7effaffff640 1 -- 172.21.15.38:0/4005757175 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7effb80bcf60 con 0x7effc8102d90 2024-09-16T00:30:19.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.989+0000 7effcd308640 1 -- 172.21.15.38:0/4005757175 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7effc8103170 con 0x7effc8102d90 2024-09-16T00:30:19.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.989+0000 7effaffff640 1 -- 172.21.15.38:0/4005757175 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v77) ==== 84+0+65 (secure 0 0 0) 0x7effb80bc860 con 0x7effc8102d90 2024-09-16T00:30:19.990 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:30:19.990 INFO:teuthology.orchestra.run.smithi038.stdout:[ 2024-09-16T00:30:19.990 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:30:19.990 INFO:teuthology.orchestra.run.smithi038.stdout: "poolnum": 1, 2024-09-16T00:30:19.990 INFO:teuthology.orchestra.run.smithi038.stdout: "poolname": ".mgr" 2024-09-16T00:30:19.991 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:30:19.991 INFO:teuthology.orchestra.run.smithi038.stdout:] 2024-09-16T00:30:19.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.991+0000 7effcd308640 1 -- 172.21.15.38:0/4005757175 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7effb00a3240 msgr2=0x7effb00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:19.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.991+0000 7effcd308640 1 --2- 172.21.15.38:0/4005757175 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7effb00a3240 0x7effb00a5700 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7effbc005fd0 tx=0x7effbc005850 comp rx=0 tx=0).stop 2024-09-16T00:30:19.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.992+0000 7effcd308640 1 -- 172.21.15.38:0/4005757175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effc8102d90 msgr2=0x7effc8198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:19.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.992+0000 7effcd308640 1 --2- 172.21.15.38:0/4005757175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effc8102d90 0x7effc8198590 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7effb8009230 tx=0x7effb8004270 comp rx=0 tx=0).stop 2024-09-16T00:30:19.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.992+0000 7effcd308640 1 -- 172.21.15.38:0/4005757175 shutdown_connections 2024-09-16T00:30:19.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.992+0000 7effcd308640 1 --2- 172.21.15.38:0/4005757175 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7effb00a3240 0x7effb00a5700 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.992+0000 7effcd308640 1 --2- 172.21.15.38:0/4005757175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effc8108290 0x7effc819ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.992+0000 7effcd308640 1 --2- 172.21.15.38:0/4005757175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7effc8103740 0x7effc8198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.992+0000 7effcd308640 1 --2- 172.21.15.38:0/4005757175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7effc8102d90 0x7effc8198590 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:19.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.992+0000 7effcd308640 1 -- 172.21.15.38:0/4005757175 >> 172.21.15.38:0/4005757175 conn(0x7effc80fc7d0 msgr2=0x7effc8105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:19.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.992+0000 7effcd308640 1 -- 172.21.15.38:0/4005757175 shutdown_connections 2024-09-16T00:30:19.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:19.992+0000 7effcd308640 1 -- 172.21.15.38:0/4005757175 wait complete. 2024-09-16T00:30:20.706 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:20 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/4005757175' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-16T00:30:20.706 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:20 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:20.706 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:20 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:20.706 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:20 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:20.706 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:20 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:30:20.706 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:20 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:20.707 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:20 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:30:20.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:20 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/4005757175' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-16T00:30:20.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:20 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:20.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:20 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:20.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:20 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:20.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:20 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:30:20.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:20 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:20.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:20 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:30:20.741 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-16T00:30:20.742 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-16T00:30:20.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:20 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/4005757175' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-09-16T00:30:20.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:20 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:20.795 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:20 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:20.795 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:20 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:20.795 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:20 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:30:20.795 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:20 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:20.795 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:20 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:30:21.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:21 smithi073 ceph-mon[35254]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:21.643 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:21.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:21 smithi038 ceph-mon[30945]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:21.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:21 smithi122 ceph-mon[35412]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:22.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.110+0000 7fefd6b8e640 1 -- 172.21.15.38:0/2189119167 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fefd010c170 msgr2=0x7fefd010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:22.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.110+0000 7fefd6b8e640 1 --2- 172.21.15.38:0/2189119167 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fefd010c170 0x7fefd010e610 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fefc8009ca0 tx=0x7fefc802a950 comp rx=0 tx=0).stop 2024-09-16T00:30:22.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.110+0000 7fefd6b8e640 1 -- 172.21.15.38:0/2189119167 shutdown_connections 2024-09-16T00:30:22.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.110+0000 7fefd6b8e640 1 --2- 172.21.15.38:0/2189119167 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fefd010c170 0x7fefd010e610 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.110+0000 7fefd6b8e640 1 --2- 172.21.15.38:0/2189119167 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefd0103660 0x7fefd010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.110+0000 7fefd6b8e640 1 --2- 172.21.15.38:0/2189119167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefd0102d40 0x7fefd0103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.110+0000 7fefd6b8e640 1 -- 172.21.15.38:0/2189119167 >> 172.21.15.38:0/2189119167 conn(0x7fefd00fc7a0 msgr2=0x7fefd00febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:22.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.110+0000 7fefd6b8e640 1 -- 172.21.15.38:0/2189119167 shutdown_connections 2024-09-16T00:30:22.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.110+0000 7fefd6b8e640 1 -- 172.21.15.38:0/2189119167 wait complete. 2024-09-16T00:30:22.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.110+0000 7fefd6b8e640 1 Processor -- start 2024-09-16T00:30:22.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.110+0000 7fefd6b8e640 1 -- start start 2024-09-16T00:30:22.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.111+0000 7fefd6b8e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefd0102d40 0x7fefd019a5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:22.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.111+0000 7fefd6b8e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefd0103660 0x7fefd019ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:22.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.111+0000 7fefd6b8e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fefd010c170 0x7fefd019eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:22.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.111+0000 7fefd6b8e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fefd0111f70 con 0x7fefd010c170 2024-09-16T00:30:22.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.111+0000 7fefd6b8e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fefd0111df0 con 0x7fefd0103660 2024-09-16T00:30:22.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.111+0000 7fefd6b8e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fefd01120f0 con 0x7fefd0102d40 2024-09-16T00:30:22.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.111+0000 7fefd512b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fefd010c170 0x7fefd019eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:22.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.111+0000 7fefd492a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefd0102d40 0x7fefd019a5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:22.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.111+0000 7fefd512b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fefd010c170 0x7fefd019eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46084/0 (socket says 172.21.15.38:46084) 2024-09-16T00:30:22.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.111+0000 7fefd492a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefd0102d40 0x7fefd019a5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.38:54916/0 (socket says 172.21.15.38:54916) 2024-09-16T00:30:22.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.111+0000 7fefd512b640 1 -- 172.21.15.38:0/4232101476 learned_addr learned my addr 172.21.15.38:0/4232101476 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:30:22.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.111+0000 7fefcffff640 1 --2- 172.21.15.38:0/4232101476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefd0103660 0x7fefd019ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:22.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.111+0000 7fefd512b640 1 -- 172.21.15.38:0/4232101476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefd0102d40 msgr2=0x7fefd019a5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:22.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.111+0000 7fefd512b640 1 --2- 172.21.15.38:0/4232101476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefd0102d40 0x7fefd019a5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.111+0000 7fefd512b640 1 -- 172.21.15.38:0/4232101476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefd0103660 msgr2=0x7fefd019ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:22.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.111+0000 7fefd512b640 1 --2- 172.21.15.38:0/4232101476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefd0103660 0x7fefd019ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.111+0000 7fefd512b640 1 -- 172.21.15.38:0/4232101476 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefd019f590 con 0x7fefd010c170 2024-09-16T00:30:22.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.111+0000 7fefd512b640 1 --2- 172.21.15.38:0/4232101476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fefd010c170 0x7fefd019eeb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fefc802ae70 tx=0x7fefc802f5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:22.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.111+0000 7fefcdffb640 1 -- 172.21.15.38:0/4232101476 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc8034040 con 0x7fefd010c170 2024-09-16T00:30:22.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.111+0000 7fefcdffb640 1 -- 172.21.15.38:0/4232101476 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fefc8030920 con 0x7fefd010c170 2024-09-16T00:30:22.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.111+0000 7fefd6b8e640 1 -- 172.21.15.38:0/4232101476 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefd019f820 con 0x7fefd010c170 2024-09-16T00:30:22.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.111+0000 7fefcdffb640 1 -- 172.21.15.38:0/4232101476 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefc8045480 con 0x7fefd010c170 2024-09-16T00:30:22.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.112+0000 7fefd6b8e640 1 -- 172.21.15.38:0/4232101476 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefd01a7160 con 0x7fefd010c170 2024-09-16T00:30:22.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.113+0000 7fefd6b8e640 1 -- 172.21.15.38:0/4232101476 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefd006b070 con 0x7fefd010c170 2024-09-16T00:30:22.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.114+0000 7fefcdffb640 1 -- 172.21.15.38:0/4232101476 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fefc8037020 con 0x7fefd010c170 2024-09-16T00:30:22.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.114+0000 7fefcdffb640 1 --2- 172.21.15.38:0/4232101476 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fefb80a3240 0x7fefb80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:22.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.114+0000 7fefd492a640 1 --2- 172.21.15.38:0/4232101476 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fefb80a3240 0x7fefb80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:22.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.114+0000 7fefcdffb640 1 -- 172.21.15.38:0/4232101476 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 7127+0+0 (secure 0 0 0) 0x7fefc80f0340 con 0x7fefd010c170 2024-09-16T00:30:22.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.115+0000 7fefd492a640 1 --2- 172.21.15.38:0/4232101476 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fefb80a3240 0x7fefb80a5700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fefc000bbe0 tx=0x7fefc000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:22.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.118+0000 7fefcdffb640 1 -- 172.21.15.38:0/4232101476 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fefc80bd4e0 con 0x7fefd010c170 2024-09-16T00:30:22.305 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.304+0000 7fefd6b8e640 1 -- 172.21.15.38:0/4232101476 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fefd01a7330 con 0x7fefd010c170 2024-09-16T00:30:22.906 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.905+0000 7fefcdffb640 1 -- 172.21.15.38:0/4232101476 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v78) ==== 138+0+0 (secure 0 0 0) 0x7fefc80bcde0 con 0x7fefd010c170 2024-09-16T00:30:22.906 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' created 2024-09-16T00:30:22.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.908+0000 7fefd6b8e640 1 -- 172.21.15.38:0/4232101476 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fefb80a3240 msgr2=0x7fefb80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:22.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.908+0000 7fefd6b8e640 1 --2- 172.21.15.38:0/4232101476 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fefb80a3240 0x7fefb80a5700 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fefc000bbe0 tx=0x7fefc000b390 comp rx=0 tx=0).stop 2024-09-16T00:30:22.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.908+0000 7fefd6b8e640 1 -- 172.21.15.38:0/4232101476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fefd010c170 msgr2=0x7fefd019eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:22.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.908+0000 7fefd6b8e640 1 --2- 172.21.15.38:0/4232101476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fefd010c170 0x7fefd019eeb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fefc802ae70 tx=0x7fefc802f5b0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.908+0000 7fefd6b8e640 1 -- 172.21.15.38:0/4232101476 shutdown_connections 2024-09-16T00:30:22.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.908+0000 7fefd6b8e640 1 --2- 172.21.15.38:0/4232101476 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fefb80a3240 0x7fefb80a5700 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.908+0000 7fefd6b8e640 1 --2- 172.21.15.38:0/4232101476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fefd010c170 0x7fefd019eeb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.908+0000 7fefd6b8e640 1 --2- 172.21.15.38:0/4232101476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fefd0103660 0x7fefd019ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.908+0000 7fefd6b8e640 1 --2- 172.21.15.38:0/4232101476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fefd0102d40 0x7fefd019a5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:22.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.908+0000 7fefd6b8e640 1 -- 172.21.15.38:0/4232101476 >> 172.21.15.38:0/4232101476 conn(0x7fefd00fc7a0 msgr2=0x7fefd00fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:22.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.908+0000 7fefd6b8e640 1 -- 172.21.15.38:0/4232101476 shutdown_connections 2024-09-16T00:30:22.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:22.908+0000 7fefd6b8e640 1 -- 172.21.15.38:0/4232101476 wait complete. 2024-09-16T00:30:23.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:22 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:23.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:22 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:23.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:22 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:23.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:22 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:23.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:22 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:23.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:22 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:23.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:22 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:23.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:22 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:23.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:22 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:23.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:22 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:23.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:22 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:23.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:22 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:23.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:22 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:23.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:22 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:23.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:22 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:23.645 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-09-16T00:30:23.907 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:23.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:23 smithi038 ceph-mon[30945]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:23.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:23 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/4232101476' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-16T00:30:23.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:23 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:23.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:23 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:23.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:23 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:23.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:23 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:30:23.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:23 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:23.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:23 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/4232101476' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-16T00:30:23.966 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:23 smithi038 ceph-mon[30945]: osdmap e78: 12 total, 12 up, 12 in 2024-09-16T00:30:24.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:23 smithi073 ceph-mon[35254]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:24.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:23 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/4232101476' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-16T00:30:24.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:23 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:24.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:23 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:24.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:23 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:24.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:23 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:30:24.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:23 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:24.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:23 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/4232101476' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-16T00:30:24.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:23 smithi073 ceph-mon[35254]: osdmap e78: 12 total, 12 up, 12 in 2024-09-16T00:30:24.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:23 smithi122 ceph-mon[35412]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:24.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:23 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/4232101476' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-09-16T00:30:24.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:23 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:24.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:23 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:24.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:23 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:30:24.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:23 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:30:24.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:23 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:30:24.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:23 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/4232101476' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-16T00:30:24.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:23 smithi122 ceph-mon[35412]: osdmap e78: 12 total, 12 up, 12 in 2024-09-16T00:30:24.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:24.421+0000 7f8e8cca5640 1 -- 172.21.15.38:0/274302487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e88102ef0 msgr2=0x7f8e8810da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:24.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:24.421+0000 7f8e8cca5640 1 --2- 172.21.15.38:0/274302487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e88102ef0 0x7f8e8810da40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f8e80009ca0 tx=0x7f8e8002a950 comp rx=0 tx=0).stop 2024-09-16T00:30:24.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:24.422+0000 7f8e8cca5640 1 -- 172.21.15.38:0/274302487 shutdown_connections 2024-09-16T00:30:24.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:24.422+0000 7f8e8cca5640 1 --2- 172.21.15.38:0/274302487 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e88102ef0 0x7f8e8810da40 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:24.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:24.422+0000 7f8e8cca5640 1 --2- 172.21.15.38:0/274302487 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e88102570 0x7f8e881029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:24.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:24.422+0000 7f8e8cca5640 1 --2- 172.21.15.38:0/274302487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e88100e20 0x7f8e88101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:24.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:24.422+0000 7f8e8cca5640 1 -- 172.21.15.38:0/274302487 >> 172.21.15.38:0/274302487 conn(0x7f8e880fc7d0 msgr2=0x7f8e880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:24.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:24.422+0000 7f8e8cca5640 1 -- 172.21.15.38:0/274302487 shutdown_connections 2024-09-16T00:30:24.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:24.422+0000 7f8e8cca5640 1 -- 172.21.15.38:0/274302487 wait complete. 2024-09-16T00:30:24.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:24.423+0000 7f8e8cca5640 1 Processor -- start 2024-09-16T00:30:24.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:24.423+0000 7f8e8cca5640 1 -- start start 2024-09-16T00:30:24.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:24.424+0000 7f8e8cca5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e88100e20 0x7f8e88116250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:24.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:24.424+0000 7f8e8cca5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e88102570 0x7f8e88116790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:24.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:24.424+0000 7f8e8cca5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e88102ef0 0x7f8e88111370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:24.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:24.424+0000 7f8e8cca5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8810fe40 con 0x7f8e88102ef0 2024-09-16T00:30:24.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:24.424+0000 7f8e8cca5640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8810fcc0 con 0x7f8e88100e20 2024-09-16T00:30:24.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:24.424+0000 7f8e8cca5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8e8810ffc0 con 0x7f8e88102570 2024-09-16T00:30:24.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:24.424+0000 7f8e86d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e88102ef0 0x7f8e88111370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:24.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:24.424+0000 7f8e86d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e88102ef0 0x7f8e88111370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46110/0 (socket says 172.21.15.38:46110) 2024-09-16T00:30:24.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:24.424+0000 7f8e86d9d640 1 -- 172.21.15.38:0/3344326263 learned_addr learned my addr 172.21.15.38:0/3344326263 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:30:24.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:24.424+0000 7f8e85d9b640 1 --2- 172.21.15.38:0/3344326263 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e88102570 0x7f8e88116790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:24.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:24.424+0000 7f8e8659c640 1 --2- 172.21.15.38:0/3344326263 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e88100e20 0x7f8e88116250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:24.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:24.424+0000 7f8e86d9d640 1 -- 172.21.15.38:0/3344326263 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e88102570 msgr2=0x7f8e88116790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:24.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:24.424+0000 7f8e86d9d640 1 --2- 172.21.15.38:0/3344326263 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e88102570 0x7f8e88116790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:24.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:24.424+0000 7f8e86d9d640 1 -- 172.21.15.38:0/3344326263 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e88100e20 msgr2=0x7f8e88116250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:24.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:24.424+0000 7f8e86d9d640 1 --2- 172.21.15.38:0/3344326263 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e88100e20 0x7f8e88116250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:24.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:24.424+0000 7f8e86d9d640 1 -- 172.21.15.38:0/3344326263 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e88111c30 con 0x7f8e88102ef0 2024-09-16T00:30:24.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:24.424+0000 7f8e86d9d640 1 --2- 172.21.15.38:0/3344326263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e88102ef0 0x7f8e88111370 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8e80004af0 tx=0x7f8e8002f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:24.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:24.424+0000 7f8e777fe640 1 -- 172.21.15.38:0/3344326263 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e80034040 con 0x7f8e88102ef0 2024-09-16T00:30:24.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:24.424+0000 7f8e777fe640 1 -- 172.21.15.38:0/3344326263 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8e80030920 con 0x7f8e88102ef0 2024-09-16T00:30:24.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:24.424+0000 7f8e8cca5640 1 -- 172.21.15.38:0/3344326263 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e88111ec0 con 0x7f8e88102ef0 2024-09-16T00:30:24.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:24.424+0000 7f8e777fe640 1 -- 172.21.15.38:0/3344326263 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e80004630 con 0x7f8e88102ef0 2024-09-16T00:30:24.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:24.425+0000 7f8e8cca5640 1 -- 172.21.15.38:0/3344326263 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e8811c140 con 0x7f8e88102ef0 2024-09-16T00:30:24.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:24.426+0000 7f8e8cca5640 1 -- 172.21.15.38:0/3344326263 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e881021b0 con 0x7f8e88102ef0 2024-09-16T00:30:24.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:24.427+0000 7f8e777fe640 1 -- 172.21.15.38:0/3344326263 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f8e80037020 con 0x7f8e88102ef0 2024-09-16T00:30:24.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:24.427+0000 7f8e777fe640 1 --2- 172.21.15.38:0/3344326263 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f8e700a3240 0x7f8e700a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:24.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:24.427+0000 7f8e8659c640 1 --2- 172.21.15.38:0/3344326263 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f8e700a3240 0x7f8e700a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:24.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:24.428+0000 7f8e777fe640 1 -- 172.21.15.38:0/3344326263 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7497+0+0 (secure 0 0 0) 0x7f8e800f0300 con 0x7f8e88102ef0 2024-09-16T00:30:24.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:24.428+0000 7f8e8659c640 1 --2- 172.21.15.38:0/3344326263 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f8e700a3240 0x7f8e700a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8e78009bd0 tx=0x7f8e78005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:24.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:24.432+0000 7f8e777fe640 1 -- 172.21.15.38:0/3344326263 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8e800bd350 con 0x7f8e88102ef0 2024-09-16T00:30:24.609 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:24.608+0000 7f8e8cca5640 1 -- 172.21.15.38:0/3344326263 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f8e881128a0 con 0x7f8e88102ef0 2024-09-16T00:30:25.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:25.051+0000 7f8e777fe640 1 -- 172.21.15.38:0/3344326263 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v80) ==== 171+0+0 (secure 0 0 0) 0x7f8e800bcc50 con 0x7f8e88102ef0 2024-09-16T00:30:25.052 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' created 2024-09-16T00:30:25.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:25.054+0000 7f8e8cca5640 1 -- 172.21.15.38:0/3344326263 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f8e700a3240 msgr2=0x7f8e700a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:25.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:25.054+0000 7f8e8cca5640 1 --2- 172.21.15.38:0/3344326263 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f8e700a3240 0x7f8e700a5700 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8e78009bd0 tx=0x7f8e78005f90 comp rx=0 tx=0).stop 2024-09-16T00:30:25.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:25.054+0000 7f8e8cca5640 1 -- 172.21.15.38:0/3344326263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e88102ef0 msgr2=0x7f8e88111370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:25.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:25.054+0000 7f8e8cca5640 1 --2- 172.21.15.38:0/3344326263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e88102ef0 0x7f8e88111370 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8e80004af0 tx=0x7f8e8002f3a0 comp rx=0 tx=0).stop 2024-09-16T00:30:25.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:25.054+0000 7f8e8cca5640 1 -- 172.21.15.38:0/3344326263 shutdown_connections 2024-09-16T00:30:25.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:25.054+0000 7f8e8cca5640 1 --2- 172.21.15.38:0/3344326263 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f8e700a3240 0x7f8e700a5700 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:25.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:25.054+0000 7f8e8cca5640 1 --2- 172.21.15.38:0/3344326263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8e88102ef0 0x7f8e88111370 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:25.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:25.054+0000 7f8e8cca5640 1 --2- 172.21.15.38:0/3344326263 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8e88102570 0x7f8e88116790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:25.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:25.054+0000 7f8e8cca5640 1 --2- 172.21.15.38:0/3344326263 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8e88100e20 0x7f8e88116250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:25.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:25.054+0000 7f8e8cca5640 1 -- 172.21.15.38:0/3344326263 >> 172.21.15.38:0/3344326263 conn(0x7f8e880fc7d0 msgr2=0x7f8e88105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:25.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:25.054+0000 7f8e8cca5640 1 -- 172.21.15.38:0/3344326263 shutdown_connections 2024-09-16T00:30:25.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:25.054+0000 7f8e8cca5640 1 -- 172.21.15.38:0/3344326263 wait complete. 2024-09-16T00:30:25.437 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:25 smithi038 ceph-mon[30945]: pgmap v279: 33 pgs: 11 creating+peering, 21 unknown, 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:25.437 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:25 smithi038 ceph-mon[30945]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T00:30:25.437 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:25 smithi038 ceph-mon[30945]: osdmap e79: 12 total, 12 up, 12 in 2024-09-16T00:30:25.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:25 smithi122 ceph-mon[35412]: pgmap v279: 33 pgs: 11 creating+peering, 21 unknown, 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:25.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:25 smithi122 ceph-mon[35412]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T00:30:25.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:25 smithi122 ceph-mon[35412]: osdmap e79: 12 total, 12 up, 12 in 2024-09-16T00:30:25.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:25 smithi073 ceph-mon[35254]: pgmap v279: 33 pgs: 11 creating+peering, 21 unknown, 1 active+clean; 577 KiB data, 364 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:25.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:25 smithi073 ceph-mon[35254]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T00:30:25.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:25 smithi073 ceph-mon[35254]: osdmap e79: 12 total, 12 up, 12 in 2024-09-16T00:30:25.801 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-16T00:30:26.060 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:26.326 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:26 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/3344326263' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-16T00:30:26.327 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:26 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/3344326263' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-16T00:30:26.327 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:26 smithi038 ceph-mon[30945]: osdmap e80: 12 total, 12 up, 12 in 2024-09-16T00:30:26.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:26 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/3344326263' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-16T00:30:26.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:26 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/3344326263' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-16T00:30:26.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:26 smithi122 ceph-mon[35412]: osdmap e80: 12 total, 12 up, 12 in 2024-09-16T00:30:26.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:26 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/3344326263' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-09-16T00:30:26.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:26 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/3344326263' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-09-16T00:30:26.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:26 smithi073 ceph-mon[35254]: osdmap e80: 12 total, 12 up, 12 in 2024-09-16T00:30:26.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.542+0000 7fb0ac806640 1 -- 172.21.15.38:0/3018772159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0a81036c0 msgr2=0x7fb0a810bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:26.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.542+0000 7fb0ac806640 1 --2- 172.21.15.38:0/3018772159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0a81036c0 0x7fb0a810bbb0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb090009900 tx=0x7fb09002a480 comp rx=0 tx=0).stop 2024-09-16T00:30:26.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.543+0000 7fb0ac806640 1 -- 172.21.15.38:0/3018772159 shutdown_connections 2024-09-16T00:30:26.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.543+0000 7fb0ac806640 1 --2- 172.21.15.38:0/3018772159 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0a810c180 0x7fb0a810e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:26.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.543+0000 7fb0ac806640 1 --2- 172.21.15.38:0/3018772159 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0a81036c0 0x7fb0a810bbb0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:26.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.543+0000 7fb0ac806640 1 --2- 172.21.15.38:0/3018772159 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0a8102da0 0x7fb0a8103180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:26.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.543+0000 7fb0ac806640 1 -- 172.21.15.38:0/3018772159 >> 172.21.15.38:0/3018772159 conn(0x7fb0a80fc7a0 msgr2=0x7fb0a80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:26.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.543+0000 7fb0ac806640 1 -- 172.21.15.38:0/3018772159 shutdown_connections 2024-09-16T00:30:26.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.543+0000 7fb0ac806640 1 -- 172.21.15.38:0/3018772159 wait complete. 2024-09-16T00:30:26.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.543+0000 7fb0ac806640 1 Processor -- start 2024-09-16T00:30:26.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.544+0000 7fb0ac806640 1 -- start start 2024-09-16T00:30:26.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.544+0000 7fb0ac806640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0a8102da0 0x7fb0a8198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:26.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.544+0000 7fb0ac806640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0a81036c0 0x7fb0a8198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:26.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.544+0000 7fb0ac806640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0a810c180 0x7fb0a819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:26.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.544+0000 7fb0ac806640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a8111f60 con 0x7fb0a8102da0 2024-09-16T00:30:26.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.544+0000 7fb0ac806640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a8111de0 con 0x7fb0a81036c0 2024-09-16T00:30:26.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.544+0000 7fb0ac806640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb0a81120e0 con 0x7fb0a810c180 2024-09-16T00:30:26.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.544+0000 7fb0a659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0a8102da0 0x7fb0a8198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:26.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.544+0000 7fb0a659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0a8102da0 0x7fb0a8198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50174/0 (socket says 172.21.15.38:50174) 2024-09-16T00:30:26.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.544+0000 7fb0a659c640 1 -- 172.21.15.38:0/2745639145 learned_addr learned my addr 172.21.15.38:0/2745639145 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:30:26.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.544+0000 7fb09dd9b640 1 --2- 172.21.15.38:0/2745639145 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0a81036c0 0x7fb0a8198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:26.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.544+0000 7fb0a6d9d640 1 --2- 172.21.15.38:0/2745639145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0a810c180 0x7fb0a819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:26.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.544+0000 7fb0a659c640 1 -- 172.21.15.38:0/2745639145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0a810c180 msgr2=0x7fb0a819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:26.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.544+0000 7fb0a659c640 1 --2- 172.21.15.38:0/2745639145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0a810c180 0x7fb0a819cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:26.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.544+0000 7fb0a659c640 1 -- 172.21.15.38:0/2745639145 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0a81036c0 msgr2=0x7fb0a8198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:26.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.544+0000 7fb0a659c640 1 --2- 172.21.15.38:0/2745639145 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0a81036c0 0x7fb0a8198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:26.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.544+0000 7fb0a659c640 1 -- 172.21.15.38:0/2745639145 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0a819d3f0 con 0x7fb0a8102da0 2024-09-16T00:30:26.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.544+0000 7fb0a6d9d640 1 --2- 172.21.15.38:0/2745639145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0a810c180 0x7fb0a819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:26.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.544+0000 7fb0a659c640 1 --2- 172.21.15.38:0/2745639145 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0a8102da0 0x7fb0a8198440 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb09800d950 tx=0x7fb09800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:26.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.545+0000 7fb09ffff640 1 -- 172.21.15.38:0/2745639145 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb098017070 con 0x7fb0a8102da0 2024-09-16T00:30:26.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.545+0000 7fb0ac806640 1 -- 172.21.15.38:0/2745639145 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0a819d6e0 con 0x7fb0a8102da0 2024-09-16T00:30:26.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.545+0000 7fb09ffff640 1 -- 172.21.15.38:0/2745639145 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb098009c30 con 0x7fb0a8102da0 2024-09-16T00:30:26.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.545+0000 7fb09ffff640 1 -- 172.21.15.38:0/2745639145 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb09800f6b0 con 0x7fb0a8102da0 2024-09-16T00:30:26.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.545+0000 7fb0ac806640 1 -- 172.21.15.38:0/2745639145 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0a81a4fc0 con 0x7fb0a8102da0 2024-09-16T00:30:26.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.546+0000 7fb0ac806640 1 -- 172.21.15.38:0/2745639145 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0a8103180 con 0x7fb0a8102da0 2024-09-16T00:30:26.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.547+0000 7fb09ffff640 1 -- 172.21.15.38:0/2745639145 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fb0980109d0 con 0x7fb0a8102da0 2024-09-16T00:30:26.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.548+0000 7fb09ffff640 1 --2- 172.21.15.38:0/2745639145 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb0940a3240 0x7fb0940a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:26.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.548+0000 7fb09dd9b640 1 --2- 172.21.15.38:0/2745639145 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb0940a3240 0x7fb0940a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:26.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.548+0000 7fb09ffff640 1 -- 172.21.15.38:0/2745639145 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7879+0+0 (secure 0 0 0) 0x7fb0980cf420 con 0x7fb0a8102da0 2024-09-16T00:30:26.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.548+0000 7fb09dd9b640 1 --2- 172.21.15.38:0/2745639145 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb0940a3240 0x7fb0940a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb090009230 tx=0x7fb0900058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:26.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.551+0000 7fb09ffff640 1 -- 172.21.15.38:0/2745639145 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb098009de0 con 0x7fb0a8102da0 2024-09-16T00:30:26.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:26.763+0000 7fb0ac806640 1 -- 172.21.15.38:0/2745639145 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fb0a81997a0 con 0x7fb0a8102da0 2024-09-16T00:30:27.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:27.060+0000 7fb09ffff640 1 -- 172.21.15.38:0/2745639145 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7fb09809c250 con 0x7fb0a8102da0 2024-09-16T00:30:27.061 INFO:teuthology.orchestra.run.smithi038.stderr:new fs with metadata pool 2 and data pool 3 2024-09-16T00:30:27.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:27.062+0000 7fb0ac806640 1 -- 172.21.15.38:0/2745639145 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb0940a3240 msgr2=0x7fb0940a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:27.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:27.062+0000 7fb0ac806640 1 --2- 172.21.15.38:0/2745639145 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb0940a3240 0x7fb0940a5700 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fb090009230 tx=0x7fb0900058d0 comp rx=0 tx=0).stop 2024-09-16T00:30:27.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:27.062+0000 7fb0ac806640 1 -- 172.21.15.38:0/2745639145 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0a8102da0 msgr2=0x7fb0a8198440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:27.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:27.062+0000 7fb0ac806640 1 --2- 172.21.15.38:0/2745639145 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0a8102da0 0x7fb0a8198440 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fb09800d950 tx=0x7fb09800de10 comp rx=0 tx=0).stop 2024-09-16T00:30:27.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:27.062+0000 7fb0ac806640 1 -- 172.21.15.38:0/2745639145 shutdown_connections 2024-09-16T00:30:27.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:27.062+0000 7fb0ac806640 1 --2- 172.21.15.38:0/2745639145 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb0940a3240 0x7fb0940a5700 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:27.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:27.062+0000 7fb0ac806640 1 --2- 172.21.15.38:0/2745639145 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb0a810c180 0x7fb0a819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:27.063 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:27.062+0000 7fb0ac806640 1 --2- 172.21.15.38:0/2745639145 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0a81036c0 0x7fb0a8198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:27.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:27.062+0000 7fb0ac806640 1 --2- 172.21.15.38:0/2745639145 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb0a8102da0 0x7fb0a8198440 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:27.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:27.062+0000 7fb0ac806640 1 -- 172.21.15.38:0/2745639145 >> 172.21.15.38:0/2745639145 conn(0x7fb0a80fc7a0 msgr2=0x7fb0a80fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:27.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:27.063+0000 7fb0ac806640 1 -- 172.21.15.38:0/2745639145 shutdown_connections 2024-09-16T00:30:27.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:27.063+0000 7fb0ac806640 1 -- 172.21.15.38:0/2745639145 wait complete. 2024-09-16T00:30:27.437 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:27 smithi038 ceph-mon[30945]: pgmap v282: 97 pgs: 8 active+clean, 42 creating+peering, 47 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:27.437 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:27 smithi038 ceph-mon[30945]: osdmap e81: 12 total, 12 up, 12 in 2024-09-16T00:30:27.437 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:27 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/2745639145' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-16T00:30:27.437 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:27 smithi038 ceph-mon[30945]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-16T00:30:27.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:27 smithi122 ceph-mon[35412]: pgmap v282: 97 pgs: 8 active+clean, 42 creating+peering, 47 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:27.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:27 smithi122 ceph-mon[35412]: osdmap e81: 12 total, 12 up, 12 in 2024-09-16T00:30:27.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:27 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/2745639145' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-16T00:30:27.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:27 smithi122 ceph-mon[35412]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-16T00:30:27.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:27 smithi073 ceph-mon[35254]: pgmap v282: 97 pgs: 8 active+clean, 42 creating+peering, 47 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:27.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:27 smithi073 ceph-mon[35254]: osdmap e81: 12 total, 12 up, 12 in 2024-09-16T00:30:27.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:27 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/2745639145' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-09-16T00:30:27.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:27 smithi073 ceph-mon[35254]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-16T00:30:27.765 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-09-16T00:30:27.765 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-09-16T00:30:28.028 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:28.334 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:28 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/2745639145' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-16T00:30:28.334 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:28 smithi038 ceph-mon[30945]: osdmap e82: 12 total, 12 up, 12 in 2024-09-16T00:30:28.334 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:28 smithi038 ceph-mon[30945]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-16T00:30:28.334 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:28 smithi038 ceph-mon[30945]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-16T00:30:28.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:28 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/2745639145' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-16T00:30:28.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:28 smithi122 ceph-mon[35412]: osdmap e82: 12 total, 12 up, 12 in 2024-09-16T00:30:28.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:28 smithi122 ceph-mon[35412]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-16T00:30:28.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:28 smithi122 ceph-mon[35412]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-16T00:30:28.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:28 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/2745639145' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-16T00:30:28.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:28 smithi073 ceph-mon[35254]: osdmap e82: 12 total, 12 up, 12 in 2024-09-16T00:30:28.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:28 smithi073 ceph-mon[35254]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-16T00:30:28.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:28 smithi073 ceph-mon[35254]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-16T00:30:28.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:28.545+0000 7f9179a13640 1 -- 172.21.15.38:0/4285799407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9174101420 msgr2=0x7f917410d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:28.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:28.545+0000 7f9179a13640 1 --2- 172.21.15.38:0/4285799407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9174101420 0x7f917410d950 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f916c009cd0 tx=0x7f916c02a8e0 comp rx=0 tx=0).stop 2024-09-16T00:30:28.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:28.545+0000 7f9179a13640 1 -- 172.21.15.38:0/4285799407 shutdown_connections 2024-09-16T00:30:28.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:28.545+0000 7f9179a13640 1 --2- 172.21.15.38:0/4285799407 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9174101420 0x7f917410d950 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:28.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:28.545+0000 7f9179a13640 1 --2- 172.21.15.38:0/4285799407 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9174100a80 0x7f9174100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:28.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:28.545+0000 7f9179a13640 1 --2- 172.21.15.38:0/4285799407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9174106a80 0x7f9174106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:28.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:28.545+0000 7f9179a13640 1 -- 172.21.15.38:0/4285799407 >> 172.21.15.38:0/4285799407 conn(0x7f91740fc7a0 msgr2=0x7f91740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:28.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:28.545+0000 7f9179a13640 1 -- 172.21.15.38:0/4285799407 shutdown_connections 2024-09-16T00:30:28.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:28.545+0000 7f9179a13640 1 -- 172.21.15.38:0/4285799407 wait complete. 2024-09-16T00:30:28.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:28.546+0000 7f9179a13640 1 Processor -- start 2024-09-16T00:30:28.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:28.546+0000 7f9179a13640 1 -- start start 2024-09-16T00:30:28.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:28.546+0000 7f9179a13640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9174100a80 0x7f917410e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:28.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:28.546+0000 7f9179a13640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9174101420 0x7f917410e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:28.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:28.546+0000 7f9179a13640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9174106a80 0x7f917410ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:28.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:28.546+0000 7f9179a13640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9174111fa0 con 0x7f9174106a80 2024-09-16T00:30:28.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:28.546+0000 7f9179a13640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9174111e20 con 0x7f9174100a80 2024-09-16T00:30:28.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:28.546+0000 7f9179a13640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9174112120 con 0x7f9174101420 2024-09-16T00:30:28.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:28.546+0000 7f91737fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9174106a80 0x7f917410ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:28.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:28.546+0000 7f91737fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9174106a80 0x7f917410ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50184/0 (socket says 172.21.15.38:50184) 2024-09-16T00:30:28.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:28.546+0000 7f91737fe640 1 -- 172.21.15.38:0/1599176884 learned_addr learned my addr 172.21.15.38:0/1599176884 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:30:28.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:28.546+0000 7f9172ffd640 1 --2- 172.21.15.38:0/1599176884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9174100a80 0x7f917410e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:28.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:28.546+0000 7f916bfff640 1 --2- 172.21.15.38:0/1599176884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9174101420 0x7f917410e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:28.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:28.546+0000 7f91737fe640 1 -- 172.21.15.38:0/1599176884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9174101420 msgr2=0x7f917410e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:28.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:28.546+0000 7f91737fe640 1 --2- 172.21.15.38:0/1599176884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9174101420 0x7f917410e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:28.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:28.546+0000 7f91737fe640 1 -- 172.21.15.38:0/1599176884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9174100a80 msgr2=0x7f917410e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:28.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:28.546+0000 7f91737fe640 1 --2- 172.21.15.38:0/1599176884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9174100a80 0x7f917410e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:28.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:28.546+0000 7f91737fe640 1 -- 172.21.15.38:0/1599176884 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91741a6870 con 0x7f9174106a80 2024-09-16T00:30:28.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:28.547+0000 7f91737fe640 1 --2- 172.21.15.38:0/1599176884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9174106a80 0x7f917410ee60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f916c005cc0 tx=0x7f916c003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:28.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:28.547+0000 7f9178a11640 1 -- 172.21.15.38:0/1599176884 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916c004040 con 0x7f9174106a80 2024-09-16T00:30:28.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:28.547+0000 7f9178a11640 1 -- 172.21.15.38:0/1599176884 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f916c0046c0 con 0x7f9174106a80 2024-09-16T00:30:28.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:28.547+0000 7f9179a13640 1 -- 172.21.15.38:0/1599176884 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91741a6b00 con 0x7f9174106a80 2024-09-16T00:30:28.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:28.547+0000 7f9178a11640 1 -- 172.21.15.38:0/1599176884 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f916c046480 con 0x7f9174106a80 2024-09-16T00:30:28.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:28.547+0000 7f9179a13640 1 -- 172.21.15.38:0/1599176884 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91741a7040 con 0x7f9174106a80 2024-09-16T00:30:28.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:28.548+0000 7f9179a13640 1 -- 172.21.15.38:0/1599176884 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9174100ee0 con 0x7f9174106a80 2024-09-16T00:30:28.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:28.549+0000 7f9178a11640 1 -- 172.21.15.38:0/1599176884 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f916c0305a0 con 0x7f9174106a80 2024-09-16T00:30:28.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:28.550+0000 7f9178a11640 1 --2- 172.21.15.38:0/1599176884 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f91600a3240 0x7f91600a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:28.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:28.550+0000 7f9172ffd640 1 --2- 172.21.15.38:0/1599176884 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f91600a3240 0x7f91600a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:28.560 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:28.550+0000 7f9178a11640 1 -- 172.21.15.38:0/1599176884 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7979+0+0 (secure 0 0 0) 0x7f916c02e030 con 0x7f9174106a80 2024-09-16T00:30:28.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:28.551+0000 7f9172ffd640 1 --2- 172.21.15.38:0/1599176884 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f91600a3240 0x7f91600a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f916400cac0 tx=0x7f916400c4a0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:28.561 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:28.553+0000 7f9178a11640 1 -- 172.21.15.38:0/1599176884 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f916c04b060 con 0x7f9174106a80 2024-09-16T00:30:28.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:28.735+0000 7f9179a13640 1 -- 172.21.15.38:0/1599176884 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f91741a7300 con 0x7f9174106a80 2024-09-16T00:30:29.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:29.100+0000 7f9178a11640 1 -- 172.21.15.38:0/1599176884 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v83) ==== 157+0+0 (secure 0 0 0) 0x7f916c0bde50 con 0x7f9174106a80 2024-09-16T00:30:29.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:29.103+0000 7f9179a13640 1 -- 172.21.15.38:0/1599176884 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f91600a3240 msgr2=0x7f91600a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:29.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:29.103+0000 7f9179a13640 1 --2- 172.21.15.38:0/1599176884 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f91600a3240 0x7f91600a5700 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f916400cac0 tx=0x7f916400c4a0 comp rx=0 tx=0).stop 2024-09-16T00:30:29.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:29.103+0000 7f9179a13640 1 -- 172.21.15.38:0/1599176884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9174106a80 msgr2=0x7f917410ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:29.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:29.103+0000 7f9179a13640 1 --2- 172.21.15.38:0/1599176884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9174106a80 0x7f917410ee60 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f916c005cc0 tx=0x7f916c003970 comp rx=0 tx=0).stop 2024-09-16T00:30:29.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:29.103+0000 7f9179a13640 1 -- 172.21.15.38:0/1599176884 shutdown_connections 2024-09-16T00:30:29.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:29.103+0000 7f9179a13640 1 --2- 172.21.15.38:0/1599176884 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f91600a3240 0x7f91600a5700 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:29.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:29.103+0000 7f9179a13640 1 --2- 172.21.15.38:0/1599176884 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9174106a80 0x7f917410ee60 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:29.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:29.103+0000 7f9179a13640 1 --2- 172.21.15.38:0/1599176884 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9174101420 0x7f917410e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:29.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:29.103+0000 7f9179a13640 1 --2- 172.21.15.38:0/1599176884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9174100a80 0x7f917410e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:29.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:29.103+0000 7f9179a13640 1 -- 172.21.15.38:0/1599176884 >> 172.21.15.38:0/1599176884 conn(0x7f91740fc7a0 msgr2=0x7f91740fceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:29.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:29.103+0000 7f9179a13640 1 -- 172.21.15.38:0/1599176884 shutdown_connections 2024-09-16T00:30:29.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:29.103+0000 7f9179a13640 1 -- 172.21.15.38:0/1599176884 wait complete. 2024-09-16T00:30:29.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:29 smithi038 ceph-mon[30945]: pgmap v285: 97 pgs: 35 active+clean, 42 creating+peering, 20 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:29.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:29 smithi038 ceph-mon[30945]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-16T00:30:29.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:29 smithi038 ceph-mon[30945]: Cluster is now healthy 2024-09-16T00:30:29.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:29 smithi038 ceph-mon[30945]: mds.? [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] up:active 2024-09-16T00:30:29.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:29 smithi038 ceph-mon[30945]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:30:29.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:29 smithi038 ceph-mon[30945]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:30:29.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:29 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/1599176884' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-16T00:30:29.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:29 smithi122 ceph-mon[35412]: pgmap v285: 97 pgs: 35 active+clean, 42 creating+peering, 20 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:29.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:29 smithi122 ceph-mon[35412]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-16T00:30:29.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:29 smithi122 ceph-mon[35412]: Cluster is now healthy 2024-09-16T00:30:29.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:29 smithi122 ceph-mon[35412]: mds.? [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] up:active 2024-09-16T00:30:29.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:29 smithi122 ceph-mon[35412]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:30:29.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:29 smithi122 ceph-mon[35412]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:30:29.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:29 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/1599176884' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-16T00:30:29.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:29 smithi073 ceph-mon[35254]: pgmap v285: 97 pgs: 35 active+clean, 42 creating+peering, 20 unknown; 577 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:29.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:29 smithi073 ceph-mon[35254]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-09-16T00:30:29.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:29 smithi073 ceph-mon[35254]: Cluster is now healthy 2024-09-16T00:30:29.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:29 smithi073 ceph-mon[35254]: mds.? [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] up:active 2024-09-16T00:30:29.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:29 smithi073 ceph-mon[35254]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:30:29.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:29 smithi073 ceph-mon[35254]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:30:29.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:29 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/1599176884' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-09-16T00:30:29.832 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-09-16T00:30:30.096 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:30.379 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:30 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/1599176884' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-16T00:30:30.379 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:30 smithi038 ceph-mon[30945]: osdmap e83: 12 total, 12 up, 12 in 2024-09-16T00:30:30.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:30 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/1599176884' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-16T00:30:30.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:30 smithi122 ceph-mon[35412]: osdmap e83: 12 total, 12 up, 12 in 2024-09-16T00:30:30.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:30 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/1599176884' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-09-16T00:30:30.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:30 smithi073 ceph-mon[35254]: osdmap e83: 12 total, 12 up, 12 in 2024-09-16T00:30:30.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:30.571+0000 7f20cf28c640 1 -- 172.21.15.38:0/1504939603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20c8102790 msgr2=0x7f20c81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:30.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:30.571+0000 7f20cf28c640 1 --2- 172.21.15.38:0/1504939603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20c8102790 0x7f20c81067e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f20b8009900 tx=0x7f20b802a410 comp rx=0 tx=0).stop 2024-09-16T00:30:30.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:30.571+0000 7f20cf28c640 1 -- 172.21.15.38:0/1504939603 shutdown_connections 2024-09-16T00:30:30.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:30.571+0000 7f20cf28c640 1 --2- 172.21.15.38:0/1504939603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20c8106f10 0x7f20c810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:30.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:30.571+0000 7f20cf28c640 1 --2- 172.21.15.38:0/1504939603 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20c8102790 0x7f20c81067e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:30.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:30.571+0000 7f20cf28c640 1 --2- 172.21.15.38:0/1504939603 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20c8101de0 0x7f20c81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:30.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:30.571+0000 7f20cf28c640 1 -- 172.21.15.38:0/1504939603 >> 172.21.15.38:0/1504939603 conn(0x7f20c80fd8c0 msgr2=0x7f20c80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:30.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:30.571+0000 7f20cf28c640 1 -- 172.21.15.38:0/1504939603 shutdown_connections 2024-09-16T00:30:30.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:30.572+0000 7f20cf28c640 1 -- 172.21.15.38:0/1504939603 wait complete. 2024-09-16T00:30:30.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:30.572+0000 7f20cf28c640 1 Processor -- start 2024-09-16T00:30:30.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:30.572+0000 7f20cf28c640 1 -- start start 2024-09-16T00:30:30.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:30.573+0000 7f20cf28c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20c8101de0 0x7f20c81983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:30.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:30.573+0000 7f20cf28c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20c8102790 0x7f20c8198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:30.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:30.573+0000 7f20cf28c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20c8106f10 0x7f20c819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:30.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:30.573+0000 7f20cf28c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f20c8111f70 con 0x7f20c8106f10 2024-09-16T00:30:30.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:30.573+0000 7f20cf28c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f20c8111df0 con 0x7f20c8101de0 2024-09-16T00:30:30.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:30.573+0000 7f20cf28c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f20c81120f0 con 0x7f20c8102790 2024-09-16T00:30:30.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:30.573+0000 7f20cd829640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20c8106f10 0x7f20c819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:30.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:30.573+0000 7f20cd829640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20c8106f10 0x7f20c819cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50214/0 (socket says 172.21.15.38:50214) 2024-09-16T00:30:30.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:30.573+0000 7f20cd829640 1 -- 172.21.15.38:0/2357227852 learned_addr learned my addr 172.21.15.38:0/2357227852 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:30:30.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:30.573+0000 7f20cc827640 1 --2- 172.21.15.38:0/2357227852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20c8102790 0x7f20c8198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:30.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:30.573+0000 7f20cd028640 1 --2- 172.21.15.38:0/2357227852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20c8101de0 0x7f20c81983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:30.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:30.573+0000 7f20cd829640 1 -- 172.21.15.38:0/2357227852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20c8102790 msgr2=0x7f20c8198910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:30.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:30.573+0000 7f20cd829640 1 --2- 172.21.15.38:0/2357227852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20c8102790 0x7f20c8198910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:30.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:30.573+0000 7f20cd829640 1 -- 172.21.15.38:0/2357227852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20c8101de0 msgr2=0x7f20c81983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:30.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:30.573+0000 7f20cd829640 1 --2- 172.21.15.38:0/2357227852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20c8101de0 0x7f20c81983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:30.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:30.573+0000 7f20cd829640 1 -- 172.21.15.38:0/2357227852 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20c819d380 con 0x7f20c8106f10 2024-09-16T00:30:30.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:30.573+0000 7f20cd829640 1 --2- 172.21.15.38:0/2357227852 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20c8106f10 0x7f20c819cca0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f20c000cce0 tx=0x7f20c000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:30.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:30.573+0000 7f20be7fc640 1 -- 172.21.15.38:0/2357227852 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c000f450 con 0x7f20c8106f10 2024-09-16T00:30:30.585 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:30.573+0000 7f20be7fc640 1 -- 172.21.15.38:0/2357227852 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20c000fad0 con 0x7f20c8106f10 2024-09-16T00:30:30.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:30.573+0000 7f20cf28c640 1 -- 172.21.15.38:0/2357227852 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20c819d670 con 0x7f20c8106f10 2024-09-16T00:30:30.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:30.573+0000 7f20be7fc640 1 -- 172.21.15.38:0/2357227852 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c0018c10 con 0x7f20c8106f10 2024-09-16T00:30:30.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:30.574+0000 7f20cf28c640 1 -- 172.21.15.38:0/2357227852 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20c81a4f50 con 0x7f20c8106f10 2024-09-16T00:30:30.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:30.575+0000 7f20cf28c640 1 -- 172.21.15.38:0/2357227852 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20c81021c0 con 0x7f20c8106f10 2024-09-16T00:30:30.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:30.576+0000 7f20be7fc640 1 -- 172.21.15.38:0/2357227852 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f20c000f5f0 con 0x7f20c8106f10 2024-09-16T00:30:30.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:30.576+0000 7f20be7fc640 1 --2- 172.21.15.38:0/2357227852 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f20b40a3240 0x7f20b40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:30.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:30.577+0000 7f20cd028640 1 --2- 172.21.15.38:0/2357227852 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f20b40a3240 0x7f20b40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:30.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:30.577+0000 7f20be7fc640 1 -- 172.21.15.38:0/2357227852 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 8277+0+0 (secure 0 0 0) 0x7f20c00cffe0 con 0x7f20c8106f10 2024-09-16T00:30:30.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:30.577+0000 7f20cd028640 1 --2- 172.21.15.38:0/2357227852 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f20b40a3240 0x7f20b40a5700 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f20c4005fd0 tx=0x7f20c4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:30.586 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:30.580+0000 7f20be7fc640 1 -- 172.21.15.38:0/2357227852 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f20c0014030 con 0x7f20c8106f10 2024-09-16T00:30:30.755 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:30.755+0000 7f20cf28c640 1 -- 172.21.15.38:0/2357227852 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f20c8199730 con 0x7f20c8106f10 2024-09-16T00:30:31.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:31 smithi038 ceph-mon[30945]: pgmap v287: 97 pgs: 62 active+clean, 28 creating+peering, 7 unknown; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-09-16T00:30:31.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:31 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/2357227852' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-16T00:30:31.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:31 smithi122 ceph-mon[35412]: pgmap v287: 97 pgs: 62 active+clean, 28 creating+peering, 7 unknown; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-09-16T00:30:31.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:31 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/2357227852' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-16T00:30:31.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:31 smithi073 ceph-mon[35254]: pgmap v287: 97 pgs: 62 active+clean, 28 creating+peering, 7 unknown; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-09-16T00:30:31.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:31 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/2357227852' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-09-16T00:30:32.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:32.128+0000 7f20be7fc640 1 -- 172.21.15.38:0/2357227852 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v85) ==== 228+0+0 (secure 0 0 0) 0x7f20c8199730 con 0x7f20c8106f10 2024-09-16T00:30:32.129 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data_ec' created 2024-09-16T00:30:32.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:32.130+0000 7f20cf28c640 1 -- 172.21.15.38:0/2357227852 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f20b40a3240 msgr2=0x7f20b40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:32.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:32.130+0000 7f20cf28c640 1 --2- 172.21.15.38:0/2357227852 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f20b40a3240 0x7f20b40a5700 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f20c4005fd0 tx=0x7f20c4005850 comp rx=0 tx=0).stop 2024-09-16T00:30:32.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:32.130+0000 7f20cf28c640 1 -- 172.21.15.38:0/2357227852 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20c8106f10 msgr2=0x7f20c819cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:32.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:32.130+0000 7f20cf28c640 1 --2- 172.21.15.38:0/2357227852 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20c8106f10 0x7f20c819cca0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f20c000cce0 tx=0x7f20c000a620 comp rx=0 tx=0).stop 2024-09-16T00:30:32.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:32.130+0000 7f20cf28c640 1 -- 172.21.15.38:0/2357227852 shutdown_connections 2024-09-16T00:30:32.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:32.130+0000 7f20cf28c640 1 --2- 172.21.15.38:0/2357227852 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f20b40a3240 0x7f20b40a5700 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:32.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:32.130+0000 7f20cf28c640 1 --2- 172.21.15.38:0/2357227852 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20c8106f10 0x7f20c819cca0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:32.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:32.130+0000 7f20cf28c640 1 --2- 172.21.15.38:0/2357227852 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20c8102790 0x7f20c8198910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:32.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:32.130+0000 7f20cf28c640 1 --2- 172.21.15.38:0/2357227852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20c8101de0 0x7f20c81983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:32.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:32.130+0000 7f20cf28c640 1 -- 172.21.15.38:0/2357227852 >> 172.21.15.38:0/2357227852 conn(0x7f20c80fd8c0 msgr2=0x7f20c80fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:32.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:32.130+0000 7f20cf28c640 1 -- 172.21.15.38:0/2357227852 shutdown_connections 2024-09-16T00:30:32.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:32.130+0000 7f20cf28c640 1 -- 172.21.15.38:0/2357227852 wait complete. 2024-09-16T00:30:32.441 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:32 smithi073 ceph-mon[35254]: osdmap e84: 12 total, 12 up, 12 in 2024-09-16T00:30:32.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:32 smithi038 ceph-mon[30945]: osdmap e84: 12 total, 12 up, 12 in 2024-09-16T00:30:32.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:32 smithi122 ceph-mon[35412]: osdmap e84: 12 total, 12 up, 12 in 2024-09-16T00:30:32.794 INFO:journalctl@ceph.mds.b.smithi073.stdout:Sep 16 00:30:32 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-b[65406]: 2024-09-16T00:30:32.440+0000 7fe7cd552640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-16T00:30:32.847 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-09-16T00:30:33.107 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:33.140 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:33 smithi038 ceph-mon[30945]: pgmap v289: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 8 op/s 2024-09-16T00:30:33.140 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:33 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/2357227852' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-16T00:30:33.140 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:33 smithi038 ceph-mon[30945]: osdmap e85: 12 total, 12 up, 12 in 2024-09-16T00:30:33.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:33 smithi122 ceph-mon[35412]: pgmap v289: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 8 op/s 2024-09-16T00:30:33.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:33 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/2357227852' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-16T00:30:33.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:33 smithi122 ceph-mon[35412]: osdmap e85: 12 total, 12 up, 12 in 2024-09-16T00:30:33.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:33 smithi073 ceph-mon[35254]: pgmap v289: 97 pgs: 97 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 8 op/s 2024-09-16T00:30:33.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:33 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/2357227852' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-16T00:30:33.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:33 smithi073 ceph-mon[35254]: osdmap e85: 12 total, 12 up, 12 in 2024-09-16T00:30:33.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.673+0000 7ffb749a2640 1 -- 172.21.15.38:0/1685804461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb70102d30 msgr2=0x7ffb70103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:33.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.673+0000 7ffb749a2640 1 --2- 172.21.15.38:0/1685804461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb70102d30 0x7ffb70103110 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ffb60009930 tx=0x7ffb6002a420 comp rx=0 tx=0).stop 2024-09-16T00:30:33.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.673+0000 7ffb749a2640 1 -- 172.21.15.38:0/1685804461 shutdown_connections 2024-09-16T00:30:33.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.673+0000 7ffb749a2640 1 --2- 172.21.15.38:0/1685804461 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffb701082a0 0x7ffb7010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:33.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.673+0000 7ffb749a2640 1 --2- 172.21.15.38:0/1685804461 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb701036e0 0x7ffb70107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:33.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.673+0000 7ffb749a2640 1 --2- 172.21.15.38:0/1685804461 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb70102d30 0x7ffb70103110 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:33.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.673+0000 7ffb749a2640 1 -- 172.21.15.38:0/1685804461 >> 172.21.15.38:0/1685804461 conn(0x7ffb700fc7d0 msgr2=0x7ffb700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:33.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.673+0000 7ffb749a2640 1 -- 172.21.15.38:0/1685804461 shutdown_connections 2024-09-16T00:30:33.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.673+0000 7ffb749a2640 1 -- 172.21.15.38:0/1685804461 wait complete. 2024-09-16T00:30:33.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.674+0000 7ffb749a2640 1 Processor -- start 2024-09-16T00:30:33.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.674+0000 7ffb749a2640 1 -- start start 2024-09-16T00:30:33.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.674+0000 7ffb749a2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb70102d30 0x7ffb7019a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:33.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.674+0000 7ffb749a2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb701036e0 0x7ffb7019ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:33.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.674+0000 7ffb749a2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffb701082a0 0x7ffb7019f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:33.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.674+0000 7ffb749a2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ffb701120e0 con 0x7ffb70102d30 2024-09-16T00:30:33.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.674+0000 7ffb749a2640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ffb70111f60 con 0x7ffb701036e0 2024-09-16T00:30:33.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.674+0000 7ffb749a2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffb70112260 con 0x7ffb701082a0 2024-09-16T00:30:33.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.674+0000 7ffb6e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb70102d30 0x7ffb7019a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:33.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.674+0000 7ffb6e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb70102d30 0x7ffb7019a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50234/0 (socket says 172.21.15.38:50234) 2024-09-16T00:30:33.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.674+0000 7ffb6e59c640 1 -- 172.21.15.38:0/811957978 learned_addr learned my addr 172.21.15.38:0/811957978 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:30:33.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.674+0000 7ffb6dd9b640 1 --2- 172.21.15.38:0/811957978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb701036e0 0x7ffb7019ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:33.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.674+0000 7ffb6ed9d640 1 --2- 172.21.15.38:0/811957978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffb701082a0 0x7ffb7019f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:33.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.674+0000 7ffb6e59c640 1 -- 172.21.15.38:0/811957978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffb701082a0 msgr2=0x7ffb7019f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:33.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.674+0000 7ffb6e59c640 1 --2- 172.21.15.38:0/811957978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffb701082a0 0x7ffb7019f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:33.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.674+0000 7ffb6e59c640 1 -- 172.21.15.38:0/811957978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb701036e0 msgr2=0x7ffb7019ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:33.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.674+0000 7ffb6e59c640 1 --2- 172.21.15.38:0/811957978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb701036e0 0x7ffb7019ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:33.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.674+0000 7ffb6e59c640 1 -- 172.21.15.38:0/811957978 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb7019f6f0 con 0x7ffb70102d30 2024-09-16T00:30:33.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.674+0000 7ffb6dd9b640 1 --2- 172.21.15.38:0/811957978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb701036e0 0x7ffb7019ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:30:33.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.674+0000 7ffb6ed9d640 1 --2- 172.21.15.38:0/811957978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffb701082a0 0x7ffb7019f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:33.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.675+0000 7ffb6e59c640 1 --2- 172.21.15.38:0/811957978 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb70102d30 0x7ffb7019a740 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ffb6002a940 tx=0x7ffb60003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:33.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.675+0000 7ffb5f7fe640 1 -- 172.21.15.38:0/811957978 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6003f070 con 0x7ffb70102d30 2024-09-16T00:30:33.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.675+0000 7ffb5f7fe640 1 -- 172.21.15.38:0/811957978 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffb6003b460 con 0x7ffb70102d30 2024-09-16T00:30:33.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.675+0000 7ffb749a2640 1 -- 172.21.15.38:0/811957978 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb7019f980 con 0x7ffb70102d30 2024-09-16T00:30:33.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.675+0000 7ffb5f7fe640 1 -- 172.21.15.38:0/811957978 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb60035030 con 0x7ffb70102d30 2024-09-16T00:30:33.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.675+0000 7ffb749a2640 1 -- 172.21.15.38:0/811957978 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb701a72c0 con 0x7ffb70102d30 2024-09-16T00:30:33.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.676+0000 7ffb749a2640 1 -- 172.21.15.38:0/811957978 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb70103110 con 0x7ffb70102d30 2024-09-16T00:30:33.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.678+0000 7ffb5f7fe640 1 -- 172.21.15.38:0/811957978 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7ffb60042070 con 0x7ffb70102d30 2024-09-16T00:30:33.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.679+0000 7ffb5f7fe640 1 --2- 172.21.15.38:0/811957978 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ffb580a32c0 0x7ffb580a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:33.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.679+0000 7ffb6dd9b640 1 --2- 172.21.15.38:0/811957978 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ffb580a32c0 0x7ffb580a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:33.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.679+0000 7ffb5f7fe640 1 -- 172.21.15.38:0/811957978 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8770+0+0 (secure 0 0 0) 0x7ffb600f0810 con 0x7ffb70102d30 2024-09-16T00:30:33.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.679+0000 7ffb6dd9b640 1 --2- 172.21.15.38:0/811957978 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ffb580a32c0 0x7ffb580a5780 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ffb700fdbc0 tx=0x7ffb6400a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:33.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.682+0000 7ffb5f7fe640 1 -- 172.21.15.38:0/811957978 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffb600bd310 con 0x7ffb70102d30 2024-09-16T00:30:33.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:33.857+0000 7ffb749a2640 1 -- 172.21.15.38:0/811957978 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7ffb7010c2c0 con 0x7ffb70102d30 2024-09-16T00:30:34.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:34.143+0000 7ffb5f7fe640 1 -- 172.21.15.38:0/811957978 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v87) ==== 169+0+0 (secure 0 0 0) 0x7ffb600bcc10 con 0x7ffb70102d30 2024-09-16T00:30:34.143 INFO:teuthology.orchestra.run.smithi038.stderr:set pool 4 allow_ec_overwrites to true 2024-09-16T00:30:34.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:34.145+0000 7ffb749a2640 1 -- 172.21.15.38:0/811957978 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ffb580a32c0 msgr2=0x7ffb580a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:34.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:34.145+0000 7ffb749a2640 1 --2- 172.21.15.38:0/811957978 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ffb580a32c0 0x7ffb580a5780 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ffb700fdbc0 tx=0x7ffb6400a5c0 comp rx=0 tx=0).stop 2024-09-16T00:30:34.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:34.145+0000 7ffb749a2640 1 -- 172.21.15.38:0/811957978 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb70102d30 msgr2=0x7ffb7019a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:34.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:34.145+0000 7ffb749a2640 1 --2- 172.21.15.38:0/811957978 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb70102d30 0x7ffb7019a740 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ffb6002a940 tx=0x7ffb60003c00 comp rx=0 tx=0).stop 2024-09-16T00:30:34.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:34.145+0000 7ffb749a2640 1 -- 172.21.15.38:0/811957978 shutdown_connections 2024-09-16T00:30:34.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:34.145+0000 7ffb749a2640 1 --2- 172.21.15.38:0/811957978 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ffb580a32c0 0x7ffb580a5780 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:34.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:34.145+0000 7ffb749a2640 1 --2- 172.21.15.38:0/811957978 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffb701082a0 0x7ffb7019f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:34.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:34.145+0000 7ffb749a2640 1 --2- 172.21.15.38:0/811957978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffb701036e0 0x7ffb7019ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:34.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:34.145+0000 7ffb749a2640 1 --2- 172.21.15.38:0/811957978 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffb70102d30 0x7ffb7019a740 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:34.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:34.145+0000 7ffb749a2640 1 -- 172.21.15.38:0/811957978 >> 172.21.15.38:0/811957978 conn(0x7ffb700fc7d0 msgr2=0x7ffb700fe4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:34.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:34.145+0000 7ffb749a2640 1 -- 172.21.15.38:0/811957978 shutdown_connections 2024-09-16T00:30:34.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:34.146+0000 7ffb749a2640 1 -- 172.21.15.38:0/811957978 wait complete. 2024-09-16T00:30:34.439 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:34 smithi038 ceph-mon[30945]: osdmap e86: 12 total, 12 up, 12 in 2024-09-16T00:30:34.439 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:34 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/811957978' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-16T00:30:34.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:34 smithi122 ceph-mon[35412]: osdmap e86: 12 total, 12 up, 12 in 2024-09-16T00:30:34.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:34 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/811957978' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-16T00:30:34.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:34 smithi073 ceph-mon[35254]: osdmap e86: 12 total, 12 up, 12 in 2024-09-16T00:30:34.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:34 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/811957978' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-09-16T00:30:34.875 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-09-16T00:30:35.137 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:35.398 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:35 smithi038 ceph-mon[30945]: pgmap v292: 129 pgs: 3 creating+activating, 17 unknown, 109 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-09-16T00:30:35.398 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:35 smithi038 ceph-mon[30945]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T00:30:35.399 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:35 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/811957978' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-16T00:30:35.399 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:35 smithi038 ceph-mon[30945]: osdmap e87: 12 total, 12 up, 12 in 2024-09-16T00:30:35.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:35 smithi122 ceph-mon[35412]: pgmap v292: 129 pgs: 3 creating+activating, 17 unknown, 109 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-09-16T00:30:35.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:35 smithi122 ceph-mon[35412]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T00:30:35.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:35 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/811957978' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-16T00:30:35.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:35 smithi122 ceph-mon[35412]: osdmap e87: 12 total, 12 up, 12 in 2024-09-16T00:30:35.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:35 smithi073 ceph-mon[35254]: pgmap v292: 129 pgs: 3 creating+activating, 17 unknown, 109 active+clean; 579 KiB data, 365 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 6 op/s 2024-09-16T00:30:35.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:35 smithi073 ceph-mon[35254]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-16T00:30:35.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:35 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/811957978' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-09-16T00:30:35.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:35 smithi073 ceph-mon[35254]: osdmap e87: 12 total, 12 up, 12 in 2024-09-16T00:30:35.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.638+0000 7fe6d41fa640 1 -- 172.21.15.38:0/646872436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6cc101380 msgr2=0x7fe6cc10dfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:35.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.638+0000 7fe6d41fa640 1 --2- 172.21.15.38:0/646872436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6cc101380 0x7fe6cc10dfb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0009900 tx=0x7fe6c002a410 comp rx=0 tx=0).stop 2024-09-16T00:30:35.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.638+0000 7fe6d41fa640 1 -- 172.21.15.38:0/646872436 shutdown_connections 2024-09-16T00:30:35.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.638+0000 7fe6d41fa640 1 --2- 172.21.15.38:0/646872436 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6cc10e4f0 0x7fe6cc110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:35.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.638+0000 7fe6d41fa640 1 --2- 172.21.15.38:0/646872436 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6cc101380 0x7fe6cc10dfb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:35.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.638+0000 7fe6d41fa640 1 --2- 172.21.15.38:0/646872436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6cc1009d0 0x7fe6cc100db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:35.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.638+0000 7fe6d41fa640 1 -- 172.21.15.38:0/646872436 >> 172.21.15.38:0/646872436 conn(0x7fe6cc0fc7d0 msgr2=0x7fe6cc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:35.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.638+0000 7fe6d41fa640 1 -- 172.21.15.38:0/646872436 shutdown_connections 2024-09-16T00:30:35.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.638+0000 7fe6d41fa640 1 -- 172.21.15.38:0/646872436 wait complete. 2024-09-16T00:30:35.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.639+0000 7fe6d41fa640 1 Processor -- start 2024-09-16T00:30:35.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.639+0000 7fe6d41fa640 1 -- start start 2024-09-16T00:30:35.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.639+0000 7fe6d41fa640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6cc1009d0 0x7fe6cc1a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:35.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.639+0000 7fe6d41fa640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6cc101380 0x7fe6cc1a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:35.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.639+0000 7fe6d41fa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6cc10e4f0 0x7fe6cc19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:35.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.639+0000 7fe6d41fa640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe6cc114260 con 0x7fe6cc101380 2024-09-16T00:30:35.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.639+0000 7fe6d41fa640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe6cc1140e0 con 0x7fe6cc1009d0 2024-09-16T00:30:35.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.639+0000 7fe6d41fa640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe6cc1143e0 con 0x7fe6cc10e4f0 2024-09-16T00:30:35.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.639+0000 7fe6d1795640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6cc101380 0x7fe6cc1a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:35.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.639+0000 7fe6d1795640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6cc101380 0x7fe6cc1a0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50254/0 (socket says 172.21.15.38:50254) 2024-09-16T00:30:35.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.639+0000 7fe6d1795640 1 -- 172.21.15.38:0/4008949947 learned_addr learned my addr 172.21.15.38:0/4008949947 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:30:35.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.639+0000 7fe6d2797640 1 --2- 172.21.15.38:0/4008949947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6cc10e4f0 0x7fe6cc19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:35.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.639+0000 7fe6d1f96640 1 --2- 172.21.15.38:0/4008949947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6cc1009d0 0x7fe6cc1a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:35.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.639+0000 7fe6d1795640 1 -- 172.21.15.38:0/4008949947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6cc10e4f0 msgr2=0x7fe6cc19a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:35.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.639+0000 7fe6d1795640 1 --2- 172.21.15.38:0/4008949947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6cc10e4f0 0x7fe6cc19a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:35.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.639+0000 7fe6d1795640 1 -- 172.21.15.38:0/4008949947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6cc1009d0 msgr2=0x7fe6cc1a0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:35.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.639+0000 7fe6d1795640 1 --2- 172.21.15.38:0/4008949947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6cc1009d0 0x7fe6cc1a0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:35.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.639+0000 7fe6d1795640 1 -- 172.21.15.38:0/4008949947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6cc19ad40 con 0x7fe6cc101380 2024-09-16T00:30:35.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.639+0000 7fe6d2797640 1 --2- 172.21.15.38:0/4008949947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6cc10e4f0 0x7fe6cc19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:30:35.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.640+0000 7fe6d1f96640 1 --2- 172.21.15.38:0/4008949947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6cc1009d0 0x7fe6cc1a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:35.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.640+0000 7fe6d1795640 1 --2- 172.21.15.38:0/4008949947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6cc101380 0x7fe6cc1a0990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0006010 tx=0x7fe6c00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:35.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.640+0000 7fe6caffd640 1 -- 172.21.15.38:0/4008949947 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c003f030 con 0x7fe6cc101380 2024-09-16T00:30:35.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.640+0000 7fe6d41fa640 1 -- 172.21.15.38:0/4008949947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6cc19afd0 con 0x7fe6cc101380 2024-09-16T00:30:35.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.640+0000 7fe6caffd640 1 -- 172.21.15.38:0/4008949947 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6c0003ce0 con 0x7fe6cc101380 2024-09-16T00:30:35.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.640+0000 7fe6caffd640 1 -- 172.21.15.38:0/4008949947 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c0035070 con 0x7fe6cc101380 2024-09-16T00:30:35.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.640+0000 7fe6d41fa640 1 -- 172.21.15.38:0/4008949947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6cc1a7320 con 0x7fe6cc101380 2024-09-16T00:30:35.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.641+0000 7fe6d41fa640 1 -- 172.21.15.38:0/4008949947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6cc100db0 con 0x7fe6cc101380 2024-09-16T00:30:35.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.642+0000 7fe6caffd640 1 -- 172.21.15.38:0/4008949947 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fe6c0042030 con 0x7fe6cc101380 2024-09-16T00:30:35.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.643+0000 7fe6caffd640 1 --2- 172.21.15.38:0/4008949947 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fe6b40a3240 0x7fe6b40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:35.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.643+0000 7fe6d1f96640 1 --2- 172.21.15.38:0/4008949947 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fe6b40a3240 0x7fe6b40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:35.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.643+0000 7fe6caffd640 1 -- 172.21.15.38:0/4008949947 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8770+0+0 (secure 0 0 0) 0x7fe6c00f03f0 con 0x7fe6cc101380 2024-09-16T00:30:35.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.643+0000 7fe6d1f96640 1 --2- 172.21.15.38:0/4008949947 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fe6b40a3240 0x7fe6b40a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc005fd0 tx=0x7fe6bc0093e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:35.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.646+0000 7fe6caffd640 1 -- 172.21.15.38:0/4008949947 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe6c00bcf50 con 0x7fe6cc101380 2024-09-16T00:30:35.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:35.863+0000 7fe6d41fa640 1 -- 172.21.15.38:0/4008949947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7fe6cc109680 con 0x7fe6cc101380 2024-09-16T00:30:36.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:36.160+0000 7fe6caffd640 1 -- 172.21.15.38:0/4008949947 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7fe6c00bc850 con 0x7fe6cc101380 2024-09-16T00:30:36.160 INFO:teuthology.orchestra.run.smithi038.stderr:added data pool 4 to fsmap 2024-09-16T00:30:36.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:36.162+0000 7fe6d41fa640 1 -- 172.21.15.38:0/4008949947 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fe6b40a3240 msgr2=0x7fe6b40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:36.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:36.162+0000 7fe6d41fa640 1 --2- 172.21.15.38:0/4008949947 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fe6b40a3240 0x7fe6b40a5700 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fe6bc005fd0 tx=0x7fe6bc0093e0 comp rx=0 tx=0).stop 2024-09-16T00:30:36.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:36.162+0000 7fe6d41fa640 1 -- 172.21.15.38:0/4008949947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6cc101380 msgr2=0x7fe6cc1a0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:36.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:36.162+0000 7fe6d41fa640 1 --2- 172.21.15.38:0/4008949947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6cc101380 0x7fe6cc1a0990 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe6c0006010 tx=0x7fe6c00042f0 comp rx=0 tx=0).stop 2024-09-16T00:30:36.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:36.162+0000 7fe6d41fa640 1 -- 172.21.15.38:0/4008949947 shutdown_connections 2024-09-16T00:30:36.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:36.162+0000 7fe6d41fa640 1 --2- 172.21.15.38:0/4008949947 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fe6b40a3240 0x7fe6b40a5700 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:36.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:36.162+0000 7fe6d41fa640 1 --2- 172.21.15.38:0/4008949947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6cc10e4f0 0x7fe6cc19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:36.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:36.162+0000 7fe6d41fa640 1 --2- 172.21.15.38:0/4008949947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6cc101380 0x7fe6cc1a0990 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:36.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:36.162+0000 7fe6d41fa640 1 --2- 172.21.15.38:0/4008949947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6cc1009d0 0x7fe6cc1a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:36.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:36.162+0000 7fe6d41fa640 1 -- 172.21.15.38:0/4008949947 >> 172.21.15.38:0/4008949947 conn(0x7fe6cc0fc7d0 msgr2=0x7fe6cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:36.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:36.163+0000 7fe6d41fa640 1 -- 172.21.15.38:0/4008949947 shutdown_connections 2024-09-16T00:30:36.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:36.163+0000 7fe6d41fa640 1 -- 172.21.15.38:0/4008949947 wait complete. 2024-09-16T00:30:36.445 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:36 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/4008949947' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-16T00:30:36.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:36 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/4008949947' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-16T00:30:36.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:36 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/4008949947' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-09-16T00:30:36.909 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:30:37.148 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:37 smithi038 ceph-mon[30945]: pgmap v294: 129 pgs: 3 creating+activating, 7 unknown, 119 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-16T00:30:37.148 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:37 smithi038 ceph-mon[30945]: osdmap e88: 12 total, 12 up, 12 in 2024-09-16T00:30:37.148 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:37 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/4008949947' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-16T00:30:37.149 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:37 smithi038 ceph-mon[30945]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:30:37.176 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:37.461 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:37 smithi073 ceph-mon[35254]: pgmap v294: 129 pgs: 3 creating+activating, 7 unknown, 119 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-16T00:30:37.462 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:37 smithi073 ceph-mon[35254]: osdmap e88: 12 total, 12 up, 12 in 2024-09-16T00:30:37.462 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:37 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/4008949947' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-16T00:30:37.462 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:37 smithi073 ceph-mon[35254]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:30:37.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:37 smithi122 ceph-mon[35412]: pgmap v294: 129 pgs: 3 creating+activating, 7 unknown, 119 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 5 op/s 2024-09-16T00:30:37.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:37 smithi122 ceph-mon[35412]: osdmap e88: 12 total, 12 up, 12 in 2024-09-16T00:30:37.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:37 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/4008949947' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-09-16T00:30:37.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:37 smithi122 ceph-mon[35412]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:30:37.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.659+0000 7f258ab8d640 1 -- 172.21.15.38:0/3546541139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2584102790 msgr2=0x7f25841067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:37.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.659+0000 7f258ab8d640 1 --2- 172.21.15.38:0/3546541139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2584102790 0x7f25841067e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f2570009900 tx=0x7f257002a410 comp rx=0 tx=0).stop 2024-09-16T00:30:37.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.659+0000 7f258ab8d640 1 -- 172.21.15.38:0/3546541139 shutdown_connections 2024-09-16T00:30:37.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.659+0000 7f258ab8d640 1 --2- 172.21.15.38:0/3546541139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2584106f10 0x7f258410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:37.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.659+0000 7f258ab8d640 1 --2- 172.21.15.38:0/3546541139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2584102790 0x7f25841067e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:37.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.659+0000 7f258ab8d640 1 --2- 172.21.15.38:0/3546541139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2584101de0 0x7f25841021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:37.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.659+0000 7f258ab8d640 1 -- 172.21.15.38:0/3546541139 >> 172.21.15.38:0/3546541139 conn(0x7f25840fd8c0 msgr2=0x7f25840ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:37.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.659+0000 7f258ab8d640 1 -- 172.21.15.38:0/3546541139 shutdown_connections 2024-09-16T00:30:37.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.659+0000 7f258ab8d640 1 -- 172.21.15.38:0/3546541139 wait complete. 2024-09-16T00:30:37.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.660+0000 7f258ab8d640 1 Processor -- start 2024-09-16T00:30:37.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.660+0000 7f258ab8d640 1 -- start start 2024-09-16T00:30:37.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.660+0000 7f258ab8d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2584101de0 0x7f25840744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:37.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.660+0000 7f258ab8d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2584102790 0x7f2584074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:37.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.660+0000 7f258ab8d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2584106f10 0x7f2584074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:37.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.660+0000 7f258ab8d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2584111f70 con 0x7f2584102790 2024-09-16T00:30:37.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.660+0000 7f258ab8d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2584111df0 con 0x7f2584101de0 2024-09-16T00:30:37.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.660+0000 7f258ab8d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f25841120f0 con 0x7f2584106f10 2024-09-16T00:30:37.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.660+0000 7f2583fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2584102790 0x7f2584074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:37.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.660+0000 7f2583fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2584102790 0x7f2584074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52748/0 (socket says 172.21.15.38:52748) 2024-09-16T00:30:37.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.660+0000 7f2583fff640 1 -- 172.21.15.38:0/1075512385 learned_addr learned my addr 172.21.15.38:0/1075512385 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:30:37.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.660+0000 7f2588929640 1 --2- 172.21.15.38:0/1075512385 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2584101de0 0x7f25840744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:37.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.661+0000 7f258912a640 1 --2- 172.21.15.38:0/1075512385 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2584106f10 0x7f2584074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:37.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.661+0000 7f2583fff640 1 -- 172.21.15.38:0/1075512385 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2584106f10 msgr2=0x7f2584074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:37.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.661+0000 7f2583fff640 1 --2- 172.21.15.38:0/1075512385 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2584106f10 0x7f2584074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:37.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.661+0000 7f2583fff640 1 -- 172.21.15.38:0/1075512385 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2584101de0 msgr2=0x7f25840744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:37.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.661+0000 7f2583fff640 1 --2- 172.21.15.38:0/1075512385 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2584101de0 0x7f25840744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:37.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.661+0000 7f2583fff640 1 -- 172.21.15.38:0/1075512385 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25841a6900 con 0x7f2584102790 2024-09-16T00:30:37.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.661+0000 7f258912a640 1 --2- 172.21.15.38:0/1075512385 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2584106f10 0x7f2584074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:37.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.661+0000 7f2583fff640 1 --2- 172.21.15.38:0/1075512385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2584102790 0x7f2584074a30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2570006010 tx=0x7f25700042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:37.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.661+0000 7f2581ffb640 1 -- 172.21.15.38:0/1075512385 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f257003f030 con 0x7f2584102790 2024-09-16T00:30:37.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.661+0000 7f258ab8d640 1 -- 172.21.15.38:0/1075512385 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25841a6b90 con 0x7f2584102790 2024-09-16T00:30:37.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.661+0000 7f2581ffb640 1 -- 172.21.15.38:0/1075512385 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2570003ce0 con 0x7f2584102790 2024-09-16T00:30:37.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.661+0000 7f2581ffb640 1 -- 172.21.15.38:0/1075512385 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2570035070 con 0x7f2584102790 2024-09-16T00:30:37.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.661+0000 7f258ab8d640 1 -- 172.21.15.38:0/1075512385 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25841a70c0 con 0x7f2584102790 2024-09-16T00:30:37.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.662+0000 7f258ab8d640 1 -- 172.21.15.38:0/1075512385 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25841021c0 con 0x7f2584102790 2024-09-16T00:30:37.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.663+0000 7f2581ffb640 1 -- 172.21.15.38:0/1075512385 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f2570042030 con 0x7f2584102790 2024-09-16T00:30:37.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.664+0000 7f2581ffb640 1 --2- 172.21.15.38:0/1075512385 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f25740a3240 0x7f25740a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:37.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.664+0000 7f2588929640 1 --2- 172.21.15.38:0/1075512385 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f25740a3240 0x7f25740a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:37.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.664+0000 7f2581ffb640 1 -- 172.21.15.38:0/1075512385 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f25700f0350 con 0x7f2584102790 2024-09-16T00:30:37.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.665+0000 7f2588929640 1 --2- 172.21.15.38:0/1075512385 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f25740a3240 0x7f25740a5700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2578005d00 tx=0x7f257800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:37.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.667+0000 7f2581ffb640 1 -- 172.21.15.38:0/1075512385 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f25700bce90 con 0x7f2584102790 2024-09-16T00:30:37.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.878+0000 7f258ab8d640 1 -- 172.21.15.38:0/1075512385 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f258410a0f0 con 0x7f2584102790 2024-09-16T00:30:37.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.879+0000 7f2581ffb640 1 -- 172.21.15.38:0/1075512385 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11836 (secure 0 0 0) 0x7f25700bc790 con 0x7f2584102790 2024-09-16T00:30:37.879 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:30:37.880 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":17,"btime":"2024-09-16T00:30:36:145982+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/367974527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":367974527},{"type":"v1","addr":"172.21.15.38:6835","nonce":367974527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/335077136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":335077136},{"type":"v1","addr":"172.21.15.38:6837","nonce":335077136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/3501211160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3501211160},{"type":"v1","addr":"172.21.15.38:6839","nonce":3501211160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6841/2623130915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":2623130915},{"type":"v1","addr":"172.21.15.38:6841","nonce":2623130915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/412277519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":412277519},{"type":"v1","addr":"172.21.15.122:6833","nonce":412277519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3987234757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3987234757},{"type":"v1","addr":"172.21.15.122:6835","nonce":3987234757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/1210328177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1210328177},{"type":"v1","addr":"172.21.15.73:6833","nonce":1210328177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1858299680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1858299680},{"type":"v1","addr":"172.21.15.122:6837","nonce":1858299680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24436,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/1983975311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1983975311},{"type":"v1","addr":"172.21.15.122:6839","nonce":1983975311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2866363113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2866363113},{"type":"v1","addr":"172.21.15.73:6835","nonce":2866363113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2681081265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2681081265},{"type":"v1","addr":"172.21.15.73:6837","nonce":2681081265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T00:30:26.764565+0000","modified":"2024-09-16T00:30:35.865257+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.73:6839/2396224525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2396224525},{"type":"v1","addr":"172.21.15.73:6839","nonce":2396224525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24494,"qdb_cluster":[24494]},"id":1}]} 2024-09-16T00:30:37.880 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 17 2024-09-16T00:30:37.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.881+0000 7f258ab8d640 1 -- 172.21.15.38:0/1075512385 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f25740a3240 msgr2=0x7f25740a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:37.881 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.881+0000 7f258ab8d640 1 --2- 172.21.15.38:0/1075512385 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f25740a3240 0x7f25740a5700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2578005d00 tx=0x7f257800b410 comp rx=0 tx=0).stop 2024-09-16T00:30:37.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.881+0000 7f258ab8d640 1 -- 172.21.15.38:0/1075512385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2584102790 msgr2=0x7f2584074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:37.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.881+0000 7f258ab8d640 1 --2- 172.21.15.38:0/1075512385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2584102790 0x7f2584074a30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f2570006010 tx=0x7f25700042f0 comp rx=0 tx=0).stop 2024-09-16T00:30:37.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.881+0000 7f258ab8d640 1 -- 172.21.15.38:0/1075512385 shutdown_connections 2024-09-16T00:30:37.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.881+0000 7f258ab8d640 1 --2- 172.21.15.38:0/1075512385 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f25740a3240 0x7f25740a5700 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:37.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.881+0000 7f258ab8d640 1 --2- 172.21.15.38:0/1075512385 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2584106f10 0x7f2584074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:37.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.881+0000 7f258ab8d640 1 --2- 172.21.15.38:0/1075512385 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2584102790 0x7f2584074a30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:37.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.881+0000 7f258ab8d640 1 --2- 172.21.15.38:0/1075512385 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2584101de0 0x7f25840744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:37.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.881+0000 7f258ab8d640 1 -- 172.21.15.38:0/1075512385 >> 172.21.15.38:0/1075512385 conn(0x7f25840fd8c0 msgr2=0x7f25840fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:37.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.881+0000 7f258ab8d640 1 -- 172.21.15.38:0/1075512385 shutdown_connections 2024-09-16T00:30:37.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:37.881+0000 7f258ab8d640 1 -- 172.21.15.38:0/1075512385 wait complete. 2024-09-16T00:30:38.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:38 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/1075512385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:30:38.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:38 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/1075512385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:30:38.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:38 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/1075512385' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:30:38.699 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:30:38.960 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:39.345 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:39 smithi038 ceph-mon[30945]: pgmap v296: 129 pgs: 3 creating+activating, 126 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-16T00:30:39.345 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:39 smithi038 ceph-mon[30945]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T00:30:39.345 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:39 smithi038 ceph-mon[30945]: Cluster is now healthy 2024-09-16T00:30:39.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.426+0000 7ff0613a3640 1 -- 172.21.15.38:0/2829678435 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff05c103660 msgr2=0x7ff05c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:39.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.426+0000 7ff0613a3640 1 --2- 172.21.15.38:0/2829678435 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff05c103660 0x7ff05c10bb50 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff050009900 tx=0x7ff05002a480 comp rx=0 tx=0).stop 2024-09-16T00:30:39.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.426+0000 7ff0613a3640 1 -- 172.21.15.38:0/2829678435 shutdown_connections 2024-09-16T00:30:39.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.426+0000 7ff0613a3640 1 --2- 172.21.15.38:0/2829678435 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff05c10c170 0x7ff05c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:39.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.426+0000 7ff0613a3640 1 --2- 172.21.15.38:0/2829678435 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff05c103660 0x7ff05c10bb50 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:39.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.426+0000 7ff0613a3640 1 --2- 172.21.15.38:0/2829678435 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff05c102d40 0x7ff05c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:39.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.426+0000 7ff0613a3640 1 -- 172.21.15.38:0/2829678435 >> 172.21.15.38:0/2829678435 conn(0x7ff05c0fc7a0 msgr2=0x7ff05c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:39.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.426+0000 7ff0613a3640 1 -- 172.21.15.38:0/2829678435 shutdown_connections 2024-09-16T00:30:39.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.426+0000 7ff0613a3640 1 -- 172.21.15.38:0/2829678435 wait complete. 2024-09-16T00:30:39.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.426+0000 7ff0613a3640 1 Processor -- start 2024-09-16T00:30:39.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.427+0000 7ff0613a3640 1 -- start start 2024-09-16T00:30:39.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.427+0000 7ff0613a3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff05c102d40 0x7ff05c19a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:39.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.427+0000 7ff0613a3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff05c103660 0x7ff05c19abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:39.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.427+0000 7ff0613a3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff05c10c170 0x7ff05c19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:39.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.427+0000 7ff0613a3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff05c111fd0 con 0x7ff05c103660 2024-09-16T00:30:39.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.427+0000 7ff0613a3640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff05c111e50 con 0x7ff05c10c170 2024-09-16T00:30:39.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.427+0000 7ff0613a3640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff05c112150 con 0x7ff05c102d40 2024-09-16T00:30:39.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.427+0000 7ff05a7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff05c103660 0x7ff05c19abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:39.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.427+0000 7ff05a7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff05c103660 0x7ff05c19abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52772/0 (socket says 172.21.15.38:52772) 2024-09-16T00:30:39.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.427+0000 7ff05a7fc640 1 -- 172.21.15.38:0/542094784 learned_addr learned my addr 172.21.15.38:0/542094784 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:30:39.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.427+0000 7ff05affd640 1 --2- 172.21.15.38:0/542094784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff05c102d40 0x7ff05c19a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:39.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.427+0000 7ff05a7fc640 1 -- 172.21.15.38:0/542094784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff05c102d40 msgr2=0x7ff05c19a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:39.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.427+0000 7ff05a7fc640 1 --2- 172.21.15.38:0/542094784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff05c102d40 0x7ff05c19a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:39.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.427+0000 7ff05a7fc640 1 -- 172.21.15.38:0/542094784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff05c10c170 msgr2=0x7ff05c19ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:39.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.427+0000 7ff05b7fe640 1 --2- 172.21.15.38:0/542094784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff05c10c170 0x7ff05c19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:39.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.427+0000 7ff05a7fc640 1 --2- 172.21.15.38:0/542094784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff05c10c170 0x7ff05c19ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:39.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.427+0000 7ff05a7fc640 1 -- 172.21.15.38:0/542094784 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff05c19f660 con 0x7ff05c103660 2024-09-16T00:30:39.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.427+0000 7ff05a7fc640 1 --2- 172.21.15.38:0/542094784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff05c103660 0x7ff05c19abf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff050006010 tx=0x7ff0500042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:39.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.428+0000 7ff043fff640 1 -- 172.21.15.38:0/542094784 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05003f030 con 0x7ff05c103660 2024-09-16T00:30:39.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.428+0000 7ff0613a3640 1 -- 172.21.15.38:0/542094784 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff05c19f8f0 con 0x7ff05c103660 2024-09-16T00:30:39.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.428+0000 7ff043fff640 1 -- 172.21.15.38:0/542094784 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff050003ce0 con 0x7ff05c103660 2024-09-16T00:30:39.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.428+0000 7ff043fff640 1 -- 172.21.15.38:0/542094784 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff050035070 con 0x7ff05c103660 2024-09-16T00:30:39.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.428+0000 7ff0613a3640 1 -- 172.21.15.38:0/542094784 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff05c1a7230 con 0x7ff05c103660 2024-09-16T00:30:39.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.429+0000 7ff0613a3640 1 -- 172.21.15.38:0/542094784 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff05c103120 con 0x7ff05c103660 2024-09-16T00:30:39.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.431+0000 7ff043fff640 1 -- 172.21.15.38:0/542094784 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7ff050042030 con 0x7ff05c103660 2024-09-16T00:30:39.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.431+0000 7ff043fff640 1 --2- 172.21.15.38:0/542094784 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ff0440a3240 0x7ff0440a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:39.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.431+0000 7ff05affd640 1 --2- 172.21.15.38:0/542094784 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ff0440a3240 0x7ff0440a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:39.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.431+0000 7ff043fff640 1 -- 172.21.15.38:0/542094784 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7ff0500f03c0 con 0x7ff05c103660 2024-09-16T00:30:39.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.432+0000 7ff05affd640 1 --2- 172.21.15.38:0/542094784 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ff0440a3240 0x7ff0440a5700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff04c005fd0 tx=0x7ff04c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:39.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.434+0000 7ff043fff640 1 -- 172.21.15.38:0/542094784 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff0500bcf60 con 0x7ff05c103660 2024-09-16T00:30:39.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:39 smithi122 ceph-mon[35412]: pgmap v296: 129 pgs: 3 creating+activating, 126 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-16T00:30:39.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:39 smithi122 ceph-mon[35412]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T00:30:39.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:39 smithi122 ceph-mon[35412]: Cluster is now healthy 2024-09-16T00:30:39.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:39 smithi073 ceph-mon[35254]: pgmap v296: 129 pgs: 3 creating+activating, 126 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-16T00:30:39.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:39 smithi073 ceph-mon[35254]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-16T00:30:39.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:39 smithi073 ceph-mon[35254]: Cluster is now healthy 2024-09-16T00:30:39.610 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.610+0000 7ff0613a3640 1 -- 172.21.15.38:0/542094784 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff05c1a7400 con 0x7ff05c103660 2024-09-16T00:30:39.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.611+0000 7ff043fff640 1 -- 172.21.15.38:0/542094784 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24174 (secure 0 0 0) 0x7ff0500bc860 con 0x7ff05c103660 2024-09-16T00:30:39.612 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:30:39.612 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":88,"fsid":"983b2072-73c1-11ef-bceb-c7b262605968","created":"2024-09-16T00:22:34.897053+0000","modified":"2024-09-16T00:30:35.865128+0000","last_up_change":"2024-09-16T00:29:34.503134+0000","last_in_change":"2024-09-16T00:29:10.135656+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:25:41.931977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:30:22.741889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:30:25.042590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:30:31.554350+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3afc20e3-869f-43a9-a911-b0e8d07de86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6803","nonce":1622028921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6805","nonce":1622028921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6809","nonce":1622028921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6807","nonce":1622028921}]},"public_addr":"172.21.15.38:6803/1622028921","cluster_addr":"172.21.15.38:6805/1622028921","heartbeat_back_addr":"172.21.15.38:6809/1622028921","heartbeat_front_addr":"172.21.15.38:6807/1622028921","state":["exists","up"]},{"osd":1,"uuid":"6d9d7acb-45ed-4543-8afc-6433fc7f096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6801","nonce":3811690336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6803","nonce":3811690336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6807","nonce":3811690336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6805","nonce":3811690336}]},"public_addr":"172.21.15.73:6801/3811690336","cluster_addr":"172.21.15.73:6803/3811690336","heartbeat_back_addr":"172.21.15.73:6807/3811690336","heartbeat_front_addr":"172.21.15.73:6805/3811690336","state":["exists","up"]},{"osd":2,"uuid":"48690f35-99e9-4f45-9bee-487857caa8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6801","nonce":1754741655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6803","nonce":1754741655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6807","nonce":1754741655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6805","nonce":1754741655}]},"public_addr":"172.21.15.122:6801/1754741655","cluster_addr":"172.21.15.122:6803/1754741655","heartbeat_back_addr":"172.21.15.122:6807/1754741655","heartbeat_front_addr":"172.21.15.122:6805/1754741655","state":["exists","up"]},{"osd":3,"uuid":"28ebb4a0-92e7-4f57-82a5-b9b761eca269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6811","nonce":3523190079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6813","nonce":3523190079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6817","nonce":3523190079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6815","nonce":3523190079}]},"public_addr":"172.21.15.38:6811/3523190079","cluster_addr":"172.21.15.38:6813/3523190079","heartbeat_back_addr":"172.21.15.38:6817/3523190079","heartbeat_front_addr":"172.21.15.38:6815/3523190079","state":["exists","up"]},{"osd":4,"uuid":"5ea363f7-7ad8-45ed-b7e4-4b010fe7f8cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6809","nonce":1469927628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6811","nonce":1469927628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6815","nonce":1469927628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6813","nonce":1469927628}]},"public_addr":"172.21.15.73:6809/1469927628","cluster_addr":"172.21.15.73:6811/1469927628","heartbeat_back_addr":"172.21.15.73:6815/1469927628","heartbeat_front_addr":"172.21.15.73:6813/1469927628","state":["exists","up"]},{"osd":5,"uuid":"e3c66d43-dd28-4bb1-a886-62a869be82cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6809","nonce":2954413058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6811","nonce":2954413058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6815","nonce":2954413058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6813","nonce":2954413058}]},"public_addr":"172.21.15.122:6809/2954413058","cluster_addr":"172.21.15.122:6811/2954413058","heartbeat_back_addr":"172.21.15.122:6815/2954413058","heartbeat_front_addr":"172.21.15.122:6813/2954413058","state":["exists","up"]},{"osd":6,"uuid":"211497ac-7c48-4095-bfdb-1d4df0149f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6819","nonce":1415719550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6821","nonce":1415719550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6825","nonce":1415719550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6823","nonce":1415719550}]},"public_addr":"172.21.15.38:6819/1415719550","cluster_addr":"172.21.15.38:6821/1415719550","heartbeat_back_addr":"172.21.15.38:6825/1415719550","heartbeat_front_addr":"172.21.15.38:6823/1415719550","state":["exists","up"]},{"osd":7,"uuid":"06512315-3f5e-44f8-976d-d3f5f1ba00e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6817","nonce":2634361473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6819","nonce":2634361473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6823","nonce":2634361473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6821","nonce":2634361473}]},"public_addr":"172.21.15.73:6817/2634361473","cluster_addr":"172.21.15.73:6819/2634361473","heartbeat_back_addr":"172.21.15.73:6823/2634361473","heartbeat_front_addr":"172.21.15.73:6821/2634361473","state":["exists","up"]},{"osd":8,"uuid":"db703c0a-1f87-46f0-be5f-f826f7f9daac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6817","nonce":1800567633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6819","nonce":1800567633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6823","nonce":1800567633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6821","nonce":1800567633}]},"public_addr":"172.21.15.122:6817/1800567633","cluster_addr":"172.21.15.122:6819/1800567633","heartbeat_back_addr":"172.21.15.122:6823/1800567633","heartbeat_front_addr":"172.21.15.122:6821/1800567633","state":["exists","up"]},{"osd":9,"uuid":"f5384200-b3b4-4936-85e1-fc57a9fe2bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6827","nonce":445727833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6829","nonce":445727833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6833","nonce":445727833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6831","nonce":445727833}]},"public_addr":"172.21.15.38:6827/445727833","cluster_addr":"172.21.15.38:6829/445727833","heartbeat_back_addr":"172.21.15.38:6833/445727833","heartbeat_front_addr":"172.21.15.38:6831/445727833","state":["exists","up"]},{"osd":10,"uuid":"c22883d9-6672-4796-a961-963d21eb37fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6825","nonce":43297139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6827","nonce":43297139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6831","nonce":43297139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6829","nonce":43297139}]},"public_addr":"172.21.15.73:6825/43297139","cluster_addr":"172.21.15.73:6827/43297139","heartbeat_back_addr":"172.21.15.73:6831/43297139","heartbeat_front_addr":"172.21.15.73:6829/43297139","state":["exists","up"]},{"osd":11,"uuid":"9f98e8cb-1bff-4b8b-aa19-56b691387383","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6825","nonce":1886446061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6827","nonce":1886446061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6831","nonce":1886446061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6829","nonce":1886446061}]},"public_addr":"172.21.15.122:6825/1886446061","cluster_addr":"172.21.15.122:6827/1886446061","heartbeat_back_addr":"172.21.15.122:6831/1886446061","heartbeat_front_addr":"172.21.15.122:6829/1886446061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:47.384012+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:12.834488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:38.770055+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:04.800466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:30.808076+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:56.646967+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:27:22.721270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:27:48.510011+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:28:13.922944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:28:40.514947+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:29:06.031052+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:29:31.882062+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6801/1779324916":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/924150447":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/141237849":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/2302081750":"2024-09-17T00:22:56.844212+0000","172.21.15.38:0/2245468594":"2024-09-17T00:22:56.844212+0000","172.21.15.38:6801/583006373":"2024-09-17T00:22:56.844212+0000","172.21.15.38:0/2435492667":"2024-09-17T00:23:21.417483+0000","172.21.15.38:6800/583006373":"2024-09-17T00:22:56.844212+0000","172.21.15.38:6800/1779324916":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/1103516837":"2024-09-17T00:22:56.844212+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:30:39.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.613+0000 7ff0613a3640 1 -- 172.21.15.38:0/542094784 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ff0440a3240 msgr2=0x7ff0440a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:39.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.613+0000 7ff0613a3640 1 --2- 172.21.15.38:0/542094784 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ff0440a3240 0x7ff0440a5700 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff04c005fd0 tx=0x7ff04c005850 comp rx=0 tx=0).stop 2024-09-16T00:30:39.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.613+0000 7ff0613a3640 1 -- 172.21.15.38:0/542094784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff05c103660 msgr2=0x7ff05c19abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:39.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.613+0000 7ff0613a3640 1 --2- 172.21.15.38:0/542094784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff05c103660 0x7ff05c19abf0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ff050006010 tx=0x7ff0500042f0 comp rx=0 tx=0).stop 2024-09-16T00:30:39.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.614+0000 7ff0613a3640 1 -- 172.21.15.38:0/542094784 shutdown_connections 2024-09-16T00:30:39.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.614+0000 7ff0613a3640 1 --2- 172.21.15.38:0/542094784 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ff0440a3240 0x7ff0440a5700 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:39.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.614+0000 7ff0613a3640 1 --2- 172.21.15.38:0/542094784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff05c10c170 0x7ff05c19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:39.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.614+0000 7ff0613a3640 1 --2- 172.21.15.38:0/542094784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff05c103660 0x7ff05c19abf0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:39.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.614+0000 7ff0613a3640 1 --2- 172.21.15.38:0/542094784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff05c102d40 0x7ff05c19a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:39.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.614+0000 7ff0613a3640 1 -- 172.21.15.38:0/542094784 >> 172.21.15.38:0/542094784 conn(0x7ff05c0fc7a0 msgr2=0x7ff05c10d490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:39.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.614+0000 7ff0613a3640 1 -- 172.21.15.38:0/542094784 shutdown_connections 2024-09-16T00:30:39.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:39.614+0000 7ff0613a3640 1 -- 172.21.15.38:0/542094784 wait complete. 2024-09-16T00:30:40.347 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:30:40.374 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:40 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/542094784' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:30:40.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:40 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/542094784' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:30:40.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:40 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/542094784' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:30:40.612 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:41.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.095+0000 7f88b8460640 1 -- 172.21.15.38:0/2747895535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88b01027a0 msgr2=0x7f88b01067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:41.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.095+0000 7f88b8460640 1 --2- 172.21.15.38:0/2747895535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88b01027a0 0x7f88b01067f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f88a0009900 tx=0x7f88a002a410 comp rx=0 tx=0).stop 2024-09-16T00:30:41.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.096+0000 7f88b8460640 1 -- 172.21.15.38:0/2747895535 shutdown_connections 2024-09-16T00:30:41.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.096+0000 7f88b8460640 1 --2- 172.21.15.38:0/2747895535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88b0106f20 0x7f88b010eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:41.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.096+0000 7f88b8460640 1 --2- 172.21.15.38:0/2747895535 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88b01027a0 0x7f88b01067f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:41.123 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.096+0000 7f88b8460640 1 --2- 172.21.15.38:0/2747895535 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88b0101df0 0x7f88b01021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:41.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.096+0000 7f88b8460640 1 -- 172.21.15.38:0/2747895535 >> 172.21.15.38:0/2747895535 conn(0x7f88b00fd8b0 msgr2=0x7f88b00ffcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:41.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.096+0000 7f88b8460640 1 -- 172.21.15.38:0/2747895535 shutdown_connections 2024-09-16T00:30:41.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.096+0000 7f88b8460640 1 -- 172.21.15.38:0/2747895535 wait complete. 2024-09-16T00:30:41.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.096+0000 7f88b8460640 1 Processor -- start 2024-09-16T00:30:41.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.096+0000 7f88b8460640 1 -- start start 2024-09-16T00:30:41.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.096+0000 7f88b8460640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88b0101df0 0x7f88b0198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:41.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.096+0000 7f88b8460640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88b01027a0 0x7f88b0198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:41.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.096+0000 7f88b8460640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88b0106f20 0x7f88b019cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:41.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.096+0000 7f88b8460640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f88b0111f80 con 0x7f88b0101df0 2024-09-16T00:30:41.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.096+0000 7f88b8460640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f88b0111e00 con 0x7f88b01027a0 2024-09-16T00:30:41.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.096+0000 7f88b8460640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f88b0112100 con 0x7f88b0106f20 2024-09-16T00:30:41.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.097+0000 7f88b61fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88b0101df0 0x7f88b0198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:41.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.097+0000 7f88b61fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88b0101df0 0x7f88b0198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52784/0 (socket says 172.21.15.38:52784) 2024-09-16T00:30:41.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.097+0000 7f88b61fc640 1 -- 172.21.15.38:0/3325026836 learned_addr learned my addr 172.21.15.38:0/3325026836 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:30:41.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.097+0000 7f88b59fb640 1 --2- 172.21.15.38:0/3325026836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88b01027a0 0x7f88b0198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:41.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.097+0000 7f88b69fd640 1 --2- 172.21.15.38:0/3325026836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88b0106f20 0x7f88b019cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:41.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.097+0000 7f88b61fc640 1 -- 172.21.15.38:0/3325026836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88b0106f20 msgr2=0x7f88b019cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:41.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.097+0000 7f88b61fc640 1 --2- 172.21.15.38:0/3325026836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88b0106f20 0x7f88b019cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:41.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.097+0000 7f88b61fc640 1 -- 172.21.15.38:0/3325026836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88b01027a0 msgr2=0x7f88b0198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:41.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.097+0000 7f88b61fc640 1 --2- 172.21.15.38:0/3325026836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88b01027a0 0x7f88b0198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:41.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.097+0000 7f88b61fc640 1 -- 172.21.15.38:0/3325026836 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88b019d2b0 con 0x7f88b0101df0 2024-09-16T00:30:41.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.097+0000 7f88b69fd640 1 --2- 172.21.15.38:0/3325026836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88b0106f20 0x7f88b019cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:41.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.097+0000 7f88b59fb640 1 --2- 172.21.15.38:0/3325026836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88b01027a0 0x7f88b0198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:30:41.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.097+0000 7f88b61fc640 1 --2- 172.21.15.38:0/3325026836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88b0101df0 0x7f88b0198300 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f88ac00c920 tx=0x7f88ac00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:41.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.097+0000 7f88a77fe640 1 -- 172.21.15.38:0/3325026836 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac007c70 con 0x7f88b0101df0 2024-09-16T00:30:41.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.097+0000 7f88a77fe640 1 -- 172.21.15.38:0/3325026836 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f88ac007e10 con 0x7f88b0101df0 2024-09-16T00:30:41.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.097+0000 7f88a77fe640 1 -- 172.21.15.38:0/3325026836 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88ac018890 con 0x7f88b0101df0 2024-09-16T00:30:41.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.097+0000 7f88b8460640 1 -- 172.21.15.38:0/3325026836 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88b019d5a0 con 0x7f88b0101df0 2024-09-16T00:30:41.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.097+0000 7f88b8460640 1 -- 172.21.15.38:0/3325026836 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88b01a4e80 con 0x7f88b0101df0 2024-09-16T00:30:41.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.098+0000 7f88b8460640 1 -- 172.21.15.38:0/3325026836 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88b01021d0 con 0x7f88b0101df0 2024-09-16T00:30:41.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.099+0000 7f88a77fe640 1 -- 172.21.15.38:0/3325026836 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f88ac018a70 con 0x7f88b0101df0 2024-09-16T00:30:41.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.100+0000 7f88a77fe640 1 --2- 172.21.15.38:0/3325026836 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f889c0a3240 0x7f889c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:41.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.100+0000 7f88b59fb640 1 --2- 172.21.15.38:0/3325026836 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f889c0a3240 0x7f889c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:41.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.100+0000 7f88a77fe640 1 -- 172.21.15.38:0/3325026836 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f88ac0cfdb0 con 0x7f88b0101df0 2024-09-16T00:30:41.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.101+0000 7f88b59fb640 1 --2- 172.21.15.38:0/3325026836 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f889c0a3240 0x7f889c0a5700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f88b01087f0 tx=0x7f88a0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:41.127 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.104+0000 7f88a77fe640 1 -- 172.21.15.38:0/3325026836 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f88ac0d4070 con 0x7f88b0101df0 2024-09-16T00:30:41.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.285+0000 7f88b8460640 1 -- 172.21.15.38:0/3325026836 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f88b01a5120 con 0x7f88b0101df0 2024-09-16T00:30:41.285 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:41 smithi038 ceph-mon[30945]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-16T00:30:41.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.286+0000 7f88a77fe640 1 -- 172.21.15.38:0/3325026836 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24174 (secure 0 0 0) 0x7f88ac014030 con 0x7f88b0101df0 2024-09-16T00:30:41.287 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:30:41.287 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":88,"fsid":"983b2072-73c1-11ef-bceb-c7b262605968","created":"2024-09-16T00:22:34.897053+0000","modified":"2024-09-16T00:30:35.865128+0000","last_up_change":"2024-09-16T00:29:34.503134+0000","last_in_change":"2024-09-16T00:29:10.135656+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:25:41.931977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:30:22.741889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:30:25.042590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:30:31.554350+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3afc20e3-869f-43a9-a911-b0e8d07de86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6803","nonce":1622028921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6805","nonce":1622028921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6809","nonce":1622028921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6807","nonce":1622028921}]},"public_addr":"172.21.15.38:6803/1622028921","cluster_addr":"172.21.15.38:6805/1622028921","heartbeat_back_addr":"172.21.15.38:6809/1622028921","heartbeat_front_addr":"172.21.15.38:6807/1622028921","state":["exists","up"]},{"osd":1,"uuid":"6d9d7acb-45ed-4543-8afc-6433fc7f096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6801","nonce":3811690336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6803","nonce":3811690336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6807","nonce":3811690336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6805","nonce":3811690336}]},"public_addr":"172.21.15.73:6801/3811690336","cluster_addr":"172.21.15.73:6803/3811690336","heartbeat_back_addr":"172.21.15.73:6807/3811690336","heartbeat_front_addr":"172.21.15.73:6805/3811690336","state":["exists","up"]},{"osd":2,"uuid":"48690f35-99e9-4f45-9bee-487857caa8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6801","nonce":1754741655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6803","nonce":1754741655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6807","nonce":1754741655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6805","nonce":1754741655}]},"public_addr":"172.21.15.122:6801/1754741655","cluster_addr":"172.21.15.122:6803/1754741655","heartbeat_back_addr":"172.21.15.122:6807/1754741655","heartbeat_front_addr":"172.21.15.122:6805/1754741655","state":["exists","up"]},{"osd":3,"uuid":"28ebb4a0-92e7-4f57-82a5-b9b761eca269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6811","nonce":3523190079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6813","nonce":3523190079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6817","nonce":3523190079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6815","nonce":3523190079}]},"public_addr":"172.21.15.38:6811/3523190079","cluster_addr":"172.21.15.38:6813/3523190079","heartbeat_back_addr":"172.21.15.38:6817/3523190079","heartbeat_front_addr":"172.21.15.38:6815/3523190079","state":["exists","up"]},{"osd":4,"uuid":"5ea363f7-7ad8-45ed-b7e4-4b010fe7f8cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6809","nonce":1469927628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6811","nonce":1469927628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6815","nonce":1469927628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6813","nonce":1469927628}]},"public_addr":"172.21.15.73:6809/1469927628","cluster_addr":"172.21.15.73:6811/1469927628","heartbeat_back_addr":"172.21.15.73:6815/1469927628","heartbeat_front_addr":"172.21.15.73:6813/1469927628","state":["exists","up"]},{"osd":5,"uuid":"e3c66d43-dd28-4bb1-a886-62a869be82cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6809","nonce":2954413058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6811","nonce":2954413058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6815","nonce":2954413058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6813","nonce":2954413058}]},"public_addr":"172.21.15.122:6809/2954413058","cluster_addr":"172.21.15.122:6811/2954413058","heartbeat_back_addr":"172.21.15.122:6815/2954413058","heartbeat_front_addr":"172.21.15.122:6813/2954413058","state":["exists","up"]},{"osd":6,"uuid":"211497ac-7c48-4095-bfdb-1d4df0149f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6819","nonce":1415719550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6821","nonce":1415719550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6825","nonce":1415719550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6823","nonce":1415719550}]},"public_addr":"172.21.15.38:6819/1415719550","cluster_addr":"172.21.15.38:6821/1415719550","heartbeat_back_addr":"172.21.15.38:6825/1415719550","heartbeat_front_addr":"172.21.15.38:6823/1415719550","state":["exists","up"]},{"osd":7,"uuid":"06512315-3f5e-44f8-976d-d3f5f1ba00e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6817","nonce":2634361473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6819","nonce":2634361473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6823","nonce":2634361473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6821","nonce":2634361473}]},"public_addr":"172.21.15.73:6817/2634361473","cluster_addr":"172.21.15.73:6819/2634361473","heartbeat_back_addr":"172.21.15.73:6823/2634361473","heartbeat_front_addr":"172.21.15.73:6821/2634361473","state":["exists","up"]},{"osd":8,"uuid":"db703c0a-1f87-46f0-be5f-f826f7f9daac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6817","nonce":1800567633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6819","nonce":1800567633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6823","nonce":1800567633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6821","nonce":1800567633}]},"public_addr":"172.21.15.122:6817/1800567633","cluster_addr":"172.21.15.122:6819/1800567633","heartbeat_back_addr":"172.21.15.122:6823/1800567633","heartbeat_front_addr":"172.21.15.122:6821/1800567633","state":["exists","up"]},{"osd":9,"uuid":"f5384200-b3b4-4936-85e1-fc57a9fe2bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6827","nonce":445727833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6829","nonce":445727833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6833","nonce":445727833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6831","nonce":445727833}]},"public_addr":"172.21.15.38:6827/445727833","cluster_addr":"172.21.15.38:6829/445727833","heartbeat_back_addr":"172.21.15.38:6833/445727833","heartbeat_front_addr":"172.21.15.38:6831/445727833","state":["exists","up"]},{"osd":10,"uuid":"c22883d9-6672-4796-a961-963d21eb37fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6825","nonce":43297139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6827","nonce":43297139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6831","nonce":43297139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6829","nonce":43297139}]},"public_addr":"172.21.15.73:6825/43297139","cluster_addr":"172.21.15.73:6827/43297139","heartbeat_back_addr":"172.21.15.73:6831/43297139","heartbeat_front_addr":"172.21.15.73:6829/43297139","state":["exists","up"]},{"osd":11,"uuid":"9f98e8cb-1bff-4b8b-aa19-56b691387383","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6825","nonce":1886446061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6827","nonce":1886446061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6831","nonce":1886446061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6829","nonce":1886446061}]},"public_addr":"172.21.15.122:6825/1886446061","cluster_addr":"172.21.15.122:6827/1886446061","heartbeat_back_addr":"172.21.15.122:6831/1886446061","heartbeat_front_addr":"172.21.15.122:6829/1886446061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:47.384012+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:12.834488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:38.770055+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:04.800466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:30.808076+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:56.646967+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:27:22.721270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:27:48.510011+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:28:13.922944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:28:40.514947+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:29:06.031052+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:29:31.882062+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6801/1779324916":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/924150447":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/141237849":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/2302081750":"2024-09-17T00:22:56.844212+0000","172.21.15.38:0/2245468594":"2024-09-17T00:22:56.844212+0000","172.21.15.38:6801/583006373":"2024-09-17T00:22:56.844212+0000","172.21.15.38:0/2435492667":"2024-09-17T00:23:21.417483+0000","172.21.15.38:6800/583006373":"2024-09-17T00:22:56.844212+0000","172.21.15.38:6800/1779324916":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/1103516837":"2024-09-17T00:22:56.844212+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:30:41.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.289+0000 7f88b8460640 1 -- 172.21.15.38:0/3325026836 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f889c0a3240 msgr2=0x7f889c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:41.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.289+0000 7f88b8460640 1 --2- 172.21.15.38:0/3325026836 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f889c0a3240 0x7f889c0a5700 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f88b01087f0 tx=0x7f88a0005a20 comp rx=0 tx=0).stop 2024-09-16T00:30:41.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.289+0000 7f88b8460640 1 -- 172.21.15.38:0/3325026836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88b0101df0 msgr2=0x7f88b0198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:41.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.289+0000 7f88b8460640 1 --2- 172.21.15.38:0/3325026836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88b0101df0 0x7f88b0198300 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f88ac00c920 tx=0x7f88ac00cde0 comp rx=0 tx=0).stop 2024-09-16T00:30:41.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.289+0000 7f88b8460640 1 -- 172.21.15.38:0/3325026836 shutdown_connections 2024-09-16T00:30:41.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.289+0000 7f88b8460640 1 --2- 172.21.15.38:0/3325026836 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f889c0a3240 0x7f889c0a5700 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:41.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.289+0000 7f88b8460640 1 --2- 172.21.15.38:0/3325026836 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f88b0106f20 0x7f88b019cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:41.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.289+0000 7f88b8460640 1 --2- 172.21.15.38:0/3325026836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88b01027a0 0x7f88b0198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:41.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.289+0000 7f88b8460640 1 --2- 172.21.15.38:0/3325026836 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f88b0101df0 0x7f88b0198300 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:41.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.289+0000 7f88b8460640 1 -- 172.21.15.38:0/3325026836 >> 172.21.15.38:0/3325026836 conn(0x7f88b00fd8b0 msgr2=0x7f88b00ffca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:41.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.290+0000 7f88b8460640 1 -- 172.21.15.38:0/3325026836 shutdown_connections 2024-09-16T00:30:41.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:41.290+0000 7f88b8460640 1 -- 172.21.15.38:0/3325026836 wait complete. 2024-09-16T00:30:41.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:41 smithi122 ceph-mon[35412]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-16T00:30:41.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:41 smithi073 ceph-mon[35254]: pgmap v297: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2024-09-16T00:30:42.154 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:30:42.438 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:42.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:42 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/3325026836' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:30:42.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:42 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/3325026836' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:30:42.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:42 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/3325026836' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:30:42.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:42.965+0000 7f8961303640 1 -- 172.21.15.38:0/120581906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f895c101420 msgr2=0x7f895c10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:42.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:42.965+0000 7f8961303640 1 --2- 172.21.15.38:0/120581906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f895c101420 0x7f895c10d950 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8954009cd0 tx=0x7f895402a8e0 comp rx=0 tx=0).stop 2024-09-16T00:30:42.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:42.965+0000 7f8961303640 1 -- 172.21.15.38:0/120581906 shutdown_connections 2024-09-16T00:30:42.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:42.965+0000 7f8961303640 1 --2- 172.21.15.38:0/120581906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f895c101420 0x7f895c10d950 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:42.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:42.965+0000 7f8961303640 1 --2- 172.21.15.38:0/120581906 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f895c100a80 0x7f895c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:42.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:42.965+0000 7f8961303640 1 --2- 172.21.15.38:0/120581906 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f895c106a80 0x7f895c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:42.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:42.965+0000 7f8961303640 1 -- 172.21.15.38:0/120581906 >> 172.21.15.38:0/120581906 conn(0x7f895c0fc7a0 msgr2=0x7f895c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:42.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:42.965+0000 7f8961303640 1 -- 172.21.15.38:0/120581906 shutdown_connections 2024-09-16T00:30:42.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:42.965+0000 7f8961303640 1 -- 172.21.15.38:0/120581906 wait complete. 2024-09-16T00:30:42.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:42.966+0000 7f8961303640 1 Processor -- start 2024-09-16T00:30:42.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:42.966+0000 7f8961303640 1 -- start start 2024-09-16T00:30:42.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:42.966+0000 7f8961303640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f895c100a80 0x7f895c198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:42.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:42.966+0000 7f8961303640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f895c101420 0x7f895c198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:42.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:42.966+0000 7f8961303640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f895c106a80 0x7f895c19ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:42.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:42.966+0000 7f8961303640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f895c110e80 con 0x7f895c100a80 2024-09-16T00:30:42.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:42.966+0000 7f8961303640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f895c110d00 con 0x7f895c106a80 2024-09-16T00:30:42.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:42.966+0000 7f8961303640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f895c111000 con 0x7f895c101420 2024-09-16T00:30:42.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:42.966+0000 7f895affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f895c100a80 0x7f895c198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:42.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:42.966+0000 7f895affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f895c100a80 0x7f895c198420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52800/0 (socket says 172.21.15.38:52800) 2024-09-16T00:30:42.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:42.966+0000 7f895affd640 1 -- 172.21.15.38:0/2434330154 learned_addr learned my addr 172.21.15.38:0/2434330154 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:30:42.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:42.966+0000 7f895b7fe640 1 --2- 172.21.15.38:0/2434330154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f895c106a80 0x7f895c19ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:42.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:42.966+0000 7f8953fff640 1 --2- 172.21.15.38:0/2434330154 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f895c101420 0x7f895c198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:42.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:42.966+0000 7f895affd640 1 -- 172.21.15.38:0/2434330154 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f895c101420 msgr2=0x7f895c198960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:42.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:42.966+0000 7f895affd640 1 --2- 172.21.15.38:0/2434330154 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f895c101420 0x7f895c198960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:42.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:42.966+0000 7f895affd640 1 -- 172.21.15.38:0/2434330154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f895c106a80 msgr2=0x7f895c19ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:42.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:42.966+0000 7f895affd640 1 --2- 172.21.15.38:0/2434330154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f895c106a80 0x7f895c19ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:42.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:42.966+0000 7f895affd640 1 -- 172.21.15.38:0/2434330154 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f895c19d3d0 con 0x7f895c100a80 2024-09-16T00:30:42.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:42.966+0000 7f8953fff640 1 --2- 172.21.15.38:0/2434330154 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f895c101420 0x7f895c198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:42.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:42.967+0000 7f895affd640 1 --2- 172.21.15.38:0/2434330154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f895c100a80 0x7f895c198420 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f894c00d950 tx=0x7f894c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:42.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:42.967+0000 7f8958ff9640 1 -- 172.21.15.38:0/2434330154 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894c017070 con 0x7f895c100a80 2024-09-16T00:30:42.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:42.967+0000 7f8958ff9640 1 -- 172.21.15.38:0/2434330154 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f894c009c30 con 0x7f895c100a80 2024-09-16T00:30:42.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:42.967+0000 7f8961303640 1 -- 172.21.15.38:0/2434330154 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f895c19d6c0 con 0x7f895c100a80 2024-09-16T00:30:42.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:42.967+0000 7f8958ff9640 1 -- 172.21.15.38:0/2434330154 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f894c00f6b0 con 0x7f895c100a80 2024-09-16T00:30:42.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:42.967+0000 7f8961303640 1 -- 172.21.15.38:0/2434330154 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f895c1a4fa0 con 0x7f895c100a80 2024-09-16T00:30:42.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:42.968+0000 7f8961303640 1 -- 172.21.15.38:0/2434330154 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f895c100ee0 con 0x7f895c100a80 2024-09-16T00:30:42.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:42.969+0000 7f8958ff9640 1 -- 172.21.15.38:0/2434330154 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f894c0109d0 con 0x7f895c100a80 2024-09-16T00:30:42.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:42.970+0000 7f8958ff9640 1 --2- 172.21.15.38:0/2434330154 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f89480a3240 0x7f89480a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:42.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:42.970+0000 7f8958ff9640 1 -- 172.21.15.38:0/2434330154 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f894c0cf700 con 0x7f895c100a80 2024-09-16T00:30:42.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:42.970+0000 7f8953fff640 1 --2- 172.21.15.38:0/2434330154 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f89480a3240 0x7f89480a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:42.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:42.970+0000 7f8953fff640 1 --2- 172.21.15.38:0/2434330154 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f89480a3240 0x7f89480a5700 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f895c06b870 tx=0x7f8944005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:42.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:42.973+0000 7f8958ff9640 1 -- 172.21.15.38:0/2434330154 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f894c009de0 con 0x7f895c100a80 2024-09-16T00:30:43.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:43.185+0000 7f8961303640 1 -- 172.21.15.38:0/2434330154 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f895c199780 con 0x7f895c100a80 2024-09-16T00:30:43.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:43.186+0000 7f8958ff9640 1 -- 172.21.15.38:0/2434330154 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11836 (secure 0 0 0) 0x7f894c09c1e0 con 0x7f895c100a80 2024-09-16T00:30:43.186 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:30:43.187 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":17,"btime":"2024-09-16T00:30:36:145982+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/367974527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":367974527},{"type":"v1","addr":"172.21.15.38:6835","nonce":367974527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/335077136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":335077136},{"type":"v1","addr":"172.21.15.38:6837","nonce":335077136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/3501211160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3501211160},{"type":"v1","addr":"172.21.15.38:6839","nonce":3501211160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6841/2623130915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":2623130915},{"type":"v1","addr":"172.21.15.38:6841","nonce":2623130915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/412277519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":412277519},{"type":"v1","addr":"172.21.15.122:6833","nonce":412277519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3987234757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3987234757},{"type":"v1","addr":"172.21.15.122:6835","nonce":3987234757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24416,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/1210328177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1210328177},{"type":"v1","addr":"172.21.15.73:6833","nonce":1210328177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24421,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6837/1858299680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1858299680},{"type":"v1","addr":"172.21.15.122:6837","nonce":1858299680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24436,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6839/1983975311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1983975311},{"type":"v1","addr":"172.21.15.122:6839","nonce":1983975311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24440,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2866363113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2866363113},{"type":"v1","addr":"172.21.15.73:6835","nonce":2866363113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24464,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2681081265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2681081265},{"type":"v1","addr":"172.21.15.73:6837","nonce":2681081265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-16T00:30:26.764565+0000","modified":"2024-09-16T00:30:35.865257+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24494},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.73:6839/2396224525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2396224525},{"type":"v1","addr":"172.21.15.73:6839","nonce":2396224525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24494,"qdb_cluster":[24494]},"id":1}]} 2024-09-16T00:30:43.187 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 17 2024-09-16T00:30:43.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:43.188+0000 7f8961303640 1 -- 172.21.15.38:0/2434330154 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f89480a3240 msgr2=0x7f89480a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:43.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:43.188+0000 7f8961303640 1 --2- 172.21.15.38:0/2434330154 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f89480a3240 0x7f89480a5700 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f895c06b870 tx=0x7f8944005e50 comp rx=0 tx=0).stop 2024-09-16T00:30:43.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:43.188+0000 7f8961303640 1 -- 172.21.15.38:0/2434330154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f895c100a80 msgr2=0x7f895c198420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:43.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:43.188+0000 7f8961303640 1 --2- 172.21.15.38:0/2434330154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f895c100a80 0x7f895c198420 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f894c00d950 tx=0x7f894c00de10 comp rx=0 tx=0).stop 2024-09-16T00:30:43.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:43.188+0000 7f8961303640 1 -- 172.21.15.38:0/2434330154 shutdown_connections 2024-09-16T00:30:43.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:43.188+0000 7f8961303640 1 --2- 172.21.15.38:0/2434330154 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f89480a3240 0x7f89480a5700 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:43.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:43.188+0000 7f8961303640 1 --2- 172.21.15.38:0/2434330154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f895c106a80 0x7f895c19ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:43.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:43.188+0000 7f8961303640 1 --2- 172.21.15.38:0/2434330154 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f895c101420 0x7f895c198960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:43.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:43.188+0000 7f8961303640 1 --2- 172.21.15.38:0/2434330154 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f895c100a80 0x7f895c198420 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:43.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:43.188+0000 7f8961303640 1 -- 172.21.15.38:0/2434330154 >> 172.21.15.38:0/2434330154 conn(0x7f895c0fc7a0 msgr2=0x7f895c1047f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:43.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:43.188+0000 7f8961303640 1 -- 172.21.15.38:0/2434330154 shutdown_connections 2024-09-16T00:30:43.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:43.188+0000 7f8961303640 1 -- 172.21.15.38:0/2434330154 wait complete. 2024-09-16T00:30:43.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:43 smithi038 ceph-mon[30945]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:43.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:43 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/2434330154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:30:43.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:43 smithi122 ceph-mon[35412]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:43.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:43 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/2434330154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:30:43.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:43 smithi073 ceph-mon[35254]: pgmap v298: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:43.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:43 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/2434330154' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:30:43.903 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:30:44.162 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:44.675 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.639+0000 7f0d021e2640 1 -- 172.21.15.38:0/1227155588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cfc101de0 msgr2=0x7f0cfc1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:44.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.639+0000 7f0d021e2640 1 --2- 172.21.15.38:0/1227155588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cfc101de0 0x7f0cfc1021c0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0cec009900 tx=0x7f0cec02a410 comp rx=0 tx=0).stop 2024-09-16T00:30:44.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.640+0000 7f0d021e2640 1 -- 172.21.15.38:0/1227155588 shutdown_connections 2024-09-16T00:30:44.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.640+0000 7f0d021e2640 1 --2- 172.21.15.38:0/1227155588 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cfc106f10 0x7f0cfc10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:44.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.640+0000 7f0d021e2640 1 --2- 172.21.15.38:0/1227155588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0cfc102790 0x7f0cfc1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:44.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.640+0000 7f0d021e2640 1 --2- 172.21.15.38:0/1227155588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cfc101de0 0x7f0cfc1021c0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:44.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.640+0000 7f0d021e2640 1 -- 172.21.15.38:0/1227155588 >> 172.21.15.38:0/1227155588 conn(0x7f0cfc0fd8c0 msgr2=0x7f0cfc0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:44.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.640+0000 7f0d021e2640 1 -- 172.21.15.38:0/1227155588 shutdown_connections 2024-09-16T00:30:44.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.640+0000 7f0d021e2640 1 -- 172.21.15.38:0/1227155588 wait complete. 2024-09-16T00:30:44.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.640+0000 7f0d021e2640 1 Processor -- start 2024-09-16T00:30:44.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.640+0000 7f0d021e2640 1 -- start start 2024-09-16T00:30:44.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.641+0000 7f0d021e2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cfc101de0 0x7f0cfc0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:44.676 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.641+0000 7f0d021e2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cfc102790 0x7f0cfc074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:44.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.641+0000 7f0d021e2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0cfc106f10 0x7f0cfc074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:44.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.641+0000 7f0d021e2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc111f70 con 0x7f0cfc102790 2024-09-16T00:30:44.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.641+0000 7f0d021e2640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc111df0 con 0x7f0cfc101de0 2024-09-16T00:30:44.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.641+0000 7f0d021e2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0cfc1120f0 con 0x7f0cfc106f10 2024-09-16T00:30:44.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.641+0000 7f0cfaffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cfc102790 0x7f0cfc074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:44.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.641+0000 7f0cfaffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cfc102790 0x7f0cfc074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:52804/0 (socket says 172.21.15.38:52804) 2024-09-16T00:30:44.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.641+0000 7f0cfaffd640 1 -- 172.21.15.38:0/2025001504 learned_addr learned my addr 172.21.15.38:0/2025001504 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:30:44.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.641+0000 7f0cfbfff640 1 --2- 172.21.15.38:0/2025001504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0cfc106f10 0x7f0cfc074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:44.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.641+0000 7f0cfb7fe640 1 --2- 172.21.15.38:0/2025001504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cfc101de0 0x7f0cfc0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:44.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.641+0000 7f0cfaffd640 1 -- 172.21.15.38:0/2025001504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0cfc106f10 msgr2=0x7f0cfc074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:44.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.641+0000 7f0cfaffd640 1 --2- 172.21.15.38:0/2025001504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0cfc106f10 0x7f0cfc074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:44.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.641+0000 7f0cfaffd640 1 -- 172.21.15.38:0/2025001504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cfc101de0 msgr2=0x7f0cfc0744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:44.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.641+0000 7f0cfaffd640 1 --2- 172.21.15.38:0/2025001504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cfc101de0 0x7f0cfc0744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:44.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.641+0000 7f0cfaffd640 1 -- 172.21.15.38:0/2025001504 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cfc1a6890 con 0x7f0cfc102790 2024-09-16T00:30:44.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.641+0000 7f0cfb7fe640 1 --2- 172.21.15.38:0/2025001504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cfc101de0 0x7f0cfc0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:30:44.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.641+0000 7f0cfaffd640 1 --2- 172.21.15.38:0/2025001504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cfc102790 0x7f0cfc074a30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0cf000d770 tx=0x7f0cf000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:44.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.641+0000 7f0cf8ff9640 1 -- 172.21.15.38:0/2025001504 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0017070 con 0x7f0cfc102790 2024-09-16T00:30:44.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.641+0000 7f0cf8ff9640 1 -- 172.21.15.38:0/2025001504 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0cf0005500 con 0x7f0cfc102790 2024-09-16T00:30:44.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.641+0000 7f0d021e2640 1 -- 172.21.15.38:0/2025001504 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cfc1a6b80 con 0x7f0cfc102790 2024-09-16T00:30:44.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.641+0000 7f0cf8ff9640 1 -- 172.21.15.38:0/2025001504 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0004e00 con 0x7f0cfc102790 2024-09-16T00:30:44.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.642+0000 7f0d021e2640 1 -- 172.21.15.38:0/2025001504 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cfc1a70c0 con 0x7f0cfc102790 2024-09-16T00:30:44.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.643+0000 7f0d021e2640 1 -- 172.21.15.38:0/2025001504 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cfc1021c0 con 0x7f0cfc102790 2024-09-16T00:30:44.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.644+0000 7f0cf8ff9640 1 -- 172.21.15.38:0/2025001504 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f0cf0004fa0 con 0x7f0cfc102790 2024-09-16T00:30:44.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.644+0000 7f0cf8ff9640 1 --2- 172.21.15.38:0/2025001504 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f0ce40a32c0 0x7f0ce40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:44.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.645+0000 7f0cf8ff9640 1 -- 172.21.15.38:0/2025001504 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f0cf00cf560 con 0x7f0cfc102790 2024-09-16T00:30:44.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.645+0000 7f0cfb7fe640 1 --2- 172.21.15.38:0/2025001504 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f0ce40a32c0 0x7f0ce40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:44.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.645+0000 7f0cfb7fe640 1 --2- 172.21.15.38:0/2025001504 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f0ce40a32c0 0x7f0ce40a5780 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0cec005ab0 tx=0x7f0cec005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:44.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.649+0000 7f0cf8ff9640 1 -- 172.21.15.38:0/2025001504 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0cf009c040 con 0x7f0cfc102790 2024-09-16T00:30:44.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.824+0000 7f0d021e2640 1 -- 172.21.15.38:0/2025001504 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0cfc10a0f0 con 0x7f0cfc102790 2024-09-16T00:30:44.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.825+0000 7f0cf8ff9640 1 -- 172.21.15.38:0/2025001504 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24174 (secure 0 0 0) 0x7f0cf009b940 con 0x7f0cfc102790 2024-09-16T00:30:44.826 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:30:44.826 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":88,"fsid":"983b2072-73c1-11ef-bceb-c7b262605968","created":"2024-09-16T00:22:34.897053+0000","modified":"2024-09-16T00:30:35.865128+0000","last_up_change":"2024-09-16T00:29:34.503134+0000","last_in_change":"2024-09-16T00:29:10.135656+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:25:41.931977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:30:22.741889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:30:25.042590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:30:31.554350+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3afc20e3-869f-43a9-a911-b0e8d07de86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6803","nonce":1622028921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6805","nonce":1622028921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6809","nonce":1622028921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6807","nonce":1622028921}]},"public_addr":"172.21.15.38:6803/1622028921","cluster_addr":"172.21.15.38:6805/1622028921","heartbeat_back_addr":"172.21.15.38:6809/1622028921","heartbeat_front_addr":"172.21.15.38:6807/1622028921","state":["exists","up"]},{"osd":1,"uuid":"6d9d7acb-45ed-4543-8afc-6433fc7f096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6801","nonce":3811690336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6803","nonce":3811690336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6807","nonce":3811690336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6805","nonce":3811690336}]},"public_addr":"172.21.15.73:6801/3811690336","cluster_addr":"172.21.15.73:6803/3811690336","heartbeat_back_addr":"172.21.15.73:6807/3811690336","heartbeat_front_addr":"172.21.15.73:6805/3811690336","state":["exists","up"]},{"osd":2,"uuid":"48690f35-99e9-4f45-9bee-487857caa8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6801","nonce":1754741655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6803","nonce":1754741655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6807","nonce":1754741655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6805","nonce":1754741655}]},"public_addr":"172.21.15.122:6801/1754741655","cluster_addr":"172.21.15.122:6803/1754741655","heartbeat_back_addr":"172.21.15.122:6807/1754741655","heartbeat_front_addr":"172.21.15.122:6805/1754741655","state":["exists","up"]},{"osd":3,"uuid":"28ebb4a0-92e7-4f57-82a5-b9b761eca269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6811","nonce":3523190079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6813","nonce":3523190079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6817","nonce":3523190079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6815","nonce":3523190079}]},"public_addr":"172.21.15.38:6811/3523190079","cluster_addr":"172.21.15.38:6813/3523190079","heartbeat_back_addr":"172.21.15.38:6817/3523190079","heartbeat_front_addr":"172.21.15.38:6815/3523190079","state":["exists","up"]},{"osd":4,"uuid":"5ea363f7-7ad8-45ed-b7e4-4b010fe7f8cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6809","nonce":1469927628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6811","nonce":1469927628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6815","nonce":1469927628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6813","nonce":1469927628}]},"public_addr":"172.21.15.73:6809/1469927628","cluster_addr":"172.21.15.73:6811/1469927628","heartbeat_back_addr":"172.21.15.73:6815/1469927628","heartbeat_front_addr":"172.21.15.73:6813/1469927628","state":["exists","up"]},{"osd":5,"uuid":"e3c66d43-dd28-4bb1-a886-62a869be82cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6809","nonce":2954413058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6811","nonce":2954413058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6815","nonce":2954413058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6813","nonce":2954413058}]},"public_addr":"172.21.15.122:6809/2954413058","cluster_addr":"172.21.15.122:6811/2954413058","heartbeat_back_addr":"172.21.15.122:6815/2954413058","heartbeat_front_addr":"172.21.15.122:6813/2954413058","state":["exists","up"]},{"osd":6,"uuid":"211497ac-7c48-4095-bfdb-1d4df0149f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6819","nonce":1415719550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6821","nonce":1415719550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6825","nonce":1415719550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6823","nonce":1415719550}]},"public_addr":"172.21.15.38:6819/1415719550","cluster_addr":"172.21.15.38:6821/1415719550","heartbeat_back_addr":"172.21.15.38:6825/1415719550","heartbeat_front_addr":"172.21.15.38:6823/1415719550","state":["exists","up"]},{"osd":7,"uuid":"06512315-3f5e-44f8-976d-d3f5f1ba00e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6817","nonce":2634361473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6819","nonce":2634361473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6823","nonce":2634361473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6821","nonce":2634361473}]},"public_addr":"172.21.15.73:6817/2634361473","cluster_addr":"172.21.15.73:6819/2634361473","heartbeat_back_addr":"172.21.15.73:6823/2634361473","heartbeat_front_addr":"172.21.15.73:6821/2634361473","state":["exists","up"]},{"osd":8,"uuid":"db703c0a-1f87-46f0-be5f-f826f7f9daac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6817","nonce":1800567633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6819","nonce":1800567633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6823","nonce":1800567633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6821","nonce":1800567633}]},"public_addr":"172.21.15.122:6817/1800567633","cluster_addr":"172.21.15.122:6819/1800567633","heartbeat_back_addr":"172.21.15.122:6823/1800567633","heartbeat_front_addr":"172.21.15.122:6821/1800567633","state":["exists","up"]},{"osd":9,"uuid":"f5384200-b3b4-4936-85e1-fc57a9fe2bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6827","nonce":445727833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6829","nonce":445727833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6833","nonce":445727833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6831","nonce":445727833}]},"public_addr":"172.21.15.38:6827/445727833","cluster_addr":"172.21.15.38:6829/445727833","heartbeat_back_addr":"172.21.15.38:6833/445727833","heartbeat_front_addr":"172.21.15.38:6831/445727833","state":["exists","up"]},{"osd":10,"uuid":"c22883d9-6672-4796-a961-963d21eb37fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6825","nonce":43297139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6827","nonce":43297139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6831","nonce":43297139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6829","nonce":43297139}]},"public_addr":"172.21.15.73:6825/43297139","cluster_addr":"172.21.15.73:6827/43297139","heartbeat_back_addr":"172.21.15.73:6831/43297139","heartbeat_front_addr":"172.21.15.73:6829/43297139","state":["exists","up"]},{"osd":11,"uuid":"9f98e8cb-1bff-4b8b-aa19-56b691387383","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6825","nonce":1886446061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6827","nonce":1886446061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6831","nonce":1886446061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6829","nonce":1886446061}]},"public_addr":"172.21.15.122:6825/1886446061","cluster_addr":"172.21.15.122:6827/1886446061","heartbeat_back_addr":"172.21.15.122:6831/1886446061","heartbeat_front_addr":"172.21.15.122:6829/1886446061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:47.384012+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:12.834488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:38.770055+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:04.800466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:30.808076+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:56.646967+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:27:22.721270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:27:48.510011+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:28:13.922944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:28:40.514947+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:29:06.031052+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:29:31.882062+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6801/1779324916":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/924150447":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/141237849":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/2302081750":"2024-09-17T00:22:56.844212+0000","172.21.15.38:0/2245468594":"2024-09-17T00:22:56.844212+0000","172.21.15.38:6801/583006373":"2024-09-17T00:22:56.844212+0000","172.21.15.38:0/2435492667":"2024-09-17T00:23:21.417483+0000","172.21.15.38:6800/583006373":"2024-09-17T00:22:56.844212+0000","172.21.15.38:6800/1779324916":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/1103516837":"2024-09-17T00:22:56.844212+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:30:44.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.828+0000 7f0d021e2640 1 -- 172.21.15.38:0/2025001504 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f0ce40a32c0 msgr2=0x7f0ce40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:44.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.828+0000 7f0d021e2640 1 --2- 172.21.15.38:0/2025001504 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f0ce40a32c0 0x7f0ce40a5780 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0cec005ab0 tx=0x7f0cec005a20 comp rx=0 tx=0).stop 2024-09-16T00:30:44.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.828+0000 7f0d021e2640 1 -- 172.21.15.38:0/2025001504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cfc102790 msgr2=0x7f0cfc074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:44.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.828+0000 7f0d021e2640 1 --2- 172.21.15.38:0/2025001504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cfc102790 0x7f0cfc074a30 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0cf000d770 tx=0x7f0cf000dc30 comp rx=0 tx=0).stop 2024-09-16T00:30:44.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.828+0000 7f0d021e2640 1 -- 172.21.15.38:0/2025001504 shutdown_connections 2024-09-16T00:30:44.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.828+0000 7f0d021e2640 1 --2- 172.21.15.38:0/2025001504 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f0ce40a32c0 0x7f0ce40a5780 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:44.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.828+0000 7f0d021e2640 1 --2- 172.21.15.38:0/2025001504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0cfc106f10 0x7f0cfc074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:44.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.828+0000 7f0d021e2640 1 --2- 172.21.15.38:0/2025001504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0cfc102790 0x7f0cfc074a30 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:44.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.828+0000 7f0d021e2640 1 --2- 172.21.15.38:0/2025001504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cfc101de0 0x7f0cfc0744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:44.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.828+0000 7f0d021e2640 1 -- 172.21.15.38:0/2025001504 >> 172.21.15.38:0/2025001504 conn(0x7f0cfc0fd8c0 msgr2=0x7f0cfc0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:44.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.828+0000 7f0d021e2640 1 -- 172.21.15.38:0/2025001504 shutdown_connections 2024-09-16T00:30:44.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:44.828+0000 7f0d021e2640 1 -- 172.21.15.38:0/2025001504 wait complete. 2024-09-16T00:30:45.463 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:45 smithi038 ceph-mon[30945]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:45.463 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:45 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/2025001504' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:30:45.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:45 smithi122 ceph-mon[35412]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:45.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:45 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/2025001504' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:30:45.493 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-16T00:30:26.764565+0000', 'modified': '2024-09-16T00:30:35.865257+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24494}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24494': {'gid': 24494, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.73:6839/2396224525', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6838', 'nonce': 2396224525}, {'type': 'v1', 'addr': '172.21.15.73:6839', 'nonce': 2396224525}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24494, 'qdb_cluster': [24494]} 2024-09-16T00:30:45.493 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-16T00:30:45.493 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-16T00:30:45.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:45 smithi073 ceph-mon[35254]: pgmap v299: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:45.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:45 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/2025001504' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:30:45.752 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:46.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.245+0000 7fe52c175640 1 -- 172.21.15.38:0/3087042168 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe524102ef0 msgr2=0x7fe52410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:46.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.245+0000 7fe52c175640 1 --2- 172.21.15.38:0/3087042168 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe524102ef0 0x7fe52410da40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe51c009ca0 tx=0x7fe51c02a950 comp rx=0 tx=0).stop 2024-09-16T00:30:46.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.245+0000 7fe52c175640 1 -- 172.21.15.38:0/3087042168 shutdown_connections 2024-09-16T00:30:46.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.245+0000 7fe52c175640 1 --2- 172.21.15.38:0/3087042168 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe524102ef0 0x7fe52410da40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.245+0000 7fe52c175640 1 --2- 172.21.15.38:0/3087042168 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe524102570 0x7fe5241029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.245+0000 7fe52c175640 1 --2- 172.21.15.38:0/3087042168 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe524100e20 0x7fe524101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.245+0000 7fe52c175640 1 -- 172.21.15.38:0/3087042168 >> 172.21.15.38:0/3087042168 conn(0x7fe5240fc7d0 msgr2=0x7fe5240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:46.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.245+0000 7fe52c175640 1 -- 172.21.15.38:0/3087042168 shutdown_connections 2024-09-16T00:30:46.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.245+0000 7fe52c175640 1 -- 172.21.15.38:0/3087042168 wait complete. 2024-09-16T00:30:46.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.245+0000 7fe52c175640 1 Processor -- start 2024-09-16T00:30:46.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.246+0000 7fe52c175640 1 -- start start 2024-09-16T00:30:46.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.246+0000 7fe52c175640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe524100e20 0x7fe524198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:46.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.246+0000 7fe52c175640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe524102570 0x7fe5241989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:46.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.246+0000 7fe52c175640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe524102ef0 0x7fe52419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:46.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.246+0000 7fe52c175640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe524110f80 con 0x7fe524102570 2024-09-16T00:30:46.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.246+0000 7fe52c175640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe524110e00 con 0x7fe524102ef0 2024-09-16T00:30:46.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.246+0000 7fe52c175640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe524111100 con 0x7fe524100e20 2024-09-16T00:30:46.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.246+0000 7fe529710640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe524102570 0x7fe5241989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:46.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.246+0000 7fe529710640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe524102570 0x7fe5241989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32776/0 (socket says 172.21.15.38:32776) 2024-09-16T00:30:46.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.246+0000 7fe529710640 1 -- 172.21.15.38:0/4206646894 learned_addr learned my addr 172.21.15.38:0/4206646894 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:30:46.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.246+0000 7fe529f11640 1 --2- 172.21.15.38:0/4206646894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe524100e20 0x7fe524198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:46.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.246+0000 7fe529710640 1 -- 172.21.15.38:0/4206646894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe524100e20 msgr2=0x7fe524198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:46.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.246+0000 7fe529710640 1 --2- 172.21.15.38:0/4206646894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe524100e20 0x7fe524198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.246+0000 7fe529710640 1 -- 172.21.15.38:0/4206646894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe524102ef0 msgr2=0x7fe52419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:46.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.246+0000 7fe529710640 1 --2- 172.21.15.38:0/4206646894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe524102ef0 0x7fe52419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.246+0000 7fe529710640 1 -- 172.21.15.38:0/4206646894 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe52419d420 con 0x7fe524102570 2024-09-16T00:30:46.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.246+0000 7fe529710640 1 --2- 172.21.15.38:0/4206646894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe524102570 0x7fe5241989b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe51400c920 tx=0x7fe51400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:46.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.247+0000 7fe51affd640 1 -- 172.21.15.38:0/4206646894 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514007c70 con 0x7fe524102570 2024-09-16T00:30:46.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.247+0000 7fe52c175640 1 -- 172.21.15.38:0/4206646894 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe52419d710 con 0x7fe524102570 2024-09-16T00:30:46.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.247+0000 7fe51affd640 1 -- 172.21.15.38:0/4206646894 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe514007e10 con 0x7fe524102570 2024-09-16T00:30:46.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.247+0000 7fe51affd640 1 -- 172.21.15.38:0/4206646894 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe514018890 con 0x7fe524102570 2024-09-16T00:30:46.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.247+0000 7fe52c175640 1 -- 172.21.15.38:0/4206646894 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5241a4ff0 con 0x7fe524102570 2024-09-16T00:30:46.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.249+0000 7fe52e54e740 1 Processor -- start 2024-09-16T00:30:46.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.249+0000 7fe52e54e740 1 -- start start 2024-09-16T00:30:46.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.249+0000 7fe51affd640 1 -- 172.21.15.38:0/4206646894 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fe514018a70 con 0x7fe524102570 2024-09-16T00:30:46.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.250+0000 7fe52e54e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613d117d8d0 0x5613d117dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:46.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.250+0000 7fe52e54e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613d117e1f0 0x5613d118ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:46.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.250+0000 7fe52e54e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613d118cfc0 0x5613d118f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:46.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.250+0000 7fe52e54e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5613d10e4090 con 0x5613d117d8d0 2024-09-16T00:30:46.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.250+0000 7fe52e54e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5613d108ba70 con 0x5613d117e1f0 2024-09-16T00:30:46.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.250+0000 7fe52e54e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5613d10e0050 con 0x5613d118cfc0 2024-09-16T00:30:46.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.250+0000 7fe52a712640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613d117d8d0 0x5613d117dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:46.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.250+0000 7fe52a712640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613d117d8d0 0x5613d117dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32792/0 (socket says 172.21.15.38:32792) 2024-09-16T00:30:46.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.250+0000 7fe52a712640 1 -- 172.21.15.38:0/2393070216 learned_addr learned my addr 172.21.15.38:0/2393070216 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:30:46.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.250+0000 7fe529f11640 1 --2- 172.21.15.38:0/2393070216 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613d117e1f0 0x5613d118ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:46.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.250+0000 7fe529710640 1 --2- 172.21.15.38:0/2393070216 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613d118cfc0 0x5613d118f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:46.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.250+0000 7fe52a712640 1 -- 172.21.15.38:0/2393070216 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613d118cfc0 msgr2=0x5613d118f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:46.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.250+0000 7fe52a712640 1 --2- 172.21.15.38:0/2393070216 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613d118cfc0 0x5613d118f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.250+0000 7fe52a712640 1 -- 172.21.15.38:0/2393070216 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613d117e1f0 msgr2=0x5613d118ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:46.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.250+0000 7fe52a712640 1 --2- 172.21.15.38:0/2393070216 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613d117e1f0 0x5613d118ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.250+0000 7fe52a712640 1 -- 172.21.15.38:0/2393070216 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613d1098130 con 0x5613d117d8d0 2024-09-16T00:30:46.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.250+0000 7fe529710640 1 --2- 172.21.15.38:0/2393070216 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613d118cfc0 0x5613d118f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:46.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.250+0000 7fe529f11640 1 --2- 172.21.15.38:0/2393070216 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613d117e1f0 0x5613d118ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:30:46.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.251+0000 7fe52a712640 1 --2- 172.21.15.38:0/2393070216 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613d117d8d0 0x5613d117dcb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe51c005e30 tx=0x7fe51c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe0cb53f706fb995 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:46.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.251+0000 7fe51affd640 1 --2- 172.21.15.38:0/4206646894 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fe5100a3240 0x7fe5100a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:46.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.251+0000 7fe50ffff640 1 -- 172.21.15.38:0/2393070216 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c034040 con 0x5613d117d8d0 2024-09-16T00:30:46.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.251+0000 7fe50ffff640 1 -- 172.21.15.38:0/2393070216 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe51c02fb30 con 0x5613d117d8d0 2024-09-16T00:30:46.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.251+0000 7fe529f11640 1 --2- 172.21.15.38:0/4206646894 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fe5100a3240 0x7fe5100a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:46.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.251+0000 7fe51affd640 1 -- 172.21.15.38:0/4206646894 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fe5140cfd00 con 0x7fe524102570 2024-09-16T00:30:46.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.251+0000 7fe50ffff640 1 -- 172.21.15.38:0/2393070216 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51c004460 con 0x5613d117d8d0 2024-09-16T00:30:46.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.252+0000 7fe52e54e740 1 -- 172.21.15.38:0/2393070216 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613d117d8d0 msgr2=0x5613d117dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:46.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.252+0000 7fe52e54e740 1 --2- 172.21.15.38:0/2393070216 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613d117d8d0 0x5613d117dcb0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe51c005e30 tx=0x7fe51c005fd0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.252+0000 7fe529f11640 1 --2- 172.21.15.38:0/4206646894 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fe5100a3240 0x7fe5100a5700 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x5613d11805c0 tx=0x7fe52000b710 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:46.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.252+0000 7fe52e54e740 1 -- 172.21.15.38:0/2393070216 shutdown_connections 2024-09-16T00:30:46.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.252+0000 7fe52e54e740 1 --2- 172.21.15.38:0/2393070216 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613d118cfc0 0x5613d118f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.252+0000 7fe52e54e740 1 --2- 172.21.15.38:0/2393070216 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613d117e1f0 0x5613d118ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.252+0000 7fe52e54e740 1 --2- 172.21.15.38:0/2393070216 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613d117d8d0 0x5613d117dcb0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.252+0000 7fe52e54e740 1 -- 172.21.15.38:0/2393070216 >> 172.21.15.38:0/2393070216 conn(0x5613d1179f60 msgr2=0x5613d117a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:46.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.252+0000 7fe52e54e740 1 -- 172.21.15.38:0/2393070216 shutdown_connections 2024-09-16T00:30:46.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.252+0000 7fe52e54e740 1 -- 172.21.15.38:0/2393070216 wait complete. 2024-09-16T00:30:46.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.252+0000 7fe52e54e740 1 -- start start 2024-09-16T00:30:46.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.253+0000 7fe52e54e740 1 Processor -- start 2024-09-16T00:30:46.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.253+0000 7fe52e54e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613d117d8d0 0x5613d118c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:46.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.253+0000 7fe52e54e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613d117e1f0 0x5613d1189b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:46.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.253+0000 7fe52e54e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613d118cfc0 0x5613d118a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:46.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.253+0000 7fe52e54e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5613d118fbe0 con 0x5613d117d8d0 2024-09-16T00:30:46.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.253+0000 7fe52e54e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5613d109ef20 con 0x5613d118cfc0 2024-09-16T00:30:46.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.253+0000 7fe52e54e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5613d118fd60 con 0x5613d117e1f0 2024-09-16T00:30:46.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.253+0000 7fe52e54e740 10 client.0 _ll_register_callbacks cb 0x5613d1068030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:30:46.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.253+0000 7fe529f11640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613d117d8d0 0x5613d118c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:46.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.253+0000 7fe529f11640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613d117d8d0 0x5613d118c810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32800/0 (socket says 172.21.15.38:32800) 2024-09-16T00:30:46.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.253+0000 7fe529f11640 1 -- 172.21.15.38:0/2758140425 learned_addr learned my addr 172.21.15.38:0/2758140425 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:30:46.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.253+0000 7fe529710640 1 --2- 172.21.15.38:0/2758140425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613d117e1f0 0x5613d1189b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:46.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.253+0000 7fe52a712640 1 --2- 172.21.15.38:0/2758140425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613d118cfc0 0x5613d118a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:46.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.254+0000 7fe529f11640 1 -- 172.21.15.38:0/2758140425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613d117e1f0 msgr2=0x5613d1189b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:46.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.254+0000 7fe529f11640 1 --2- 172.21.15.38:0/2758140425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613d117e1f0 0x5613d1189b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.254+0000 7fe529f11640 1 -- 172.21.15.38:0/2758140425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613d118cfc0 msgr2=0x5613d118a0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:46.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.254+0000 7fe529f11640 1 --2- 172.21.15.38:0/2758140425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613d118cfc0 0x5613d118a0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.254+0000 7fe529f11640 1 -- 172.21.15.38:0/2758140425 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613d0e552b0 con 0x5613d117d8d0 2024-09-16T00:30:46.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.254+0000 7fe529f11640 1 --2- 172.21.15.38:0/2758140425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613d117d8d0 0x5613d118c810 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe52000b950 tx=0x7fe5200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4636007e6dde2eaf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:46.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.254+0000 7fe50d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:30:46.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.254+0000 7fe50d7fa640 1 -- 172.21.15.38:0/2758140425 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe52001f8f0 con 0x5613d117d8d0 2024-09-16T00:30:46.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.254+0000 7fe50d7fa640 1 -- 172.21.15.38:0/2758140425 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fe52001fa90 con 0x5613d117d8d0 2024-09-16T00:30:46.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.254+0000 7fe52e54e740 1 -- 172.21.15.38:0/2758140425 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe51c02fb30 con 0x5613d117d8d0 2024-09-16T00:30:46.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.254+0000 7fe50d7fa640 1 -- 172.21.15.38:0/2758140425 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fe52001fc90 con 0x5613d117d8d0 2024-09-16T00:30:46.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.254+0000 7fe52e54e740 10 client.14715 fetch_fsmap learned FSMap version 17 2024-09-16T00:30:46.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.254+0000 7fe52e54e740 1 -- 172.21.15.38:0/2758140425 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5613d119b550 con 0x5613d117d8d0 2024-09-16T00:30:46.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.254+0000 7fe50d7fa640 1 -- 172.21.15.38:0/2758140425 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe51c02fb30 con 0x5613d117d8d0 2024-09-16T00:30:46.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.255+0000 7fe50d7fa640 1 -- 172.21.15.38:0/2758140425 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7fe52001e4d0 con 0x5613d117d8d0 2024-09-16T00:30:46.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.255+0000 7fe52e54e740 10 client.14715 fetch_fsmap finished waiting for FSMap version 17 2024-09-16T00:30:46.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.255+0000 7fe52e54e740 10 client.14715 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T00:30:46.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.255+0000 7fe52e54e740 20 client.14715 populate_metadata read hostname 'smithi038' 2024-09-16T00:30:46.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.255+0000 7fe52e54e740 1 --2- 172.21.15.38:0/2758140425 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5613d119db20 0x5613d119ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:46.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.255+0000 7fe52e54e740 4 client.14715 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:30:46.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.255+0000 7fe52e54e740 1 -- 172.21.15.38:0/2758140425 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5613d10f1600 con 0x5613d119db20 2024-09-16T00:30:46.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.255+0000 7fe529710640 1 --2- 172.21.15.38:0/2758140425 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5613d119db20 0x5613d119ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:46.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.256+0000 7fe529710640 1 --2- 172.21.15.38:0/2758140425 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5613d119db20 0x5613d119ef90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=964abd823096f4ee server_cookie=f2d2ccdb0fc71384 in_seq=0 out_seq=0 2024-09-16T00:30:46.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.256+0000 7fe50d7fa640 10 client.14715 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:30:46.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.256+0000 7fe50d7fa640 1 -- 172.21.15.38:0/2758140425 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5613d10f1600 con 0x5613d119db20 2024-09-16T00:30:46.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.256+0000 7fe50d7fa640 10 client.14715 handle_command_reply: tid=0 2024-09-16T00:30:46.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.256+0000 7fe52e54e740 1 client.14715 shutdown 2024-09-16T00:30:46.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.257+0000 7fe52e54e740 1 -- 172.21.15.38:0/2758140425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613d117d8d0 msgr2=0x5613d118c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:46.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.257+0000 7fe52e54e740 1 --2- 172.21.15.38:0/2758140425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613d117d8d0 0x5613d118c810 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe52000b950 tx=0x7fe5200046c0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.257+0000 7fe52e54e740 1 -- 172.21.15.38:0/2758140425 shutdown_connections 2024-09-16T00:30:46.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.257+0000 7fe52e54e740 1 --2- 172.21.15.38:0/2758140425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613d118cfc0 0x5613d118a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.257+0000 7fe52e54e740 1 --2- 172.21.15.38:0/2758140425 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613d117e1f0 0x5613d1189b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.257+0000 7fe52e54e740 1 --2- 172.21.15.38:0/2758140425 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5613d119db20 0x5613d119ef90 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.257+0000 7fe52e54e740 1 --2- 172.21.15.38:0/2758140425 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613d117d8d0 0x5613d118c810 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.257+0000 7fe52e54e740 1 -- 172.21.15.38:0/2758140425 >> 172.21.15.38:0/2758140425 conn(0x5613d1179f60 msgr2=0x5613d117fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:46.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.257+0000 7fe50d7fa640 0 client.14715 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:30:46.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.257+0000 7fe52e54e740 1 -- 172.21.15.38:0/2758140425 shutdown_connections 2024-09-16T00:30:46.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.257+0000 7fe52e54e740 1 -- 172.21.15.38:0/2758140425 wait complete. 2024-09-16T00:30:46.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.257+0000 7fe52e54e740 20 client.14715 trim_cache size 0 max 16384 2024-09-16T00:30:46.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.275+0000 7fe52e54e740 1 Processor -- start 2024-09-16T00:30:46.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.275+0000 7fe52e54e740 1 -- start start 2024-09-16T00:30:46.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.276+0000 7fe52e54e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613d117d8d0 0x5613d119ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:46.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.276+0000 7fe52e54e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613d117e1f0 0x5613d1185130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:46.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.276+0000 7fe52e54e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613d118cfc0 0x5613d1186dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:46.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.276+0000 7fe52e54e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5613d119d2d0 con 0x5613d117e1f0 2024-09-16T00:30:46.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.276+0000 7fe52e54e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5613d119d9a0 con 0x5613d117d8d0 2024-09-16T00:30:46.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.276+0000 7fe52e54e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5613d119e070 con 0x5613d118cfc0 2024-09-16T00:30:46.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.276+0000 7fe529710640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613d117e1f0 0x5613d1185130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:46.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.276+0000 7fe529710640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613d117e1f0 0x5613d1185130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32808/0 (socket says 172.21.15.38:32808) 2024-09-16T00:30:46.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.276+0000 7fe529710640 1 -- 172.21.15.38:0/4107698934 learned_addr learned my addr 172.21.15.38:0/4107698934 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:30:46.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.276+0000 7fe52a712640 1 --2- 172.21.15.38:0/4107698934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613d118cfc0 0x5613d1186dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:46.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.276+0000 7fe529f11640 1 --2- 172.21.15.38:0/4107698934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613d117d8d0 0x5613d119ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:46.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.276+0000 7fe529710640 1 -- 172.21.15.38:0/4107698934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613d118cfc0 msgr2=0x5613d1186dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:46.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.276+0000 7fe529710640 1 --2- 172.21.15.38:0/4107698934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613d118cfc0 0x5613d1186dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.276+0000 7fe529710640 1 -- 172.21.15.38:0/4107698934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613d117d8d0 msgr2=0x5613d119ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:46.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.276+0000 7fe529710640 1 --2- 172.21.15.38:0/4107698934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613d117d8d0 0x5613d119ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.276+0000 7fe529710640 1 -- 172.21.15.38:0/4107698934 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613d117e020 con 0x5613d117e1f0 2024-09-16T00:30:46.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.276+0000 7fe529f11640 1 --2- 172.21.15.38:0/4107698934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613d117d8d0 0x5613d119ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:46.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.276+0000 7fe529710640 1 --2- 172.21.15.38:0/4107698934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613d117e1f0 0x5613d1185130 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe51409c070 tx=0x7fe5140bc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9d492821e6e3f18 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:46.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.276+0000 7fe50f7fe640 1 -- 172.21.15.38:0/4107698934 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5140b4070 con 0x5613d117e1f0 2024-09-16T00:30:46.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.276+0000 7fe50f7fe640 1 -- 172.21.15.38:0/4107698934 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5140a4d50 con 0x5613d117e1f0 2024-09-16T00:30:46.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.277+0000 7fe50f7fe640 1 -- 172.21.15.38:0/4107698934 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5140bc0b0 con 0x5613d117e1f0 2024-09-16T00:30:46.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.277+0000 7fe52e54e740 1 -- 172.21.15.38:0/4107698934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613d117e1f0 msgr2=0x5613d1185130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:46.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.277+0000 7fe52e54e740 1 --2- 172.21.15.38:0/4107698934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613d117e1f0 0x5613d1185130 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe51409c070 tx=0x7fe5140bc040 comp rx=0 tx=0).stop 2024-09-16T00:30:46.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.277+0000 7fe52e54e740 1 -- 172.21.15.38:0/4107698934 shutdown_connections 2024-09-16T00:30:46.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.277+0000 7fe52e54e740 1 --2- 172.21.15.38:0/4107698934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613d118cfc0 0x5613d1186dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.277+0000 7fe52e54e740 1 --2- 172.21.15.38:0/4107698934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613d117e1f0 0x5613d1185130 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.277+0000 7fe52e54e740 1 --2- 172.21.15.38:0/4107698934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613d117d8d0 0x5613d119ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.277+0000 7fe52e54e740 1 -- 172.21.15.38:0/4107698934 >> 172.21.15.38:0/4107698934 conn(0x5613d1179f60 msgr2=0x5613d1184a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:46.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.277+0000 7fe52e54e740 1 -- 172.21.15.38:0/4107698934 shutdown_connections 2024-09-16T00:30:46.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.277+0000 7fe52e54e740 1 -- 172.21.15.38:0/4107698934 wait complete. 2024-09-16T00:30:46.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.278+0000 7fe52e54e740 1 -- start start 2024-09-16T00:30:46.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.278+0000 7fe52e54e740 1 Processor -- start 2024-09-16T00:30:46.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.278+0000 7fe52e54e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613d117d8d0 0x5613d11a75c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:46.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.278+0000 7fe52e54e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613d117e1f0 0x5613d11a7b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:46.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.278+0000 7fe52e54e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613d118cfc0 0x5613d11a0af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:46.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.278+0000 7fe52e54e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5613d119ee10 con 0x5613d117d8d0 2024-09-16T00:30:46.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.278+0000 7fe52e54e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5613d119e740 con 0x5613d118cfc0 2024-09-16T00:30:46.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.278+0000 7fe52e54e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5613d119f4e0 con 0x5613d117e1f0 2024-09-16T00:30:46.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.278+0000 7fe529f11640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613d117d8d0 0x5613d11a75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:46.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.278+0000 7fe52e54e740 10 client.0 _ll_register_callbacks cb 0x7fe52000be90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:30:46.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.278+0000 7fe529f11640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613d117d8d0 0x5613d11a75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32820/0 (socket says 172.21.15.38:32820) 2024-09-16T00:30:46.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.278+0000 7fe529f11640 1 -- 172.21.15.38:0/2515099986 learned_addr learned my addr 172.21.15.38:0/2515099986 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:30:46.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.279+0000 7fe52a712640 1 --2- 172.21.15.38:0/2515099986 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613d118cfc0 0x5613d11a0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:46.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.279+0000 7fe529710640 1 --2- 172.21.15.38:0/2515099986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613d117e1f0 0x5613d11a7b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:46.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.279+0000 7fe529f11640 1 -- 172.21.15.38:0/2515099986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613d117e1f0 msgr2=0x5613d11a7b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:46.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.279+0000 7fe529f11640 1 --2- 172.21.15.38:0/2515099986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613d117e1f0 0x5613d11a7b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.279+0000 7fe529f11640 1 -- 172.21.15.38:0/2515099986 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613d118cfc0 msgr2=0x5613d11a0af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:30:46.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.279+0000 7fe529f11640 1 --2- 172.21.15.38:0/2515099986 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613d118cfc0 0x5613d11a0af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.279+0000 7fe529f11640 1 -- 172.21.15.38:0/2515099986 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613d11a1320 con 0x5613d117d8d0 2024-09-16T00:30:46.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.279+0000 7fe529710640 1 --2- 172.21.15.38:0/2515099986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613d117e1f0 0x5613d11a7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:46.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.279+0000 7fe529f11640 1 --2- 172.21.15.38:0/2515099986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613d117d8d0 0x5613d11a75c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe52000b650 tx=0x7fe52001e660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7006404994dffb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:46.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.279+0000 7fe50d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:30:46.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.279+0000 7fe50d7fa640 1 -- 172.21.15.38:0/2515099986 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe520011cc0 con 0x5613d117d8d0 2024-09-16T00:30:46.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.279+0000 7fe52e54e740 1 -- 172.21.15.38:0/2515099986 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5140a4d50 con 0x5613d117d8d0 2024-09-16T00:30:46.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.279+0000 7fe50d7fa640 1 -- 172.21.15.38:0/2515099986 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fe520005040 con 0x5613d117d8d0 2024-09-16T00:30:46.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.279+0000 7fe50d7fa640 1 -- 172.21.15.38:0/2515099986 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fe520005240 con 0x5613d117d8d0 2024-09-16T00:30:46.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.279+0000 7fe52e54e740 10 client.14721 fetch_fsmap learned FSMap version 17 2024-09-16T00:30:46.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.279+0000 7fe52e54e740 1 -- 172.21.15.38:0/2515099986 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x5613d11a5d30 con 0x5613d117d8d0 2024-09-16T00:30:46.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.280+0000 7fe50d7fa640 1 -- 172.21.15.38:0/2515099986 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe5140a4d50 con 0x5613d117d8d0 2024-09-16T00:30:46.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.280+0000 7fe50d7fa640 1 -- 172.21.15.38:0/2515099986 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 17) ==== 8350+0+0 (secure 0 0 0) 0x7fe52002d430 con 0x5613d117d8d0 2024-09-16T00:30:46.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.280+0000 7fe52e54e740 10 client.14721 fetch_fsmap finished waiting for FSMap version 17 2024-09-16T00:30:46.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.280+0000 7fe52e54e740 10 client.14721 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T00:30:46.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.280+0000 7fe52e54e740 20 client.14721 populate_metadata read hostname 'smithi038' 2024-09-16T00:30:46.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.280+0000 7fe52e54e740 1 --2- 172.21.15.38:0/2515099986 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5613d120cb70 0x5613d120b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:46.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.280+0000 7fe52e54e740 4 client.14721 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:30:46.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.280+0000 7fe52e54e740 1 -- 172.21.15.38:0/2515099986 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "status"}) -- 0x5613d10ebb10 con 0x5613d120cb70 2024-09-16T00:30:46.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.280+0000 7fe529710640 1 --2- 172.21.15.38:0/2515099986 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5613d120cb70 0x5613d120b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:46.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.280+0000 7fe50d7fa640 10 client.14721 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:30:46.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.280+0000 7fe529710640 1 --2- 172.21.15.38:0/2515099986 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5613d120cb70 0x5613d120b880 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dea4f4c3c4f28186 server_cookie=124c8b3d75610cb3 in_seq=0 out_seq=0 2024-09-16T00:30:46.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.281+0000 7fe50d7fa640 1 -- 172.21.15.38:0/2515099986 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5613d10ebb10 con 0x5613d120cb70 2024-09-16T00:30:46.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.281+0000 7fe50d7fa640 10 client.14721 handle_command_reply: tid=0 2024-09-16T00:30:46.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.281+0000 7fe52e54e740 1 client.14721 shutdown 2024-09-16T00:30:46.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.281+0000 7fe52e54e740 1 -- 172.21.15.38:0/2515099986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613d117d8d0 msgr2=0x5613d11a75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:46.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.281+0000 7fe52e54e740 1 --2- 172.21.15.38:0/2515099986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613d117d8d0 0x5613d11a75c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe52000b650 tx=0x7fe52001e660 comp rx=0 tx=0).stop 2024-09-16T00:30:46.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.281+0000 7fe52e54e740 1 -- 172.21.15.38:0/2515099986 shutdown_connections 2024-09-16T00:30:46.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.281+0000 7fe52e54e740 1 --2- 172.21.15.38:0/2515099986 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5613d118cfc0 0x5613d11a0af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.281+0000 7fe52e54e740 1 --2- 172.21.15.38:0/2515099986 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5613d117e1f0 0x5613d11a7b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.281+0000 7fe52e54e740 1 --2- 172.21.15.38:0/2515099986 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5613d120cb70 0x5613d120b880 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.281+0000 7fe52e54e740 1 --2- 172.21.15.38:0/2515099986 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5613d117d8d0 0x5613d11a75c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.281+0000 7fe52e54e740 1 -- 172.21.15.38:0/2515099986 >> 172.21.15.38:0/2515099986 conn(0x5613d1179f60 msgr2=0x5613d117f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:46.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.281+0000 7fe50d7fa640 0 client.14721 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:30:46.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.281+0000 7fe52e54e740 1 -- 172.21.15.38:0/2515099986 shutdown_connections 2024-09-16T00:30:46.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.281+0000 7fe52e54e740 1 -- 172.21.15.38:0/2515099986 wait complete. 2024-09-16T00:30:46.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.281+0000 7fe52e54e740 20 client.14721 trim_cache size 0 max 16384 2024-09-16T00:30:46.298 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-16T00:30:46.298 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "983b2072-73c1-11ef-bceb-c7b262605968", 2024-09-16T00:30:46.298 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-09-16T00:30:46.298 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24494, 2024-09-16T00:30:46.299 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-16T00:30:46.299 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-16T00:30:46.299 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-16T00:30:46.299 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 19.218669884000001, 2024-09-16T00:30:46.299 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 17, 2024-09-16T00:30:46.299 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 88, 2024-09-16T00:30:46.299 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 88, 2024-09-16T00:30:46.299 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 29.137220627000001 2024-09-16T00:30:46.299 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-16T00:30:46.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.282+0000 7fe52c175640 1 -- 172.21.15.38:0/4206646894 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fe5100a3240 msgr2=0x7fe5100a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:46.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.282+0000 7fe52c175640 1 --2- 172.21.15.38:0/4206646894 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fe5100a3240 0x7fe5100a5700 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x5613d11805c0 tx=0x7fe52000b710 comp rx=0 tx=0).stop 2024-09-16T00:30:46.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.282+0000 7fe52c175640 1 -- 172.21.15.38:0/4206646894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe524102570 msgr2=0x7fe5241989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:46.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.282+0000 7fe52c175640 1 --2- 172.21.15.38:0/4206646894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe524102570 0x7fe5241989b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe51400c920 tx=0x7fe51400cde0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.282+0000 7fe52c175640 1 -- 172.21.15.38:0/4206646894 shutdown_connections 2024-09-16T00:30:46.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.282+0000 7fe52c175640 1 --2- 172.21.15.38:0/4206646894 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fe5100a3240 0x7fe5100a5700 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.282+0000 7fe52c175640 1 --2- 172.21.15.38:0/4206646894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe524102ef0 0x7fe52419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.282+0000 7fe52c175640 1 --2- 172.21.15.38:0/4206646894 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe524102570 0x7fe5241989b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.282+0000 7fe52c175640 1 --2- 172.21.15.38:0/4206646894 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe524100e20 0x7fe524198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:46.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.282+0000 7fe52c175640 1 -- 172.21.15.38:0/4206646894 >> 172.21.15.38:0/4206646894 conn(0x7fe5240fc7d0 msgr2=0x7fe52406a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:46.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.282+0000 7fe52c175640 1 -- 172.21.15.38:0/4206646894 shutdown_connections 2024-09-16T00:30:46.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:46.282+0000 7fe52c175640 1 -- 172.21.15.38:0/4206646894 wait complete. 2024-09-16T00:30:47.017 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-16T00:30:47.017 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-16T00:30:47.018 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-16T00:30:47.272 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:47.393 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:47 smithi038 ceph-mon[30945]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:47.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:47 smithi122 ceph-mon[35412]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:47.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:47 smithi073 ceph-mon[35254]: pgmap v300: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:47.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.741+0000 7fdaab46c640 1 -- 172.21.15.38:0/3439122545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdaa4102d30 msgr2=0x7fdaa4103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:47.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.741+0000 7fdaab46c640 1 --2- 172.21.15.38:0/3439122545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdaa4102d30 0x7fdaa4103110 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0009930 tx=0x7fdaa002a420 comp rx=0 tx=0).stop 2024-09-16T00:30:47.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.741+0000 7fdaab46c640 1 -- 172.21.15.38:0/3439122545 shutdown_connections 2024-09-16T00:30:47.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.741+0000 7fdaab46c640 1 --2- 172.21.15.38:0/3439122545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdaa41082a0 0x7fdaa410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:47.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.741+0000 7fdaab46c640 1 --2- 172.21.15.38:0/3439122545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaa41036e0 0x7fdaa4107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:47.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.741+0000 7fdaab46c640 1 --2- 172.21.15.38:0/3439122545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdaa4102d30 0x7fdaa4103110 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:47.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.741+0000 7fdaab46c640 1 -- 172.21.15.38:0/3439122545 >> 172.21.15.38:0/3439122545 conn(0x7fdaa40fc7d0 msgr2=0x7fdaa40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:47.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.741+0000 7fdaab46c640 1 -- 172.21.15.38:0/3439122545 shutdown_connections 2024-09-16T00:30:47.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.741+0000 7fdaab46c640 1 -- 172.21.15.38:0/3439122545 wait complete. 2024-09-16T00:30:47.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.742+0000 7fdaab46c640 1 Processor -- start 2024-09-16T00:30:47.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.742+0000 7fdaab46c640 1 -- start start 2024-09-16T00:30:47.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.742+0000 7fdaab46c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaa4102d30 0x7fdaa4198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:47.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.742+0000 7fdaab46c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdaa41036e0 0x7fdaa4198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:47.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.742+0000 7fdaab46c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdaa41082a0 0x7fdaa419ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:47.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.742+0000 7fdaab46c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa4112100 con 0x7fdaa41082a0 2024-09-16T00:30:47.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.742+0000 7fdaab46c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa4111f80 con 0x7fdaa4102d30 2024-09-16T00:30:47.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.742+0000 7fdaab46c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa4112280 con 0x7fdaa41036e0 2024-09-16T00:30:47.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.742+0000 7fdaa9a09640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdaa41082a0 0x7fdaa419ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:47.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.742+0000 7fdaa9a09640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdaa41082a0 0x7fdaa419ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32840/0 (socket says 172.21.15.38:32840) 2024-09-16T00:30:47.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.742+0000 7fdaa9a09640 1 -- 172.21.15.38:0/845443709 learned_addr learned my addr 172.21.15.38:0/845443709 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:30:47.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.742+0000 7fdaa9208640 1 --2- 172.21.15.38:0/845443709 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaa4102d30 0x7fdaa4198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:47.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.742+0000 7fdaa9a09640 1 -- 172.21.15.38:0/845443709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdaa41036e0 msgr2=0x7fdaa4198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:47.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.742+0000 7fdaa8a07640 1 --2- 172.21.15.38:0/845443709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdaa41036e0 0x7fdaa4198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:47.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.742+0000 7fdaa9a09640 1 --2- 172.21.15.38:0/845443709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdaa41036e0 0x7fdaa4198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:47.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.742+0000 7fdaa9a09640 1 -- 172.21.15.38:0/845443709 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaa4102d30 msgr2=0x7fdaa4198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:47.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.742+0000 7fdaa9a09640 1 --2- 172.21.15.38:0/845443709 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaa4102d30 0x7fdaa4198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:47.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.742+0000 7fdaa9a09640 1 -- 172.21.15.38:0/845443709 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa419d5b0 con 0x7fdaa41082a0 2024-09-16T00:30:47.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.742+0000 7fdaa9208640 1 --2- 172.21.15.38:0/845443709 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaa4102d30 0x7fdaa4198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:47.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.742+0000 7fdaa8a07640 1 --2- 172.21.15.38:0/845443709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdaa41036e0 0x7fdaa4198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:30:47.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.743+0000 7fdaa9a09640 1 --2- 172.21.15.38:0/845443709 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdaa41082a0 0x7fdaa419ced0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fda9c00b970 tx=0x7fda9c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:47.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.743+0000 7fda9a7fc640 1 -- 172.21.15.38:0/845443709 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c019070 con 0x7fdaa41082a0 2024-09-16T00:30:47.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.743+0000 7fdaab46c640 1 -- 172.21.15.38:0/845443709 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaa419d8a0 con 0x7fdaa41082a0 2024-09-16T00:30:47.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.743+0000 7fda9a7fc640 1 -- 172.21.15.38:0/845443709 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda9c00cb40 con 0x7fdaa41082a0 2024-09-16T00:30:47.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.743+0000 7fda9a7fc640 1 -- 172.21.15.38:0/845443709 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c007610 con 0x7fdaa41082a0 2024-09-16T00:30:47.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.743+0000 7fdaab46c640 1 -- 172.21.15.38:0/845443709 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaa41a5180 con 0x7fdaa41082a0 2024-09-16T00:30:47.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.744+0000 7fdaab46c640 1 -- 172.21.15.38:0/845443709 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaa406a850 con 0x7fdaa41082a0 2024-09-16T00:30:47.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.745+0000 7fda9a7fc640 1 -- 172.21.15.38:0/845443709 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fda9c01c070 con 0x7fdaa41082a0 2024-09-16T00:30:47.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.746+0000 7fda9a7fc640 1 --2- 172.21.15.38:0/845443709 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fda900a3240 0x7fda900a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:47.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.746+0000 7fdaa9208640 1 --2- 172.21.15.38:0/845443709 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fda900a3240 0x7fda900a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:47.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.746+0000 7fda9a7fc640 1 -- 172.21.15.38:0/845443709 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fda9c0cf6a0 con 0x7fdaa41082a0 2024-09-16T00:30:47.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.746+0000 7fdaa9208640 1 --2- 172.21.15.38:0/845443709 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fda900a3240 0x7fda900a5700 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0000c00 tx=0x7fdaa0032c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:47.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.749+0000 7fda9a7fc640 1 -- 172.21.15.38:0/845443709 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fda9c09c120 con 0x7fdaa41082a0 2024-09-16T00:30:47.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.961+0000 7fdaab46c640 1 -- 172.21.15.38:0/845443709 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fdaa4103110 con 0x7fdaa41082a0 2024-09-16T00:30:47.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.962+0000 7fda9a7fc640 1 -- 172.21.15.38:0/845443709 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7fdaa4103110 con 0x7fdaa41082a0 2024-09-16T00:30:47.962 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-16T00:30:47.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.964+0000 7fdaab46c640 1 -- 172.21.15.38:0/845443709 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fda900a3240 msgr2=0x7fda900a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:47.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.964+0000 7fdaab46c640 1 --2- 172.21.15.38:0/845443709 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fda900a3240 0x7fda900a5700 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0000c00 tx=0x7fdaa0032c70 comp rx=0 tx=0).stop 2024-09-16T00:30:47.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.964+0000 7fdaab46c640 1 -- 172.21.15.38:0/845443709 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdaa41082a0 msgr2=0x7fdaa419ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:47.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.964+0000 7fdaab46c640 1 --2- 172.21.15.38:0/845443709 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdaa41082a0 0x7fdaa419ced0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fda9c00b970 tx=0x7fda9c00be30 comp rx=0 tx=0).stop 2024-09-16T00:30:47.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.964+0000 7fdaab46c640 1 -- 172.21.15.38:0/845443709 shutdown_connections 2024-09-16T00:30:47.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.964+0000 7fdaab46c640 1 --2- 172.21.15.38:0/845443709 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fda900a3240 0x7fda900a5700 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:47.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.964+0000 7fdaab46c640 1 --2- 172.21.15.38:0/845443709 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdaa41082a0 0x7fdaa419ced0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:47.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.964+0000 7fdaab46c640 1 --2- 172.21.15.38:0/845443709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdaa41036e0 0x7fdaa4198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:47.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.964+0000 7fdaab46c640 1 --2- 172.21.15.38:0/845443709 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdaa4102d30 0x7fdaa4198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:47.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.964+0000 7fdaab46c640 1 -- 172.21.15.38:0/845443709 >> 172.21.15.38:0/845443709 conn(0x7fdaa40fc7d0 msgr2=0x7fdaa4105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:47.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.964+0000 7fdaab46c640 1 -- 172.21.15.38:0/845443709 shutdown_connections 2024-09-16T00:30:47.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:47.964+0000 7fdaab46c640 1 -- 172.21.15.38:0/845443709 wait complete. 2024-09-16T00:30:48.214 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:48 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/845443709' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T00:30:48.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:48 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/845443709' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T00:30:48.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:48 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/845443709' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T00:30:48.780 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-16T00:30:48.780 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T00:30:48.780 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-09-16T00:30:48.781 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-09-16T00:30:48.781 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-09-16T00:30:48.781 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-16T00:30:48.781 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-16T00:30:48.781 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T00:30:48.781 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-09-16T00:30:48.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:30:48.781 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-09-16T00:30:48.799 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T00:30:48.799 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T00:30:48.800 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T00:30:48.800 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-16T00:30:48.800 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-09-16T00:30:48.800 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-16T00:30:48.800 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T00:30:48.800 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-09-16T00:30:48.800 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-09-16T00:30:48.800 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T00:30:48.800 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-09-16T00:30:48.800 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-09-16T00:30:48.800 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T00:30:48.800 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-09-16T00:30:48.800 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-09-16T00:30:48.800 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-16T00:30:48.801 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42246sec preferred_lft 42246sec 2024-09-16T00:30:48.801 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link 2024-09-16T00:30:48.801 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-16T00:30:48.801 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T00:30:48.801 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-09-16T00:30:48.801 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-09-16T00:30:48.802 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-16T00:30:48.802 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:30:48.802 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-16T00:30:48.802 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link add name ceph-brx type bridge 2024-09-16T00:30:48.802 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip addr flush dev ceph-brx 2024-09-16T00:30:48.802 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set ceph-brx up 2024-09-16T00:30:48.802 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-16T00:30:48.802 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-16T00:30:48.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:48 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:30:49.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:30:49.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:30:49.029 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T00:30:49.056 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-09-16T00:30:49.058 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:30:49.059 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-09-16T00:30:49.127 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-09-16T00:30:49.128 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T00:30:49.128 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-16T00:30:49.128 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-16T00:30:49.128 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T00:30:49.128 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:30:49.128 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-16T00:30:49.128 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-16T00:30:49.128 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-16T00:30:49.129 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-16T00:30:49.129 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-16T00:30:49.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:30:49.328 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:49 smithi038 ceph-mon[30945]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:49.417 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:30:49.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:30:49.425 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-09-16T00:30:49.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:30:49.444 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list-id 2024-09-16T00:30:49.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:49 smithi122 ceph-mon[35412]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:49.502 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:30:49.503 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-16T00:30:49.503 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-16T00:30:49.503 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-09-16T00:30:49.503 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-16T00:30:49.551 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:49 smithi073 ceph-mon[35254]: pgmap v301: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:49.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:30:49.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:30:49.631 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-09-16T00:30:49.631 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:30:49.631 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-16T00:30:49.631 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-09-16T00:30:49.631 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-16T00:30:49.632 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-09-16T00:30:49.632 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-09-16T00:30:49.632 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-09-16T00:30:49.632 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-16T00:30:49.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:30:49.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:30:49.759 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:30:49.759 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-16T00:30:49.759 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-09-16T00:30:49.759 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set dev brx.0 master ceph-brx 2024-09-16T00:30:49.759 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-16T00:30:49.813 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:49 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:30:49.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:49 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:30:49.848 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-09-16T00:30:49.849 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T00:30:49.849 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-09-16T00:30:49.868 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-09-16T00:30:49.868 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T00:30:49.869 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-09-16T00:30:49.923 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-09-16T00:30:49.990 DEBUG:teuthology.orchestra.run.smithi038:> cat /proc/self/mounts | awk '{print $2}' 2024-09-16T00:30:50.048 INFO:teuthology.orchestra.run.smithi038.stdout:/proc 2024-09-16T00:30:50.048 INFO:teuthology.orchestra.run.smithi038.stdout:/sys 2024-09-16T00:30:50.048 INFO:teuthology.orchestra.run.smithi038.stdout:/dev 2024-09-16T00:30:50.048 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/security 2024-09-16T00:30:50.048 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/shm 2024-09-16T00:30:50.048 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/pts 2024-09-16T00:30:50.048 INFO:teuthology.orchestra.run.smithi038.stdout:/run 2024-09-16T00:30:50.048 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/cgroup 2024-09-16T00:30:50.048 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/pstore 2024-09-16T00:30:50.049 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/bpf 2024-09-16T00:30:50.049 INFO:teuthology.orchestra.run.smithi038.stdout:/ 2024-09-16T00:30:50.049 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/selinux 2024-09-16T00:30:50.049 INFO:teuthology.orchestra.run.smithi038.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T00:30:50.049 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/hugepages 2024-09-16T00:30:50.049 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/mqueue 2024-09-16T00:30:50.049 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/debug 2024-09-16T00:30:50.049 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/tracing 2024-09-16T00:30:50.049 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-16T00:30:50.049 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/fuse/connections 2024-09-16T00:30:50.049 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysctl.service 2024-09-16T00:30:50.049 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/config 2024-09-16T00:30:50.049 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-16T00:30:50.050 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/nfs/rpc_pipefs 2024-09-16T00:30:50.050 INFO:teuthology.orchestra.run.smithi038.stdout:/run/user/1000 2024-09-16T00:30:50.050 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph 2024-09-16T00:30:50.050 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay 2024-09-16T00:30:50.050 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay/1c8075ee511df27a09580ddac8cee87af030cf43afc864c7cd41bc0303eb33a0/merged 2024-09-16T00:30:50.050 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay/cfda78c2e16bb9c4abc821b23fb1006f63ce4cbcae11b64624d38771e2012ef8/merged 2024-09-16T00:30:50.050 INFO:teuthology.orchestra.run.smithi038.stdout:/run/user/0 2024-09-16T00:30:50.050 INFO:teuthology.orchestra.run.smithi038.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T00:30:50.050 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay/f3232ca818faf958fc18aaba34f0915d6305825a194956dad07ba07f0110e614/merged 2024-09-16T00:30:50.050 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay/3048131403de1b2a94ecc38766f3472e834e7e08314d86a095362d3cb6c7ff0e/merged 2024-09-16T00:30:50.050 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay/a9153252bd01cf108321ca8c25d18b4673443c8ea9e7e9d695dd94cbc6705587/merged 2024-09-16T00:30:50.050 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay/f0ff27f971d0c1ce7044cd118b98c8959d3ea1cec551e67360664a2baf373460/merged 2024-09-16T00:30:50.050 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay/cc7cd273260416c39262440b64e94b827183f8103f584d0a4537b4a9abc09e4a/merged 2024-09-16T00:30:50.050 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay/898281c91ffc2d8b1d1515082c6a50700ffb4d592026d3477105c20ec27fff81/merged 2024-09-16T00:30:50.050 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay/948233fdce93cedc942fef5b522809606380c4c65b6ac423e38ec82264ebe61b/merged 2024-09-16T00:30:50.051 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay/10978d81bd466ba9858cc8843bd7f474372aa6062a4d1e8f80c60e48515202e5/merged 2024-09-16T00:30:50.051 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns 2024-09-16T00:30:50.051 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-16T00:30:50.051 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-16T00:30:50.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:30:50.051 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-09-16T00:30:50.104 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-09-16T00:30:50.104 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-09-16T00:30:50.147 DEBUG:teuthology.orchestra.run.smithi038:> sudo modprobe fuse 2024-09-16T00:30:50.172 DEBUG:teuthology.orchestra.run.smithi038:> cat /proc/self/mounts | awk '{print $2}' 2024-09-16T00:30:50.230 INFO:teuthology.orchestra.run.smithi038.stdout:/proc 2024-09-16T00:30:50.230 INFO:teuthology.orchestra.run.smithi038.stdout:/sys 2024-09-16T00:30:50.230 INFO:teuthology.orchestra.run.smithi038.stdout:/dev 2024-09-16T00:30:50.230 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/security 2024-09-16T00:30:50.231 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/shm 2024-09-16T00:30:50.231 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/pts 2024-09-16T00:30:50.231 INFO:teuthology.orchestra.run.smithi038.stdout:/run 2024-09-16T00:30:50.231 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/cgroup 2024-09-16T00:30:50.231 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/pstore 2024-09-16T00:30:50.231 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/bpf 2024-09-16T00:30:50.231 INFO:teuthology.orchestra.run.smithi038.stdout:/ 2024-09-16T00:30:50.231 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/selinux 2024-09-16T00:30:50.231 INFO:teuthology.orchestra.run.smithi038.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T00:30:50.231 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/hugepages 2024-09-16T00:30:50.231 INFO:teuthology.orchestra.run.smithi038.stdout:/dev/mqueue 2024-09-16T00:30:50.231 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/debug 2024-09-16T00:30:50.231 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/tracing 2024-09-16T00:30:50.231 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-09-16T00:30:50.231 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/fs/fuse/connections 2024-09-16T00:30:50.232 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-sysctl.service 2024-09-16T00:30:50.232 INFO:teuthology.orchestra.run.smithi038.stdout:/sys/kernel/config 2024-09-16T00:30:50.232 INFO:teuthology.orchestra.run.smithi038.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-09-16T00:30:50.232 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/nfs/rpc_pipefs 2024-09-16T00:30:50.232 INFO:teuthology.orchestra.run.smithi038.stdout:/run/user/1000 2024-09-16T00:30:50.232 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/ceph 2024-09-16T00:30:50.232 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay 2024-09-16T00:30:50.232 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay/1c8075ee511df27a09580ddac8cee87af030cf43afc864c7cd41bc0303eb33a0/merged 2024-09-16T00:30:50.232 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay/cfda78c2e16bb9c4abc821b23fb1006f63ce4cbcae11b64624d38771e2012ef8/merged 2024-09-16T00:30:50.232 INFO:teuthology.orchestra.run.smithi038.stdout:/run/user/0 2024-09-16T00:30:50.232 INFO:teuthology.orchestra.run.smithi038.stdout:/proc/sys/fs/binfmt_misc 2024-09-16T00:30:50.232 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay/f3232ca818faf958fc18aaba34f0915d6305825a194956dad07ba07f0110e614/merged 2024-09-16T00:30:50.232 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay/3048131403de1b2a94ecc38766f3472e834e7e08314d86a095362d3cb6c7ff0e/merged 2024-09-16T00:30:50.232 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay/a9153252bd01cf108321ca8c25d18b4673443c8ea9e7e9d695dd94cbc6705587/merged 2024-09-16T00:30:50.233 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay/f0ff27f971d0c1ce7044cd118b98c8959d3ea1cec551e67360664a2baf373460/merged 2024-09-16T00:30:50.233 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay/cc7cd273260416c39262440b64e94b827183f8103f584d0a4537b4a9abc09e4a/merged 2024-09-16T00:30:50.233 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay/898281c91ffc2d8b1d1515082c6a50700ffb4d592026d3477105c20ec27fff81/merged 2024-09-16T00:30:50.233 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay/948233fdce93cedc942fef5b522809606380c4c65b6ac423e38ec82264ebe61b/merged 2024-09-16T00:30:50.233 INFO:teuthology.orchestra.run.smithi038.stdout:/var/lib/containers/storage/overlay/10978d81bd466ba9858cc8843bd7f474372aa6062a4d1e8f80c60e48515202e5/merged 2024-09-16T00:30:50.233 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns 2024-09-16T00:30:50.233 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-16T00:30:50.233 INFO:teuthology.orchestra.run.smithi038.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-16T00:30:50.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:30:50.233 DEBUG:teuthology.orchestra.run.smithi038:> ls /sys/fs/fuse/connections 2024-09-16T00:30:50.268 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi038.stderr:2024-09-16T00:30:50.267+0000 7f604aad3480 -1 init, newargv = 0x56172d441c40 newargc=15 2024-09-16T00:30:50.268 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi038.stderr:ceph-fuse[74751]: starting ceph client 2024-09-16T00:30:50.276 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi038.stderr:ceph-fuse[74751]: starting fuse 2024-09-16T00:30:50.284 INFO:teuthology.orchestra.run.smithi038.stdout:96 2024-09-16T00:30:50.284 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [96] 2024-09-16T00:30:50.284 DEBUG:teuthology.orchestra.run.smithi038:> sudo stdin-killer -- python3 -c ' 2024-09-16T00:30:50.285 DEBUG:teuthology.orchestra.run.smithi038:> import glob 2024-09-16T00:30:50.285 DEBUG:teuthology.orchestra.run.smithi038:> import re 2024-09-16T00:30:50.285 DEBUG:teuthology.orchestra.run.smithi038:> import os 2024-09-16T00:30:50.285 DEBUG:teuthology.orchestra.run.smithi038:> import subprocess 2024-09-16T00:30:50.285 DEBUG:teuthology.orchestra.run.smithi038:> 2024-09-16T00:30:50.285 DEBUG:teuthology.orchestra.run.smithi038:> def _find_admin_socket(client_name): 2024-09-16T00:30:50.285 DEBUG:teuthology.orchestra.run.smithi038:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-09-16T00:30:50.285 DEBUG:teuthology.orchestra.run.smithi038:> files = glob.glob(asok_path) 2024-09-16T00:30:50.285 DEBUG:teuthology.orchestra.run.smithi038:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-09-16T00:30:50.285 DEBUG:teuthology.orchestra.run.smithi038:> 2024-09-16T00:30:50.285 DEBUG:teuthology.orchestra.run.smithi038:> # Given a non-glob path, it better be there 2024-09-16T00:30:50.285 DEBUG:teuthology.orchestra.run.smithi038:> if "*" not in asok_path: 2024-09-16T00:30:50.285 DEBUG:teuthology.orchestra.run.smithi038:> assert(len(files) == 1) 2024-09-16T00:30:50.285 DEBUG:teuthology.orchestra.run.smithi038:> return files[0] 2024-09-16T00:30:50.286 DEBUG:teuthology.orchestra.run.smithi038:> 2024-09-16T00:30:50.286 DEBUG:teuthology.orchestra.run.smithi038:> for f in files: 2024-09-16T00:30:50.286 DEBUG:teuthology.orchestra.run.smithi038:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-09-16T00:30:50.286 DEBUG:teuthology.orchestra.run.smithi038:> if os.path.exists("/proc/{0}".format(pid)): 2024-09-16T00:30:50.286 DEBUG:teuthology.orchestra.run.smithi038:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-09-16T00:30:50.286 DEBUG:teuthology.orchestra.run.smithi038:> contents = proc_f.read() 2024-09-16T00:30:50.286 DEBUG:teuthology.orchestra.run.smithi038:> if mountpoint in contents: 2024-09-16T00:30:50.286 DEBUG:teuthology.orchestra.run.smithi038:> return f 2024-09-16T00:30:50.286 DEBUG:teuthology.orchestra.run.smithi038:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-09-16T00:30:50.286 DEBUG:teuthology.orchestra.run.smithi038:> 2024-09-16T00:30:50.286 DEBUG:teuthology.orchestra.run.smithi038:> print(_find_admin_socket("client.admin")) 2024-09-16T00:30:50.286 DEBUG:teuthology.orchestra.run.smithi038:> ' 2024-09-16T00:30:50.405 INFO:teuthology.orchestra.run.smithi038.stdout:/var/run/ceph/ceph-client.admin.74751.asok 2024-09-16T00:30:50.408 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:30:50.416 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.74751.asok 2024-09-16T00:30:50.416 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:30:50.416 DEBUG:teuthology.orchestra.run.smithi038:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.74751.asok status 2024-09-16T00:30:50.553 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-16T00:30:50.553 INFO:teuthology.orchestra.run.smithi038.stdout: "metadata": { 2024-09-16T00:30:50.553 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_sha1": "8921c8ef947807b717db041d85f0761b7914d5fc", 2024-09-16T00:30:50.553 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_version": "ceph version 19.1.1-326-g8921c8ef (8921c8ef947807b717db041d85f0761b7914d5fc) squid (stable)", 2024-09-16T00:30:50.553 INFO:teuthology.orchestra.run.smithi038.stdout: "entity_id": "admin", 2024-09-16T00:30:50.554 INFO:teuthology.orchestra.run.smithi038.stdout: "hostname": "smithi038", 2024-09-16T00:30:50.554 INFO:teuthology.orchestra.run.smithi038.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-09-16T00:30:50.554 INFO:teuthology.orchestra.run.smithi038.stdout: "pid": "74751", 2024-09-16T00:30:50.554 INFO:teuthology.orchestra.run.smithi038.stdout: "root": "/" 2024-09-16T00:30:50.554 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:30:50.554 INFO:teuthology.orchestra.run.smithi038.stdout: "dentry_count": 0, 2024-09-16T00:30:50.554 INFO:teuthology.orchestra.run.smithi038.stdout: "dentry_pinned_count": 0, 2024-09-16T00:30:50.554 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 14733, 2024-09-16T00:30:50.554 INFO:teuthology.orchestra.run.smithi038.stdout: "inst": { 2024-09-16T00:30:50.554 INFO:teuthology.orchestra.run.smithi038.stdout: "name": { 2024-09-16T00:30:50.554 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "client", 2024-09-16T00:30:50.554 INFO:teuthology.orchestra.run.smithi038.stdout: "num": 14733 2024-09-16T00:30:50.554 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:30:50.554 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": { 2024-09-16T00:30:50.555 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-09-16T00:30:50.555 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "192.168.0.1:0", 2024-09-16T00:30:50.555 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 130884574 2024-09-16T00:30:50.555 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:30:50.555 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:30:50.555 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": { 2024-09-16T00:30:50.555 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-09-16T00:30:50.555 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "192.168.0.1:0", 2024-09-16T00:30:50.555 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 130884574 2024-09-16T00:30:50.555 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:30:50.555 INFO:teuthology.orchestra.run.smithi038.stdout: "inst_str": "client.14733 192.168.0.1:0/130884574", 2024-09-16T00:30:50.555 INFO:teuthology.orchestra.run.smithi038.stdout: "addr_str": "192.168.0.1:0/130884574", 2024-09-16T00:30:50.555 INFO:teuthology.orchestra.run.smithi038.stdout: "inode_count": 1, 2024-09-16T00:30:50.555 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_epoch": 17, 2024-09-16T00:30:50.556 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_epoch": 88, 2024-09-16T00:30:50.556 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_epoch_barrier": 0, 2024-09-16T00:30:50.556 INFO:teuthology.orchestra.run.smithi038.stdout: "blocklisted": false, 2024-09-16T00:30:50.556 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs" 2024-09-16T00:30:50.556 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-16T00:30:50.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:30:50.562 DEBUG:teuthology.orchestra.run.smithi038:> stat --file-system '--printf=%T 2024-09-16T00:30:50.562 DEBUG:teuthology.orchestra.run.smithi038:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-09-16T00:30:50.578 INFO:teuthology.orchestra.run.smithi038.stdout:fuseblk 2024-09-16T00:30:50.578 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-09-16T00:30:50.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:30:50.579 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-09-16T00:30:50.645 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-09-16T00:30:50.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:50 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T00:30:50.755 INFO:teuthology.orchestra.run.smithi038.stdout:# file: . 2024-09-16T00:30:50.756 INFO:teuthology.orchestra.run.smithi038.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-09-16T00:30:50.756 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:30:50.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:50 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:30:50.762 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-16T00:30:50.762 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T00:30:50.780 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-16T00:30:51.078 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:51.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:51 smithi038 ceph-mon[30945]: pgmap v302: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:51.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:51 smithi073 ceph-mon[35254]: pgmap v302: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:51.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.582+0000 7f15c1385640 1 -- 172.21.15.38:0/3041906630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15bc108290 msgr2=0x7f15bc10ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:51.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.582+0000 7f15c1385640 1 --2- 172.21.15.38:0/3041906630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15bc108290 0x7f15bc10ec30 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f15b4009cd0 tx=0x7f15b402a8e0 comp rx=0 tx=0).stop 2024-09-16T00:30:51.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.582+0000 7f15c1385640 1 -- 172.21.15.38:0/3041906630 shutdown_connections 2024-09-16T00:30:51.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.582+0000 7f15c1385640 1 --2- 172.21.15.38:0/3041906630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15bc108290 0x7f15bc10ec30 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:51.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.582+0000 7f15c1385640 1 --2- 172.21.15.38:0/3041906630 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15bc103740 0x7f15bc107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:51.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.582+0000 7f15c1385640 1 --2- 172.21.15.38:0/3041906630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15bc102d90 0x7f15bc103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:51.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.582+0000 7f15c1385640 1 -- 172.21.15.38:0/3041906630 >> 172.21.15.38:0/3041906630 conn(0x7f15bc0fc7d0 msgr2=0x7f15bc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:51.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.582+0000 7f15c1385640 1 -- 172.21.15.38:0/3041906630 shutdown_connections 2024-09-16T00:30:51.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.582+0000 7f15c1385640 1 -- 172.21.15.38:0/3041906630 wait complete. 2024-09-16T00:30:51.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.582+0000 7f15c1385640 1 Processor -- start 2024-09-16T00:30:51.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.583+0000 7f15c1385640 1 -- start start 2024-09-16T00:30:51.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.583+0000 7f15c1385640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15bc102d90 0x7f15bc1985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:51.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.583+0000 7f15c1385640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15bc103740 0x7f15bc198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:51.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.583+0000 7f15c1385640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15bc108290 0x7f15bc19ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:51.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.583+0000 7f15c1385640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc1120d0 con 0x7f15bc103740 2024-09-16T00:30:51.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.583+0000 7f15c1385640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc111f50 con 0x7f15bc108290 2024-09-16T00:30:51.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.583+0000 7f15c1385640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc112250 con 0x7f15bc102d90 2024-09-16T00:30:51.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.583+0000 7f15ba7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15bc103740 0x7f15bc198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:51.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.583+0000 7f15ba7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15bc103740 0x7f15bc198af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32858/0 (socket says 172.21.15.38:32858) 2024-09-16T00:30:51.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.583+0000 7f15ba7fc640 1 -- 172.21.15.38:0/485194820 learned_addr learned my addr 172.21.15.38:0/485194820 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:30:51.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.583+0000 7f15ba7fc640 1 -- 172.21.15.38:0/485194820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15bc102d90 msgr2=0x7f15bc1985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:51.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.583+0000 7f15baffd640 1 --2- 172.21.15.38:0/485194820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15bc102d90 0x7f15bc1985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:51.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.583+0000 7f15bb7fe640 1 --2- 172.21.15.38:0/485194820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15bc108290 0x7f15bc19ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:51.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.583+0000 7f15ba7fc640 1 --2- 172.21.15.38:0/485194820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15bc102d90 0x7f15bc1985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:51.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.583+0000 7f15ba7fc640 1 -- 172.21.15.38:0/485194820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15bc108290 msgr2=0x7f15bc19ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:51.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.583+0000 7f15ba7fc640 1 --2- 172.21.15.38:0/485194820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15bc108290 0x7f15bc19ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:51.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.583+0000 7f15ba7fc640 1 -- 172.21.15.38:0/485194820 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15bc19d560 con 0x7f15bc103740 2024-09-16T00:30:51.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.584+0000 7f15ba7fc640 1 --2- 172.21.15.38:0/485194820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15bc103740 0x7f15bc198af0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f15a400b450 tx=0x7f15a400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:51.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.584+0000 7f15b3fff640 1 -- 172.21.15.38:0/485194820 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a4015020 con 0x7f15bc103740 2024-09-16T00:30:51.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.584+0000 7f15b3fff640 1 -- 172.21.15.38:0/485194820 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15a4011d70 con 0x7f15bc103740 2024-09-16T00:30:51.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.584+0000 7f15c1385640 1 -- 172.21.15.38:0/485194820 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15bc19d850 con 0x7f15bc103740 2024-09-16T00:30:51.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.584+0000 7f15b3fff640 1 -- 172.21.15.38:0/485194820 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15a400f340 con 0x7f15bc103740 2024-09-16T00:30:51.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.584+0000 7f15c1385640 1 -- 172.21.15.38:0/485194820 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15bc1a5130 con 0x7f15bc103740 2024-09-16T00:30:51.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.585+0000 7f15c1385640 1 -- 172.21.15.38:0/485194820 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15bc103170 con 0x7f15bc103740 2024-09-16T00:30:51.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.586+0000 7f15b3fff640 1 -- 172.21.15.38:0/485194820 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f15a4018020 con 0x7f15bc103740 2024-09-16T00:30:51.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.587+0000 7f15b3fff640 1 --2- 172.21.15.38:0/485194820 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f15a80a3240 0x7f15a80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:51.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.587+0000 7f15baffd640 1 --2- 172.21.15.38:0/485194820 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f15a80a3240 0x7f15a80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:51.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.587+0000 7f15b3fff640 1 -- 172.21.15.38:0/485194820 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f15a40cfc40 con 0x7f15bc103740 2024-09-16T00:30:51.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.587+0000 7f15baffd640 1 --2- 172.21.15.38:0/485194820 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f15a80a3240 0x7f15a80a5700 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f15ac005fd0 tx=0x7f15ac005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:51.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.590+0000 7f15b3fff640 1 -- 172.21.15.38:0/485194820 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f15a401f020 con 0x7f15bc103740 2024-09-16T00:30:51.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:51 smithi122 ceph-mon[35412]: pgmap v302: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:51.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.777+0000 7f15c1385640 1 -- 172.21.15.38:0/485194820 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f15bc199910 con 0x7f15bc103740 2024-09-16T00:30:51.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.778+0000 7f15b3fff640 1 -- 172.21.15.38:0/485194820 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) ==== 81+0+588 (secure 0 0 0) 0x7f15bc199910 con 0x7f15bc103740 2024-09-16T00:30:51.778 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6801/1779324916 2024-09-17T00:23:21.417483+0000 2024-09-16T00:30:51.778 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/924150447 2024-09-17T00:23:21.417483+0000 2024-09-16T00:30:51.779 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/141237849 2024-09-17T00:23:21.417483+0000 2024-09-16T00:30:51.779 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/2302081750 2024-09-17T00:22:56.844212+0000 2024-09-16T00:30:51.779 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/2245468594 2024-09-17T00:22:56.844212+0000 2024-09-16T00:30:51.779 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6801/583006373 2024-09-17T00:22:56.844212+0000 2024-09-16T00:30:51.779 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/2435492667 2024-09-17T00:23:21.417483+0000 2024-09-16T00:30:51.779 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6800/583006373 2024-09-17T00:22:56.844212+0000 2024-09-16T00:30:51.779 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6800/1779324916 2024-09-17T00:23:21.417483+0000 2024-09-16T00:30:51.779 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/1103516837 2024-09-17T00:22:56.844212+0000 2024-09-16T00:30:51.779 INFO:teuthology.orchestra.run.smithi038.stderr:listed 10 entries 2024-09-16T00:30:51.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.780+0000 7f15c1385640 1 -- 172.21.15.38:0/485194820 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f15a80a3240 msgr2=0x7f15a80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:51.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.780+0000 7f15c1385640 1 --2- 172.21.15.38:0/485194820 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f15a80a3240 0x7f15a80a5700 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f15ac005fd0 tx=0x7f15ac005850 comp rx=0 tx=0).stop 2024-09-16T00:30:51.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.780+0000 7f15c1385640 1 -- 172.21.15.38:0/485194820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15bc103740 msgr2=0x7f15bc198af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:51.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.780+0000 7f15c1385640 1 --2- 172.21.15.38:0/485194820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15bc103740 0x7f15bc198af0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f15a400b450 tx=0x7f15a400b910 comp rx=0 tx=0).stop 2024-09-16T00:30:51.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.780+0000 7f15c1385640 1 -- 172.21.15.38:0/485194820 shutdown_connections 2024-09-16T00:30:51.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.780+0000 7f15c1385640 1 --2- 172.21.15.38:0/485194820 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f15a80a3240 0x7f15a80a5700 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:51.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.780+0000 7f15c1385640 1 --2- 172.21.15.38:0/485194820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15bc108290 0x7f15bc19ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:51.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.780+0000 7f15c1385640 1 --2- 172.21.15.38:0/485194820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15bc103740 0x7f15bc198af0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:51.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.780+0000 7f15c1385640 1 --2- 172.21.15.38:0/485194820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15bc102d90 0x7f15bc1985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:51.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.780+0000 7f15c1385640 1 -- 172.21.15.38:0/485194820 >> 172.21.15.38:0/485194820 conn(0x7f15bc0fc7d0 msgr2=0x7f15bc105d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:51.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.780+0000 7f15c1385640 1 -- 172.21.15.38:0/485194820 shutdown_connections 2024-09-16T00:30:51.781 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:51.780+0000 7f15c1385640 1 -- 172.21.15.38:0/485194820 wait complete. 2024-09-16T00:30:52.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:52 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/485194820' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:30:52.659 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:52 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/485194820' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:30:52.693 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-16T00:30:52.693 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T00:30:52.713 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-16T00:30:52.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:52 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/485194820' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:30:53.014 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:53.405 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:53 smithi038 ceph-mon[30945]: pgmap v303: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:53.504 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.487+0000 7f7acb122640 1 -- 172.21.15.38:0/1963888760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ac4101280 msgr2=0x7f7ac410df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:53.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.487+0000 7f7acb122640 1 --2- 172.21.15.38:0/1963888760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ac4101280 0x7f7ac410df40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0009900 tx=0x7f7ab002a410 comp rx=0 tx=0).stop 2024-09-16T00:30:53.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.487+0000 7f7acb122640 1 -- 172.21.15.38:0/1963888760 shutdown_connections 2024-09-16T00:30:53.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.487+0000 7f7acb122640 1 --2- 172.21.15.38:0/1963888760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ac410e480 0x7f7ac4110870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:53.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.487+0000 7f7acb122640 1 --2- 172.21.15.38:0/1963888760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ac4101280 0x7f7ac410df40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:53.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.487+0000 7f7acb122640 1 --2- 172.21.15.38:0/1963888760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ac41008d0 0x7f7ac4100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:53.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.487+0000 7f7acb122640 1 -- 172.21.15.38:0/1963888760 >> 172.21.15.38:0/1963888760 conn(0x7f7ac40fc7a0 msgr2=0x7f7ac40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:53.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.487+0000 7f7acb122640 1 -- 172.21.15.38:0/1963888760 shutdown_connections 2024-09-16T00:30:53.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.487+0000 7f7acb122640 1 -- 172.21.15.38:0/1963888760 wait complete. 2024-09-16T00:30:53.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.488+0000 7f7acb122640 1 Processor -- start 2024-09-16T00:30:53.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.488+0000 7f7acb122640 1 -- start start 2024-09-16T00:30:53.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.488+0000 7f7acb122640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ac41008d0 0x7f7ac419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:53.505 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.488+0000 7f7acb122640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ac4101280 0x7f7ac419abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:53.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.488+0000 7f7acb122640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ac410e480 0x7f7ac419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:53.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.488+0000 7f7acb122640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac4114190 con 0x7f7ac4101280 2024-09-16T00:30:53.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.488+0000 7f7acb122640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac4114010 con 0x7f7ac410e480 2024-09-16T00:30:53.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.488+0000 7f7acb122640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7ac4114310 con 0x7f7ac41008d0 2024-09-16T00:30:53.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.489+0000 7f7ac3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ac4101280 0x7f7ac419abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:53.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.489+0000 7f7ac3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ac4101280 0x7f7ac419abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32880/0 (socket says 172.21.15.38:32880) 2024-09-16T00:30:53.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.489+0000 7f7ac3fff640 1 -- 172.21.15.38:0/4125153850 learned_addr learned my addr 172.21.15.38:0/4125153850 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:30:53.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.489+0000 7f7ac96bf640 1 --2- 172.21.15.38:0/4125153850 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ac410e480 0x7f7ac419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:53.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.489+0000 7f7ac8ebe640 1 --2- 172.21.15.38:0/4125153850 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ac41008d0 0x7f7ac419a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:53.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.489+0000 7f7ac3fff640 1 -- 172.21.15.38:0/4125153850 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ac41008d0 msgr2=0x7f7ac419a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:53.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.489+0000 7f7ac3fff640 1 --2- 172.21.15.38:0/4125153850 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ac41008d0 0x7f7ac419a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:53.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.489+0000 7f7ac3fff640 1 -- 172.21.15.38:0/4125153850 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ac410e480 msgr2=0x7f7ac419ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:53.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.489+0000 7f7ac3fff640 1 --2- 172.21.15.38:0/4125153850 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ac410e480 0x7f7ac419ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:53.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.489+0000 7f7ac3fff640 1 -- 172.21.15.38:0/4125153850 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ac419f630 con 0x7f7ac4101280 2024-09-16T00:30:53.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.489+0000 7f7ac3fff640 1 --2- 172.21.15.38:0/4125153850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ac4101280 0x7f7ac419abc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0006010 tx=0x7f7ab00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:53.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.489+0000 7f7ac1ffb640 1 -- 172.21.15.38:0/4125153850 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab003f030 con 0x7f7ac4101280 2024-09-16T00:30:53.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.489+0000 7f7ac1ffb640 1 -- 172.21.15.38:0/4125153850 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ab0003ce0 con 0x7f7ac4101280 2024-09-16T00:30:53.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.489+0000 7f7acb122640 1 -- 172.21.15.38:0/4125153850 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ac419f8c0 con 0x7f7ac4101280 2024-09-16T00:30:53.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.489+0000 7f7ac1ffb640 1 -- 172.21.15.38:0/4125153850 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab0035070 con 0x7f7ac4101280 2024-09-16T00:30:53.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.490+0000 7f7acb122640 1 -- 172.21.15.38:0/4125153850 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ac41a7200 con 0x7f7ac4101280 2024-09-16T00:30:53.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.491+0000 7f7acb122640 1 -- 172.21.15.38:0/4125153850 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ac4100cb0 con 0x7f7ac4101280 2024-09-16T00:30:53.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.492+0000 7f7ac1ffb640 1 -- 172.21.15.38:0/4125153850 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f7ab0042030 con 0x7f7ac4101280 2024-09-16T00:30:53.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.492+0000 7f7ac1ffb640 1 --2- 172.21.15.38:0/4125153850 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f7ab40a32c0 0x7f7ab40a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:53.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.493+0000 7f7ac1ffb640 1 -- 172.21.15.38:0/4125153850 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f7ab00f0430 con 0x7f7ac4101280 2024-09-16T00:30:53.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.493+0000 7f7ac8ebe640 1 --2- 172.21.15.38:0/4125153850 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f7ab40a32c0 0x7f7ab40a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:53.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.493+0000 7f7ac8ebe640 1 --2- 172.21.15.38:0/4125153850 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f7ab40a32c0 0x7f7ab40a5780 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8005d00 tx=0x7f7ab800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:53.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.496+0000 7f7ac1ffb640 1 -- 172.21.15.38:0/4125153850 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7ab00bc920 con 0x7f7ac4101280 2024-09-16T00:30:53.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:53 smithi073 ceph-mon[35254]: pgmap v303: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:53.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.683+0000 7f7acb122640 1 -- 172.21.15.38:0/4125153850 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f7ac419b9e0 con 0x7f7ac4101280 2024-09-16T00:30:53.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.683+0000 7f7ac1ffb640 1 -- 172.21.15.38:0/4125153850 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) ==== 81+0+588 (secure 0 0 0) 0x7f7ac419b9e0 con 0x7f7ac4101280 2024-09-16T00:30:53.684 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6801/1779324916 2024-09-17T00:23:21.417483+0000 2024-09-16T00:30:53.684 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/924150447 2024-09-17T00:23:21.417483+0000 2024-09-16T00:30:53.684 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/141237849 2024-09-17T00:23:21.417483+0000 2024-09-16T00:30:53.684 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/2302081750 2024-09-17T00:22:56.844212+0000 2024-09-16T00:30:53.684 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/2245468594 2024-09-17T00:22:56.844212+0000 2024-09-16T00:30:53.684 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6801/583006373 2024-09-17T00:22:56.844212+0000 2024-09-16T00:30:53.684 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/2435492667 2024-09-17T00:23:21.417483+0000 2024-09-16T00:30:53.685 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6800/583006373 2024-09-17T00:22:56.844212+0000 2024-09-16T00:30:53.685 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6800/1779324916 2024-09-17T00:23:21.417483+0000 2024-09-16T00:30:53.685 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/1103516837 2024-09-17T00:22:56.844212+0000 2024-09-16T00:30:53.685 INFO:teuthology.orchestra.run.smithi038.stderr:listed 10 entries 2024-09-16T00:30:53.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.685+0000 7f7acb122640 1 -- 172.21.15.38:0/4125153850 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f7ab40a32c0 msgr2=0x7f7ab40a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:53.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.685+0000 7f7acb122640 1 --2- 172.21.15.38:0/4125153850 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f7ab40a32c0 0x7f7ab40a5780 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f7ab8005d00 tx=0x7f7ab800b410 comp rx=0 tx=0).stop 2024-09-16T00:30:53.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.685+0000 7f7acb122640 1 -- 172.21.15.38:0/4125153850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ac4101280 msgr2=0x7f7ac419abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:53.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.685+0000 7f7acb122640 1 --2- 172.21.15.38:0/4125153850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ac4101280 0x7f7ac419abc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7ab0006010 tx=0x7f7ab00042f0 comp rx=0 tx=0).stop 2024-09-16T00:30:53.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.685+0000 7f7acb122640 1 -- 172.21.15.38:0/4125153850 shutdown_connections 2024-09-16T00:30:53.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.685+0000 7f7acb122640 1 --2- 172.21.15.38:0/4125153850 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f7ab40a32c0 0x7f7ab40a5780 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:53.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.685+0000 7f7acb122640 1 --2- 172.21.15.38:0/4125153850 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ac410e480 0x7f7ac419ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:53.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.685+0000 7f7acb122640 1 --2- 172.21.15.38:0/4125153850 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ac4101280 0x7f7ac419abc0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:53.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.685+0000 7f7acb122640 1 --2- 172.21.15.38:0/4125153850 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ac41008d0 0x7f7ac419a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:53.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.685+0000 7f7acb122640 1 -- 172.21.15.38:0/4125153850 >> 172.21.15.38:0/4125153850 conn(0x7f7ac40fc7a0 msgr2=0x7f7ac40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:53.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.686+0000 7f7acb122640 1 -- 172.21.15.38:0/4125153850 shutdown_connections 2024-09-16T00:30:53.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:53.686+0000 7f7acb122640 1 -- 172.21.15.38:0/4125153850 wait complete. 2024-09-16T00:30:53.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:53 smithi122 ceph-mon[35412]: pgmap v303: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:54.535 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:54 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/4125153850' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:30:54.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:54 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/4125153850' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:30:54.570 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi038.front.sepia.ceph.com... 2024-09-16T00:30:54.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:30:54.570 DEBUG:teuthology.orchestra.run.smithi038:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-09-16T00:30:54.599 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi038.stderr:ceph-fuse[74751]: fuse finished with error 0 and tester_r 0 2024-09-16T00:30:54.601 INFO:teuthology.orchestra.run:waiting for 300 2024-09-16T00:30:54.644 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-16T00:30:54.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:30:54.644 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-09-16T00:30:54.667 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:30:54.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:54 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/4125153850' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:30:54.965 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:55.348 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:55 smithi038 ceph-mon[30945]: pgmap v304: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:55.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.430+0000 7f6b8672d640 1 -- 172.21.15.38:0/3302880829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b80102790 msgr2=0x7f6b801067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:55.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.430+0000 7f6b8672d640 1 --2- 172.21.15.38:0/3302880829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b80102790 0x7f6b801067e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f6b74009900 tx=0x7f6b7402a410 comp rx=0 tx=0).stop 2024-09-16T00:30:55.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.430+0000 7f6b8672d640 1 -- 172.21.15.38:0/3302880829 shutdown_connections 2024-09-16T00:30:55.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.430+0000 7f6b8672d640 1 --2- 172.21.15.38:0/3302880829 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b80106f10 0x7f6b8010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:55.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.430+0000 7f6b8672d640 1 --2- 172.21.15.38:0/3302880829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b80102790 0x7f6b801067e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:55.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.430+0000 7f6b8672d640 1 --2- 172.21.15.38:0/3302880829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b80101de0 0x7f6b801021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:55.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.430+0000 7f6b8672d640 1 -- 172.21.15.38:0/3302880829 >> 172.21.15.38:0/3302880829 conn(0x7f6b800fd8c0 msgr2=0x7f6b800ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:55.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.430+0000 7f6b8672d640 1 -- 172.21.15.38:0/3302880829 shutdown_connections 2024-09-16T00:30:55.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.430+0000 7f6b8672d640 1 -- 172.21.15.38:0/3302880829 wait complete. 2024-09-16T00:30:55.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.431+0000 7f6b8672d640 1 Processor -- start 2024-09-16T00:30:55.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.431+0000 7f6b8672d640 1 -- start start 2024-09-16T00:30:55.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.431+0000 7f6b8672d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b80101de0 0x7f6b800744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:55.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.431+0000 7f6b8672d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b80102790 0x7f6b80074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:55.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.431+0000 7f6b8672d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b80106f10 0x7f6b80074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:55.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.431+0000 7f6b8672d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6b80112010 con 0x7f6b80106f10 2024-09-16T00:30:55.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.431+0000 7f6b8672d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6b80111e90 con 0x7f6b80101de0 2024-09-16T00:30:55.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.431+0000 7f6b8672d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6b80112190 con 0x7f6b80102790 2024-09-16T00:30:55.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.431+0000 7f6b84cca640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b80106f10 0x7f6b80074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:55.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.431+0000 7f6b84cca640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b80106f10 0x7f6b80074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:32890/0 (socket says 172.21.15.38:32890) 2024-09-16T00:30:55.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.431+0000 7f6b84cca640 1 -- 172.21.15.38:0/1458369450 learned_addr learned my addr 172.21.15.38:0/1458369450 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:30:55.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.432+0000 7f6b7ffff640 1 --2- 172.21.15.38:0/1458369450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b80101de0 0x7f6b800744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:55.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.432+0000 7f6b7f7fe640 1 --2- 172.21.15.38:0/1458369450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b80102790 0x7f6b80074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:55.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.432+0000 7f6b84cca640 1 -- 172.21.15.38:0/1458369450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b80102790 msgr2=0x7f6b80074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:55.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.432+0000 7f6b84cca640 1 --2- 172.21.15.38:0/1458369450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b80102790 0x7f6b80074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:55.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.432+0000 7f6b84cca640 1 -- 172.21.15.38:0/1458369450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b80101de0 msgr2=0x7f6b800744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:55.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.432+0000 7f6b84cca640 1 --2- 172.21.15.38:0/1458369450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b80101de0 0x7f6b800744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:55.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.432+0000 7f6b84cca640 1 -- 172.21.15.38:0/1458369450 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b801a6a70 con 0x7f6b80106f10 2024-09-16T00:30:55.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.432+0000 7f6b84cca640 1 --2- 172.21.15.38:0/1458369450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b80106f10 0x7f6b80074f70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6b7800b970 tx=0x7f6b7800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:55.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.432+0000 7f6b7d7fa640 1 -- 172.21.15.38:0/1458369450 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b78019070 con 0x7f6b80106f10 2024-09-16T00:30:55.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.432+0000 7f6b7d7fa640 1 -- 172.21.15.38:0/1458369450 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6b78014440 con 0x7f6b80106f10 2024-09-16T00:30:55.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.432+0000 7f6b8672d640 1 -- 172.21.15.38:0/1458369450 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b801a6d60 con 0x7f6b80106f10 2024-09-16T00:30:55.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.432+0000 7f6b7d7fa640 1 -- 172.21.15.38:0/1458369450 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b780075d0 con 0x7f6b80106f10 2024-09-16T00:30:55.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.432+0000 7f6b8672d640 1 -- 172.21.15.38:0/1458369450 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b801a72a0 con 0x7f6b80106f10 2024-09-16T00:30:55.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.433+0000 7f6b8672d640 1 -- 172.21.15.38:0/1458369450 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b80102240 con 0x7f6b80106f10 2024-09-16T00:30:55.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.434+0000 7f6b7d7fa640 1 -- 172.21.15.38:0/1458369450 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f6b7801c070 con 0x7f6b80106f10 2024-09-16T00:30:55.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.435+0000 7f6b7d7fa640 1 --2- 172.21.15.38:0/1458369450 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f6b680a32c0 0x7f6b680a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:55.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.435+0000 7f6b7d7fa640 1 -- 172.21.15.38:0/1458369450 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f6b780cf5c0 con 0x7f6b80106f10 2024-09-16T00:30:55.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.435+0000 7f6b7ffff640 1 --2- 172.21.15.38:0/1458369450 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f6b680a32c0 0x7f6b680a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:55.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.436+0000 7f6b7ffff640 1 --2- 172.21.15.38:0/1458369450 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f6b680a32c0 0x7f6b680a5780 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6b70009990 tx=0x7f6b70009110 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:55.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.438+0000 7f6b7d7fa640 1 -- 172.21.15.38:0/1458369450 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6b7809c0a0 con 0x7f6b80106f10 2024-09-16T00:30:55.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:55 smithi073 ceph-mon[35254]: pgmap v304: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:55.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.614+0000 7f6b8672d640 1 -- 172.21.15.38:0/1458369450 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6b8010a0f0 con 0x7f6b80106f10 2024-09-16T00:30:55.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.615+0000 7f6b7d7fa640 1 -- 172.21.15.38:0/1458369450 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24174 (secure 0 0 0) 0x7f6b7809b9a0 con 0x7f6b80106f10 2024-09-16T00:30:55.616 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:30:55.616 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":88,"fsid":"983b2072-73c1-11ef-bceb-c7b262605968","created":"2024-09-16T00:22:34.897053+0000","modified":"2024-09-16T00:30:35.865128+0000","last_up_change":"2024-09-16T00:29:34.503134+0000","last_in_change":"2024-09-16T00:29:10.135656+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:25:41.931977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:30:22.741889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:30:25.042590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:30:31.554350+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3afc20e3-869f-43a9-a911-b0e8d07de86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6803","nonce":1622028921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6805","nonce":1622028921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6809","nonce":1622028921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6807","nonce":1622028921}]},"public_addr":"172.21.15.38:6803/1622028921","cluster_addr":"172.21.15.38:6805/1622028921","heartbeat_back_addr":"172.21.15.38:6809/1622028921","heartbeat_front_addr":"172.21.15.38:6807/1622028921","state":["exists","up"]},{"osd":1,"uuid":"6d9d7acb-45ed-4543-8afc-6433fc7f096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6801","nonce":3811690336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6803","nonce":3811690336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6807","nonce":3811690336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6805","nonce":3811690336}]},"public_addr":"172.21.15.73:6801/3811690336","cluster_addr":"172.21.15.73:6803/3811690336","heartbeat_back_addr":"172.21.15.73:6807/3811690336","heartbeat_front_addr":"172.21.15.73:6805/3811690336","state":["exists","up"]},{"osd":2,"uuid":"48690f35-99e9-4f45-9bee-487857caa8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6801","nonce":1754741655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6803","nonce":1754741655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6807","nonce":1754741655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6805","nonce":1754741655}]},"public_addr":"172.21.15.122:6801/1754741655","cluster_addr":"172.21.15.122:6803/1754741655","heartbeat_back_addr":"172.21.15.122:6807/1754741655","heartbeat_front_addr":"172.21.15.122:6805/1754741655","state":["exists","up"]},{"osd":3,"uuid":"28ebb4a0-92e7-4f57-82a5-b9b761eca269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6811","nonce":3523190079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6813","nonce":3523190079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6817","nonce":3523190079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6815","nonce":3523190079}]},"public_addr":"172.21.15.38:6811/3523190079","cluster_addr":"172.21.15.38:6813/3523190079","heartbeat_back_addr":"172.21.15.38:6817/3523190079","heartbeat_front_addr":"172.21.15.38:6815/3523190079","state":["exists","up"]},{"osd":4,"uuid":"5ea363f7-7ad8-45ed-b7e4-4b010fe7f8cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6809","nonce":1469927628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6811","nonce":1469927628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6815","nonce":1469927628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6813","nonce":1469927628}]},"public_addr":"172.21.15.73:6809/1469927628","cluster_addr":"172.21.15.73:6811/1469927628","heartbeat_back_addr":"172.21.15.73:6815/1469927628","heartbeat_front_addr":"172.21.15.73:6813/1469927628","state":["exists","up"]},{"osd":5,"uuid":"e3c66d43-dd28-4bb1-a886-62a869be82cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6809","nonce":2954413058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6811","nonce":2954413058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6815","nonce":2954413058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6813","nonce":2954413058}]},"public_addr":"172.21.15.122:6809/2954413058","cluster_addr":"172.21.15.122:6811/2954413058","heartbeat_back_addr":"172.21.15.122:6815/2954413058","heartbeat_front_addr":"172.21.15.122:6813/2954413058","state":["exists","up"]},{"osd":6,"uuid":"211497ac-7c48-4095-bfdb-1d4df0149f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6819","nonce":1415719550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6821","nonce":1415719550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6825","nonce":1415719550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6823","nonce":1415719550}]},"public_addr":"172.21.15.38:6819/1415719550","cluster_addr":"172.21.15.38:6821/1415719550","heartbeat_back_addr":"172.21.15.38:6825/1415719550","heartbeat_front_addr":"172.21.15.38:6823/1415719550","state":["exists","up"]},{"osd":7,"uuid":"06512315-3f5e-44f8-976d-d3f5f1ba00e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6817","nonce":2634361473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6819","nonce":2634361473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6823","nonce":2634361473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6821","nonce":2634361473}]},"public_addr":"172.21.15.73:6817/2634361473","cluster_addr":"172.21.15.73:6819/2634361473","heartbeat_back_addr":"172.21.15.73:6823/2634361473","heartbeat_front_addr":"172.21.15.73:6821/2634361473","state":["exists","up"]},{"osd":8,"uuid":"db703c0a-1f87-46f0-be5f-f826f7f9daac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6817","nonce":1800567633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6819","nonce":1800567633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6823","nonce":1800567633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6821","nonce":1800567633}]},"public_addr":"172.21.15.122:6817/1800567633","cluster_addr":"172.21.15.122:6819/1800567633","heartbeat_back_addr":"172.21.15.122:6823/1800567633","heartbeat_front_addr":"172.21.15.122:6821/1800567633","state":["exists","up"]},{"osd":9,"uuid":"f5384200-b3b4-4936-85e1-fc57a9fe2bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6827","nonce":445727833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6829","nonce":445727833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6833","nonce":445727833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6831","nonce":445727833}]},"public_addr":"172.21.15.38:6827/445727833","cluster_addr":"172.21.15.38:6829/445727833","heartbeat_back_addr":"172.21.15.38:6833/445727833","heartbeat_front_addr":"172.21.15.38:6831/445727833","state":["exists","up"]},{"osd":10,"uuid":"c22883d9-6672-4796-a961-963d21eb37fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6825","nonce":43297139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6827","nonce":43297139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6831","nonce":43297139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6829","nonce":43297139}]},"public_addr":"172.21.15.73:6825/43297139","cluster_addr":"172.21.15.73:6827/43297139","heartbeat_back_addr":"172.21.15.73:6831/43297139","heartbeat_front_addr":"172.21.15.73:6829/43297139","state":["exists","up"]},{"osd":11,"uuid":"9f98e8cb-1bff-4b8b-aa19-56b691387383","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6825","nonce":1886446061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6827","nonce":1886446061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6831","nonce":1886446061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6829","nonce":1886446061}]},"public_addr":"172.21.15.122:6825/1886446061","cluster_addr":"172.21.15.122:6827/1886446061","heartbeat_back_addr":"172.21.15.122:6831/1886446061","heartbeat_front_addr":"172.21.15.122:6829/1886446061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:47.384012+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:12.834488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:38.770055+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:04.800466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:30.808076+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:56.646967+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:27:22.721270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:27:48.510011+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:28:13.922944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:28:40.514947+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:29:06.031052+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:29:31.882062+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6801/1779324916":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/924150447":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/141237849":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/2302081750":"2024-09-17T00:22:56.844212+0000","172.21.15.38:0/2245468594":"2024-09-17T00:22:56.844212+0000","172.21.15.38:6801/583006373":"2024-09-17T00:22:56.844212+0000","172.21.15.38:0/2435492667":"2024-09-17T00:23:21.417483+0000","172.21.15.38:6800/583006373":"2024-09-17T00:22:56.844212+0000","172.21.15.38:6800/1779324916":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/1103516837":"2024-09-17T00:22:56.844212+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:30:55.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.618+0000 7f6b8672d640 1 -- 172.21.15.38:0/1458369450 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f6b680a32c0 msgr2=0x7f6b680a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:55.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.618+0000 7f6b8672d640 1 --2- 172.21.15.38:0/1458369450 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f6b680a32c0 0x7f6b680a5780 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f6b70009990 tx=0x7f6b70009110 comp rx=0 tx=0).stop 2024-09-16T00:30:55.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.618+0000 7f6b8672d640 1 -- 172.21.15.38:0/1458369450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b80106f10 msgr2=0x7f6b80074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:55.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.618+0000 7f6b8672d640 1 --2- 172.21.15.38:0/1458369450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b80106f10 0x7f6b80074f70 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6b7800b970 tx=0x7f6b7800bd70 comp rx=0 tx=0).stop 2024-09-16T00:30:55.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.618+0000 7f6b8672d640 1 -- 172.21.15.38:0/1458369450 shutdown_connections 2024-09-16T00:30:55.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.618+0000 7f6b8672d640 1 --2- 172.21.15.38:0/1458369450 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f6b680a32c0 0x7f6b680a5780 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:55.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.618+0000 7f6b8672d640 1 --2- 172.21.15.38:0/1458369450 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6b80106f10 0x7f6b80074f70 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:55.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.618+0000 7f6b8672d640 1 --2- 172.21.15.38:0/1458369450 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6b80102790 0x7f6b80074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:55.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.618+0000 7f6b8672d640 1 --2- 172.21.15.38:0/1458369450 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b80101de0 0x7f6b800744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:55.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.618+0000 7f6b8672d640 1 -- 172.21.15.38:0/1458369450 >> 172.21.15.38:0/1458369450 conn(0x7f6b800fd8c0 msgr2=0x7f6b800fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:55.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.618+0000 7f6b8672d640 1 -- 172.21.15.38:0/1458369450 shutdown_connections 2024-09-16T00:30:55.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:55.618+0000 7f6b8672d640 1 -- 172.21.15.38:0/1458369450 wait complete. 2024-09-16T00:30:55.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:55 smithi122 ceph-mon[35412]: pgmap v304: 129 pgs: 129 active+clean; 579 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:30:56.393 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:30:56.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:56 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/1458369450' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:30:56.651 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:56.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:56 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/1458369450' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:30:56.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:56 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/1458369450' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:30:57.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.124+0000 7fc8d3d46640 1 -- 172.21.15.38:0/1341306704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8cc102d40 msgr2=0x7fc8cc103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:57.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.124+0000 7fc8d3d46640 1 --2- 172.21.15.38:0/1341306704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8cc102d40 0x7fc8cc103120 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0009900 tx=0x7fc8c002a480 comp rx=0 tx=0).stop 2024-09-16T00:30:57.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.124+0000 7fc8d3d46640 1 -- 172.21.15.38:0/1341306704 shutdown_connections 2024-09-16T00:30:57.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.124+0000 7fc8d3d46640 1 --2- 172.21.15.38:0/1341306704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc8cc10c170 0x7fc8cc10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:57.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.124+0000 7fc8d3d46640 1 --2- 172.21.15.38:0/1341306704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8cc103660 0x7fc8cc10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:57.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.124+0000 7fc8d3d46640 1 --2- 172.21.15.38:0/1341306704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8cc102d40 0x7fc8cc103120 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:57.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.124+0000 7fc8d3d46640 1 -- 172.21.15.38:0/1341306704 >> 172.21.15.38:0/1341306704 conn(0x7fc8cc0fc7a0 msgr2=0x7fc8cc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:57.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.124+0000 7fc8d3d46640 1 -- 172.21.15.38:0/1341306704 shutdown_connections 2024-09-16T00:30:57.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.124+0000 7fc8d3d46640 1 -- 172.21.15.38:0/1341306704 wait complete. 2024-09-16T00:30:57.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.125+0000 7fc8d3d46640 1 Processor -- start 2024-09-16T00:30:57.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.125+0000 7fc8d3d46640 1 -- start start 2024-09-16T00:30:57.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.125+0000 7fc8d3d46640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc8cc102d40 0x7fc8cc1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:57.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.125+0000 7fc8d3d46640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8cc103660 0x7fc8cc198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:57.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.125+0000 7fc8d3d46640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8cc10c170 0x7fc8cc19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:57.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.125+0000 7fc8d3d46640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc8cc111f70 con 0x7fc8cc10c170 2024-09-16T00:30:57.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.125+0000 7fc8d3d46640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc8cc111df0 con 0x7fc8cc103660 2024-09-16T00:30:57.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.125+0000 7fc8d3d46640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc8cc1120f0 con 0x7fc8cc102d40 2024-09-16T00:30:57.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.125+0000 7fc8d22e3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8cc10c170 0x7fc8cc19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:57.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.125+0000 7fc8d22e3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8cc10c170 0x7fc8cc19ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50980/0 (socket says 172.21.15.38:50980) 2024-09-16T00:30:57.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.125+0000 7fc8d22e3640 1 -- 172.21.15.38:0/4089234568 learned_addr learned my addr 172.21.15.38:0/4089234568 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:30:57.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.125+0000 7fc8d12e1640 1 --2- 172.21.15.38:0/4089234568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8cc103660 0x7fc8cc198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:57.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.125+0000 7fc8d1ae2640 1 --2- 172.21.15.38:0/4089234568 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc8cc102d40 0x7fc8cc1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:57.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.125+0000 7fc8d22e3640 1 -- 172.21.15.38:0/4089234568 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc8cc102d40 msgr2=0x7fc8cc1983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:57.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.125+0000 7fc8d22e3640 1 --2- 172.21.15.38:0/4089234568 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc8cc102d40 0x7fc8cc1983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:57.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.125+0000 7fc8d22e3640 1 -- 172.21.15.38:0/4089234568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8cc103660 msgr2=0x7fc8cc198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:57.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.125+0000 7fc8d22e3640 1 --2- 172.21.15.38:0/4089234568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8cc103660 0x7fc8cc198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:57.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.125+0000 7fc8d22e3640 1 -- 172.21.15.38:0/4089234568 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8cc19d3a0 con 0x7fc8cc10c170 2024-09-16T00:30:57.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.125+0000 7fc8d1ae2640 1 --2- 172.21.15.38:0/4089234568 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc8cc102d40 0x7fc8cc1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:30:57.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.126+0000 7fc8d22e3640 1 --2- 172.21.15.38:0/4089234568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8cc10c170 0x7fc8cc19ccc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc8c400d750 tx=0x7fc8c400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:57.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.126+0000 7fc8beffd640 1 -- 172.21.15.38:0/4089234568 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c4017070 con 0x7fc8cc10c170 2024-09-16T00:30:57.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.126+0000 7fc8beffd640 1 -- 172.21.15.38:0/4089234568 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc8c4005500 con 0x7fc8cc10c170 2024-09-16T00:30:57.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.126+0000 7fc8d3d46640 1 -- 172.21.15.38:0/4089234568 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8cc19d690 con 0x7fc8cc10c170 2024-09-16T00:30:57.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.126+0000 7fc8beffd640 1 -- 172.21.15.38:0/4089234568 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8c4004d90 con 0x7fc8cc10c170 2024-09-16T00:30:57.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.126+0000 7fc8d3d46640 1 -- 172.21.15.38:0/4089234568 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8cc1a4f70 con 0x7fc8cc10c170 2024-09-16T00:30:57.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.127+0000 7fc8d3d46640 1 -- 172.21.15.38:0/4089234568 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8cc103120 con 0x7fc8cc10c170 2024-09-16T00:30:57.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.128+0000 7fc8beffd640 1 -- 172.21.15.38:0/4089234568 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fc8c4004f30 con 0x7fc8cc10c170 2024-09-16T00:30:57.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.129+0000 7fc8beffd640 1 --2- 172.21.15.38:0/4089234568 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fc8b80a3240 0x7fc8b80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:57.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.129+0000 7fc8d1ae2640 1 --2- 172.21.15.38:0/4089234568 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fc8b80a3240 0x7fc8b80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:57.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.129+0000 7fc8beffd640 1 -- 172.21.15.38:0/4089234568 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fc8c40cf4b0 con 0x7fc8cc10c170 2024-09-16T00:30:57.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.129+0000 7fc8d1ae2640 1 --2- 172.21.15.38:0/4089234568 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fc8b80a3240 0x7fc8b80a5700 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0005ab0 tx=0x7fc8c0005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:57.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.133+0000 7fc8beffd640 1 -- 172.21.15.38:0/4089234568 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc8c409bf30 con 0x7fc8cc10c170 2024-09-16T00:30:57.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.314+0000 7fc8d3d46640 1 -- 172.21.15.38:0/4089234568 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc8cc106c80 con 0x7fc8cc10c170 2024-09-16T00:30:57.314 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:57 smithi038 ceph-mon[30945]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-16T00:30:57.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.316+0000 7fc8beffd640 1 -- 172.21.15.38:0/4089234568 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24174 (secure 0 0 0) 0x7fc8c409b830 con 0x7fc8cc10c170 2024-09-16T00:30:57.316 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:30:57.316 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":88,"fsid":"983b2072-73c1-11ef-bceb-c7b262605968","created":"2024-09-16T00:22:34.897053+0000","modified":"2024-09-16T00:30:35.865128+0000","last_up_change":"2024-09-16T00:29:34.503134+0000","last_in_change":"2024-09-16T00:29:10.135656+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:25:41.931977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:30:22.741889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:30:25.042590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:30:31.554350+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3afc20e3-869f-43a9-a911-b0e8d07de86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6803","nonce":1622028921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6805","nonce":1622028921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6809","nonce":1622028921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6807","nonce":1622028921}]},"public_addr":"172.21.15.38:6803/1622028921","cluster_addr":"172.21.15.38:6805/1622028921","heartbeat_back_addr":"172.21.15.38:6809/1622028921","heartbeat_front_addr":"172.21.15.38:6807/1622028921","state":["exists","up"]},{"osd":1,"uuid":"6d9d7acb-45ed-4543-8afc-6433fc7f096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6801","nonce":3811690336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6803","nonce":3811690336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6807","nonce":3811690336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6805","nonce":3811690336}]},"public_addr":"172.21.15.73:6801/3811690336","cluster_addr":"172.21.15.73:6803/3811690336","heartbeat_back_addr":"172.21.15.73:6807/3811690336","heartbeat_front_addr":"172.21.15.73:6805/3811690336","state":["exists","up"]},{"osd":2,"uuid":"48690f35-99e9-4f45-9bee-487857caa8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6801","nonce":1754741655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6803","nonce":1754741655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6807","nonce":1754741655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6805","nonce":1754741655}]},"public_addr":"172.21.15.122:6801/1754741655","cluster_addr":"172.21.15.122:6803/1754741655","heartbeat_back_addr":"172.21.15.122:6807/1754741655","heartbeat_front_addr":"172.21.15.122:6805/1754741655","state":["exists","up"]},{"osd":3,"uuid":"28ebb4a0-92e7-4f57-82a5-b9b761eca269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6811","nonce":3523190079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6813","nonce":3523190079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6817","nonce":3523190079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6815","nonce":3523190079}]},"public_addr":"172.21.15.38:6811/3523190079","cluster_addr":"172.21.15.38:6813/3523190079","heartbeat_back_addr":"172.21.15.38:6817/3523190079","heartbeat_front_addr":"172.21.15.38:6815/3523190079","state":["exists","up"]},{"osd":4,"uuid":"5ea363f7-7ad8-45ed-b7e4-4b010fe7f8cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6809","nonce":1469927628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6811","nonce":1469927628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6815","nonce":1469927628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6813","nonce":1469927628}]},"public_addr":"172.21.15.73:6809/1469927628","cluster_addr":"172.21.15.73:6811/1469927628","heartbeat_back_addr":"172.21.15.73:6815/1469927628","heartbeat_front_addr":"172.21.15.73:6813/1469927628","state":["exists","up"]},{"osd":5,"uuid":"e3c66d43-dd28-4bb1-a886-62a869be82cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6809","nonce":2954413058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6811","nonce":2954413058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6815","nonce":2954413058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6813","nonce":2954413058}]},"public_addr":"172.21.15.122:6809/2954413058","cluster_addr":"172.21.15.122:6811/2954413058","heartbeat_back_addr":"172.21.15.122:6815/2954413058","heartbeat_front_addr":"172.21.15.122:6813/2954413058","state":["exists","up"]},{"osd":6,"uuid":"211497ac-7c48-4095-bfdb-1d4df0149f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6819","nonce":1415719550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6821","nonce":1415719550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6825","nonce":1415719550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6823","nonce":1415719550}]},"public_addr":"172.21.15.38:6819/1415719550","cluster_addr":"172.21.15.38:6821/1415719550","heartbeat_back_addr":"172.21.15.38:6825/1415719550","heartbeat_front_addr":"172.21.15.38:6823/1415719550","state":["exists","up"]},{"osd":7,"uuid":"06512315-3f5e-44f8-976d-d3f5f1ba00e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6817","nonce":2634361473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6819","nonce":2634361473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6823","nonce":2634361473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6821","nonce":2634361473}]},"public_addr":"172.21.15.73:6817/2634361473","cluster_addr":"172.21.15.73:6819/2634361473","heartbeat_back_addr":"172.21.15.73:6823/2634361473","heartbeat_front_addr":"172.21.15.73:6821/2634361473","state":["exists","up"]},{"osd":8,"uuid":"db703c0a-1f87-46f0-be5f-f826f7f9daac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6817","nonce":1800567633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6819","nonce":1800567633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6823","nonce":1800567633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6821","nonce":1800567633}]},"public_addr":"172.21.15.122:6817/1800567633","cluster_addr":"172.21.15.122:6819/1800567633","heartbeat_back_addr":"172.21.15.122:6823/1800567633","heartbeat_front_addr":"172.21.15.122:6821/1800567633","state":["exists","up"]},{"osd":9,"uuid":"f5384200-b3b4-4936-85e1-fc57a9fe2bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6827","nonce":445727833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6829","nonce":445727833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6833","nonce":445727833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6831","nonce":445727833}]},"public_addr":"172.21.15.38:6827/445727833","cluster_addr":"172.21.15.38:6829/445727833","heartbeat_back_addr":"172.21.15.38:6833/445727833","heartbeat_front_addr":"172.21.15.38:6831/445727833","state":["exists","up"]},{"osd":10,"uuid":"c22883d9-6672-4796-a961-963d21eb37fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6825","nonce":43297139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6827","nonce":43297139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6831","nonce":43297139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6829","nonce":43297139}]},"public_addr":"172.21.15.73:6825/43297139","cluster_addr":"172.21.15.73:6827/43297139","heartbeat_back_addr":"172.21.15.73:6831/43297139","heartbeat_front_addr":"172.21.15.73:6829/43297139","state":["exists","up"]},{"osd":11,"uuid":"9f98e8cb-1bff-4b8b-aa19-56b691387383","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6825","nonce":1886446061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6827","nonce":1886446061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6831","nonce":1886446061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6829","nonce":1886446061}]},"public_addr":"172.21.15.122:6825/1886446061","cluster_addr":"172.21.15.122:6827/1886446061","heartbeat_back_addr":"172.21.15.122:6831/1886446061","heartbeat_front_addr":"172.21.15.122:6829/1886446061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:47.384012+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:12.834488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:38.770055+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:04.800466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:30.808076+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:56.646967+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:27:22.721270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:27:48.510011+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:28:13.922944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:28:40.514947+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:29:06.031052+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:29:31.882062+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6801/1779324916":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/924150447":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/141237849":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/2302081750":"2024-09-17T00:22:56.844212+0000","172.21.15.38:0/2245468594":"2024-09-17T00:22:56.844212+0000","172.21.15.38:6801/583006373":"2024-09-17T00:22:56.844212+0000","172.21.15.38:0/2435492667":"2024-09-17T00:23:21.417483+0000","172.21.15.38:6800/583006373":"2024-09-17T00:22:56.844212+0000","172.21.15.38:6800/1779324916":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/1103516837":"2024-09-17T00:22:56.844212+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:30:57.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.318+0000 7fc8d3d46640 1 -- 172.21.15.38:0/4089234568 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fc8b80a3240 msgr2=0x7fc8b80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:57.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.318+0000 7fc8d3d46640 1 --2- 172.21.15.38:0/4089234568 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fc8b80a3240 0x7fc8b80a5700 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc8c0005ab0 tx=0x7fc8c0005a20 comp rx=0 tx=0).stop 2024-09-16T00:30:57.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.318+0000 7fc8d3d46640 1 -- 172.21.15.38:0/4089234568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8cc10c170 msgr2=0x7fc8cc19ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:57.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.318+0000 7fc8d3d46640 1 --2- 172.21.15.38:0/4089234568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8cc10c170 0x7fc8cc19ccc0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc8c400d750 tx=0x7fc8c400dc10 comp rx=0 tx=0).stop 2024-09-16T00:30:57.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.318+0000 7fc8d3d46640 1 -- 172.21.15.38:0/4089234568 shutdown_connections 2024-09-16T00:30:57.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.318+0000 7fc8d3d46640 1 --2- 172.21.15.38:0/4089234568 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fc8b80a3240 0x7fc8b80a5700 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:57.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.318+0000 7fc8d3d46640 1 --2- 172.21.15.38:0/4089234568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc8cc10c170 0x7fc8cc19ccc0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:57.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.318+0000 7fc8d3d46640 1 --2- 172.21.15.38:0/4089234568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8cc103660 0x7fc8cc198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:57.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.318+0000 7fc8d3d46640 1 --2- 172.21.15.38:0/4089234568 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc8cc102d40 0x7fc8cc1983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:57.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.318+0000 7fc8d3d46640 1 -- 172.21.15.38:0/4089234568 >> 172.21.15.38:0/4089234568 conn(0x7fc8cc0fc7a0 msgr2=0x7fc8cc0fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:57.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.318+0000 7fc8d3d46640 1 -- 172.21.15.38:0/4089234568 shutdown_connections 2024-09-16T00:30:57.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:57.318+0000 7fc8d3d46640 1 -- 172.21.15.38:0/4089234568 wait complete. 2024-09-16T00:30:57.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:57 smithi073 ceph-mon[35254]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-16T00:30:57.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:57 smithi122 ceph-mon[35412]: pgmap v305: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-16T00:30:57.981 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-09-16T00:30:58.246 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:30:58.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:58 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/4089234568' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:30:58.561 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:58 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/4089234568' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:30:58.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:58 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/4089234568' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:30:58.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:58.775+0000 7f9c5780b640 1 -- 172.21.15.38:0/164717814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c50106a80 msgr2=0x7f9c50106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:58.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:58.775+0000 7f9c5780b640 1 --2- 172.21.15.38:0/164717814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c50106a80 0x7f9c50106e60 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c009930 tx=0x7f9c4c02a420 comp rx=0 tx=0).stop 2024-09-16T00:30:58.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:58.776+0000 7f9c5780b640 1 -- 172.21.15.38:0/164717814 shutdown_connections 2024-09-16T00:30:58.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:58.776+0000 7f9c5780b640 1 --2- 172.21.15.38:0/164717814 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c50101420 0x7f9c5010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:58.776+0000 7f9c5780b640 1 --2- 172.21.15.38:0/164717814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c50100a80 0x7f9c50100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:58.776+0000 7f9c5780b640 1 --2- 172.21.15.38:0/164717814 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c50106a80 0x7f9c50106e60 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:58.776+0000 7f9c5780b640 1 -- 172.21.15.38:0/164717814 >> 172.21.15.38:0/164717814 conn(0x7f9c500fc7a0 msgr2=0x7f9c500febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:58.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:58.776+0000 7f9c5780b640 1 -- 172.21.15.38:0/164717814 shutdown_connections 2024-09-16T00:30:58.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:58.776+0000 7f9c5780b640 1 -- 172.21.15.38:0/164717814 wait complete. 2024-09-16T00:30:58.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:58.776+0000 7f9c5780b640 1 Processor -- start 2024-09-16T00:30:58.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:58.776+0000 7f9c5780b640 1 -- start start 2024-09-16T00:30:58.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:58.777+0000 7f9c5780b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c50100a80 0x7f9c500fcdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:58.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:58.777+0000 7f9c5780b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c50101420 0x7f9c500fd330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:58.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:58.777+0000 7f9c5780b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c50106a80 0x7f9c500fd870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:58.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:58.777+0000 7f9c5780b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9c50110ec0 con 0x7f9c50106a80 2024-09-16T00:30:58.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:58.777+0000 7f9c5780b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9c50110d40 con 0x7f9c50100a80 2024-09-16T00:30:58.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:58.777+0000 7f9c5780b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9c50111040 con 0x7f9c50101420 2024-09-16T00:30:58.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:58.777+0000 7f9c55da8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c50106a80 0x7f9c500fd870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:58.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:58.777+0000 7f9c54da6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c50101420 0x7f9c500fd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:58.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:58.777+0000 7f9c55da8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c50106a80 0x7f9c500fd870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:50992/0 (socket says 172.21.15.38:50992) 2024-09-16T00:30:58.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:58.777+0000 7f9c55da8640 1 -- 172.21.15.38:0/875144496 learned_addr learned my addr 172.21.15.38:0/875144496 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:30:58.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:58.777+0000 7f9c555a7640 1 --2- 172.21.15.38:0/875144496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c50100a80 0x7f9c500fcdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:58.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:58.777+0000 7f9c54da6640 1 -- 172.21.15.38:0/875144496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c50100a80 msgr2=0x7f9c500fcdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:58.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:58.777+0000 7f9c54da6640 1 --2- 172.21.15.38:0/875144496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c50100a80 0x7f9c500fcdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:58.777+0000 7f9c54da6640 1 -- 172.21.15.38:0/875144496 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c50106a80 msgr2=0x7f9c500fd870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:58.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:58.777+0000 7f9c54da6640 1 --2- 172.21.15.38:0/875144496 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c50106a80 0x7f9c500fd870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:58.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:58.777+0000 7f9c54da6640 1 -- 172.21.15.38:0/875144496 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c501a6980 con 0x7f9c50101420 2024-09-16T00:30:58.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:58.777+0000 7f9c55da8640 1 --2- 172.21.15.38:0/875144496 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c50106a80 0x7f9c500fd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:30:58.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:58.777+0000 7f9c54da6640 1 --2- 172.21.15.38:0/875144496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c50101420 0x7f9c500fd330 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9c4000b3f0 tx=0x7f9c4000b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:58.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:58.778+0000 7f9c467fc640 1 -- 172.21.15.38:0/875144496 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c40015020 con 0x7f9c50101420 2024-09-16T00:30:58.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:58.778+0000 7f9c5780b640 1 -- 172.21.15.38:0/875144496 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c501a6c70 con 0x7f9c50101420 2024-09-16T00:30:58.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:58.778+0000 7f9c467fc640 1 -- 172.21.15.38:0/875144496 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c40011d10 con 0x7f9c50101420 2024-09-16T00:30:58.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:58.778+0000 7f9c467fc640 1 -- 172.21.15.38:0/875144496 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c4000f2e0 con 0x7f9c50101420 2024-09-16T00:30:58.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:58.778+0000 7f9c5780b640 1 -- 172.21.15.38:0/875144496 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c501a71b0 con 0x7f9c50101420 2024-09-16T00:30:58.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:58.779+0000 7f9c5780b640 1 -- 172.21.15.38:0/875144496 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c50100ee0 con 0x7f9c50101420 2024-09-16T00:30:58.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:58.780+0000 7f9c467fc640 1 -- 172.21.15.38:0/875144496 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f9c40018020 con 0x7f9c50101420 2024-09-16T00:30:58.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:58.781+0000 7f9c467fc640 1 --2- 172.21.15.38:0/875144496 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f9c3c0a3240 0x7f9c3c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:30:58.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:58.781+0000 7f9c555a7640 1 --2- 172.21.15.38:0/875144496 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f9c3c0a3240 0x7f9c3c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:30:58.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:58.781+0000 7f9c467fc640 1 -- 172.21.15.38:0/875144496 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f9c400cf870 con 0x7f9c50101420 2024-09-16T00:30:58.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:58.781+0000 7f9c555a7640 1 --2- 172.21.15.38:0/875144496 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f9c3c0a3240 0x7f9c3c0a5700 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c009900 tx=0x7f9c4c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:30:58.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:58.784+0000 7f9c467fc640 1 -- 172.21.15.38:0/875144496 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9c4001f020 con 0x7f9c50101420 2024-09-16T00:30:59.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:59.001+0000 7f9c5780b640 1 -- 172.21.15.38:0/875144496 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f9c5010a000 con 0x7f9c50101420 2024-09-16T00:30:59.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:59.271+0000 7f9c467fc640 1 -- 172.21.15.38:0/875144496 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f9c4009c2f0 con 0x7f9c50101420 2024-09-16T00:30:59.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:59.274+0000 7f9c5780b640 1 -- 172.21.15.38:0/875144496 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f9c3c0a3240 msgr2=0x7f9c3c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:59.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:59.274+0000 7f9c5780b640 1 --2- 172.21.15.38:0/875144496 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f9c3c0a3240 0x7f9c3c0a5700 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9c4c009900 tx=0x7f9c4c0058d0 comp rx=0 tx=0).stop 2024-09-16T00:30:59.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:59.274+0000 7f9c5780b640 1 -- 172.21.15.38:0/875144496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c50101420 msgr2=0x7f9c500fd330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:30:59.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:59.274+0000 7f9c5780b640 1 --2- 172.21.15.38:0/875144496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c50101420 0x7f9c500fd330 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9c4000b3f0 tx=0x7f9c4000b8b0 comp rx=0 tx=0).stop 2024-09-16T00:30:59.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:59.274+0000 7f9c5780b640 1 -- 172.21.15.38:0/875144496 shutdown_connections 2024-09-16T00:30:59.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:59.274+0000 7f9c5780b640 1 --2- 172.21.15.38:0/875144496 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f9c3c0a3240 0x7f9c3c0a5700 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:59.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:59.274+0000 7f9c5780b640 1 --2- 172.21.15.38:0/875144496 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c50106a80 0x7f9c500fd870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:59.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:59.274+0000 7f9c5780b640 1 --2- 172.21.15.38:0/875144496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c50101420 0x7f9c500fd330 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:59.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:59.274+0000 7f9c5780b640 1 --2- 172.21.15.38:0/875144496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c50100a80 0x7f9c500fcdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:30:59.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:59.274+0000 7f9c5780b640 1 -- 172.21.15.38:0/875144496 >> 172.21.15.38:0/875144496 conn(0x7f9c500fc7a0 msgr2=0x7f9c5006aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:30:59.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:59.274+0000 7f9c5780b640 1 -- 172.21.15.38:0/875144496 shutdown_connections 2024-09-16T00:30:59.275 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:30:59.274+0000 7f9c5780b640 1 -- 172.21.15.38:0/875144496 wait complete. 2024-09-16T00:30:59.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:59 smithi073 ceph-mon[35254]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-16T00:30:59.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:59 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/875144496' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-16T00:30:59.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:30:59 smithi073 ceph-mon[35254]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-16T00:30:59.646 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:59 smithi038 ceph-mon[30945]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-16T00:30:59.646 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:59 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/875144496' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-16T00:30:59.646 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:30:59 smithi038 ceph-mon[30945]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-16T00:30:59.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:59 smithi122 ceph-mon[35412]: pgmap v306: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-09-16T00:30:59.979 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:59 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/875144496' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-16T00:30:59.979 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:30:59 smithi122 ceph-mon[35412]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-09-16T00:31:00.084 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-09-16T00:31:00.085 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs set cephfs max_mds 3 2024-09-16T00:31:00.350 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:00.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:00 smithi038 ceph-mon[30945]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-16T00:31:00.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:00 smithi038 ceph-mon[30945]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:31:00.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:00 smithi073 ceph-mon[35254]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-16T00:31:00.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:00 smithi073 ceph-mon[35254]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:31:00.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:00 smithi122 ceph-mon[35412]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-16T00:31:00.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:00 smithi122 ceph-mon[35412]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:31:00.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:00.850+0000 7f79603b1640 1 -- 172.21.15.38:0/1200027349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79581082e0 msgr2=0x7f795810ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:00.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:00.850+0000 7f79603b1640 1 --2- 172.21.15.38:0/1200027349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79581082e0 0x7f795810ec50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f7950009cd0 tx=0x7f795002a8e0 comp rx=0 tx=0).stop 2024-09-16T00:31:00.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:00.850+0000 7f79603b1640 1 -- 172.21.15.38:0/1200027349 shutdown_connections 2024-09-16T00:31:00.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:00.850+0000 7f79603b1640 1 --2- 172.21.15.38:0/1200027349 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79581082e0 0x7f795810ec50 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:00.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:00.850+0000 7f79603b1640 1 --2- 172.21.15.38:0/1200027349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7958103760 0x7f7958107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:00.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:00.850+0000 7f79603b1640 1 --2- 172.21.15.38:0/1200027349 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7958102db0 0x7f7958103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:00.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:00.850+0000 7f79603b1640 1 -- 172.21.15.38:0/1200027349 >> 172.21.15.38:0/1200027349 conn(0x7f79580fc7d0 msgr2=0x7f79580febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:00.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:00.850+0000 7f79603b1640 1 -- 172.21.15.38:0/1200027349 shutdown_connections 2024-09-16T00:31:00.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:00.850+0000 7f79603b1640 1 -- 172.21.15.38:0/1200027349 wait complete. 2024-09-16T00:31:00.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:00.850+0000 7f79603b1640 1 Processor -- start 2024-09-16T00:31:00.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:00.850+0000 7f79603b1640 1 -- start start 2024-09-16T00:31:00.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:00.851+0000 7f79603b1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7958102db0 0x7f7958100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:00.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:00.851+0000 7f79603b1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7958103760 0x7f7958100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:00.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:00.851+0000 7f79603b1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79581082e0 0x7f7958101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:00.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:00.851+0000 7f79603b1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f79581120b0 con 0x7f79581082e0 2024-09-16T00:31:00.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:00.851+0000 7f79603b1640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7958111f30 con 0x7f7958102db0 2024-09-16T00:31:00.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:00.851+0000 7f79603b1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7958112230 con 0x7f7958103760 2024-09-16T00:31:00.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:00.851+0000 7f795e94e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79581082e0 0x7f7958101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:00.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:00.851+0000 7f795e94e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79581082e0 0x7f7958101490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51012/0 (socket says 172.21.15.38:51012) 2024-09-16T00:31:00.875 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:00.851+0000 7f795e94e640 1 -- 172.21.15.38:0/226606930 learned_addr learned my addr 172.21.15.38:0/226606930 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:31:00.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:00.851+0000 7f795e14d640 1 --2- 172.21.15.38:0/226606930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7958102db0 0x7f7958100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:00.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:00.851+0000 7f795d94c640 1 --2- 172.21.15.38:0/226606930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7958103760 0x7f7958100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:00.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:00.851+0000 7f795e94e640 1 -- 172.21.15.38:0/226606930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7958103760 msgr2=0x7f7958100f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:00.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:00.851+0000 7f795e94e640 1 --2- 172.21.15.38:0/226606930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7958103760 0x7f7958100f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:00.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:00.851+0000 7f795e94e640 1 -- 172.21.15.38:0/226606930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7958102db0 msgr2=0x7f7958100a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:00.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:00.851+0000 7f795e94e640 1 --2- 172.21.15.38:0/226606930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7958102db0 0x7f7958100a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:00.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:00.851+0000 7f795e94e640 1 -- 172.21.15.38:0/226606930 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79581a6a70 con 0x7f79581082e0 2024-09-16T00:31:00.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:00.851+0000 7f795e14d640 1 --2- 172.21.15.38:0/226606930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7958102db0 0x7f7958100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:31:00.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:00.851+0000 7f795e94e640 1 --2- 172.21.15.38:0/226606930 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79581082e0 0x7f7958101490 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7950005e30 tx=0x7f7950003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:00.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:00.852+0000 7f794f7fe640 1 -- 172.21.15.38:0/226606930 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7950004040 con 0x7f79581082e0 2024-09-16T00:31:00.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:00.852+0000 7f794f7fe640 1 -- 172.21.15.38:0/226606930 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f79500046c0 con 0x7f79581082e0 2024-09-16T00:31:00.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:00.852+0000 7f79603b1640 1 -- 172.21.15.38:0/226606930 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79581a6d00 con 0x7f79581082e0 2024-09-16T00:31:00.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:00.852+0000 7f794f7fe640 1 -- 172.21.15.38:0/226606930 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f795003a400 con 0x7f79581082e0 2024-09-16T00:31:00.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:00.852+0000 7f79603b1640 1 -- 172.21.15.38:0/226606930 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79581a7290 con 0x7f79581082e0 2024-09-16T00:31:00.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:00.853+0000 7f79603b1640 1 -- 172.21.15.38:0/226606930 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7958103190 con 0x7f79581082e0 2024-09-16T00:31:00.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:00.854+0000 7f794f7fe640 1 -- 172.21.15.38:0/226606930 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f7950030630 con 0x7f79581082e0 2024-09-16T00:31:00.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:00.855+0000 7f794f7fe640 1 --2- 172.21.15.38:0/226606930 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f79440a32c0 0x7f79440a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:00.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:00.855+0000 7f794f7fe640 1 -- 172.21.15.38:0/226606930 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f795002e030 con 0x7f79581082e0 2024-09-16T00:31:00.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:00.855+0000 7f795e14d640 1 --2- 172.21.15.38:0/226606930 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f79440a32c0 0x7f79440a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:00.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:00.855+0000 7f795e14d640 1 --2- 172.21.15.38:0/226606930 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f79440a32c0 0x7f79440a5780 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f79540058c0 tx=0x7f7954005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:00.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:00.858+0000 7f794f7fe640 1 -- 172.21.15.38:0/226606930 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f79500bd800 con 0x7f79581082e0 2024-09-16T00:31:01.072 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:01.072+0000 7f79603b1640 1 -- 172.21.15.38:0/226606930 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) -- 0x7f795810a2c0 con 0x7f79581082e0 2024-09-16T00:31:01.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:01.286+0000 7f794f7fe640 1 -- 172.21.15.38:0/226606930 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f79500bd100 con 0x7f79581082e0 2024-09-16T00:31:01.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:01.289+0000 7f79603b1640 1 -- 172.21.15.38:0/226606930 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f79440a32c0 msgr2=0x7f79440a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:01.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:01.289+0000 7f79603b1640 1 --2- 172.21.15.38:0/226606930 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f79440a32c0 0x7f79440a5780 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f79540058c0 tx=0x7f7954005850 comp rx=0 tx=0).stop 2024-09-16T00:31:01.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:01.289+0000 7f79603b1640 1 -- 172.21.15.38:0/226606930 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79581082e0 msgr2=0x7f7958101490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:01.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:01.289+0000 7f79603b1640 1 --2- 172.21.15.38:0/226606930 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79581082e0 0x7f7958101490 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f7950005e30 tx=0x7f7950003970 comp rx=0 tx=0).stop 2024-09-16T00:31:01.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:01.289+0000 7f79603b1640 1 -- 172.21.15.38:0/226606930 shutdown_connections 2024-09-16T00:31:01.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:01.289+0000 7f79603b1640 1 --2- 172.21.15.38:0/226606930 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f79440a32c0 0x7f79440a5780 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:01.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:01.289+0000 7f79603b1640 1 --2- 172.21.15.38:0/226606930 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f79581082e0 0x7f7958101490 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:01.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:01.289+0000 7f79603b1640 1 --2- 172.21.15.38:0/226606930 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7958103760 0x7f7958100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:01.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:01.289+0000 7f79603b1640 1 --2- 172.21.15.38:0/226606930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7958102db0 0x7f7958100a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:01.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:01.289+0000 7f79603b1640 1 -- 172.21.15.38:0/226606930 >> 172.21.15.38:0/226606930 conn(0x7f79580fc7d0 msgr2=0x7f7958106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:01.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:01.289+0000 7f79603b1640 1 -- 172.21.15.38:0/226606930 shutdown_connections 2024-09-16T00:31:01.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:01.289+0000 7f79603b1640 1 -- 172.21.15.38:0/226606930 wait complete. 2024-09-16T00:31:01.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:01 smithi038 ceph-mon[30945]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T00:31:01.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:01 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/226606930' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-16T00:31:01.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:01 smithi073 ceph-mon[35254]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T00:31:01.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:01 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/226606930' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-16T00:31:01.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:01 smithi122 ceph-mon[35412]: pgmap v307: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T00:31:01.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:01 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/226606930' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2024-09-16T00:31:02.090 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-09-16T00:31:02.361 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:02.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:02 smithi038 ceph-mon[30945]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-16T00:31:02.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:02 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/226606930' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-16T00:31:02.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:02 smithi038 ceph-mon[30945]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-16T00:31:02.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:02 smithi038 ceph-mon[30945]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:31:02.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:02 smithi038 ceph-mon[30945]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-16T00:31:02.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:02 smithi038 ceph-mon[30945]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-16T00:31:02.544 INFO:journalctl@ceph.mds.b.smithi073.stdout:Sep 16 00:31:02 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-b[65406]: 2024-09-16T00:31:02.441+0000 7fe7cd552640 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-09-16T00:31:02.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:02 smithi073 ceph-mon[35254]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-16T00:31:02.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:02 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/226606930' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-16T00:31:02.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:02 smithi073 ceph-mon[35254]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-16T00:31:02.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:02 smithi073 ceph-mon[35254]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:31:02.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:02 smithi073 ceph-mon[35254]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-16T00:31:02.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:02 smithi073 ceph-mon[35254]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-16T00:31:02.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:02 smithi122 ceph-mon[35412]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-09-16T00:31:02.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:02 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/226606930' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2024-09-16T00:31:02.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:02 smithi122 ceph-mon[35412]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-09-16T00:31:02.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:02 smithi122 ceph-mon[35412]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-16T00:31:02.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:02 smithi122 ceph-mon[35412]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-09-16T00:31:02.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:02 smithi122 ceph-mon[35412]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-09-16T00:31:02.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:02.834+0000 7fdcc8fc2640 1 -- 172.21.15.38:0/2845458149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcc4106a80 msgr2=0x7fdcc4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:02.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:02.834+0000 7fdcc8fc2640 1 --2- 172.21.15.38:0/2845458149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcc4106a80 0x7fdcc4106e60 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4009930 tx=0x7fdcb402a420 comp rx=0 tx=0).stop 2024-09-16T00:31:02.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:02.834+0000 7fdcc8fc2640 1 -- 172.21.15.38:0/2845458149 shutdown_connections 2024-09-16T00:31:02.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:02.834+0000 7fdcc8fc2640 1 --2- 172.21.15.38:0/2845458149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcc4101420 0x7fdcc410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:02.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:02.834+0000 7fdcc8fc2640 1 --2- 172.21.15.38:0/2845458149 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcc4100a80 0x7fdcc4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:02.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:02.834+0000 7fdcc8fc2640 1 --2- 172.21.15.38:0/2845458149 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcc4106a80 0x7fdcc4106e60 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:02.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:02.834+0000 7fdcc8fc2640 1 -- 172.21.15.38:0/2845458149 >> 172.21.15.38:0/2845458149 conn(0x7fdcc40fc7a0 msgr2=0x7fdcc40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:02.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:02.834+0000 7fdcc8fc2640 1 -- 172.21.15.38:0/2845458149 shutdown_connections 2024-09-16T00:31:02.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:02.834+0000 7fdcc8fc2640 1 -- 172.21.15.38:0/2845458149 wait complete. 2024-09-16T00:31:02.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:02.835+0000 7fdcc8fc2640 1 Processor -- start 2024-09-16T00:31:02.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:02.835+0000 7fdcc8fc2640 1 -- start start 2024-09-16T00:31:02.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:02.835+0000 7fdcc8fc2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcc4100a80 0x7fdcc4197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:02.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:02.836+0000 7fdcc8fc2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcc4101420 0x7fdcc4197870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:02.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:02.836+0000 7fdcc8fc2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcc4106a80 0x7fdcc419bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:02.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:02.836+0000 7fdcc8fc2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc4110e40 con 0x7fdcc4101420 2024-09-16T00:31:02.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:02.836+0000 7fdcc8fc2640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc4110cc0 con 0x7fdcc4106a80 2024-09-16T00:31:02.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:02.836+0000 7fdcc8fc2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc4110fc0 con 0x7fdcc4100a80 2024-09-16T00:31:02.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:02.836+0000 7fdcc1d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcc4101420 0x7fdcc4197870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:02.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:02.836+0000 7fdcc1d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcc4101420 0x7fdcc4197870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51026/0 (socket says 172.21.15.38:51026) 2024-09-16T00:31:02.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:02.836+0000 7fdcc1d9b640 1 -- 172.21.15.38:0/334941573 learned_addr learned my addr 172.21.15.38:0/334941573 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:31:02.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:02.836+0000 7fdcc259c640 1 --2- 172.21.15.38:0/334941573 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcc4100a80 0x7fdcc4197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:02.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:02.836+0000 7fdcc1d9b640 1 -- 172.21.15.38:0/334941573 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcc4100a80 msgr2=0x7fdcc4197330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:02.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:02.836+0000 7fdcc1d9b640 1 --2- 172.21.15.38:0/334941573 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcc4100a80 0x7fdcc4197330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:02.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:02.836+0000 7fdcc1d9b640 1 -- 172.21.15.38:0/334941573 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcc4106a80 msgr2=0x7fdcc419bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:02.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:02.836+0000 7fdcc2d9d640 1 --2- 172.21.15.38:0/334941573 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcc4106a80 0x7fdcc419bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:02.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:02.836+0000 7fdcc1d9b640 1 --2- 172.21.15.38:0/334941573 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcc4106a80 0x7fdcc419bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:02.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:02.836+0000 7fdcc1d9b640 1 -- 172.21.15.38:0/334941573 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc419c2e0 con 0x7fdcc4101420 2024-09-16T00:31:02.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:02.836+0000 7fdcc2d9d640 1 --2- 172.21.15.38:0/334941573 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcc4106a80 0x7fdcc419bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:31:02.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:02.836+0000 7fdcc1d9b640 1 --2- 172.21.15.38:0/334941573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcc4101420 0x7fdcc4197870 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdcac00c920 tx=0x7fdcac00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:02.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:02.836+0000 7fdcbb7fe640 1 -- 172.21.15.38:0/334941573 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcac007c70 con 0x7fdcc4101420 2024-09-16T00:31:02.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:02.836+0000 7fdcbb7fe640 1 -- 172.21.15.38:0/334941573 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdcac007e10 con 0x7fdcc4101420 2024-09-16T00:31:02.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:02.836+0000 7fdcc8fc2640 1 -- 172.21.15.38:0/334941573 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc419c5d0 con 0x7fdcc4101420 2024-09-16T00:31:02.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:02.836+0000 7fdcbb7fe640 1 -- 172.21.15.38:0/334941573 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcac018890 con 0x7fdcc4101420 2024-09-16T00:31:02.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:02.836+0000 7fdcc8fc2640 1 -- 172.21.15.38:0/334941573 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc41a3eb0 con 0x7fdcc4101420 2024-09-16T00:31:02.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:02.837+0000 7fdcc8fc2640 1 -- 172.21.15.38:0/334941573 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcc4100ee0 con 0x7fdcc4101420 2024-09-16T00:31:02.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:02.838+0000 7fdcbb7fe640 1 -- 172.21.15.38:0/334941573 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7fdcac018a70 con 0x7fdcc4101420 2024-09-16T00:31:02.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:02.839+0000 7fdcbb7fe640 1 --2- 172.21.15.38:0/334941573 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fdcb00a32c0 0x7fdcb00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:02.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:02.839+0000 7fdcbb7fe640 1 -- 172.21.15.38:0/334941573 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fdcac0cfc40 con 0x7fdcc4101420 2024-09-16T00:31:02.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:02.839+0000 7fdcc259c640 1 --2- 172.21.15.38:0/334941573 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fdcb00a32c0 0x7fdcb00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:02.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:02.840+0000 7fdcc259c640 1 --2- 172.21.15.38:0/334941573 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fdcb00a32c0 0x7fdcb00a5780 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4009900 tx=0x7fdcb40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:02.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:02.844+0000 7fdcbb7fe640 1 -- 172.21.15.38:0/334941573 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdcac014030 con 0x7fdcc4101420 2024-09-16T00:31:03.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:03.055+0000 7fdcc8fc2640 1 -- 172.21.15.38:0/334941573 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fdcc402d7c0 con 0x7fdcc4101420 2024-09-16T00:31:03.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:03.311+0000 7fdcbb7fe640 1 -- 172.21.15.38:0/334941573 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v23) ==== 121+0+0 (secure 0 0 0) 0x7fdcac09c720 con 0x7fdcc4101420 2024-09-16T00:31:03.313 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:03 smithi038 ceph-mon[30945]: pgmap v308: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T00:31:03.313 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:03 smithi038 ceph-mon[30945]: mds.? [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] up:active 2024-09-16T00:31:03.313 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:03 smithi038 ceph-mon[30945]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-16T00:31:03.313 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:03 smithi038 ceph-mon[30945]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-16T00:31:03.313 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:03 smithi038 ceph-mon[30945]: Cluster is now healthy 2024-09-16T00:31:03.313 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:03 smithi038 ceph-mon[30945]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-16T00:31:03.313 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:03 smithi038 ceph-mon[30945]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-09-16T00:31:03.313 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:03 smithi038 ceph-mon[30945]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-09-16T00:31:03.313 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:03 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/334941573' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-16T00:31:03.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:03.313+0000 7fdcc8fc2640 1 -- 172.21.15.38:0/334941573 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fdcb00a32c0 msgr2=0x7fdcb00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:03.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:03.313+0000 7fdcc8fc2640 1 --2- 172.21.15.38:0/334941573 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fdcb00a32c0 0x7fdcb00a5780 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4009900 tx=0x7fdcb40058d0 comp rx=0 tx=0).stop 2024-09-16T00:31:03.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:03.313+0000 7fdcc8fc2640 1 -- 172.21.15.38:0/334941573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcc4101420 msgr2=0x7fdcc4197870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:03.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:03.313+0000 7fdcc8fc2640 1 --2- 172.21.15.38:0/334941573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcc4101420 0x7fdcc4197870 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fdcac00c920 tx=0x7fdcac00cde0 comp rx=0 tx=0).stop 2024-09-16T00:31:03.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:03.313+0000 7fdcc8fc2640 1 -- 172.21.15.38:0/334941573 shutdown_connections 2024-09-16T00:31:03.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:03.313+0000 7fdcc8fc2640 1 --2- 172.21.15.38:0/334941573 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fdcb00a32c0 0x7fdcb00a5780 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:03.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:03.313+0000 7fdcc8fc2640 1 --2- 172.21.15.38:0/334941573 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdcc4106a80 0x7fdcc419bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:03.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:03.313+0000 7fdcc8fc2640 1 --2- 172.21.15.38:0/334941573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdcc4101420 0x7fdcc4197870 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:03.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:03.313+0000 7fdcc8fc2640 1 --2- 172.21.15.38:0/334941573 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdcc4100a80 0x7fdcc4197330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:03.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:03.313+0000 7fdcc8fc2640 1 -- 172.21.15.38:0/334941573 >> 172.21.15.38:0/334941573 conn(0x7fdcc40fc7a0 msgr2=0x7fdcc40fd050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:03.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:03.313+0000 7fdcc8fc2640 1 -- 172.21.15.38:0/334941573 shutdown_connections 2024-09-16T00:31:03.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:03.313+0000 7fdcc8fc2640 1 -- 172.21.15.38:0/334941573 wait complete. 2024-09-16T00:31:03.618 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:03 smithi122 ceph-mon[35412]: pgmap v308: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T00:31:03.618 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:03 smithi122 ceph-mon[35412]: mds.? [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] up:active 2024-09-16T00:31:03.618 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:03 smithi122 ceph-mon[35412]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-16T00:31:03.618 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:03 smithi122 ceph-mon[35412]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-16T00:31:03.618 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:03 smithi122 ceph-mon[35412]: Cluster is now healthy 2024-09-16T00:31:03.618 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:03 smithi122 ceph-mon[35412]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-16T00:31:03.618 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:03 smithi122 ceph-mon[35412]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-09-16T00:31:03.619 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:03 smithi122 ceph-mon[35412]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-09-16T00:31:03.619 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:03 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/334941573' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-16T00:31:03.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:03 smithi073 ceph-mon[35254]: pgmap v308: 129 pgs: 129 active+clean; 581 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-09-16T00:31:03.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:03 smithi073 ceph-mon[35254]: mds.? [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] up:active 2024-09-16T00:31:03.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:03 smithi073 ceph-mon[35254]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-09-16T00:31:03.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:03 smithi073 ceph-mon[35254]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-09-16T00:31:03.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:03 smithi073 ceph-mon[35254]: Cluster is now healthy 2024-09-16T00:31:03.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:03 smithi073 ceph-mon[35254]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-09-16T00:31:03.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:03 smithi073 ceph-mon[35254]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:creating} 9 up:standby 2024-09-16T00:31:03.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:03 smithi073 ceph-mon[35254]: daemon mds.e is now active in filesystem cephfs as rank 2 2024-09-16T00:31:03.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:03 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/334941573' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-09-16T00:31:04.083 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-09-16T00:31:04.343 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:04.607 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:04 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/334941573' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-16T00:31:04.607 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:04 smithi038 ceph-mon[30945]: mds.? [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] up:active 2024-09-16T00:31:04.607 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:04 smithi038 ceph-mon[30945]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-09-16T00:31:04.607 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:04 smithi038 ceph-mon[30945]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-16T00:31:04.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:04 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/334941573' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-16T00:31:04.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:04 smithi122 ceph-mon[35412]: mds.? [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] up:active 2024-09-16T00:31:04.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:04 smithi122 ceph-mon[35412]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-09-16T00:31:04.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:04 smithi122 ceph-mon[35412]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-16T00:31:04.793 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:04 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/334941573' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-16T00:31:04.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:04 smithi073 ceph-mon[35254]: mds.? [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] up:active 2024-09-16T00:31:04.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:04 smithi073 ceph-mon[35254]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 9 up:standby 2024-09-16T00:31:04.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:04 smithi073 ceph-mon[35254]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-16T00:31:04.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.854+0000 7f85a9872640 1 -- 172.21.15.38:0/3958869515 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85a4106a80 msgr2=0x7f85a4106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:04.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.854+0000 7f85a9872640 1 --2- 172.21.15.38:0/3958869515 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85a4106a80 0x7f85a4106e60 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8594009930 tx=0x7f859402a420 comp rx=0 tx=0).stop 2024-09-16T00:31:04.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.854+0000 7f85a9872640 1 -- 172.21.15.38:0/3958869515 shutdown_connections 2024-09-16T00:31:04.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.854+0000 7f85a9872640 1 --2- 172.21.15.38:0/3958869515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85a4101420 0x7f85a410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:04.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.854+0000 7f85a9872640 1 --2- 172.21.15.38:0/3958869515 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85a4100a80 0x7f85a4100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:04.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.854+0000 7f85a9872640 1 --2- 172.21.15.38:0/3958869515 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85a4106a80 0x7f85a4106e60 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:04.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.854+0000 7f85a9872640 1 -- 172.21.15.38:0/3958869515 >> 172.21.15.38:0/3958869515 conn(0x7f85a40fc7a0 msgr2=0x7f85a40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:04.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.854+0000 7f85a9872640 1 -- 172.21.15.38:0/3958869515 shutdown_connections 2024-09-16T00:31:04.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.854+0000 7f85a9872640 1 -- 172.21.15.38:0/3958869515 wait complete. 2024-09-16T00:31:04.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.855+0000 7f85a9872640 1 Processor -- start 2024-09-16T00:31:04.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.855+0000 7f85a9872640 1 -- start start 2024-09-16T00:31:04.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.855+0000 7f85a9872640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85a4100a80 0x7f85a4198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:04.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.855+0000 7f85a9872640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85a4101420 0x7f85a4198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:04.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.855+0000 7f85a9872640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85a4106a80 0x7f85a419cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:04.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.855+0000 7f85a9872640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f85a4110e40 con 0x7f85a4106a80 2024-09-16T00:31:04.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.855+0000 7f85a9872640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f85a4110cc0 con 0x7f85a4100a80 2024-09-16T00:31:04.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.855+0000 7f85a9872640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f85a4110fc0 con 0x7f85a4101420 2024-09-16T00:31:04.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.855+0000 7f85a37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85a4106a80 0x7f85a419cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:04.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.855+0000 7f85a37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85a4106a80 0x7f85a419cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51038/0 (socket says 172.21.15.38:51038) 2024-09-16T00:31:04.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.855+0000 7f85a37fe640 1 -- 172.21.15.38:0/162328499 learned_addr learned my addr 172.21.15.38:0/162328499 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:31:04.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.856+0000 7f85a27fc640 1 --2- 172.21.15.38:0/162328499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85a4101420 0x7f85a4198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:04.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.856+0000 7f85a2ffd640 1 --2- 172.21.15.38:0/162328499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85a4100a80 0x7f85a4198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:04.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.856+0000 7f85a37fe640 1 -- 172.21.15.38:0/162328499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85a4101420 msgr2=0x7f85a4198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:04.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.856+0000 7f85a37fe640 1 --2- 172.21.15.38:0/162328499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85a4101420 0x7f85a4198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:04.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.856+0000 7f85a37fe640 1 -- 172.21.15.38:0/162328499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85a4100a80 msgr2=0x7f85a4198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:04.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.856+0000 7f85a37fe640 1 --2- 172.21.15.38:0/162328499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85a4100a80 0x7f85a4198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:04.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.856+0000 7f85a37fe640 1 -- 172.21.15.38:0/162328499 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85a419d3c0 con 0x7f85a4106a80 2024-09-16T00:31:04.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.856+0000 7f85a27fc640 1 --2- 172.21.15.38:0/162328499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85a4101420 0x7f85a4198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:31:04.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.856+0000 7f85a2ffd640 1 --2- 172.21.15.38:0/162328499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85a4100a80 0x7f85a4198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:31:04.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.856+0000 7f85a37fe640 1 --2- 172.21.15.38:0/162328499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85a4106a80 0x7f85a419cce0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f859c00d750 tx=0x7f859c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:04.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.857+0000 7f85a8870640 1 -- 172.21.15.38:0/162328499 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c017070 con 0x7f85a4106a80 2024-09-16T00:31:04.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.857+0000 7f85a9872640 1 -- 172.21.15.38:0/162328499 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85a419d6b0 con 0x7f85a4106a80 2024-09-16T00:31:04.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.857+0000 7f85a9872640 1 -- 172.21.15.38:0/162328499 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85a41a4f90 con 0x7f85a4106a80 2024-09-16T00:31:04.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.857+0000 7f85a8870640 1 -- 172.21.15.38:0/162328499 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f859c005500 con 0x7f85a4106a80 2024-09-16T00:31:04.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.857+0000 7f85a8870640 1 -- 172.21.15.38:0/162328499 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f859c020060 con 0x7f85a4106a80 2024-09-16T00:31:04.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.858+0000 7f85a9872640 1 -- 172.21.15.38:0/162328499 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f85a4100ee0 con 0x7f85a4106a80 2024-09-16T00:31:04.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.859+0000 7f85a8870640 1 -- 172.21.15.38:0/162328499 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f859c009c60 con 0x7f85a4106a80 2024-09-16T00:31:04.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.860+0000 7f85a8870640 1 --2- 172.21.15.38:0/162328499 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f858c0a3240 0x7f858c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:04.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.860+0000 7f85a2ffd640 1 --2- 172.21.15.38:0/162328499 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f858c0a3240 0x7f858c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:04.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.860+0000 7f85a8870640 1 -- 172.21.15.38:0/162328499 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f859c0cf610 con 0x7f85a4106a80 2024-09-16T00:31:04.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.861+0000 7f85a2ffd640 1 --2- 172.21.15.38:0/162328499 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f858c0a3240 0x7f858c0a5700 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8594000c00 tx=0x7f8594005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:04.873 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:04.864+0000 7f85a8870640 1 -- 172.21.15.38:0/162328499 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f859c09c0f0 con 0x7f85a4106a80 2024-09-16T00:31:05.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:05.080+0000 7f85a9872640 1 -- 172.21.15.38:0/162328499 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f85a410a000 con 0x7f85a4106a80 2024-09-16T00:31:05.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:05.329+0000 7f85a8870640 1 -- 172.21.15.38:0/162328499 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7f859c09b9f0 con 0x7f85a4106a80 2024-09-16T00:31:05.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:05.332+0000 7f85a9872640 1 -- 172.21.15.38:0/162328499 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f858c0a3240 msgr2=0x7f858c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:05.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:05.332+0000 7f85a9872640 1 --2- 172.21.15.38:0/162328499 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f858c0a3240 0x7f858c0a5700 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8594000c00 tx=0x7f8594005a20 comp rx=0 tx=0).stop 2024-09-16T00:31:05.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:05.332+0000 7f85a9872640 1 -- 172.21.15.38:0/162328499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85a4106a80 msgr2=0x7f85a419cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:05.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:05.332+0000 7f85a9872640 1 --2- 172.21.15.38:0/162328499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85a4106a80 0x7f85a419cce0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f859c00d750 tx=0x7f859c00dc10 comp rx=0 tx=0).stop 2024-09-16T00:31:05.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:05.332+0000 7f85a9872640 1 -- 172.21.15.38:0/162328499 shutdown_connections 2024-09-16T00:31:05.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:05.332+0000 7f85a9872640 1 --2- 172.21.15.38:0/162328499 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f858c0a3240 0x7f858c0a5700 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:05.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:05.332+0000 7f85a9872640 1 --2- 172.21.15.38:0/162328499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f85a4106a80 0x7f85a419cce0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:05.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:05.332+0000 7f85a9872640 1 --2- 172.21.15.38:0/162328499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f85a4101420 0x7f85a4198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:05.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:05.332+0000 7f85a9872640 1 --2- 172.21.15.38:0/162328499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85a4100a80 0x7f85a4198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:05.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:05.332+0000 7f85a9872640 1 -- 172.21.15.38:0/162328499 >> 172.21.15.38:0/162328499 conn(0x7f85a40fc7a0 msgr2=0x7f85a406aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:05.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:05.333+0000 7f85a9872640 1 -- 172.21.15.38:0/162328499 shutdown_connections 2024-09-16T00:31:05.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:05.333+0000 7f85a9872640 1 -- 172.21.15.38:0/162328499 wait complete. 2024-09-16T00:31:05.631 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:05 smithi038 ceph-mon[30945]: pgmap v309: 129 pgs: 129 active+clean; 583 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 2 op/s 2024-09-16T00:31:05.631 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:05 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/162328499' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-16T00:31:05.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:05 smithi122 ceph-mon[35412]: pgmap v309: 129 pgs: 129 active+clean; 583 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 2 op/s 2024-09-16T00:31:05.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:05 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/162328499' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-16T00:31:05.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:05 smithi073 ceph-mon[35254]: pgmap v309: 129 pgs: 129 active+clean; 583 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s wr, 2 op/s 2024-09-16T00:31:05.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:05 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/162328499' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-09-16T00:31:06.180 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-09-16T00:31:06.180 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-09-16T00:31:06.442 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:06.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:06 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/162328499' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-16T00:31:06.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:06 smithi038 ceph-mon[30945]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-16T00:31:06.695 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:06 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/162328499' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-16T00:31:06.695 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:06 smithi073 ceph-mon[35254]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-16T00:31:06.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:06 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/162328499' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-16T00:31:06.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:06 smithi122 ceph-mon[35412]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-16T00:31:06.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.950+0000 7f8212311640 1 -- 172.21.15.38:0/3382842552 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f820c102790 msgr2=0x7f820c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:06.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.950+0000 7f8212311640 1 --2- 172.21.15.38:0/3382842552 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f820c102790 0x7f820c1067e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f81f4009900 tx=0x7f81f402a410 comp rx=0 tx=0).stop 2024-09-16T00:31:06.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.950+0000 7f8212311640 1 -- 172.21.15.38:0/3382842552 shutdown_connections 2024-09-16T00:31:06.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.951+0000 7f8212311640 1 --2- 172.21.15.38:0/3382842552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f820c106f10 0x7f820c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:06.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.951+0000 7f8212311640 1 --2- 172.21.15.38:0/3382842552 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f820c102790 0x7f820c1067e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:06.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.951+0000 7f8212311640 1 --2- 172.21.15.38:0/3382842552 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f820c101de0 0x7f820c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:06.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.951+0000 7f8212311640 1 -- 172.21.15.38:0/3382842552 >> 172.21.15.38:0/3382842552 conn(0x7f820c0fd8c0 msgr2=0x7f820c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:06.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.951+0000 7f8212311640 1 -- 172.21.15.38:0/3382842552 shutdown_connections 2024-09-16T00:31:06.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.951+0000 7f8212311640 1 -- 172.21.15.38:0/3382842552 wait complete. 2024-09-16T00:31:06.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.951+0000 7f8212311640 1 Processor -- start 2024-09-16T00:31:06.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.951+0000 7f8212311640 1 -- start start 2024-09-16T00:31:06.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.951+0000 7f8212311640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f820c101de0 0x7f820c1983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:06.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.951+0000 7f8212311640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f820c102790 0x7f820c198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:06.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.951+0000 7f8212311640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f820c106f10 0x7f820c19cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:06.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.951+0000 7f8212311640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f820c111f70 con 0x7f820c102790 2024-09-16T00:31:06.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.951+0000 7f8212311640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f820c111df0 con 0x7f820c106f10 2024-09-16T00:31:06.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.951+0000 7f8212311640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f820c1120f0 con 0x7f820c101de0 2024-09-16T00:31:06.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.951+0000 7f8203fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f820c102790 0x7f820c198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:06.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.952+0000 7f8203fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f820c102790 0x7f820c198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39574/0 (socket says 172.21.15.38:39574) 2024-09-16T00:31:06.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.952+0000 7f8203fff640 1 -- 172.21.15.38:0/3795107205 learned_addr learned my addr 172.21.15.38:0/3795107205 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:31:06.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.952+0000 7f820bfff640 1 --2- 172.21.15.38:0/3795107205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f820c101de0 0x7f820c1983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:06.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.952+0000 7f82108ae640 1 --2- 172.21.15.38:0/3795107205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f820c106f10 0x7f820c19cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:06.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.952+0000 7f8203fff640 1 -- 172.21.15.38:0/3795107205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f820c101de0 msgr2=0x7f820c1983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:06.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.952+0000 7f8203fff640 1 --2- 172.21.15.38:0/3795107205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f820c101de0 0x7f820c1983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:06.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.952+0000 7f8203fff640 1 -- 172.21.15.38:0/3795107205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f820c106f10 msgr2=0x7f820c19cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:06.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.952+0000 7f8203fff640 1 --2- 172.21.15.38:0/3795107205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f820c106f10 0x7f820c19cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:06.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.952+0000 7f8203fff640 1 -- 172.21.15.38:0/3795107205 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f820c19d380 con 0x7f820c102790 2024-09-16T00:31:06.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.952+0000 7f82108ae640 1 --2- 172.21.15.38:0/3795107205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f820c106f10 0x7f820c19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:31:06.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.952+0000 7f820bfff640 1 --2- 172.21.15.38:0/3795107205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f820c101de0 0x7f820c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:31:06.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.952+0000 7f8203fff640 1 --2- 172.21.15.38:0/3795107205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f820c102790 0x7f820c198910 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f81f4006010 tx=0x7f81f40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:06.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.952+0000 7f8209ffb640 1 -- 172.21.15.38:0/3795107205 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f403f030 con 0x7f820c102790 2024-09-16T00:31:06.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.952+0000 7f8212311640 1 -- 172.21.15.38:0/3795107205 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f820c19d610 con 0x7f820c102790 2024-09-16T00:31:06.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.952+0000 7f8209ffb640 1 -- 172.21.15.38:0/3795107205 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81f4003ce0 con 0x7f820c102790 2024-09-16T00:31:06.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.952+0000 7f8212311640 1 -- 172.21.15.38:0/3795107205 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f820c1a4f50 con 0x7f820c102790 2024-09-16T00:31:06.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.952+0000 7f8209ffb640 1 -- 172.21.15.38:0/3795107205 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81f4035070 con 0x7f820c102790 2024-09-16T00:31:06.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.953+0000 7f8212311640 1 -- 172.21.15.38:0/3795107205 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f820c1021c0 con 0x7f820c102790 2024-09-16T00:31:06.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.954+0000 7f8209ffb640 1 -- 172.21.15.38:0/3795107205 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 16) ==== 138355+0+0 (secure 0 0 0) 0x7f81f4042030 con 0x7f820c102790 2024-09-16T00:31:06.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.955+0000 7f8209ffb640 1 --2- 172.21.15.38:0/3795107205 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f81f80a32c0 0x7f81f80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:06.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.956+0000 7f820bfff640 1 --2- 172.21.15.38:0/3795107205 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f81f80a32c0 0x7f81f80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:06.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.956+0000 7f8209ffb640 1 -- 172.21.15.38:0/3795107205 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f81f40efe00 con 0x7f820c102790 2024-09-16T00:31:06.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.956+0000 7f820bfff640 1 --2- 172.21.15.38:0/3795107205 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f81f80a32c0 0x7f81f80a5780 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f81fc0058c0 tx=0x7f81fc005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:06.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:06.960+0000 7f8209ffb640 1 -- 172.21.15.38:0/3795107205 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f81f40bc880 con 0x7f820c102790 2024-09-16T00:31:07.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:07.161+0000 7f8212311640 1 -- 172.21.15.38:0/3795107205 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f820c062630 con 0x7f81f80a32c0 2024-09-16T00:31:07.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:07.188+0000 7f8209ffb640 1 -- 172.21.15.38:0/3795107205 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f820c062630 con 0x7f81f80a32c0 2024-09-16T00:31:07.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:07.190+0000 7f8212311640 1 -- 172.21.15.38:0/3795107205 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f81f80a32c0 msgr2=0x7f81f80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:07.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:07.190+0000 7f8212311640 1 --2- 172.21.15.38:0/3795107205 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f81f80a32c0 0x7f81f80a5780 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f81fc0058c0 tx=0x7f81fc005850 comp rx=0 tx=0).stop 2024-09-16T00:31:07.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:07.190+0000 7f8212311640 1 -- 172.21.15.38:0/3795107205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f820c102790 msgr2=0x7f820c198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:07.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:07.190+0000 7f8212311640 1 --2- 172.21.15.38:0/3795107205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f820c102790 0x7f820c198910 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f81f4006010 tx=0x7f81f40042f0 comp rx=0 tx=0).stop 2024-09-16T00:31:07.191 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:07.190+0000 7f8212311640 1 -- 172.21.15.38:0/3795107205 shutdown_connections 2024-09-16T00:31:07.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:07.190+0000 7f8212311640 1 --2- 172.21.15.38:0/3795107205 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f81f80a32c0 0x7f81f80a5780 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:07.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:07.190+0000 7f8212311640 1 --2- 172.21.15.38:0/3795107205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f820c106f10 0x7f820c19cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:07.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:07.190+0000 7f8212311640 1 --2- 172.21.15.38:0/3795107205 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f820c102790 0x7f820c198910 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:07.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:07.190+0000 7f8212311640 1 --2- 172.21.15.38:0/3795107205 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f820c101de0 0x7f820c1983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:07.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:07.190+0000 7f8212311640 1 -- 172.21.15.38:0/3795107205 >> 172.21.15.38:0/3795107205 conn(0x7f820c0fd8c0 msgr2=0x7f820c0fec70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:07.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:07.190+0000 7f8212311640 1 -- 172.21.15.38:0/3795107205 shutdown_connections 2024-09-16T00:31:07.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:07.190+0000 7f8212311640 1 -- 172.21.15.38:0/3795107205 wait complete. 2024-09-16T00:31:07.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:07 smithi038 ceph-mon[30945]: pgmap v310: 129 pgs: 129 active+clean; 583 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1.6 KiB/s wr, 5 op/s 2024-09-16T00:31:07.465 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:31:07 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:31:07.167+0000 7f1b6169f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:31:07.465 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:31:07 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:31:07.167+0000 7f1b6169f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:31:07.465 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:31:07 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:31:07.167+0000 7f1b6169f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:31:07.465 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:31:07 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:31:07.167+0000 7f1b6169f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:31:07.465 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:31:07 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:31:07.167+0000 7f1b6169f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:31:07.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:07 smithi122 ceph-mon[35412]: pgmap v310: 129 pgs: 129 active+clean; 583 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1.6 KiB/s wr, 5 op/s 2024-09-16T00:31:07.793 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:07 smithi073 ceph-mon[35254]: pgmap v310: 129 pgs: 129 active+clean; 583 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1.6 KiB/s wr, 5 op/s 2024-09-16T00:31:07.794 INFO:journalctl@ceph.mds.b.smithi073.stdout:Sep 16 00:31:07 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-b[65406]: 2024-09-16T00:31:07.442+0000 7fe7cd552640 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-09-16T00:31:07.918 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-16T00:31:08.177 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:08.558 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:08 smithi038 ceph-mon[30945]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:31:08.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.646+0000 7f02bd83a640 1 -- 172.21.15.38:0/2765087085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02b8106a80 msgr2=0x7f02b8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:08.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.646+0000 7f02bd83a640 1 --2- 172.21.15.38:0/2765087085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02b8106a80 0x7f02b8106e60 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f02a8009930 tx=0x7f02a802a420 comp rx=0 tx=0).stop 2024-09-16T00:31:08.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.646+0000 7f02bd83a640 1 -- 172.21.15.38:0/2765087085 shutdown_connections 2024-09-16T00:31:08.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.646+0000 7f02bd83a640 1 --2- 172.21.15.38:0/2765087085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02b8101420 0x7f02b810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.646+0000 7f02bd83a640 1 --2- 172.21.15.38:0/2765087085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02b8100a80 0x7f02b8100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.646+0000 7f02bd83a640 1 --2- 172.21.15.38:0/2765087085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02b8106a80 0x7f02b8106e60 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.646+0000 7f02bd83a640 1 -- 172.21.15.38:0/2765087085 >> 172.21.15.38:0/2765087085 conn(0x7f02b80fc7a0 msgr2=0x7f02b80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:08.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.646+0000 7f02bd83a640 1 -- 172.21.15.38:0/2765087085 shutdown_connections 2024-09-16T00:31:08.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.646+0000 7f02bd83a640 1 -- 172.21.15.38:0/2765087085 wait complete. 2024-09-16T00:31:08.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.647+0000 7f02bd83a640 1 Processor -- start 2024-09-16T00:31:08.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.647+0000 7f02bd83a640 1 -- start start 2024-09-16T00:31:08.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.647+0000 7f02bd83a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02b8100a80 0x7f02b8197300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:08.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.647+0000 7f02bd83a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02b8101420 0x7f02b8197840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:08.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.647+0000 7f02bd83a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02b8106a80 0x7f02b819bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:08.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.647+0000 7f02bd83a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f02b8110e80 con 0x7f02b8101420 2024-09-16T00:31:08.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.647+0000 7f02bd83a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f02b8110d00 con 0x7f02b8100a80 2024-09-16T00:31:08.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.647+0000 7f02bd83a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f02b8111000 con 0x7f02b8106a80 2024-09-16T00:31:08.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.647+0000 7f02affff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02b8101420 0x7f02b8197840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:08.668 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.647+0000 7f02affff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02b8101420 0x7f02b8197840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39612/0 (socket says 172.21.15.38:39612) 2024-09-16T00:31:08.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.648+0000 7f02affff640 1 -- 172.21.15.38:0/911972738 learned_addr learned my addr 172.21.15.38:0/911972738 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:31:08.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.648+0000 7f02b6ffd640 1 --2- 172.21.15.38:0/911972738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02b8100a80 0x7f02b8197300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:08.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.648+0000 7f02b77fe640 1 --2- 172.21.15.38:0/911972738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02b8106a80 0x7f02b819bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:08.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.648+0000 7f02affff640 1 -- 172.21.15.38:0/911972738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02b8106a80 msgr2=0x7f02b819bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:08.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.648+0000 7f02affff640 1 --2- 172.21.15.38:0/911972738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02b8106a80 0x7f02b819bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.648+0000 7f02affff640 1 -- 172.21.15.38:0/911972738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02b8100a80 msgr2=0x7f02b8197300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:08.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.648+0000 7f02affff640 1 --2- 172.21.15.38:0/911972738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02b8100a80 0x7f02b8197300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.648+0000 7f02affff640 1 -- 172.21.15.38:0/911972738 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b819c2b0 con 0x7f02b8101420 2024-09-16T00:31:08.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.648+0000 7f02affff640 1 --2- 172.21.15.38:0/911972738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02b8101420 0x7f02b8197840 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f02a000d770 tx=0x7f02a000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:08.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.648+0000 7f02bc838640 1 -- 172.21.15.38:0/911972738 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a0017070 con 0x7f02b8101420 2024-09-16T00:31:08.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.648+0000 7f02bc838640 1 -- 172.21.15.38:0/911972738 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f02a0005500 con 0x7f02b8101420 2024-09-16T00:31:08.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.648+0000 7f02bd83a640 1 -- 172.21.15.38:0/911972738 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02b819c5a0 con 0x7f02b8101420 2024-09-16T00:31:08.669 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.648+0000 7f02bc838640 1 -- 172.21.15.38:0/911972738 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02a0004e00 con 0x7f02b8101420 2024-09-16T00:31:08.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.648+0000 7f02bd83a640 1 -- 172.21.15.38:0/911972738 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02b81a3e80 con 0x7f02b8101420 2024-09-16T00:31:08.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.649+0000 7f02bd83a640 1 -- 172.21.15.38:0/911972738 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02b8100ee0 con 0x7f02b8101420 2024-09-16T00:31:08.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.650+0000 7f02bc838640 1 -- 172.21.15.38:0/911972738 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f02a0004fa0 con 0x7f02b8101420 2024-09-16T00:31:08.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.651+0000 7f02bc838640 1 --2- 172.21.15.38:0/911972738 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f02a40a32d0 0x7f02a40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:08.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.651+0000 7f02b6ffd640 1 --2- 172.21.15.38:0/911972738 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f02a40a32d0 0x7f02a40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:08.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.651+0000 7f02bc838640 1 -- 172.21.15.38:0/911972738 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f02a00cf7c0 con 0x7f02b8101420 2024-09-16T00:31:08.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.652+0000 7f02b6ffd640 1 --2- 172.21.15.38:0/911972738 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f02a40a32d0 0x7f02a40a5790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f02a8009900 tx=0x7f02a80058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:08.670 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.655+0000 7f02bc838640 1 -- 172.21.15.38:0/911972738 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f02a009c0e0 con 0x7f02b8101420 2024-09-16T00:31:08.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:08 smithi122 ceph-mon[35412]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:31:08.793 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:08 smithi073 ceph-mon[35254]: from='client.14787 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:31:08.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.856+0000 7f02bd83a640 1 -- 172.21.15.38:0/911972738 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f02b8062630 con 0x7f02a40a32d0 2024-09-16T00:31:08.882 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.881+0000 7f02bc838640 1 -- 172.21.15.38:0/911972738 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f02b8062630 con 0x7f02a40a32d0 2024-09-16T00:31:08.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.884+0000 7f02bd83a640 1 -- 172.21.15.38:0/911972738 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f02a40a32d0 msgr2=0x7f02a40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:08.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.884+0000 7f02bd83a640 1 --2- 172.21.15.38:0/911972738 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f02a40a32d0 0x7f02a40a5790 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f02a8009900 tx=0x7f02a80058d0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.884+0000 7f02bd83a640 1 -- 172.21.15.38:0/911972738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02b8101420 msgr2=0x7f02b8197840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:08.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.884+0000 7f02bd83a640 1 --2- 172.21.15.38:0/911972738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02b8101420 0x7f02b8197840 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f02a000d770 tx=0x7f02a000dc30 comp rx=0 tx=0).stop 2024-09-16T00:31:08.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.884+0000 7f02bd83a640 1 -- 172.21.15.38:0/911972738 shutdown_connections 2024-09-16T00:31:08.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.884+0000 7f02bd83a640 1 --2- 172.21.15.38:0/911972738 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f02a40a32d0 0x7f02a40a5790 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.884+0000 7f02bd83a640 1 --2- 172.21.15.38:0/911972738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f02b8106a80 0x7f02b819bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.884+0000 7f02bd83a640 1 --2- 172.21.15.38:0/911972738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f02b8101420 0x7f02b8197840 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.884+0000 7f02bd83a640 1 --2- 172.21.15.38:0/911972738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02b8100a80 0x7f02b8197300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:08.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.884+0000 7f02bd83a640 1 -- 172.21.15.38:0/911972738 >> 172.21.15.38:0/911972738 conn(0x7f02b80fc7a0 msgr2=0x7f02b80fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:08.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.884+0000 7f02bd83a640 1 -- 172.21.15.38:0/911972738 shutdown_connections 2024-09-16T00:31:08.885 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:08.884+0000 7f02bd83a640 1 -- 172.21.15.38:0/911972738 wait complete. 2024-09-16T00:31:09.576 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:09 smithi038 ceph-mon[30945]: pgmap v311: 129 pgs: 129 active+clean; 584 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-09-16T00:31:09.576 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:09 smithi038 ceph-mon[30945]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-16T00:31:09.577 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-16T00:31:09.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:09 smithi122 ceph-mon[35412]: pgmap v311: 129 pgs: 129 active+clean; 584 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-09-16T00:31:09.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:09 smithi122 ceph-mon[35412]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-16T00:31:09.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:09 smithi073 ceph-mon[35254]: pgmap v311: 129 pgs: 129 active+clean; 584 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2.1 KiB/s wr, 7 op/s 2024-09-16T00:31:09.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:09 smithi073 ceph-mon[35254]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-09-16T00:31:09.835 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:10.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.322+0000 7f84e1d28640 1 -- 172.21.15.38:0/2904338490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84dc1009d0 msgr2=0x7f84dc100db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:10.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.322+0000 7f84e1d28640 1 --2- 172.21.15.38:0/2904338490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84dc1009d0 0x7f84dc100db0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f84cc009900 tx=0x7f84cc02a410 comp rx=0 tx=0).stop 2024-09-16T00:31:10.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.322+0000 7f84e1d28640 1 -- 172.21.15.38:0/2904338490 shutdown_connections 2024-09-16T00:31:10.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.322+0000 7f84e1d28640 1 --2- 172.21.15.38:0/2904338490 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84dc10e4f0 0x7f84dc110900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:10.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.322+0000 7f84e1d28640 1 --2- 172.21.15.38:0/2904338490 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84dc101380 0x7f84dc10dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:10.360 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.322+0000 7f84e1d28640 1 --2- 172.21.15.38:0/2904338490 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84dc1009d0 0x7f84dc100db0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:10.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.322+0000 7f84e1d28640 1 -- 172.21.15.38:0/2904338490 >> 172.21.15.38:0/2904338490 conn(0x7f84dc0fc7d0 msgr2=0x7f84dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:10.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.323+0000 7f84e1d28640 1 -- 172.21.15.38:0/2904338490 shutdown_connections 2024-09-16T00:31:10.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.323+0000 7f84e1d28640 1 -- 172.21.15.38:0/2904338490 wait complete. 2024-09-16T00:31:10.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.323+0000 7f84e1d28640 1 Processor -- start 2024-09-16T00:31:10.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.323+0000 7f84e1d28640 1 -- start start 2024-09-16T00:31:10.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.323+0000 7f84e1d28640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84dc1009d0 0x7f84dc1a0450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:10.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.323+0000 7f84e1d28640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84dc101380 0x7f84dc1a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:10.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.323+0000 7f84e1d28640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84dc10e4f0 0x7f84dc19a610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:10.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.323+0000 7f84e1d28640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc1142c0 con 0x7f84dc101380 2024-09-16T00:31:10.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.323+0000 7f84e1d28640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc114140 con 0x7f84dc1009d0 2024-09-16T00:31:10.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.323+0000 7f84e1d28640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f84dc114440 con 0x7f84dc10e4f0 2024-09-16T00:31:10.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.324+0000 7f84daffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84dc101380 0x7f84dc1a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:10.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.324+0000 7f84daffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84dc101380 0x7f84dc1a0990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39632/0 (socket says 172.21.15.38:39632) 2024-09-16T00:31:10.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.324+0000 7f84daffd640 1 -- 172.21.15.38:0/2647998358 learned_addr learned my addr 172.21.15.38:0/2647998358 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:31:10.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.324+0000 7f84dbfff640 1 --2- 172.21.15.38:0/2647998358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84dc10e4f0 0x7f84dc19a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:10.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.324+0000 7f84db7fe640 1 --2- 172.21.15.38:0/2647998358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84dc1009d0 0x7f84dc1a0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:10.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.324+0000 7f84daffd640 1 -- 172.21.15.38:0/2647998358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84dc10e4f0 msgr2=0x7f84dc19a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:10.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.324+0000 7f84daffd640 1 --2- 172.21.15.38:0/2647998358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84dc10e4f0 0x7f84dc19a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:10.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.324+0000 7f84daffd640 1 -- 172.21.15.38:0/2647998358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84dc1009d0 msgr2=0x7f84dc1a0450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:10.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.324+0000 7f84daffd640 1 --2- 172.21.15.38:0/2647998358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84dc1009d0 0x7f84dc1a0450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:10.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.324+0000 7f84daffd640 1 -- 172.21.15.38:0/2647998358 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84dc19ad40 con 0x7f84dc101380 2024-09-16T00:31:10.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.324+0000 7f84dbfff640 1 --2- 172.21.15.38:0/2647998358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84dc10e4f0 0x7f84dc19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:31:10.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.324+0000 7f84db7fe640 1 --2- 172.21.15.38:0/2647998358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84dc1009d0 0x7f84dc1a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:31:10.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.324+0000 7f84daffd640 1 --2- 172.21.15.38:0/2647998358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84dc101380 0x7f84dc1a0990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f84d000b4a0 tx=0x7f84d000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:10.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.324+0000 7f84d8ff9640 1 -- 172.21.15.38:0/2647998358 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d0014070 con 0x7f84dc101380 2024-09-16T00:31:10.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.324+0000 7f84e1d28640 1 -- 172.21.15.38:0/2647998358 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84dc19b030 con 0x7f84dc101380 2024-09-16T00:31:10.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.324+0000 7f84d8ff9640 1 -- 172.21.15.38:0/2647998358 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f84d0010af0 con 0x7f84dc101380 2024-09-16T00:31:10.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.324+0000 7f84d8ff9640 1 -- 172.21.15.38:0/2647998358 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d0017d10 con 0x7f84dc101380 2024-09-16T00:31:10.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.324+0000 7f84e1d28640 1 -- 172.21.15.38:0/2647998358 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84dc1a7320 con 0x7f84dc101380 2024-09-16T00:31:10.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.325+0000 7f84e1d28640 1 -- 172.21.15.38:0/2647998358 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84dc06a7d0 con 0x7f84dc101380 2024-09-16T00:31:10.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.327+0000 7f84d8ff9640 1 -- 172.21.15.38:0/2647998358 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f84d0017070 con 0x7f84dc101380 2024-09-16T00:31:10.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.327+0000 7f84d8ff9640 1 --2- 172.21.15.38:0/2647998358 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f84c40a32d0 0x7f84c40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:10.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.327+0000 7f84d8ff9640 1 -- 172.21.15.38:0/2647998358 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f84d00cf050 con 0x7f84dc101380 2024-09-16T00:31:10.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.327+0000 7f84db7fe640 1 --2- 172.21.15.38:0/2647998358 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f84c40a32d0 0x7f84c40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:10.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.328+0000 7f84db7fe640 1 --2- 172.21.15.38:0/2647998358 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f84c40a32d0 0x7f84c40a5790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f84cc005ab0 tx=0x7f84cc005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:10.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.332+0000 7f84d8ff9640 1 -- 172.21.15.38:0/2647998358 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f84d009bae0 con 0x7f84dc101380 2024-09-16T00:31:10.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.533+0000 7f84e1d28640 1 -- 172.21.15.38:0/2647998358 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f84dc062630 con 0x7f84c40a32d0 2024-09-16T00:31:10.534 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:10 smithi038 ceph-mon[30945]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:31:10.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.538+0000 7f84d8ff9640 1 -- 172.21.15.38:0/2647998358 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f84dc062630 con 0x7f84c40a32d0 2024-09-16T00:31:10.539 INFO:teuthology.orchestra.run.smithi038.stdout:/volumes/qa/sv_0/659583b1-4baa-42f0-b480-3c8f7c19cef6 2024-09-16T00:31:10.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.540+0000 7f84e1d28640 1 -- 172.21.15.38:0/2647998358 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f84c40a32d0 msgr2=0x7f84c40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:10.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.540+0000 7f84e1d28640 1 --2- 172.21.15.38:0/2647998358 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f84c40a32d0 0x7f84c40a5790 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f84cc005ab0 tx=0x7f84cc005a20 comp rx=0 tx=0).stop 2024-09-16T00:31:10.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.540+0000 7f84e1d28640 1 -- 172.21.15.38:0/2647998358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84dc101380 msgr2=0x7f84dc1a0990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:10.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.540+0000 7f84e1d28640 1 --2- 172.21.15.38:0/2647998358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84dc101380 0x7f84dc1a0990 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f84d000b4a0 tx=0x7f84d000b960 comp rx=0 tx=0).stop 2024-09-16T00:31:10.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.540+0000 7f84e1d28640 1 -- 172.21.15.38:0/2647998358 shutdown_connections 2024-09-16T00:31:10.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.540+0000 7f84e1d28640 1 --2- 172.21.15.38:0/2647998358 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f84c40a32d0 0x7f84c40a5790 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:10.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.540+0000 7f84e1d28640 1 --2- 172.21.15.38:0/2647998358 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f84dc10e4f0 0x7f84dc19a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:10.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.540+0000 7f84e1d28640 1 --2- 172.21.15.38:0/2647998358 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f84dc101380 0x7f84dc1a0990 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:10.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.540+0000 7f84e1d28640 1 --2- 172.21.15.38:0/2647998358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84dc1009d0 0x7f84dc1a0450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:10.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.540+0000 7f84e1d28640 1 -- 172.21.15.38:0/2647998358 >> 172.21.15.38:0/2647998358 conn(0x7f84dc0fc7d0 msgr2=0x7f84dc10f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:10.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.541+0000 7f84e1d28640 1 -- 172.21.15.38:0/2647998358 shutdown_connections 2024-09-16T00:31:10.542 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:10.541+0000 7f84e1d28640 1 -- 172.21.15.38:0/2647998358 wait complete. 2024-09-16T00:31:10.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:10 smithi122 ceph-mon[35412]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:31:10.793 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:10 smithi073 ceph-mon[35254]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:31:11.255 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-16T00:31:11.515 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:11.636 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:11 smithi038 ceph-mon[30945]: pgmap v312: 129 pgs: 129 active+clean; 591 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-09-16T00:31:11.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:11 smithi122 ceph-mon[35412]: pgmap v312: 129 pgs: 129 active+clean; 591 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-09-16T00:31:11.793 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:11 smithi073 ceph-mon[35254]: pgmap v312: 129 pgs: 129 active+clean; 591 KiB data, 370 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.0 KiB/s wr, 11 op/s 2024-09-16T00:31:12.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.023+0000 7f95a3a4d640 1 -- 172.21.15.38:0/1044445210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f959c106a80 msgr2=0x7f959c106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:12.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.023+0000 7f95a3a4d640 1 --2- 172.21.15.38:0/1044445210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f959c106a80 0x7f959c106e60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9598009930 tx=0x7f959802a420 comp rx=0 tx=0).stop 2024-09-16T00:31:12.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.023+0000 7f95a3a4d640 1 -- 172.21.15.38:0/1044445210 shutdown_connections 2024-09-16T00:31:12.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.023+0000 7f95a3a4d640 1 --2- 172.21.15.38:0/1044445210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f959c101420 0x7f959c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.023+0000 7f95a3a4d640 1 --2- 172.21.15.38:0/1044445210 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f959c100a80 0x7f959c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.023+0000 7f95a3a4d640 1 --2- 172.21.15.38:0/1044445210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f959c106a80 0x7f959c106e60 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.023+0000 7f95a3a4d640 1 -- 172.21.15.38:0/1044445210 >> 172.21.15.38:0/1044445210 conn(0x7f959c0fc7a0 msgr2=0x7f959c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:12.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.023+0000 7f95a3a4d640 1 -- 172.21.15.38:0/1044445210 shutdown_connections 2024-09-16T00:31:12.036 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.023+0000 7f95a3a4d640 1 -- 172.21.15.38:0/1044445210 wait complete. 2024-09-16T00:31:12.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.024+0000 7f95a3a4d640 1 Processor -- start 2024-09-16T00:31:12.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.024+0000 7f95a3a4d640 1 -- start start 2024-09-16T00:31:12.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.024+0000 7f95a3a4d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f959c100a80 0x7f959c199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:12.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.024+0000 7f95a3a4d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f959c101420 0x7f959c1997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:12.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.024+0000 7f95a3a4d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f959c106a80 0x7f959c1a4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:12.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.024+0000 7f95a3a4d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f959c110e40 con 0x7f959c100a80 2024-09-16T00:31:12.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.024+0000 7f95a3a4d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f959c110cc0 con 0x7f959c106a80 2024-09-16T00:31:12.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.024+0000 7f95a3a4d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f959c110fc0 con 0x7f959c101420 2024-09-16T00:31:12.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.024+0000 7f95a17e9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f959c100a80 0x7f959c199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:12.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.024+0000 7f95a17e9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f959c100a80 0x7f959c199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39650/0 (socket says 172.21.15.38:39650) 2024-09-16T00:31:12.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.024+0000 7f95a17e9640 1 -- 172.21.15.38:0/322752146 learned_addr learned my addr 172.21.15.38:0/322752146 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:31:12.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.025+0000 7f95a0fe8640 1 --2- 172.21.15.38:0/322752146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f959c101420 0x7f959c1997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:12.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.025+0000 7f95a1fea640 1 --2- 172.21.15.38:0/322752146 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f959c106a80 0x7f959c1a4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:12.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.025+0000 7f95a17e9640 1 -- 172.21.15.38:0/322752146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f959c101420 msgr2=0x7f959c1997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:12.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.025+0000 7f95a17e9640 1 --2- 172.21.15.38:0/322752146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f959c101420 0x7f959c1997c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.025+0000 7f95a17e9640 1 -- 172.21.15.38:0/322752146 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f959c106a80 msgr2=0x7f959c1a4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:12.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.025+0000 7f95a17e9640 1 --2- 172.21.15.38:0/322752146 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f959c106a80 0x7f959c1a4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.025+0000 7f95a17e9640 1 -- 172.21.15.38:0/322752146 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f959c1a4840 con 0x7f959c100a80 2024-09-16T00:31:12.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.025+0000 7f95a1fea640 1 --2- 172.21.15.38:0/322752146 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f959c106a80 0x7f959c1a4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:31:12.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.025+0000 7f95a17e9640 1 --2- 172.21.15.38:0/322752146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f959c100a80 0x7f959c199280 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9598009a50 tx=0x7f9598004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:12.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.025+0000 7f95927fc640 1 -- 172.21.15.38:0/322752146 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f959803f030 con 0x7f959c100a80 2024-09-16T00:31:12.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.025+0000 7f95927fc640 1 -- 172.21.15.38:0/322752146 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9598003ce0 con 0x7f959c100a80 2024-09-16T00:31:12.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.025+0000 7f95a3a4d640 1 -- 172.21.15.38:0/322752146 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f959c1a4a10 con 0x7f959c100a80 2024-09-16T00:31:12.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.025+0000 7f95927fc640 1 -- 172.21.15.38:0/322752146 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9598035070 con 0x7f959c100a80 2024-09-16T00:31:12.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.025+0000 7f95a3a4d640 1 -- 172.21.15.38:0/322752146 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f959c1a4ec0 con 0x7f959c100a80 2024-09-16T00:31:12.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.026+0000 7f95a3a4d640 1 -- 172.21.15.38:0/322752146 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f959c100ee0 con 0x7f959c100a80 2024-09-16T00:31:12.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.027+0000 7f95927fc640 1 -- 172.21.15.38:0/322752146 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f9598042030 con 0x7f959c100a80 2024-09-16T00:31:12.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.028+0000 7f95927fc640 1 --2- 172.21.15.38:0/322752146 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f95880a3350 0x7f95880a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:12.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.028+0000 7f95a0fe8640 1 --2- 172.21.15.38:0/322752146 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f95880a3350 0x7f95880a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:12.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.028+0000 7f95927fc640 1 -- 172.21.15.38:0/322752146 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f95980eff80 con 0x7f959c100a80 2024-09-16T00:31:12.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.029+0000 7f95a0fe8640 1 --2- 172.21.15.38:0/322752146 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f95880a3350 0x7f95880a5810 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f958c005e10 tx=0x7f958c005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:12.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.032+0000 7f95927fc640 1 -- 172.21.15.38:0/322752146 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f95980bc900 con 0x7f959c100a80 2024-09-16T00:31:12.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.233+0000 7f95a3a4d640 1 -- 172.21.15.38:0/322752146 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f959c109060 con 0x7f95880a3350 2024-09-16T00:31:12.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.257+0000 7f95927fc640 1 -- 172.21.15.38:0/322752146 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f958c00eac0 con 0x7f95880a3350 2024-09-16T00:31:12.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.259+0000 7f95a3a4d640 1 -- 172.21.15.38:0/322752146 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f95880a3350 msgr2=0x7f95880a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:12.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.259+0000 7f95a3a4d640 1 --2- 172.21.15.38:0/322752146 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f95880a3350 0x7f95880a5810 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f958c005e10 tx=0x7f958c005da0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.259+0000 7f95a3a4d640 1 -- 172.21.15.38:0/322752146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f959c100a80 msgr2=0x7f959c199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:12.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.259+0000 7f95a3a4d640 1 --2- 172.21.15.38:0/322752146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f959c100a80 0x7f959c199280 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f9598009a50 tx=0x7f9598004270 comp rx=0 tx=0).stop 2024-09-16T00:31:12.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.259+0000 7f95a3a4d640 1 -- 172.21.15.38:0/322752146 shutdown_connections 2024-09-16T00:31:12.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.259+0000 7f95a3a4d640 1 --2- 172.21.15.38:0/322752146 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f95880a3350 0x7f95880a5810 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.259+0000 7f95a3a4d640 1 --2- 172.21.15.38:0/322752146 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f959c106a80 0x7f959c1a4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.259+0000 7f95a3a4d640 1 --2- 172.21.15.38:0/322752146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f959c101420 0x7f959c1997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.259+0000 7f95a3a4d640 1 --2- 172.21.15.38:0/322752146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f959c100a80 0x7f959c199280 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:12.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.259+0000 7f95a3a4d640 1 -- 172.21.15.38:0/322752146 >> 172.21.15.38:0/322752146 conn(0x7f959c0fc7a0 msgr2=0x7f959c0fcf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:12.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.260+0000 7f95a3a4d640 1 -- 172.21.15.38:0/322752146 shutdown_connections 2024-09-16T00:31:12.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:12.260+0000 7f95a3a4d640 1 -- 172.21.15.38:0/322752146 wait complete. 2024-09-16T00:31:12.610 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:12 smithi038 ceph-mon[30945]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:31:12.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:12 smithi122 ceph-mon[35412]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:31:12.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:12 smithi073 ceph-mon[35254]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:31:12.922 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-16T00:31:13.181 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:13.581 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:13 smithi038 ceph-mon[30945]: pgmap v313: 129 pgs: 129 active+clean; 591 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-09-16T00:31:13.581 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:13 smithi038 ceph-mon[30945]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:31:13.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.668+0000 7fe16dfba640 1 -- 172.21.15.38:0/1675773051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe168106a80 msgr2=0x7fe168106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:13.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.668+0000 7fe16dfba640 1 --2- 172.21.15.38:0/1675773051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe168106a80 0x7fe168106e60 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe158009930 tx=0x7fe15802a420 comp rx=0 tx=0).stop 2024-09-16T00:31:13.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.669+0000 7fe16dfba640 1 -- 172.21.15.38:0/1675773051 shutdown_connections 2024-09-16T00:31:13.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.669+0000 7fe16dfba640 1 --2- 172.21.15.38:0/1675773051 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe168101420 0x7fe16810d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:13.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.669+0000 7fe16dfba640 1 --2- 172.21.15.38:0/1675773051 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe168100a80 0x7fe168100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:13.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.669+0000 7fe16dfba640 1 --2- 172.21.15.38:0/1675773051 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe168106a80 0x7fe168106e60 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:13.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.669+0000 7fe16dfba640 1 -- 172.21.15.38:0/1675773051 >> 172.21.15.38:0/1675773051 conn(0x7fe1680fc7a0 msgr2=0x7fe1680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:13.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.669+0000 7fe16dfba640 1 -- 172.21.15.38:0/1675773051 shutdown_connections 2024-09-16T00:31:13.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.669+0000 7fe16dfba640 1 -- 172.21.15.38:0/1675773051 wait complete. 2024-09-16T00:31:13.682 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.669+0000 7fe16dfba640 1 Processor -- start 2024-09-16T00:31:13.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.669+0000 7fe16dfba640 1 -- start start 2024-09-16T00:31:13.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.670+0000 7fe16dfba640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe168100a80 0x7fe1680fce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:13.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.670+0000 7fe16dfba640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe168101420 0x7fe1680fd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:13.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.670+0000 7fe16dfba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe168106a80 0x7fe1680fd910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:13.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.670+0000 7fe16dfba640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe168110e40 con 0x7fe168100a80 2024-09-16T00:31:13.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.670+0000 7fe16dfba640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe168110cc0 con 0x7fe168101420 2024-09-16T00:31:13.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.670+0000 7fe16dfba640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe168110fc0 con 0x7fe168106a80 2024-09-16T00:31:13.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.670+0000 7fe1677fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe168100a80 0x7fe1680fce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:13.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.670+0000 7fe1677fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe168100a80 0x7fe1680fce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39664/0 (socket says 172.21.15.38:39664) 2024-09-16T00:31:13.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.670+0000 7fe1677fe640 1 -- 172.21.15.38:0/3420891886 learned_addr learned my addr 172.21.15.38:0/3420891886 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:31:13.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.670+0000 7fe166ffd640 1 --2- 172.21.15.38:0/3420891886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe168101420 0x7fe1680fd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:13.683 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.670+0000 7fe167fff640 1 --2- 172.21.15.38:0/3420891886 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe168106a80 0x7fe1680fd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:13.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.670+0000 7fe1677fe640 1 -- 172.21.15.38:0/3420891886 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe168106a80 msgr2=0x7fe1680fd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:13.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.670+0000 7fe1677fe640 1 --2- 172.21.15.38:0/3420891886 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe168106a80 0x7fe1680fd910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:13.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.670+0000 7fe1677fe640 1 -- 172.21.15.38:0/3420891886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe168101420 msgr2=0x7fe1680fd3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:13.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.670+0000 7fe1677fe640 1 --2- 172.21.15.38:0/3420891886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe168101420 0x7fe1680fd3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:13.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.670+0000 7fe1677fe640 1 -- 172.21.15.38:0/3420891886 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1681a6910 con 0x7fe168100a80 2024-09-16T00:31:13.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.670+0000 7fe1677fe640 1 --2- 172.21.15.38:0/3420891886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe168100a80 0x7fe1680fce90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe158009230 tx=0x7fe158004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:13.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.670+0000 7fe164ff9640 1 -- 172.21.15.38:0/3420891886 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15803f030 con 0x7fe168100a80 2024-09-16T00:31:13.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.670+0000 7fe164ff9640 1 -- 172.21.15.38:0/3420891886 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe158003ad0 con 0x7fe168100a80 2024-09-16T00:31:13.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.670+0000 7fe16dfba640 1 -- 172.21.15.38:0/3420891886 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1681a6ba0 con 0x7fe168100a80 2024-09-16T00:31:13.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.670+0000 7fe164ff9640 1 -- 172.21.15.38:0/3420891886 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe158035070 con 0x7fe168100a80 2024-09-16T00:31:13.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.671+0000 7fe16dfba640 1 -- 172.21.15.38:0/3420891886 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1681a7080 con 0x7fe168100a80 2024-09-16T00:31:13.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.672+0000 7fe16dfba640 1 -- 172.21.15.38:0/3420891886 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe168100ee0 con 0x7fe168100a80 2024-09-16T00:31:13.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.673+0000 7fe164ff9640 1 -- 172.21.15.38:0/3420891886 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fe158042030 con 0x7fe168100a80 2024-09-16T00:31:13.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.674+0000 7fe164ff9640 1 --2- 172.21.15.38:0/3420891886 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fe1500a32d0 0x7fe1500a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:13.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.674+0000 7fe166ffd640 1 --2- 172.21.15.38:0/3420891886 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fe1500a32d0 0x7fe1500a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:13.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.674+0000 7fe164ff9640 1 -- 172.21.15.38:0/3420891886 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fe1580f0130 con 0x7fe168100a80 2024-09-16T00:31:13.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.674+0000 7fe166ffd640 1 --2- 172.21.15.38:0/3420891886 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fe1500a32d0 0x7fe1500a5790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe1680fe920 tx=0x7fe15c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:13.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.678+0000 7fe164ff9640 1 -- 172.21.15.38:0/3420891886 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fe1580bc460 con 0x7fe168100a80 2024-09-16T00:31:13.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:13 smithi122 ceph-mon[35412]: pgmap v313: 129 pgs: 129 active+clean; 591 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-09-16T00:31:13.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:13 smithi122 ceph-mon[35412]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:31:13.793 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:13 smithi073 ceph-mon[35254]: pgmap v313: 129 pgs: 129 active+clean; 591 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.0 KiB/s wr, 12 op/s 2024-09-16T00:31:13.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:13 smithi073 ceph-mon[35254]: from='client.14811 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:31:13.880 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.879+0000 7fe16dfba640 1 -- 172.21.15.38:0/3420891886 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7fe168062630 con 0x7fe1500a32d0 2024-09-16T00:31:13.884 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.884+0000 7fe164ff9640 1 -- 172.21.15.38:0/3420891886 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fe168062630 con 0x7fe1500a32d0 2024-09-16T00:31:13.885 INFO:teuthology.orchestra.run.smithi038.stdout:/volumes/qa/sv_1/aaa8e7e9-698e-40fc-9502-7256606ca4b9 2024-09-16T00:31:13.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.886+0000 7fe16dfba640 1 -- 172.21.15.38:0/3420891886 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fe1500a32d0 msgr2=0x7fe1500a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:13.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.886+0000 7fe16dfba640 1 --2- 172.21.15.38:0/3420891886 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fe1500a32d0 0x7fe1500a5790 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fe1680fe920 tx=0x7fe15c00b390 comp rx=0 tx=0).stop 2024-09-16T00:31:13.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.886+0000 7fe16dfba640 1 -- 172.21.15.38:0/3420891886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe168100a80 msgr2=0x7fe1680fce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:13.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.886+0000 7fe16dfba640 1 --2- 172.21.15.38:0/3420891886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe168100a80 0x7fe1680fce90 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe158009230 tx=0x7fe158004270 comp rx=0 tx=0).stop 2024-09-16T00:31:13.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.886+0000 7fe16dfba640 1 -- 172.21.15.38:0/3420891886 shutdown_connections 2024-09-16T00:31:13.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.886+0000 7fe16dfba640 1 --2- 172.21.15.38:0/3420891886 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fe1500a32d0 0x7fe1500a5790 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:13.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.886+0000 7fe16dfba640 1 --2- 172.21.15.38:0/3420891886 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe168106a80 0x7fe1680fd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:13.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.886+0000 7fe16dfba640 1 --2- 172.21.15.38:0/3420891886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe168101420 0x7fe1680fd3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:13.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.886+0000 7fe16dfba640 1 --2- 172.21.15.38:0/3420891886 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe168100a80 0x7fe1680fce90 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:13.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.886+0000 7fe16dfba640 1 -- 172.21.15.38:0/3420891886 >> 172.21.15.38:0/3420891886 conn(0x7fe1680fc7a0 msgr2=0x7fe16806aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:13.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.887+0000 7fe16dfba640 1 -- 172.21.15.38:0/3420891886 shutdown_connections 2024-09-16T00:31:13.888 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:13.887+0000 7fe16dfba640 1 -- 172.21.15.38:0/3420891886 wait complete. 2024-09-16T00:31:14.549 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:31:14.804 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:15.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.299+0000 7f357546e640 1 -- 172.21.15.38:0/2807624480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f357010c170 msgr2=0x7f357010e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:15.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.299+0000 7f357546e640 1 --2- 172.21.15.38:0/2807624480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f357010c170 0x7f357010e610 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3568009ca0 tx=0x7f356802a950 comp rx=0 tx=0).stop 2024-09-16T00:31:15.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.299+0000 7f357546e640 1 -- 172.21.15.38:0/2807624480 shutdown_connections 2024-09-16T00:31:15.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.299+0000 7f357546e640 1 --2- 172.21.15.38:0/2807624480 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f357010c170 0x7f357010e610 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.299+0000 7f357546e640 1 --2- 172.21.15.38:0/2807624480 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3570103660 0x7f357010bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.299+0000 7f357546e640 1 --2- 172.21.15.38:0/2807624480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3570102d40 0x7f3570103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.299+0000 7f357546e640 1 -- 172.21.15.38:0/2807624480 >> 172.21.15.38:0/2807624480 conn(0x7f35700fc7a0 msgr2=0x7f35700febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:15.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.299+0000 7f357546e640 1 -- 172.21.15.38:0/2807624480 shutdown_connections 2024-09-16T00:31:15.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.299+0000 7f357546e640 1 -- 172.21.15.38:0/2807624480 wait complete. 2024-09-16T00:31:15.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.300+0000 7f357546e640 1 Processor -- start 2024-09-16T00:31:15.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.300+0000 7f357546e640 1 -- start start 2024-09-16T00:31:15.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.300+0000 7f357546e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3570102d40 0x7f35701983d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:15.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.300+0000 7f357546e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3570103660 0x7f3570198910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:15.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.300+0000 7f357546e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f357010c170 0x7f357019cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:15.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.300+0000 7f357546e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3570111fa0 con 0x7f3570103660 2024-09-16T00:31:15.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.300+0000 7f357546e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3570111e20 con 0x7f357010c170 2024-09-16T00:31:15.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.300+0000 7f357546e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3570112120 con 0x7f3570102d40 2024-09-16T00:31:15.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.300+0000 7f35667fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3570103660 0x7f3570198910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:15.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.300+0000 7f35667fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3570103660 0x7f3570198910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:39676/0 (socket says 172.21.15.38:39676) 2024-09-16T00:31:15.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.301+0000 7f35667fc640 1 -- 172.21.15.38:0/1173055767 learned_addr learned my addr 172.21.15.38:0/1173055767 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:31:15.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.301+0000 7f356effd640 1 --2- 172.21.15.38:0/1173055767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3570102d40 0x7f35701983d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:15.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.301+0000 7f356f7fe640 1 --2- 172.21.15.38:0/1173055767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f357010c170 0x7f357019cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:15.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.301+0000 7f35667fc640 1 -- 172.21.15.38:0/1173055767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3570102d40 msgr2=0x7f35701983d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:15.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.301+0000 7f35667fc640 1 --2- 172.21.15.38:0/1173055767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3570102d40 0x7f35701983d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.301+0000 7f35667fc640 1 -- 172.21.15.38:0/1173055767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f357010c170 msgr2=0x7f357019cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:15.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.301+0000 7f35667fc640 1 --2- 172.21.15.38:0/1173055767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f357010c170 0x7f357019cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.301+0000 7f35667fc640 1 -- 172.21.15.38:0/1173055767 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f357019d380 con 0x7f3570103660 2024-09-16T00:31:15.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.301+0000 7f356f7fe640 1 --2- 172.21.15.38:0/1173055767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f357010c170 0x7f357019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:31:15.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.301+0000 7f35667fc640 1 --2- 172.21.15.38:0/1173055767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3570103660 0x7f3570198910 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f355800c970 tx=0x7f355800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:15.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.301+0000 7f356cff9640 1 -- 172.21.15.38:0/1173055767 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3558007bb0 con 0x7f3570103660 2024-09-16T00:31:15.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.301+0000 7f356cff9640 1 -- 172.21.15.38:0/1173055767 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3558007d50 con 0x7f3570103660 2024-09-16T00:31:15.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.301+0000 7f357546e640 1 -- 172.21.15.38:0/1173055767 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f357019d670 con 0x7f3570103660 2024-09-16T00:31:15.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.301+0000 7f356cff9640 1 -- 172.21.15.38:0/1173055767 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f355800f450 con 0x7f3570103660 2024-09-16T00:31:15.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.301+0000 7f357546e640 1 -- 172.21.15.38:0/1173055767 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35701a4f50 con 0x7f3570103660 2024-09-16T00:31:15.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.302+0000 7f357546e640 1 -- 172.21.15.38:0/1173055767 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3570103120 con 0x7f3570103660 2024-09-16T00:31:15.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.303+0000 7f356cff9640 1 -- 172.21.15.38:0/1173055767 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f3558004f40 con 0x7f3570103660 2024-09-16T00:31:15.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.304+0000 7f356cff9640 1 --2- 172.21.15.38:0/1173055767 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f355c0a3350 0x7f355c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:15.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.304+0000 7f356effd640 1 --2- 172.21.15.38:0/1173055767 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f355c0a3350 0x7f355c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:15.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.304+0000 7f356cff9640 1 -- 172.21.15.38:0/1173055767 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f35580cf970 con 0x7f3570103660 2024-09-16T00:31:15.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.305+0000 7f356effd640 1 --2- 172.21.15.38:0/1173055767 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f355c0a3350 0x7f355c0a5810 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3560005fd0 tx=0x7f356000b590 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:15.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.308+0000 7f356cff9640 1 -- 172.21.15.38:0/1173055767 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3558014030 con 0x7f3570103660 2024-09-16T00:31:15.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.520+0000 7f357546e640 1 -- 172.21.15.38:0/1173055767 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f35701a52e0 con 0x7f3570103660 2024-09-16T00:31:15.521 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:15 smithi038 ceph-mon[30945]: pgmap v314: 129 pgs: 129 active+clean; 591 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-09-16T00:31:15.521 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:15 smithi038 ceph-mon[30945]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:31:15.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.520+0000 7f356cff9640 1 -- 172.21.15.38:0/1173055767 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11909 (secure 0 0 0) 0x7f355809c2f0 con 0x7f3570103660 2024-09-16T00:31:15.522 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:31:15.522 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":25,"btime":"2024-09-16T00:31:05:330385+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/367974527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":367974527},{"type":"v1","addr":"172.21.15.38:6835","nonce":367974527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/335077136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":335077136},{"type":"v1","addr":"172.21.15.38:6837","nonce":335077136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/3501211160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3501211160},{"type":"v1","addr":"172.21.15.38:6839","nonce":3501211160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6841/2623130915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":2623130915},{"type":"v1","addr":"172.21.15.38:6841","nonce":2623130915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/412277519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":412277519},{"type":"v1","addr":"172.21.15.122:6833","nonce":412277519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3987234757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3987234757},{"type":"v1","addr":"172.21.15.122:6835","nonce":3987234757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:30:26.764565+0000","modified":"2024-09-16T00:31:05.083019+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24440},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1210328177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1210328177},{"type":"v1","addr":"172.21.15.73:6833","nonce":1210328177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6837/1858299680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1858299680},{"type":"v1","addr":"172.21.15.122:6837","nonce":1858299680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1983975311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1983975311},{"type":"v1","addr":"172.21.15.122:6839","nonce":1983975311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.73:6835/2866363113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2866363113},{"type":"v1","addr":"172.21.15.73:6835","nonce":2866363113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.73:6837/2681081265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2681081265},{"type":"v1","addr":"172.21.15.73:6837","nonce":2681081265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.73:6839/2396224525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2396224525},{"type":"v1","addr":"172.21.15.73:6839","nonce":2396224525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24440,24464,24494]},"id":1}]} 2024-09-16T00:31:15.522 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 25 2024-09-16T00:31:15.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.522+0000 7f357546e640 1 -- 172.21.15.38:0/1173055767 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f355c0a3350 msgr2=0x7f355c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:15.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.523+0000 7f357546e640 1 --2- 172.21.15.38:0/1173055767 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f355c0a3350 0x7f355c0a5810 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f3560005fd0 tx=0x7f356000b590 comp rx=0 tx=0).stop 2024-09-16T00:31:15.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.523+0000 7f357546e640 1 -- 172.21.15.38:0/1173055767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3570103660 msgr2=0x7f3570198910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:15.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.523+0000 7f357546e640 1 --2- 172.21.15.38:0/1173055767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3570103660 0x7f3570198910 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f355800c970 tx=0x7f355800ce30 comp rx=0 tx=0).stop 2024-09-16T00:31:15.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.523+0000 7f357546e640 1 -- 172.21.15.38:0/1173055767 shutdown_connections 2024-09-16T00:31:15.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.523+0000 7f357546e640 1 --2- 172.21.15.38:0/1173055767 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f355c0a3350 0x7f355c0a5810 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.523+0000 7f357546e640 1 --2- 172.21.15.38:0/1173055767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f357010c170 0x7f357019cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.523+0000 7f357546e640 1 --2- 172.21.15.38:0/1173055767 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3570103660 0x7f3570198910 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.523+0000 7f357546e640 1 --2- 172.21.15.38:0/1173055767 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3570102d40 0x7f35701983d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:15.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.523+0000 7f357546e640 1 -- 172.21.15.38:0/1173055767 >> 172.21.15.38:0/1173055767 conn(0x7f35700fc7a0 msgr2=0x7f35700fde20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:15.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.523+0000 7f357546e640 1 -- 172.21.15.38:0/1173055767 shutdown_connections 2024-09-16T00:31:15.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:15.523+0000 7f357546e640 1 -- 172.21.15.38:0/1173055767 wait complete. 2024-09-16T00:31:15.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:15 smithi122 ceph-mon[35412]: pgmap v314: 129 pgs: 129 active+clean; 591 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-09-16T00:31:15.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:15 smithi122 ceph-mon[35412]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:31:15.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:15 smithi073 ceph-mon[35254]: pgmap v314: 129 pgs: 129 active+clean; 591 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 3.2 KiB/s wr, 13 op/s 2024-09-16T00:31:15.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:15 smithi073 ceph-mon[35254]: from='client.14817 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:31:16.186 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:31:16.442 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:16.708 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:16 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/1173055767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:31:16.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:16 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/1173055767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:31:16.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:16 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/1173055767' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:31:16.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:16.943+0000 7f244aad0640 1 -- 172.21.15.38:0/3796591749 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2444100a60 msgr2=0x7f2444100ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:16.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:16.943+0000 7f244aad0640 1 --2- 172.21.15.38:0/3796591749 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2444100a60 0x7f2444100ec0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2438009930 tx=0x7f243802a420 comp rx=0 tx=0).stop 2024-09-16T00:31:16.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:16.943+0000 7f244aad0640 1 -- 172.21.15.38:0/3796591749 shutdown_connections 2024-09-16T00:31:16.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:16.943+0000 7f244aad0640 1 --2- 172.21.15.38:0/3796591749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2444101400 0x7f244410d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:16.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:16.943+0000 7f244aad0640 1 --2- 172.21.15.38:0/3796591749 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2444100a60 0x7f2444100ec0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:16.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:16.943+0000 7f244aad0640 1 --2- 172.21.15.38:0/3796591749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2444106a60 0x7f2444106e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:16.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:16.943+0000 7f244aad0640 1 -- 172.21.15.38:0/3796591749 >> 172.21.15.38:0/3796591749 conn(0x7f24440fc780 msgr2=0x7f24440feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:16.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:16.943+0000 7f244aad0640 1 -- 172.21.15.38:0/3796591749 shutdown_connections 2024-09-16T00:31:16.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:16.943+0000 7f244aad0640 1 -- 172.21.15.38:0/3796591749 wait complete. 2024-09-16T00:31:16.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:16.943+0000 7f244aad0640 1 Processor -- start 2024-09-16T00:31:16.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:16.943+0000 7f244aad0640 1 -- start start 2024-09-16T00:31:16.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:16.943+0000 7f244aad0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2444100a60 0x7f24441970c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:16.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:16.943+0000 7f244aad0640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2444101400 0x7f2444197600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:16.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:16.943+0000 7f244aad0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2444106a60 0x7f2444197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:16.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:16.943+0000 7f244aad0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2444110e80 con 0x7f2444100a60 2024-09-16T00:31:16.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:16.943+0000 7f244aad0640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2444110d00 con 0x7f2444101400 2024-09-16T00:31:16.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:16.943+0000 7f244aad0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2444111000 con 0x7f2444106a60 2024-09-16T00:31:16.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:16.944+0000 7f244886c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2444100a60 0x7f24441970c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:16.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:16.944+0000 7f244886c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2444100a60 0x7f24441970c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44060/0 (socket says 172.21.15.38:44060) 2024-09-16T00:31:16.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:16.944+0000 7f244886c640 1 -- 172.21.15.38:0/1795567270 learned_addr learned my addr 172.21.15.38:0/1795567270 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:31:16.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:16.944+0000 7f244906d640 1 --2- 172.21.15.38:0/1795567270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2444106a60 0x7f2444197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:16.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:16.944+0000 7f2443fff640 1 --2- 172.21.15.38:0/1795567270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2444101400 0x7f2444197600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:16.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:16.944+0000 7f244886c640 1 -- 172.21.15.38:0/1795567270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2444106a60 msgr2=0x7f2444197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:16.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:16.944+0000 7f244886c640 1 --2- 172.21.15.38:0/1795567270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2444106a60 0x7f2444197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:16.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:16.944+0000 7f244886c640 1 -- 172.21.15.38:0/1795567270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2444101400 msgr2=0x7f2444197600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:16.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:16.944+0000 7f244886c640 1 --2- 172.21.15.38:0/1795567270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2444101400 0x7f2444197600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:16.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:16.944+0000 7f244886c640 1 -- 172.21.15.38:0/1795567270 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24441a6930 con 0x7f2444100a60 2024-09-16T00:31:16.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:16.944+0000 7f244906d640 1 --2- 172.21.15.38:0/1795567270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2444106a60 0x7f2444197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:31:16.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:16.944+0000 7f244886c640 1 --2- 172.21.15.38:0/1795567270 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2444100a60 0x7f24441970c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f243c00ba50 tx=0x7f243c00bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:16.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:16.944+0000 7f2441ffb640 1 -- 172.21.15.38:0/1795567270 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243c019070 con 0x7f2444100a60 2024-09-16T00:31:16.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:16.944+0000 7f2441ffb640 1 -- 172.21.15.38:0/1795567270 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f243c00cbe0 con 0x7f2444100a60 2024-09-16T00:31:16.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:16.944+0000 7f244aad0640 1 -- 172.21.15.38:0/1795567270 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24441a6c20 con 0x7f2444100a60 2024-09-16T00:31:16.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:16.944+0000 7f2441ffb640 1 -- 172.21.15.38:0/1795567270 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f243c0076e0 con 0x7f2444100a60 2024-09-16T00:31:16.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:16.945+0000 7f244aad0640 1 -- 172.21.15.38:0/1795567270 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24441a7130 con 0x7f2444100a60 2024-09-16T00:31:16.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:16.945+0000 7f244aad0640 1 -- 172.21.15.38:0/1795567270 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2444100ec0 con 0x7f2444100a60 2024-09-16T00:31:16.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:16.946+0000 7f2441ffb640 1 -- 172.21.15.38:0/1795567270 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f243c01c070 con 0x7f2444100a60 2024-09-16T00:31:16.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:16.947+0000 7f2441ffb640 1 --2- 172.21.15.38:0/1795567270 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f242c0a32d0 0x7f242c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:16.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:16.947+0000 7f2443fff640 1 --2- 172.21.15.38:0/1795567270 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f242c0a32d0 0x7f242c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:16.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:16.947+0000 7f2441ffb640 1 -- 172.21.15.38:0/1795567270 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f243c0cf230 con 0x7f2444100a60 2024-09-16T00:31:16.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:16.948+0000 7f2443fff640 1 --2- 172.21.15.38:0/1795567270 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f242c0a32d0 0x7f242c0a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2438000c00 tx=0x7f2438031490 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:16.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:16.951+0000 7f2441ffb640 1 -- 172.21.15.38:0/1795567270 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f243c09bcc0 con 0x7f2444100a60 2024-09-16T00:31:17.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:17.128+0000 7f244aad0640 1 -- 172.21.15.38:0/1795567270 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f244402d7c0 con 0x7f2444100a60 2024-09-16T00:31:17.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:17.129+0000 7f2441ffb640 1 -- 172.21.15.38:0/1795567270 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24174 (secure 0 0 0) 0x7f243c09b5c0 con 0x7f2444100a60 2024-09-16T00:31:17.130 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:31:17.130 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":88,"fsid":"983b2072-73c1-11ef-bceb-c7b262605968","created":"2024-09-16T00:22:34.897053+0000","modified":"2024-09-16T00:30:35.865128+0000","last_up_change":"2024-09-16T00:29:34.503134+0000","last_in_change":"2024-09-16T00:29:10.135656+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:25:41.931977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:30:22.741889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:30:25.042590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:30:31.554350+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3afc20e3-869f-43a9-a911-b0e8d07de86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6803","nonce":1622028921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6805","nonce":1622028921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6809","nonce":1622028921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6807","nonce":1622028921}]},"public_addr":"172.21.15.38:6803/1622028921","cluster_addr":"172.21.15.38:6805/1622028921","heartbeat_back_addr":"172.21.15.38:6809/1622028921","heartbeat_front_addr":"172.21.15.38:6807/1622028921","state":["exists","up"]},{"osd":1,"uuid":"6d9d7acb-45ed-4543-8afc-6433fc7f096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6801","nonce":3811690336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6803","nonce":3811690336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6807","nonce":3811690336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6805","nonce":3811690336}]},"public_addr":"172.21.15.73:6801/3811690336","cluster_addr":"172.21.15.73:6803/3811690336","heartbeat_back_addr":"172.21.15.73:6807/3811690336","heartbeat_front_addr":"172.21.15.73:6805/3811690336","state":["exists","up"]},{"osd":2,"uuid":"48690f35-99e9-4f45-9bee-487857caa8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6801","nonce":1754741655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6803","nonce":1754741655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6807","nonce":1754741655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6805","nonce":1754741655}]},"public_addr":"172.21.15.122:6801/1754741655","cluster_addr":"172.21.15.122:6803/1754741655","heartbeat_back_addr":"172.21.15.122:6807/1754741655","heartbeat_front_addr":"172.21.15.122:6805/1754741655","state":["exists","up"]},{"osd":3,"uuid":"28ebb4a0-92e7-4f57-82a5-b9b761eca269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6811","nonce":3523190079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6813","nonce":3523190079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6817","nonce":3523190079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6815","nonce":3523190079}]},"public_addr":"172.21.15.38:6811/3523190079","cluster_addr":"172.21.15.38:6813/3523190079","heartbeat_back_addr":"172.21.15.38:6817/3523190079","heartbeat_front_addr":"172.21.15.38:6815/3523190079","state":["exists","up"]},{"osd":4,"uuid":"5ea363f7-7ad8-45ed-b7e4-4b010fe7f8cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6809","nonce":1469927628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6811","nonce":1469927628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6815","nonce":1469927628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6813","nonce":1469927628}]},"public_addr":"172.21.15.73:6809/1469927628","cluster_addr":"172.21.15.73:6811/1469927628","heartbeat_back_addr":"172.21.15.73:6815/1469927628","heartbeat_front_addr":"172.21.15.73:6813/1469927628","state":["exists","up"]},{"osd":5,"uuid":"e3c66d43-dd28-4bb1-a886-62a869be82cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6809","nonce":2954413058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6811","nonce":2954413058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6815","nonce":2954413058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6813","nonce":2954413058}]},"public_addr":"172.21.15.122:6809/2954413058","cluster_addr":"172.21.15.122:6811/2954413058","heartbeat_back_addr":"172.21.15.122:6815/2954413058","heartbeat_front_addr":"172.21.15.122:6813/2954413058","state":["exists","up"]},{"osd":6,"uuid":"211497ac-7c48-4095-bfdb-1d4df0149f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6819","nonce":1415719550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6821","nonce":1415719550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6825","nonce":1415719550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6823","nonce":1415719550}]},"public_addr":"172.21.15.38:6819/1415719550","cluster_addr":"172.21.15.38:6821/1415719550","heartbeat_back_addr":"172.21.15.38:6825/1415719550","heartbeat_front_addr":"172.21.15.38:6823/1415719550","state":["exists","up"]},{"osd":7,"uuid":"06512315-3f5e-44f8-976d-d3f5f1ba00e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6817","nonce":2634361473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6819","nonce":2634361473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6823","nonce":2634361473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6821","nonce":2634361473}]},"public_addr":"172.21.15.73:6817/2634361473","cluster_addr":"172.21.15.73:6819/2634361473","heartbeat_back_addr":"172.21.15.73:6823/2634361473","heartbeat_front_addr":"172.21.15.73:6821/2634361473","state":["exists","up"]},{"osd":8,"uuid":"db703c0a-1f87-46f0-be5f-f826f7f9daac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6817","nonce":1800567633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6819","nonce":1800567633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6823","nonce":1800567633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6821","nonce":1800567633}]},"public_addr":"172.21.15.122:6817/1800567633","cluster_addr":"172.21.15.122:6819/1800567633","heartbeat_back_addr":"172.21.15.122:6823/1800567633","heartbeat_front_addr":"172.21.15.122:6821/1800567633","state":["exists","up"]},{"osd":9,"uuid":"f5384200-b3b4-4936-85e1-fc57a9fe2bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6827","nonce":445727833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6829","nonce":445727833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6833","nonce":445727833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6831","nonce":445727833}]},"public_addr":"172.21.15.38:6827/445727833","cluster_addr":"172.21.15.38:6829/445727833","heartbeat_back_addr":"172.21.15.38:6833/445727833","heartbeat_front_addr":"172.21.15.38:6831/445727833","state":["exists","up"]},{"osd":10,"uuid":"c22883d9-6672-4796-a961-963d21eb37fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6825","nonce":43297139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6827","nonce":43297139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6831","nonce":43297139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6829","nonce":43297139}]},"public_addr":"172.21.15.73:6825/43297139","cluster_addr":"172.21.15.73:6827/43297139","heartbeat_back_addr":"172.21.15.73:6831/43297139","heartbeat_front_addr":"172.21.15.73:6829/43297139","state":["exists","up"]},{"osd":11,"uuid":"9f98e8cb-1bff-4b8b-aa19-56b691387383","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6825","nonce":1886446061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6827","nonce":1886446061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6831","nonce":1886446061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6829","nonce":1886446061}]},"public_addr":"172.21.15.122:6825/1886446061","cluster_addr":"172.21.15.122:6827/1886446061","heartbeat_back_addr":"172.21.15.122:6831/1886446061","heartbeat_front_addr":"172.21.15.122:6829/1886446061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:47.384012+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:12.834488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:38.770055+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:04.800466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:30.808076+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:56.646967+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:27:22.721270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:27:48.510011+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:28:13.922944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:28:40.514947+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:29:06.031052+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:29:31.882062+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6801/1779324916":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/924150447":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/141237849":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/2302081750":"2024-09-17T00:22:56.844212+0000","172.21.15.38:0/2245468594":"2024-09-17T00:22:56.844212+0000","172.21.15.38:6801/583006373":"2024-09-17T00:22:56.844212+0000","172.21.15.38:0/2435492667":"2024-09-17T00:23:21.417483+0000","172.21.15.38:6800/583006373":"2024-09-17T00:22:56.844212+0000","172.21.15.38:6800/1779324916":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/1103516837":"2024-09-17T00:22:56.844212+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:31:17.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:17.131+0000 7f244aad0640 1 -- 172.21.15.38:0/1795567270 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f242c0a32d0 msgr2=0x7f242c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:17.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:17.131+0000 7f244aad0640 1 --2- 172.21.15.38:0/1795567270 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f242c0a32d0 0x7f242c0a5790 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2438000c00 tx=0x7f2438031490 comp rx=0 tx=0).stop 2024-09-16T00:31:17.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:17.132+0000 7f244aad0640 1 -- 172.21.15.38:0/1795567270 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2444100a60 msgr2=0x7f24441970c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:17.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:17.132+0000 7f244aad0640 1 --2- 172.21.15.38:0/1795567270 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2444100a60 0x7f24441970c0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f243c00ba50 tx=0x7f243c00bf10 comp rx=0 tx=0).stop 2024-09-16T00:31:17.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:17.132+0000 7f244aad0640 1 -- 172.21.15.38:0/1795567270 shutdown_connections 2024-09-16T00:31:17.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:17.132+0000 7f244aad0640 1 --2- 172.21.15.38:0/1795567270 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f242c0a32d0 0x7f242c0a5790 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:17.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:17.132+0000 7f244aad0640 1 --2- 172.21.15.38:0/1795567270 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2444106a60 0x7f2444197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:17.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:17.132+0000 7f244aad0640 1 --2- 172.21.15.38:0/1795567270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2444101400 0x7f2444197600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:17.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:17.132+0000 7f244aad0640 1 --2- 172.21.15.38:0/1795567270 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2444100a60 0x7f24441970c0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:17.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:17.132+0000 7f244aad0640 1 -- 172.21.15.38:0/1795567270 >> 172.21.15.38:0/1795567270 conn(0x7f24440fc780 msgr2=0x7f24440fcf10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:17.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:17.132+0000 7f244aad0640 1 -- 172.21.15.38:0/1795567270 shutdown_connections 2024-09-16T00:31:17.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:17.132+0000 7f244aad0640 1 -- 172.21.15.38:0/1795567270 wait complete. 2024-09-16T00:31:17.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:17 smithi038 ceph-mon[30945]: pgmap v315: 129 pgs: 129 active+clean; 623 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.2 KiB/s wr, 13 op/s 2024-09-16T00:31:17.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:17 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/1795567270' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:31:17.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:17 smithi122 ceph-mon[35412]: pgmap v315: 129 pgs: 129 active+clean; 623 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.2 KiB/s wr, 13 op/s 2024-09-16T00:31:17.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:17 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/1795567270' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:31:17.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:17 smithi073 ceph-mon[35254]: pgmap v315: 129 pgs: 129 active+clean; 623 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.2 KiB/s wr, 13 op/s 2024-09-16T00:31:17.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:17 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/1795567270' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:31:17.799 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-16T00:31:17.799 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T00:31:18.056 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:18.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.546+0000 7f30a6cbe640 1 -- 172.21.15.38:0/1006409222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30a0074f40 msgr2=0x7f30a010de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:18.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.546+0000 7f30a6cbe640 1 --2- 172.21.15.38:0/1006409222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30a0074f40 0x7f30a010de80 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f308c009900 tx=0x7f308c02a480 comp rx=0 tx=0).stop 2024-09-16T00:31:18.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.546+0000 7f30a6cbe640 1 -- 172.21.15.38:0/1006409222 shutdown_connections 2024-09-16T00:31:18.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.546+0000 7f30a6cbe640 1 --2- 172.21.15.38:0/1006409222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30a010e3c0 0x7f30a0110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:18.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.546+0000 7f30a6cbe640 1 --2- 172.21.15.38:0/1006409222 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30a0074f40 0x7f30a010de80 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:18.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.546+0000 7f30a6cbe640 1 --2- 172.21.15.38:0/1006409222 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30a0074620 0x7f30a0074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:18.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.546+0000 7f30a6cbe640 1 -- 172.21.15.38:0/1006409222 >> 172.21.15.38:0/1006409222 conn(0x7f30a00fd8c0 msgr2=0x7f30a00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:18.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.546+0000 7f30a6cbe640 1 -- 172.21.15.38:0/1006409222 shutdown_connections 2024-09-16T00:31:18.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.546+0000 7f30a6cbe640 1 -- 172.21.15.38:0/1006409222 wait complete. 2024-09-16T00:31:18.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.547+0000 7f30a6cbe640 1 Processor -- start 2024-09-16T00:31:18.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.547+0000 7f30a6cbe640 1 -- start start 2024-09-16T00:31:18.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.547+0000 7f30a6cbe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30a0074620 0x7f30a019a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:18.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.547+0000 7f30a6cbe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30a0074f40 0x7f30a019abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:18.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.547+0000 7f30a6cbe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30a010e3c0 0x7f30a019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:18.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.547+0000 7f30a6cbe640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f30a0114160 con 0x7f30a0074f40 2024-09-16T00:31:18.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.547+0000 7f30a6cbe640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f30a0113fe0 con 0x7f30a0074620 2024-09-16T00:31:18.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.547+0000 7f30a6cbe640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f30a01142e0 con 0x7f30a010e3c0 2024-09-16T00:31:18.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.547+0000 7f309ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30a0074f40 0x7f30a019abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:18.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.547+0000 7f309ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30a0074f40 0x7f30a019abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44076/0 (socket says 172.21.15.38:44076) 2024-09-16T00:31:18.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.547+0000 7f309ffff640 1 -- 172.21.15.38:0/2014666257 learned_addr learned my addr 172.21.15.38:0/2014666257 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:31:18.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.547+0000 7f30a4a5a640 1 --2- 172.21.15.38:0/2014666257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30a0074620 0x7f30a019a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:18.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.547+0000 7f30a525b640 1 --2- 172.21.15.38:0/2014666257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30a010e3c0 0x7f30a019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:18.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.548+0000 7f309ffff640 1 -- 172.21.15.38:0/2014666257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30a010e3c0 msgr2=0x7f30a019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:18.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.548+0000 7f309ffff640 1 --2- 172.21.15.38:0/2014666257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30a010e3c0 0x7f30a019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:18.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.548+0000 7f309ffff640 1 -- 172.21.15.38:0/2014666257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30a0074620 msgr2=0x7f30a019a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:18.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.548+0000 7f309ffff640 1 --2- 172.21.15.38:0/2014666257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30a0074620 0x7f30a019a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:18.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.548+0000 7f309ffff640 1 -- 172.21.15.38:0/2014666257 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30a019f650 con 0x7f30a0074f40 2024-09-16T00:31:18.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.548+0000 7f309ffff640 1 --2- 172.21.15.38:0/2014666257 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30a0074f40 0x7f30a019abe0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f308c006010 tx=0x7f308c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:18.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.548+0000 7f309dffb640 1 -- 172.21.15.38:0/2014666257 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308c03f030 con 0x7f30a0074f40 2024-09-16T00:31:18.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.548+0000 7f309dffb640 1 -- 172.21.15.38:0/2014666257 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f308c003ce0 con 0x7f30a0074f40 2024-09-16T00:31:18.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.548+0000 7f309dffb640 1 -- 172.21.15.38:0/2014666257 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f308c035070 con 0x7f30a0074f40 2024-09-16T00:31:18.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.548+0000 7f30a6cbe640 1 -- 172.21.15.38:0/2014666257 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30a019f8e0 con 0x7f30a0074f40 2024-09-16T00:31:18.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.548+0000 7f30a6cbe640 1 -- 172.21.15.38:0/2014666257 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30a01a7220 con 0x7f30a0074f40 2024-09-16T00:31:18.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.549+0000 7f30a6cbe640 1 -- 172.21.15.38:0/2014666257 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30a0074a00 con 0x7f30a0074f40 2024-09-16T00:31:18.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.550+0000 7f309dffb640 1 -- 172.21.15.38:0/2014666257 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f308c042030 con 0x7f30a0074f40 2024-09-16T00:31:18.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.551+0000 7f309dffb640 1 --2- 172.21.15.38:0/2014666257 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f30900a3350 0x7f30900a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:18.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.551+0000 7f309dffb640 1 -- 172.21.15.38:0/2014666257 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f308c0f0220 con 0x7f30a0074f40 2024-09-16T00:31:18.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.551+0000 7f30a4a5a640 1 --2- 172.21.15.38:0/2014666257 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f30900a3350 0x7f30900a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:18.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.552+0000 7f30a4a5a640 1 --2- 172.21.15.38:0/2014666257 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f30900a3350 0x7f30900a5810 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f30940058c0 tx=0x7f309400a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:18.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.555+0000 7f309dffb640 1 -- 172.21.15.38:0/2014666257 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f308c0bc5b0 con 0x7f30a0074f40 2024-09-16T00:31:18.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.729+0000 7f30a6cbe640 1 -- 172.21.15.38:0/2014666257 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f30a0109570 con 0x7f30900a3350 2024-09-16T00:31:18.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.738+0000 7f309dffb640 1 -- 172.21.15.38:0/2014666257 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340339 (secure 0 0 0) 0x7f30a0109570 con 0x7f30900a3350 2024-09-16T00:31:18.742 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:31:18.744 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-16T00:31:18.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.742+0000 7f30a6cbe640 1 -- 172.21.15.38:0/2014666257 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f30900a3350 msgr2=0x7f30900a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:18.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.742+0000 7f30a6cbe640 1 --2- 172.21.15.38:0/2014666257 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f30900a3350 0x7f30900a5810 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f30940058c0 tx=0x7f309400a250 comp rx=0 tx=0).stop 2024-09-16T00:31:18.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.742+0000 7f30a6cbe640 1 -- 172.21.15.38:0/2014666257 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30a0074f40 msgr2=0x7f30a019abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:18.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.742+0000 7f30a6cbe640 1 --2- 172.21.15.38:0/2014666257 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30a0074f40 0x7f30a019abe0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f308c006010 tx=0x7f308c0042f0 comp rx=0 tx=0).stop 2024-09-16T00:31:18.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.743+0000 7f30a6cbe640 1 -- 172.21.15.38:0/2014666257 shutdown_connections 2024-09-16T00:31:18.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.743+0000 7f30a6cbe640 1 --2- 172.21.15.38:0/2014666257 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f30900a3350 0x7f30900a5810 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:18.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.743+0000 7f30a6cbe640 1 --2- 172.21.15.38:0/2014666257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30a010e3c0 0x7f30a019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:18.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.743+0000 7f30a6cbe640 1 --2- 172.21.15.38:0/2014666257 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30a0074f40 0x7f30a019abe0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:18.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.743+0000 7f30a6cbe640 1 --2- 172.21.15.38:0/2014666257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30a0074620 0x7f30a019a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:18.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.743+0000 7f30a6cbe640 1 -- 172.21.15.38:0/2014666257 >> 172.21.15.38:0/2014666257 conn(0x7f30a00fd8c0 msgr2=0x7f30a00fe5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:18.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.743+0000 7f30a6cbe640 1 -- 172.21.15.38:0/2014666257 shutdown_connections 2024-09-16T00:31:18.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:18.743+0000 7f30a6cbe640 1 -- 172.21.15.38:0/2014666257 wait complete. 2024-09-16T00:31:19.441 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":316,"stamp":"2024-09-16T00:31:17.518578+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":637950,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":215,"num_read_kb":207,"num_write":332,"num_write_kb":3031,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":269,"ondisk_log_size":269,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379492,"kb_used_data":13604,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562332,"statfs":{"total":1151940427776,"available":1151551827968,"internally_reserved":0,"allocated":13930496,"data_stored":7716536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":40844,"num_objects":15,"num_object_clones":0,"num_object_copies":47,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":82,"num_read_kb":89,"num_write":48,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003185"},"pg_stats":[{"pgid":"2.1f","version":"88'9","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:02.316003+0000","last_change":"2024-09-16T00:30:23.933355+0000","last_active":"2024-09-16T00:31:02.316003+0000","last_peered":"2024-09-16T00:31:02.316003+0000","last_clean":"2024-09-16T00:31:02.316003+0000","last_became_active":"2024-09-16T00:30:23.933057+0000","last_became_peered":"2024-09-16T00:30:23.933057+0000","last_unstale":"2024-09-16T00:31:02.316003+0000","last_undegraded":"2024-09-16T00:31:02.316003+0000","last_fullsized":"2024-09-16T00:31:02.316003+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:23:01.713440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765629+0000","last_change":"2024-09-16T00:30:26.084537+0000","last_active":"2024-09-16T00:30:36.765629+0000","last_peered":"2024-09-16T00:30:36.765629+0000","last_clean":"2024-09-16T00:30:36.765629+0000","last_became_active":"2024-09-16T00:30:26.084226+0000","last_became_peered":"2024-09-16T00:30:26.084226+0000","last_unstale":"2024-09-16T00:30:36.765629+0000","last_undegraded":"2024-09-16T00:30:36.765629+0000","last_fullsized":"2024-09-16T00:30:36.765629+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:43:41.628877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765535+0000","last_change":"2024-09-16T00:30:33.154873+0000","last_active":"2024-09-16T00:30:36.765535+0000","last_peered":"2024-09-16T00:30:36.765535+0000","last_clean":"2024-09-16T00:30:36.765535+0000","last_became_active":"2024-09-16T00:30:33.154433+0000","last_became_peered":"2024-09-16T00:30:33.154433+0000","last_unstale":"2024-09-16T00:30:36.765535+0000","last_undegraded":"2024-09-16T00:30:36.765535+0000","last_fullsized":"2024-09-16T00:30:36.765535+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:50.805806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766149+0000","last_change":"2024-09-16T00:30:24.422274+0000","last_active":"2024-09-16T00:30:36.766149+0000","last_peered":"2024-09-16T00:30:36.766149+0000","last_clean":"2024-09-16T00:30:36.766149+0000","last_became_active":"2024-09-16T00:30:24.421902+0000","last_became_peered":"2024-09-16T00:30:24.421902+0000","last_unstale":"2024-09-16T00:30:36.766149+0000","last_undegraded":"2024-09-16T00:30:36.766149+0000","last_fullsized":"2024-09-16T00:30:36.766149+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:01.910187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.767336+0000","last_change":"2024-09-16T00:30:26.088165+0000","last_active":"2024-09-16T00:30:36.767336+0000","last_peered":"2024-09-16T00:30:36.767336+0000","last_clean":"2024-09-16T00:30:36.767336+0000","last_became_active":"2024-09-16T00:30:26.087593+0000","last_became_peered":"2024-09-16T00:30:26.087593+0000","last_unstale":"2024-09-16T00:30:36.767336+0000","last_undegraded":"2024-09-16T00:30:36.767336+0000","last_fullsized":"2024-09-16T00:30:36.767336+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:54.149930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.764467+0000","last_change":"2024-09-16T00:30:33.154586+0000","last_active":"2024-09-16T00:30:36.764467+0000","last_peered":"2024-09-16T00:30:36.764467+0000","last_clean":"2024-09-16T00:30:36.764467+0000","last_became_active":"2024-09-16T00:30:33.154209+0000","last_became_peered":"2024-09-16T00:30:33.154209+0000","last_unstale":"2024-09-16T00:30:36.764467+0000","last_undegraded":"2024-09-16T00:30:36.764467+0000","last_fullsized":"2024-09-16T00:30:36.764467+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:45.776895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:02.314625+0000","last_change":"2024-09-16T00:30:23.932442+0000","last_active":"2024-09-16T00:31:02.314625+0000","last_peered":"2024-09-16T00:31:02.314625+0000","last_clean":"2024-09-16T00:31:02.314625+0000","last_became_active":"2024-09-16T00:30:23.932009+0000","last_became_peered":"2024-09-16T00:30:23.932009+0000","last_unstale":"2024-09-16T00:31:02.314625+0000","last_undegraded":"2024-09-16T00:31:02.314625+0000","last_fullsized":"2024-09-16T00:31:02.314625+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:09:27.078187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765329+0000","last_change":"2024-09-16T00:30:26.087715+0000","last_active":"2024-09-16T00:30:36.765329+0000","last_peered":"2024-09-16T00:30:36.765329+0000","last_clean":"2024-09-16T00:30:36.765329+0000","last_became_active":"2024-09-16T00:30:26.087415+0000","last_became_peered":"2024-09-16T00:30:26.087415+0000","last_unstale":"2024-09-16T00:30:36.765329+0000","last_undegraded":"2024-09-16T00:30:36.765329+0000","last_fullsized":"2024-09-16T00:30:36.765329+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:52.373992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765223+0000","last_change":"2024-09-16T00:30:33.152328+0000","last_active":"2024-09-16T00:30:36.765223+0000","last_peered":"2024-09-16T00:30:36.765223+0000","last_clean":"2024-09-16T00:30:36.765223+0000","last_became_active":"2024-09-16T00:30:33.151954+0000","last_became_peered":"2024-09-16T00:30:33.151954+0000","last_unstale":"2024-09-16T00:30:36.765223+0000","last_undegraded":"2024-09-16T00:30:36.765223+0000","last_fullsized":"2024-09-16T00:30:36.765223+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:36.546729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765938+0000","last_change":"2024-09-16T00:30:23.928988+0000","last_active":"2024-09-16T00:30:36.765938+0000","last_peered":"2024-09-16T00:30:36.765938+0000","last_clean":"2024-09-16T00:30:36.765938+0000","last_became_active":"2024-09-16T00:30:23.928638+0000","last_became_peered":"2024-09-16T00:30:23.928638+0000","last_unstale":"2024-09-16T00:30:36.765938+0000","last_undegraded":"2024-09-16T00:30:36.765938+0000","last_fullsized":"2024-09-16T00:30:36.765938+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:45:58.203455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.767026+0000","last_change":"2024-09-16T00:30:26.086911+0000","last_active":"2024-09-16T00:30:36.767026+0000","last_peered":"2024-09-16T00:30:36.767026+0000","last_clean":"2024-09-16T00:30:36.767026+0000","last_became_active":"2024-09-16T00:30:26.086589+0000","last_became_peered":"2024-09-16T00:30:26.086589+0000","last_unstale":"2024-09-16T00:30:36.767026+0000","last_undegraded":"2024-09-16T00:30:36.767026+0000","last_fullsized":"2024-09-16T00:30:36.767026+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:13:10.389476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.764558+0000","last_change":"2024-09-16T00:30:33.147564+0000","last_active":"2024-09-16T00:30:36.764558+0000","last_peered":"2024-09-16T00:30:36.764558+0000","last_clean":"2024-09-16T00:30:36.764558+0000","last_became_active":"2024-09-16T00:30:33.147119+0000","last_became_peered":"2024-09-16T00:30:33.147119+0000","last_unstale":"2024-09-16T00:30:36.764558+0000","last_undegraded":"2024-09-16T00:30:36.764558+0000","last_fullsized":"2024-09-16T00:30:36.764558+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:00.509122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.634602+0000","last_change":"2024-09-16T00:30:26.080954+0000","last_active":"2024-09-16T00:30:36.634602+0000","last_peered":"2024-09-16T00:30:36.634602+0000","last_clean":"2024-09-16T00:30:36.634602+0000","last_became_active":"2024-09-16T00:30:26.080624+0000","last_became_peered":"2024-09-16T00:30:26.080624+0000","last_unstale":"2024-09-16T00:30:36.634602+0000","last_undegraded":"2024-09-16T00:30:36.634602+0000","last_fullsized":"2024-09-16T00:30:36.634602+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:05:10.440980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:01.303450+0000","last_change":"2024-09-16T00:30:23.933715+0000","last_active":"2024-09-16T00:31:01.303450+0000","last_peered":"2024-09-16T00:31:01.303450+0000","last_clean":"2024-09-16T00:31:01.303450+0000","last_became_active":"2024-09-16T00:30:23.933275+0000","last_became_peered":"2024-09-16T00:30:23.933275+0000","last_unstale":"2024-09-16T00:31:01.303450+0000","last_undegraded":"2024-09-16T00:31:01.303450+0000","last_fullsized":"2024-09-16T00:31:01.303450+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:40:44.846053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.764833+0000","last_change":"2024-09-16T00:30:33.604602+0000","last_active":"2024-09-16T00:30:36.764833+0000","last_peered":"2024-09-16T00:30:36.764833+0000","last_clean":"2024-09-16T00:30:36.764833+0000","last_became_active":"2024-09-16T00:30:33.604107+0000","last_became_peered":"2024-09-16T00:30:33.604107+0000","last_unstale":"2024-09-16T00:30:36.764833+0000","last_undegraded":"2024-09-16T00:30:36.764833+0000","last_fullsized":"2024-09-16T00:30:36.764833+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:41.254445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765248+0000","last_change":"2024-09-16T00:30:26.080612+0000","last_active":"2024-09-16T00:30:36.765248+0000","last_peered":"2024-09-16T00:30:36.765248+0000","last_clean":"2024-09-16T00:30:36.765248+0000","last_became_active":"2024-09-16T00:30:26.080317+0000","last_became_peered":"2024-09-16T00:30:26.080317+0000","last_unstale":"2024-09-16T00:30:36.765248+0000","last_undegraded":"2024-09-16T00:30:36.765248+0000","last_fullsized":"2024-09-16T00:30:36.765248+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:23.814241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766325+0000","last_change":"2024-09-16T00:30:23.936778+0000","last_active":"2024-09-16T00:30:36.766325+0000","last_peered":"2024-09-16T00:30:36.766325+0000","last_clean":"2024-09-16T00:30:36.766325+0000","last_became_active":"2024-09-16T00:30:23.936265+0000","last_became_peered":"2024-09-16T00:30:23.936265+0000","last_unstale":"2024-09-16T00:30:36.766325+0000","last_undegraded":"2024-09-16T00:30:36.766325+0000","last_fullsized":"2024-09-16T00:30:36.766325+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:35:28.288476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765465+0000","last_change":"2024-09-16T00:30:33.148340+0000","last_active":"2024-09-16T00:30:36.765465+0000","last_peered":"2024-09-16T00:30:36.765465+0000","last_clean":"2024-09-16T00:30:36.765465+0000","last_became_active":"2024-09-16T00:30:33.147289+0000","last_became_peered":"2024-09-16T00:30:33.147289+0000","last_unstale":"2024-09-16T00:30:36.765465+0000","last_undegraded":"2024-09-16T00:30:36.765465+0000","last_fullsized":"2024-09-16T00:30:36.765465+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:21.259254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.149061+0000","last_change":"2024-09-16T00:30:26.077325+0000","last_active":"2024-09-16T00:30:36.149061+0000","last_peered":"2024-09-16T00:30:36.149061+0000","last_clean":"2024-09-16T00:30:36.149061+0000","last_became_active":"2024-09-16T00:30:26.077002+0000","last_became_peered":"2024-09-16T00:30:26.077002+0000","last_unstale":"2024-09-16T00:30:36.149061+0000","last_undegraded":"2024-09-16T00:30:36.149061+0000","last_fullsized":"2024-09-16T00:30:36.149061+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:12:38.569247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:01.304089+0000","last_change":"2024-09-16T00:30:23.933025+0000","last_active":"2024-09-16T00:31:01.304089+0000","last_peered":"2024-09-16T00:31:01.304089+0000","last_clean":"2024-09-16T00:31:01.304089+0000","last_became_active":"2024-09-16T00:30:23.932669+0000","last_became_peered":"2024-09-16T00:30:23.932669+0000","last_unstale":"2024-09-16T00:31:01.304089+0000","last_undegraded":"2024-09-16T00:31:01.304089+0000","last_fullsized":"2024-09-16T00:31:01.304089+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:20.593665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.153862+0000","last_change":"2024-09-16T00:30:33.151844+0000","last_active":"2024-09-16T00:30:36.153862+0000","last_peered":"2024-09-16T00:30:36.153862+0000","last_clean":"2024-09-16T00:30:36.153862+0000","last_became_active":"2024-09-16T00:30:33.151311+0000","last_became_peered":"2024-09-16T00:30:33.151311+0000","last_unstale":"2024-09-16T00:30:36.153862+0000","last_undegraded":"2024-09-16T00:30:36.153862+0000","last_fullsized":"2024-09-16T00:30:36.153862+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:51.785233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765023+0000","last_change":"2024-09-16T00:30:26.076104+0000","last_active":"2024-09-16T00:30:36.765023+0000","last_peered":"2024-09-16T00:30:36.765023+0000","last_clean":"2024-09-16T00:30:36.765023+0000","last_became_active":"2024-09-16T00:30:26.075764+0000","last_became_peered":"2024-09-16T00:30:26.075764+0000","last_unstale":"2024-09-16T00:30:36.765023+0000","last_undegraded":"2024-09-16T00:30:36.765023+0000","last_fullsized":"2024-09-16T00:30:36.765023+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:31.228533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:01.301696+0000","last_change":"2024-09-16T00:30:24.422411+0000","last_active":"2024-09-16T00:31:01.301696+0000","last_peered":"2024-09-16T00:31:01.301696+0000","last_clean":"2024-09-16T00:31:01.301696+0000","last_became_active":"2024-09-16T00:30:24.422102+0000","last_became_peered":"2024-09-16T00:30:24.422102+0000","last_unstale":"2024-09-16T00:31:01.301696+0000","last_undegraded":"2024-09-16T00:31:01.301696+0000","last_fullsized":"2024-09-16T00:31:01.301696+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:06:00.572070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765253+0000","last_change":"2024-09-16T00:30:33.605397+0000","last_active":"2024-09-16T00:30:36.765253+0000","last_peered":"2024-09-16T00:30:36.765253+0000","last_clean":"2024-09-16T00:30:36.765253+0000","last_became_active":"2024-09-16T00:30:33.604978+0000","last_became_peered":"2024-09-16T00:30:33.604978+0000","last_unstale":"2024-09-16T00:30:36.765253+0000","last_undegraded":"2024-09-16T00:30:36.765253+0000","last_fullsized":"2024-09-16T00:30:36.765253+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:35:16.766056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766891+0000","last_change":"2024-09-16T00:30:26.082684+0000","last_active":"2024-09-16T00:30:36.766891+0000","last_peered":"2024-09-16T00:30:36.766891+0000","last_clean":"2024-09-16T00:30:36.766891+0000","last_became_active":"2024-09-16T00:30:26.082335+0000","last_became_peered":"2024-09-16T00:30:26.082335+0000","last_unstale":"2024-09-16T00:30:36.766891+0000","last_undegraded":"2024-09-16T00:30:36.766891+0000","last_fullsized":"2024-09-16T00:30:36.766891+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:51:21.389627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:01.302667+0000","last_change":"2024-09-16T00:30:24.587840+0000","last_active":"2024-09-16T00:31:01.302667+0000","last_peered":"2024-09-16T00:31:01.302667+0000","last_clean":"2024-09-16T00:31:01.302667+0000","last_became_active":"2024-09-16T00:30:24.587436+0000","last_became_peered":"2024-09-16T00:30:24.587436+0000","last_unstale":"2024-09-16T00:31:01.302667+0000","last_undegraded":"2024-09-16T00:31:01.302667+0000","last_fullsized":"2024-09-16T00:31:01.302667+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:06.758585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765780+0000","last_change":"2024-09-16T00:30:33.145051+0000","last_active":"2024-09-16T00:30:36.765780+0000","last_peered":"2024-09-16T00:30:36.765780+0000","last_clean":"2024-09-16T00:30:36.765780+0000","last_became_active":"2024-09-16T00:30:33.144585+0000","last_became_peered":"2024-09-16T00:30:33.144585+0000","last_unstale":"2024-09-16T00:30:36.765780+0000","last_undegraded":"2024-09-16T00:30:36.765780+0000","last_fullsized":"2024-09-16T00:30:36.765780+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:48:23.588495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766177+0000","last_change":"2024-09-16T00:30:26.078830+0000","last_active":"2024-09-16T00:30:36.766177+0000","last_peered":"2024-09-16T00:30:36.766177+0000","last_clean":"2024-09-16T00:30:36.766177+0000","last_became_active":"2024-09-16T00:30:26.078520+0000","last_became_peered":"2024-09-16T00:30:26.078520+0000","last_unstale":"2024-09-16T00:30:36.766177+0000","last_undegraded":"2024-09-16T00:30:36.766177+0000","last_fullsized":"2024-09-16T00:30:36.766177+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:26:07.790519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'6","reported_seq":36,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:09.074962+0000","last_change":"2024-09-16T00:30:23.933284+0000","last_active":"2024-09-16T00:31:09.074962+0000","last_peered":"2024-09-16T00:31:09.074962+0000","last_clean":"2024-09-16T00:31:09.074962+0000","last_became_active":"2024-09-16T00:30:23.932920+0000","last_became_peered":"2024-09-16T00:30:23.932920+0000","last_unstale":"2024-09-16T00:31:09.074962+0000","last_undegraded":"2024-09-16T00:31:09.074962+0000","last_fullsized":"2024-09-16T00:31:09.074962+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:38.199391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.764616+0000","last_change":"2024-09-16T00:30:33.604308+0000","last_active":"2024-09-16T00:30:36.764616+0000","last_peered":"2024-09-16T00:30:36.764616+0000","last_clean":"2024-09-16T00:30:36.764616+0000","last_became_active":"2024-09-16T00:30:33.603889+0000","last_became_peered":"2024-09-16T00:30:33.603889+0000","last_unstale":"2024-09-16T00:30:36.764616+0000","last_undegraded":"2024-09-16T00:30:36.764616+0000","last_fullsized":"2024-09-16T00:30:36.764616+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:32:55.173063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766489+0000","last_change":"2024-09-16T00:30:26.078515+0000","last_active":"2024-09-16T00:30:36.766489+0000","last_peered":"2024-09-16T00:30:36.766489+0000","last_clean":"2024-09-16T00:30:36.766489+0000","last_became_active":"2024-09-16T00:30:26.078173+0000","last_became_peered":"2024-09-16T00:30:26.078173+0000","last_unstale":"2024-09-16T00:30:36.766489+0000","last_undegraded":"2024-09-16T00:30:36.766489+0000","last_fullsized":"2024-09-16T00:30:36.766489+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:40:31.471154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:01.300620+0000","last_change":"2024-09-16T00:30:24.423209+0000","last_active":"2024-09-16T00:31:01.300620+0000","last_peered":"2024-09-16T00:31:01.300620+0000","last_clean":"2024-09-16T00:31:01.300620+0000","last_became_active":"2024-09-16T00:30:24.422157+0000","last_became_peered":"2024-09-16T00:30:24.422157+0000","last_unstale":"2024-09-16T00:31:01.300620+0000","last_undegraded":"2024-09-16T00:31:01.300620+0000","last_fullsized":"2024-09-16T00:31:01.300620+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:28.262392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.149166+0000","last_change":"2024-09-16T00:30:33.146204+0000","last_active":"2024-09-16T00:30:36.149166+0000","last_peered":"2024-09-16T00:30:36.149166+0000","last_clean":"2024-09-16T00:30:36.149166+0000","last_became_active":"2024-09-16T00:30:33.145553+0000","last_became_peered":"2024-09-16T00:30:33.145553+0000","last_unstale":"2024-09-16T00:30:36.149166+0000","last_undegraded":"2024-09-16T00:30:36.149166+0000","last_fullsized":"2024-09-16T00:30:36.149166+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:22.096050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.153876+0000","last_change":"2024-09-16T00:30:26.070153+0000","last_active":"2024-09-16T00:30:36.153876+0000","last_peered":"2024-09-16T00:30:36.153876+0000","last_clean":"2024-09-16T00:30:36.153876+0000","last_became_active":"2024-09-16T00:30:26.069792+0000","last_became_peered":"2024-09-16T00:30:26.069792+0000","last_unstale":"2024-09-16T00:30:36.153876+0000","last_undegraded":"2024-09-16T00:30:36.153876+0000","last_fullsized":"2024-09-16T00:30:36.153876+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:28.164242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.153735+0000","last_change":"2024-09-16T00:30:24.423261+0000","last_active":"2024-09-16T00:30:36.153735+0000","last_peered":"2024-09-16T00:30:36.153735+0000","last_clean":"2024-09-16T00:30:36.153735+0000","last_became_active":"2024-09-16T00:30:24.422650+0000","last_became_peered":"2024-09-16T00:30:24.422650+0000","last_unstale":"2024-09-16T00:30:36.153735+0000","last_undegraded":"2024-09-16T00:30:36.153735+0000","last_fullsized":"2024-09-16T00:30:36.153735+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:00.211744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765204+0000","last_change":"2024-09-16T00:30:33.604492+0000","last_active":"2024-09-16T00:30:36.765204+0000","last_peered":"2024-09-16T00:30:36.765204+0000","last_clean":"2024-09-16T00:30:36.765204+0000","last_became_active":"2024-09-16T00:30:33.604015+0000","last_became_peered":"2024-09-16T00:30:33.604015+0000","last_unstale":"2024-09-16T00:30:36.765204+0000","last_undegraded":"2024-09-16T00:30:36.765204+0000","last_fullsized":"2024-09-16T00:30:36.765204+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:40:22.516583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.153854+0000","last_change":"2024-09-16T00:30:26.075517+0000","last_active":"2024-09-16T00:30:36.153854+0000","last_peered":"2024-09-16T00:30:36.153854+0000","last_clean":"2024-09-16T00:30:36.153854+0000","last_became_active":"2024-09-16T00:30:26.075113+0000","last_became_peered":"2024-09-16T00:30:26.075113+0000","last_unstale":"2024-09-16T00:30:36.153854+0000","last_undegraded":"2024-09-16T00:30:36.153854+0000","last_fullsized":"2024-09-16T00:30:36.153854+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:11.427010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.634735+0000","last_change":"2024-09-16T00:30:23.933639+0000","last_active":"2024-09-16T00:30:36.634735+0000","last_peered":"2024-09-16T00:30:36.634735+0000","last_clean":"2024-09-16T00:30:36.634735+0000","last_became_active":"2024-09-16T00:30:23.933322+0000","last_became_peered":"2024-09-16T00:30:23.933322+0000","last_unstale":"2024-09-16T00:30:36.634735+0000","last_undegraded":"2024-09-16T00:30:36.634735+0000","last_fullsized":"2024-09-16T00:30:36.634735+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:59:59.082702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766116+0000","last_change":"2024-09-16T00:30:33.148508+0000","last_active":"2024-09-16T00:30:36.766116+0000","last_peered":"2024-09-16T00:30:36.766116+0000","last_clean":"2024-09-16T00:30:36.766116+0000","last_became_active":"2024-09-16T00:30:33.148091+0000","last_became_peered":"2024-09-16T00:30:33.148091+0000","last_unstale":"2024-09-16T00:30:36.766116+0000","last_undegraded":"2024-09-16T00:30:36.766116+0000","last_fullsized":"2024-09-16T00:30:36.766116+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:50.052526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766765+0000","last_change":"2024-09-16T00:30:26.080374+0000","last_active":"2024-09-16T00:30:36.766765+0000","last_peered":"2024-09-16T00:30:36.766765+0000","last_clean":"2024-09-16T00:30:36.766765+0000","last_became_active":"2024-09-16T00:30:26.080044+0000","last_became_peered":"2024-09-16T00:30:26.080044+0000","last_unstale":"2024-09-16T00:30:36.766765+0000","last_undegraded":"2024-09-16T00:30:36.766765+0000","last_fullsized":"2024-09-16T00:30:36.766765+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:38:36.034903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:13.345304+0000","last_change":"2024-09-16T00:30:24.422137+0000","last_active":"2024-09-16T00:31:13.345304+0000","last_peered":"2024-09-16T00:31:13.345304+0000","last_clean":"2024-09-16T00:31:13.345304+0000","last_became_active":"2024-09-16T00:30:24.421778+0000","last_became_peered":"2024-09-16T00:30:24.421778+0000","last_unstale":"2024-09-16T00:31:13.345304+0000","last_undegraded":"2024-09-16T00:31:13.345304+0000","last_fullsized":"2024-09-16T00:31:13.345304+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:54:21.964099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766397+0000","last_change":"2024-09-16T00:30:33.604263+0000","last_active":"2024-09-16T00:30:36.766397+0000","last_peered":"2024-09-16T00:30:36.766397+0000","last_clean":"2024-09-16T00:30:36.766397+0000","last_became_active":"2024-09-16T00:30:33.603804+0000","last_became_peered":"2024-09-16T00:30:33.603804+0000","last_unstale":"2024-09-16T00:30:36.766397+0000","last_undegraded":"2024-09-16T00:30:36.766397+0000","last_fullsized":"2024-09-16T00:30:36.766397+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:08:37.590827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765524+0000","last_change":"2024-09-16T00:30:26.082916+0000","last_active":"2024-09-16T00:30:36.765524+0000","last_peered":"2024-09-16T00:30:36.765524+0000","last_clean":"2024-09-16T00:30:36.765524+0000","last_became_active":"2024-09-16T00:30:26.082606+0000","last_became_peered":"2024-09-16T00:30:26.082606+0000","last_unstale":"2024-09-16T00:30:36.765524+0000","last_undegraded":"2024-09-16T00:30:36.765524+0000","last_fullsized":"2024-09-16T00:30:36.765524+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:56.777640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:02.314276+0000","last_change":"2024-09-16T00:30:23.937602+0000","last_active":"2024-09-16T00:31:02.314276+0000","last_peered":"2024-09-16T00:31:02.314276+0000","last_clean":"2024-09-16T00:31:02.314276+0000","last_became_active":"2024-09-16T00:30:23.937261+0000","last_became_peered":"2024-09-16T00:30:23.937261+0000","last_unstale":"2024-09-16T00:31:02.314276+0000","last_undegraded":"2024-09-16T00:31:02.314276+0000","last_fullsized":"2024-09-16T00:31:02.314276+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:24.698758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765812+0000","last_change":"2024-09-16T00:30:33.605536+0000","last_active":"2024-09-16T00:30:36.765812+0000","last_peered":"2024-09-16T00:30:36.765812+0000","last_clean":"2024-09-16T00:30:36.765812+0000","last_became_active":"2024-09-16T00:30:33.605073+0000","last_became_peered":"2024-09-16T00:30:33.605073+0000","last_unstale":"2024-09-16T00:30:36.765812+0000","last_undegraded":"2024-09-16T00:30:36.765812+0000","last_fullsized":"2024-09-16T00:30:36.765812+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:11.143052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765181+0000","last_change":"2024-09-16T00:30:26.084507+0000","last_active":"2024-09-16T00:30:36.765181+0000","last_peered":"2024-09-16T00:30:36.765181+0000","last_clean":"2024-09-16T00:30:36.765181+0000","last_became_active":"2024-09-16T00:30:26.084152+0000","last_became_peered":"2024-09-16T00:30:26.084152+0000","last_unstale":"2024-09-16T00:30:36.765181+0000","last_undegraded":"2024-09-16T00:30:36.765181+0000","last_fullsized":"2024-09-16T00:30:36.765181+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:27:21.557409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:03.327725+0000","last_change":"2024-09-16T00:30:23.932686+0000","last_active":"2024-09-16T00:31:03.327725+0000","last_peered":"2024-09-16T00:31:03.327725+0000","last_clean":"2024-09-16T00:31:03.327725+0000","last_became_active":"2024-09-16T00:30:23.932343+0000","last_became_peered":"2024-09-16T00:30:23.932343+0000","last_unstale":"2024-09-16T00:31:03.327725+0000","last_undegraded":"2024-09-16T00:31:03.327725+0000","last_fullsized":"2024-09-16T00:31:03.327725+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:13:44.185712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":342,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.634509+0000","last_change":"2024-09-16T00:29:11.143642+0000","last_active":"2024-09-16T00:30:36.634509+0000","last_peered":"2024-09-16T00:30:36.634509+0000","last_clean":"2024-09-16T00:30:36.634509+0000","last_became_active":"2024-09-16T00:29:11.143227+0000","last_became_peered":"2024-09-16T00:29:11.143227+0000","last_unstale":"2024-09-16T00:30:36.634509+0000","last_undegraded":"2024-09-16T00:30:36.634509+0000","last_fullsized":"2024-09-16T00:30:36.634509+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:25:42.361316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:25:42.361316+0000","last_clean_scrub_stamp":"2024-09-16T00:25:42.361316+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:29:12.740474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.634923+0000","last_change":"2024-09-16T00:30:33.155555+0000","last_active":"2024-09-16T00:30:36.634923+0000","last_peered":"2024-09-16T00:30:36.634923+0000","last_clean":"2024-09-16T00:30:36.634923+0000","last_became_active":"2024-09-16T00:30:33.155170+0000","last_became_peered":"2024-09-16T00:30:33.155170+0000","last_unstale":"2024-09-16T00:30:36.634923+0000","last_undegraded":"2024-09-16T00:30:36.634923+0000","last_fullsized":"2024-09-16T00:30:36.634923+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:34:46.214422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.634987+0000","last_change":"2024-09-16T00:30:26.076003+0000","last_active":"2024-09-16T00:30:36.634987+0000","last_peered":"2024-09-16T00:30:36.634987+0000","last_clean":"2024-09-16T00:30:36.634987+0000","last_became_active":"2024-09-16T00:30:26.075666+0000","last_became_peered":"2024-09-16T00:30:26.075666+0000","last_unstale":"2024-09-16T00:30:36.634987+0000","last_undegraded":"2024-09-16T00:30:36.634987+0000","last_fullsized":"2024-09-16T00:30:36.634987+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:30.113380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:02.315587+0000","last_change":"2024-09-16T00:30:24.587469+0000","last_active":"2024-09-16T00:31:02.315587+0000","last_peered":"2024-09-16T00:31:02.315587+0000","last_clean":"2024-09-16T00:31:02.315587+0000","last_became_active":"2024-09-16T00:30:24.587049+0000","last_became_peered":"2024-09-16T00:30:24.587049+0000","last_unstale":"2024-09-16T00:31:02.315587+0000","last_undegraded":"2024-09-16T00:31:02.315587+0000","last_fullsized":"2024-09-16T00:31:02.315587+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:17:45.934505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.764736+0000","last_change":"2024-09-16T00:30:33.145921+0000","last_active":"2024-09-16T00:30:36.764736+0000","last_peered":"2024-09-16T00:30:36.764736+0000","last_clean":"2024-09-16T00:30:36.764736+0000","last_became_active":"2024-09-16T00:30:33.145450+0000","last_became_peered":"2024-09-16T00:30:33.145450+0000","last_unstale":"2024-09-16T00:30:36.764736+0000","last_undegraded":"2024-09-16T00:30:36.764736+0000","last_fullsized":"2024-09-16T00:30:36.764736+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:53:51.698686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.634824+0000","last_change":"2024-09-16T00:30:26.087940+0000","last_active":"2024-09-16T00:30:36.634824+0000","last_peered":"2024-09-16T00:30:36.634824+0000","last_clean":"2024-09-16T00:30:36.634824+0000","last_became_active":"2024-09-16T00:30:26.087599+0000","last_became_peered":"2024-09-16T00:30:26.087599+0000","last_unstale":"2024-09-16T00:30:36.634824+0000","last_undegraded":"2024-09-16T00:30:36.634824+0000","last_fullsized":"2024-09-16T00:30:36.634824+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:43.934839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.767180+0000","last_change":"2024-09-16T00:30:23.930986+0000","last_active":"2024-09-16T00:30:36.767180+0000","last_peered":"2024-09-16T00:30:36.767180+0000","last_clean":"2024-09-16T00:30:36.767180+0000","last_became_active":"2024-09-16T00:30:23.930588+0000","last_became_peered":"2024-09-16T00:30:23.930588+0000","last_unstale":"2024-09-16T00:30:36.767180+0000","last_undegraded":"2024-09-16T00:30:36.767180+0000","last_fullsized":"2024-09-16T00:30:36.767180+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:04:41.902358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.767280+0000","last_change":"2024-09-16T00:30:33.154894+0000","last_active":"2024-09-16T00:30:36.767280+0000","last_peered":"2024-09-16T00:30:36.767280+0000","last_clean":"2024-09-16T00:30:36.767280+0000","last_became_active":"2024-09-16T00:30:33.154490+0000","last_became_peered":"2024-09-16T00:30:33.154490+0000","last_unstale":"2024-09-16T00:30:36.767280+0000","last_undegraded":"2024-09-16T00:30:36.767280+0000","last_fullsized":"2024-09-16T00:30:36.767280+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:59:53.854212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.634981+0000","last_change":"2024-09-16T00:30:26.077069+0000","last_active":"2024-09-16T00:30:36.634981+0000","last_peered":"2024-09-16T00:30:36.634981+0000","last_clean":"2024-09-16T00:30:36.634981+0000","last_became_active":"2024-09-16T00:30:26.076718+0000","last_became_peered":"2024-09-16T00:30:26.076718+0000","last_unstale":"2024-09-16T00:30:36.634981+0000","last_undegraded":"2024-09-16T00:30:36.634981+0000","last_fullsized":"2024-09-16T00:30:36.634981+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:25:37.541704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766245+0000","last_change":"2024-09-16T00:30:23.932081+0000","last_active":"2024-09-16T00:30:36.766245+0000","last_peered":"2024-09-16T00:30:36.766245+0000","last_clean":"2024-09-16T00:30:36.766245+0000","last_became_active":"2024-09-16T00:30:23.931729+0000","last_became_peered":"2024-09-16T00:30:23.931729+0000","last_unstale":"2024-09-16T00:30:36.766245+0000","last_undegraded":"2024-09-16T00:30:36.766245+0000","last_fullsized":"2024-09-16T00:30:36.766245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:20:05.178580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766427+0000","last_change":"2024-09-16T00:30:33.147738+0000","last_active":"2024-09-16T00:30:36.766427+0000","last_peered":"2024-09-16T00:30:36.766427+0000","last_clean":"2024-09-16T00:30:36.766427+0000","last_became_active":"2024-09-16T00:30:33.147326+0000","last_became_peered":"2024-09-16T00:30:33.147326+0000","last_unstale":"2024-09-16T00:30:36.766427+0000","last_undegraded":"2024-09-16T00:30:36.766427+0000","last_fullsized":"2024-09-16T00:30:36.766427+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:44:55.123980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.149267+0000","last_change":"2024-09-16T00:30:26.079312+0000","last_active":"2024-09-16T00:30:36.149267+0000","last_peered":"2024-09-16T00:30:36.149267+0000","last_clean":"2024-09-16T00:30:36.149267+0000","last_became_active":"2024-09-16T00:30:26.078986+0000","last_became_peered":"2024-09-16T00:30:26.078986+0000","last_unstale":"2024-09-16T00:30:36.149267+0000","last_undegraded":"2024-09-16T00:30:36.149267+0000","last_fullsized":"2024-09-16T00:30:36.149267+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:02:06.602850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:03.328650+0000","last_change":"2024-09-16T00:30:23.931313+0000","last_active":"2024-09-16T00:31:03.328650+0000","last_peered":"2024-09-16T00:31:03.328650+0000","last_clean":"2024-09-16T00:31:03.328650+0000","last_became_active":"2024-09-16T00:30:23.930296+0000","last_became_peered":"2024-09-16T00:30:23.930296+0000","last_unstale":"2024-09-16T00:31:03.328650+0000","last_undegraded":"2024-09-16T00:31:03.328650+0000","last_fullsized":"2024-09-16T00:31:03.328650+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:30.006861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766732+0000","last_change":"2024-09-16T00:30:33.149154+0000","last_active":"2024-09-16T00:30:36.766732+0000","last_peered":"2024-09-16T00:30:36.766732+0000","last_clean":"2024-09-16T00:30:36.766732+0000","last_became_active":"2024-09-16T00:30:33.148763+0000","last_became_peered":"2024-09-16T00:30:33.148763+0000","last_unstale":"2024-09-16T00:30:36.766732+0000","last_undegraded":"2024-09-16T00:30:36.766732+0000","last_fullsized":"2024-09-16T00:30:36.766732+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:58:05.884919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766775+0000","last_change":"2024-09-16T00:30:26.086905+0000","last_active":"2024-09-16T00:30:36.766775+0000","last_peered":"2024-09-16T00:30:36.766775+0000","last_clean":"2024-09-16T00:30:36.766775+0000","last_became_active":"2024-09-16T00:30:26.086519+0000","last_became_peered":"2024-09-16T00:30:26.086519+0000","last_unstale":"2024-09-16T00:30:36.766775+0000","last_undegraded":"2024-09-16T00:30:36.766775+0000","last_fullsized":"2024-09-16T00:30:36.766775+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:51:08.787679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:02.315689+0000","last_change":"2024-09-16T00:30:24.422893+0000","last_active":"2024-09-16T00:31:02.315689+0000","last_peered":"2024-09-16T00:31:02.315689+0000","last_clean":"2024-09-16T00:31:02.315689+0000","last_became_active":"2024-09-16T00:30:24.422073+0000","last_became_peered":"2024-09-16T00:30:24.422073+0000","last_unstale":"2024-09-16T00:31:02.315689+0000","last_undegraded":"2024-09-16T00:31:02.315689+0000","last_fullsized":"2024-09-16T00:31:02.315689+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:15:11.375957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765949+0000","last_change":"2024-09-16T00:30:33.152582+0000","last_active":"2024-09-16T00:30:36.765949+0000","last_peered":"2024-09-16T00:30:36.765949+0000","last_clean":"2024-09-16T00:30:36.765949+0000","last_became_active":"2024-09-16T00:30:33.152180+0000","last_became_peered":"2024-09-16T00:30:33.152180+0000","last_unstale":"2024-09-16T00:30:36.765949+0000","last_undegraded":"2024-09-16T00:30:36.765949+0000","last_fullsized":"2024-09-16T00:30:36.765949+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:41:10.499391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.634299+0000","last_change":"2024-09-16T00:30:26.086052+0000","last_active":"2024-09-16T00:30:36.634299+0000","last_peered":"2024-09-16T00:30:36.634299+0000","last_clean":"2024-09-16T00:30:36.634299+0000","last_became_active":"2024-09-16T00:30:26.085756+0000","last_became_peered":"2024-09-16T00:30:26.085756+0000","last_unstale":"2024-09-16T00:30:36.634299+0000","last_undegraded":"2024-09-16T00:30:36.634299+0000","last_fullsized":"2024-09-16T00:30:36.634299+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:52.582166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:11.342848+0000","last_change":"2024-09-16T00:30:23.933528+0000","last_active":"2024-09-16T00:31:11.342848+0000","last_peered":"2024-09-16T00:31:11.342848+0000","last_clean":"2024-09-16T00:31:11.342848+0000","last_became_active":"2024-09-16T00:30:23.933186+0000","last_became_peered":"2024-09-16T00:30:23.933186+0000","last_unstale":"2024-09-16T00:31:11.342848+0000","last_undegraded":"2024-09-16T00:31:11.342848+0000","last_fullsized":"2024-09-16T00:31:11.342848+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:11:00.489676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.153953+0000","last_change":"2024-09-16T00:30:33.154711+0000","last_active":"2024-09-16T00:30:36.153953+0000","last_peered":"2024-09-16T00:30:36.153953+0000","last_clean":"2024-09-16T00:30:36.153953+0000","last_became_active":"2024-09-16T00:30:33.154112+0000","last_became_peered":"2024-09-16T00:30:33.154112+0000","last_unstale":"2024-09-16T00:30:36.153953+0000","last_undegraded":"2024-09-16T00:30:36.153953+0000","last_fullsized":"2024-09-16T00:30:36.153953+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:54:50.767406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.634689+0000","last_change":"2024-09-16T00:30:26.084509+0000","last_active":"2024-09-16T00:30:36.634689+0000","last_peered":"2024-09-16T00:30:36.634689+0000","last_clean":"2024-09-16T00:30:36.634689+0000","last_became_active":"2024-09-16T00:30:26.084188+0000","last_became_peered":"2024-09-16T00:30:26.084188+0000","last_unstale":"2024-09-16T00:30:36.634689+0000","last_undegraded":"2024-09-16T00:30:36.634689+0000","last_fullsized":"2024-09-16T00:30:36.634689+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:21:48.981277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:01.303518+0000","last_change":"2024-09-16T00:30:23.933499+0000","last_active":"2024-09-16T00:31:01.303518+0000","last_peered":"2024-09-16T00:31:01.303518+0000","last_clean":"2024-09-16T00:31:01.303518+0000","last_became_active":"2024-09-16T00:30:23.933082+0000","last_became_peered":"2024-09-16T00:30:23.933082+0000","last_unstale":"2024-09-16T00:31:01.303518+0000","last_undegraded":"2024-09-16T00:31:01.303518+0000","last_fullsized":"2024-09-16T00:31:01.303518+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:42:18.998433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.149255+0000","last_change":"2024-09-16T00:30:33.154748+0000","last_active":"2024-09-16T00:30:36.149255+0000","last_peered":"2024-09-16T00:30:36.149255+0000","last_clean":"2024-09-16T00:30:36.149255+0000","last_became_active":"2024-09-16T00:30:33.154305+0000","last_became_peered":"2024-09-16T00:30:33.154305+0000","last_unstale":"2024-09-16T00:30:36.149255+0000","last_undegraded":"2024-09-16T00:30:36.149255+0000","last_fullsized":"2024-09-16T00:30:36.149255+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:57:34.651844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765961+0000","last_change":"2024-09-16T00:30:26.080517+0000","last_active":"2024-09-16T00:30:36.765961+0000","last_peered":"2024-09-16T00:30:36.765961+0000","last_clean":"2024-09-16T00:30:36.765961+0000","last_became_active":"2024-09-16T00:30:26.080197+0000","last_became_peered":"2024-09-16T00:30:26.080197+0000","last_unstale":"2024-09-16T00:30:36.765961+0000","last_undegraded":"2024-09-16T00:30:36.765961+0000","last_fullsized":"2024-09-16T00:30:36.765961+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:15.508562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:02.314135+0000","last_change":"2024-09-16T00:30:23.933275+0000","last_active":"2024-09-16T00:31:02.314135+0000","last_peered":"2024-09-16T00:31:02.314135+0000","last_clean":"2024-09-16T00:31:02.314135+0000","last_became_active":"2024-09-16T00:30:23.932609+0000","last_became_peered":"2024-09-16T00:30:23.932609+0000","last_unstale":"2024-09-16T00:31:02.314135+0000","last_undegraded":"2024-09-16T00:31:02.314135+0000","last_fullsized":"2024-09-16T00:31:02.314135+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:59:19.896758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766830+0000","last_change":"2024-09-16T00:30:33.145803+0000","last_active":"2024-09-16T00:30:36.766830+0000","last_peered":"2024-09-16T00:30:36.766830+0000","last_clean":"2024-09-16T00:30:36.766830+0000","last_became_active":"2024-09-16T00:30:33.145366+0000","last_became_peered":"2024-09-16T00:30:33.145366+0000","last_unstale":"2024-09-16T00:30:36.766830+0000","last_undegraded":"2024-09-16T00:30:36.766830+0000","last_fullsized":"2024-09-16T00:30:36.766830+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:47:28.689305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765528+0000","last_change":"2024-09-16T00:30:26.079415+0000","last_active":"2024-09-16T00:30:36.765528+0000","last_peered":"2024-09-16T00:30:36.765528+0000","last_clean":"2024-09-16T00:30:36.765528+0000","last_became_active":"2024-09-16T00:30:26.079075+0000","last_became_peered":"2024-09-16T00:30:26.079075+0000","last_unstale":"2024-09-16T00:30:36.765528+0000","last_undegraded":"2024-09-16T00:30:36.765528+0000","last_fullsized":"2024-09-16T00:30:36.765528+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:18:18.236342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'25","reported_seq":82,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:13.347174+0000","last_change":"2024-09-16T00:30:23.933172+0000","last_active":"2024-09-16T00:31:13.347174+0000","last_peered":"2024-09-16T00:31:13.347174+0000","last_clean":"2024-09-16T00:31:13.347174+0000","last_became_active":"2024-09-16T00:30:23.932492+0000","last_became_peered":"2024-09-16T00:30:23.932492+0000","last_unstale":"2024-09-16T00:31:13.347174+0000","last_undegraded":"2024-09-16T00:31:13.347174+0000","last_fullsized":"2024-09-16T00:31:13.347174+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:29.169448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":43345,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":27,"num_read_kb":56,"num_write":26,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.153383+0000","last_change":"2024-09-16T00:30:33.151089+0000","last_active":"2024-09-16T00:30:36.153383+0000","last_peered":"2024-09-16T00:30:36.153383+0000","last_clean":"2024-09-16T00:30:36.153383+0000","last_became_active":"2024-09-16T00:30:33.150642+0000","last_became_peered":"2024-09-16T00:30:33.150642+0000","last_unstale":"2024-09-16T00:30:36.153383+0000","last_undegraded":"2024-09-16T00:30:36.153383+0000","last_fullsized":"2024-09-16T00:30:36.153383+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:24:31.022949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765334+0000","last_change":"2024-09-16T00:30:26.085419+0000","last_active":"2024-09-16T00:30:36.765334+0000","last_peered":"2024-09-16T00:30:36.765334+0000","last_clean":"2024-09-16T00:30:36.765334+0000","last_became_active":"2024-09-16T00:30:26.085103+0000","last_became_peered":"2024-09-16T00:30:26.085103+0000","last_unstale":"2024-09-16T00:30:36.765334+0000","last_undegraded":"2024-09-16T00:30:36.765334+0000","last_fullsized":"2024-09-16T00:30:36.765334+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:56:26.352779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:01.304909+0000","last_change":"2024-09-16T00:30:24.422646+0000","last_active":"2024-09-16T00:31:01.304909+0000","last_peered":"2024-09-16T00:31:01.304909+0000","last_clean":"2024-09-16T00:31:01.304909+0000","last_became_active":"2024-09-16T00:30:24.422065+0000","last_became_peered":"2024-09-16T00:30:24.422065+0000","last_unstale":"2024-09-16T00:31:01.304909+0000","last_undegraded":"2024-09-16T00:31:01.304909+0000","last_fullsized":"2024-09-16T00:31:01.304909+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:53:12.401613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.764965+0000","last_change":"2024-09-16T00:30:33.147692+0000","last_active":"2024-09-16T00:30:36.764965+0000","last_peered":"2024-09-16T00:30:36.764965+0000","last_clean":"2024-09-16T00:30:36.764965+0000","last_became_active":"2024-09-16T00:30:33.147115+0000","last_became_peered":"2024-09-16T00:30:33.147115+0000","last_unstale":"2024-09-16T00:30:36.764965+0000","last_undegraded":"2024-09-16T00:30:36.764965+0000","last_fullsized":"2024-09-16T00:30:36.764965+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:44:04.536894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.767413+0000","last_change":"2024-09-16T00:30:26.088427+0000","last_active":"2024-09-16T00:30:36.767413+0000","last_peered":"2024-09-16T00:30:36.767413+0000","last_clean":"2024-09-16T00:30:36.767413+0000","last_became_active":"2024-09-16T00:30:26.086454+0000","last_became_peered":"2024-09-16T00:30:26.086454+0000","last_unstale":"2024-09-16T00:30:36.767413+0000","last_undegraded":"2024-09-16T00:30:36.767413+0000","last_fullsized":"2024-09-16T00:30:36.767413+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:54:29.679959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:01.302823+0000","last_change":"2024-09-16T00:30:23.936793+0000","last_active":"2024-09-16T00:31:01.302823+0000","last_peered":"2024-09-16T00:31:01.302823+0000","last_clean":"2024-09-16T00:31:01.302823+0000","last_became_active":"2024-09-16T00:30:23.936445+0000","last_became_peered":"2024-09-16T00:30:23.936445+0000","last_unstale":"2024-09-16T00:31:01.302823+0000","last_undegraded":"2024-09-16T00:31:01.302823+0000","last_fullsized":"2024-09-16T00:31:01.302823+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:33:06.645927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.153293+0000","last_change":"2024-09-16T00:30:33.156510+0000","last_active":"2024-09-16T00:30:36.153293+0000","last_peered":"2024-09-16T00:30:36.153293+0000","last_clean":"2024-09-16T00:30:36.153293+0000","last_became_active":"2024-09-16T00:30:33.156088+0000","last_became_peered":"2024-09-16T00:30:33.156088+0000","last_unstale":"2024-09-16T00:30:36.153293+0000","last_undegraded":"2024-09-16T00:30:36.153293+0000","last_fullsized":"2024-09-16T00:30:36.153293+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:34.230901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765614+0000","last_change":"2024-09-16T00:30:26.082598+0000","last_active":"2024-09-16T00:30:36.765614+0000","last_peered":"2024-09-16T00:30:36.765614+0000","last_clean":"2024-09-16T00:30:36.765614+0000","last_became_active":"2024-09-16T00:30:26.082286+0000","last_became_peered":"2024-09-16T00:30:26.082286+0000","last_unstale":"2024-09-16T00:30:36.765614+0000","last_undegraded":"2024-09-16T00:30:36.765614+0000","last_fullsized":"2024-09-16T00:30:36.765614+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:22:18.579149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:12.344332+0000","last_change":"2024-09-16T00:30:23.931097+0000","last_active":"2024-09-16T00:31:12.344332+0000","last_peered":"2024-09-16T00:31:12.344332+0000","last_clean":"2024-09-16T00:31:12.344332+0000","last_became_active":"2024-09-16T00:30:23.930719+0000","last_became_peered":"2024-09-16T00:30:23.930719+0000","last_unstale":"2024-09-16T00:31:12.344332+0000","last_undegraded":"2024-09-16T00:31:12.344332+0000","last_fullsized":"2024-09-16T00:31:12.344332+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:13:47.887754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.764905+0000","last_change":"2024-09-16T00:30:33.142581+0000","last_active":"2024-09-16T00:30:36.764905+0000","last_peered":"2024-09-16T00:30:36.764905+0000","last_clean":"2024-09-16T00:30:36.764905+0000","last_became_active":"2024-09-16T00:30:33.142263+0000","last_became_peered":"2024-09-16T00:30:33.142263+0000","last_unstale":"2024-09-16T00:30:36.764905+0000","last_undegraded":"2024-09-16T00:30:36.764905+0000","last_fullsized":"2024-09-16T00:30:36.764905+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:10:06.456011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766715+0000","last_change":"2024-09-16T00:30:33.152084+0000","last_active":"2024-09-16T00:30:36.766715+0000","last_peered":"2024-09-16T00:30:36.766715+0000","last_clean":"2024-09-16T00:30:36.766715+0000","last_became_active":"2024-09-16T00:30:33.151681+0000","last_became_peered":"2024-09-16T00:30:33.151681+0000","last_unstale":"2024-09-16T00:30:36.766715+0000","last_undegraded":"2024-09-16T00:30:36.766715+0000","last_fullsized":"2024-09-16T00:30:36.766715+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:36.966498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.153316+0000","last_change":"2024-09-16T00:30:26.084200+0000","last_active":"2024-09-16T00:30:36.153316+0000","last_peered":"2024-09-16T00:30:36.153316+0000","last_clean":"2024-09-16T00:30:36.153316+0000","last_became_active":"2024-09-16T00:30:26.083858+0000","last_became_peered":"2024-09-16T00:30:26.083858+0000","last_unstale":"2024-09-16T00:30:36.153316+0000","last_undegraded":"2024-09-16T00:30:36.153316+0000","last_fullsized":"2024-09-16T00:30:36.153316+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:42.712624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.767012+0000","last_change":"2024-09-16T00:30:23.932399+0000","last_active":"2024-09-16T00:30:36.767012+0000","last_peered":"2024-09-16T00:30:36.767012+0000","last_clean":"2024-09-16T00:30:36.767012+0000","last_became_active":"2024-09-16T00:30:23.931996+0000","last_became_peered":"2024-09-16T00:30:23.931996+0000","last_unstale":"2024-09-16T00:30:36.767012+0000","last_undegraded":"2024-09-16T00:30:36.767012+0000","last_fullsized":"2024-09-16T00:30:36.767012+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:47:09.634507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.149263+0000","last_change":"2024-09-16T00:30:33.154563+0000","last_active":"2024-09-16T00:30:36.149263+0000","last_peered":"2024-09-16T00:30:36.149263+0000","last_clean":"2024-09-16T00:30:36.149263+0000","last_became_active":"2024-09-16T00:30:33.154105+0000","last_became_peered":"2024-09-16T00:30:33.154105+0000","last_unstale":"2024-09-16T00:30:36.149263+0000","last_undegraded":"2024-09-16T00:30:36.149263+0000","last_fullsized":"2024-09-16T00:30:36.149263+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:59.966255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.764688+0000","last_change":"2024-09-16T00:30:26.075126+0000","last_active":"2024-09-16T00:30:36.764688+0000","last_peered":"2024-09-16T00:30:36.764688+0000","last_clean":"2024-09-16T00:30:36.764688+0000","last_became_active":"2024-09-16T00:30:26.074819+0000","last_became_peered":"2024-09-16T00:30:26.074819+0000","last_unstale":"2024-09-16T00:30:36.764688+0000","last_undegraded":"2024-09-16T00:30:36.764688+0000","last_fullsized":"2024-09-16T00:30:36.764688+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:36.996346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:02.315266+0000","last_change":"2024-09-16T00:30:23.929943+0000","last_active":"2024-09-16T00:31:02.315266+0000","last_peered":"2024-09-16T00:31:02.315266+0000","last_clean":"2024-09-16T00:31:02.315266+0000","last_became_active":"2024-09-16T00:30:23.929578+0000","last_became_peered":"2024-09-16T00:30:23.929578+0000","last_unstale":"2024-09-16T00:31:02.315266+0000","last_undegraded":"2024-09-16T00:31:02.315266+0000","last_fullsized":"2024-09-16T00:31:02.315266+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:05:12.505543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:02.313911+0000","last_change":"2024-09-16T00:30:23.935758+0000","last_active":"2024-09-16T00:31:02.313911+0000","last_peered":"2024-09-16T00:31:02.313911+0000","last_clean":"2024-09-16T00:31:02.313911+0000","last_became_active":"2024-09-16T00:30:23.935321+0000","last_became_peered":"2024-09-16T00:30:23.935321+0000","last_unstale":"2024-09-16T00:31:02.313911+0000","last_undegraded":"2024-09-16T00:31:02.313911+0000","last_fullsized":"2024-09-16T00:31:02.313911+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:35:40.513435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765532+0000","last_change":"2024-09-16T00:30:26.086316+0000","last_active":"2024-09-16T00:30:36.765532+0000","last_peered":"2024-09-16T00:30:36.765532+0000","last_clean":"2024-09-16T00:30:36.765532+0000","last_became_active":"2024-09-16T00:30:26.085800+0000","last_became_peered":"2024-09-16T00:30:26.085800+0000","last_unstale":"2024-09-16T00:30:36.765532+0000","last_undegraded":"2024-09-16T00:30:36.765532+0000","last_fullsized":"2024-09-16T00:30:36.765532+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:05:36.277127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766520+0000","last_change":"2024-09-16T00:30:33.155665+0000","last_active":"2024-09-16T00:30:36.766520+0000","last_peered":"2024-09-16T00:30:36.766520+0000","last_clean":"2024-09-16T00:30:36.766520+0000","last_became_active":"2024-09-16T00:30:33.155253+0000","last_became_peered":"2024-09-16T00:30:33.155253+0000","last_unstale":"2024-09-16T00:30:36.766520+0000","last_undegraded":"2024-09-16T00:30:36.766520+0000","last_fullsized":"2024-09-16T00:30:36.766520+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:37.545232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:11.343150+0000","last_change":"2024-09-16T00:30:23.934487+0000","last_active":"2024-09-16T00:31:11.343150+0000","last_peered":"2024-09-16T00:31:11.343150+0000","last_clean":"2024-09-16T00:31:11.343150+0000","last_became_active":"2024-09-16T00:30:23.934137+0000","last_became_peered":"2024-09-16T00:30:23.934137+0000","last_unstale":"2024-09-16T00:31:11.343150+0000","last_undegraded":"2024-09-16T00:31:11.343150+0000","last_fullsized":"2024-09-16T00:31:11.343150+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:14:33.088776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":13,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765355+0000","last_change":"2024-09-16T00:30:26.077533+0000","last_active":"2024-09-16T00:30:36.765355+0000","last_peered":"2024-09-16T00:30:36.765355+0000","last_clean":"2024-09-16T00:30:36.765355+0000","last_became_active":"2024-09-16T00:30:26.076443+0000","last_became_peered":"2024-09-16T00:30:26.076443+0000","last_unstale":"2024-09-16T00:30:36.765355+0000","last_undegraded":"2024-09-16T00:30:36.765355+0000","last_fullsized":"2024-09-16T00:30:36.765355+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:39:17.495521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766308+0000","last_change":"2024-09-16T00:30:33.146123+0000","last_active":"2024-09-16T00:30:36.766308+0000","last_peered":"2024-09-16T00:30:36.766308+0000","last_clean":"2024-09-16T00:30:36.766308+0000","last_became_active":"2024-09-16T00:30:33.145716+0000","last_became_peered":"2024-09-16T00:30:33.145716+0000","last_unstale":"2024-09-16T00:30:36.766308+0000","last_undegraded":"2024-09-16T00:30:36.766308+0000","last_fullsized":"2024-09-16T00:30:36.766308+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:14:27.281367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.148977+0000","last_change":"2024-09-16T00:30:26.083155+0000","last_active":"2024-09-16T00:30:36.148977+0000","last_peered":"2024-09-16T00:30:36.148977+0000","last_clean":"2024-09-16T00:30:36.148977+0000","last_became_active":"2024-09-16T00:30:26.082782+0000","last_became_peered":"2024-09-16T00:30:26.082782+0000","last_unstale":"2024-09-16T00:30:36.148977+0000","last_undegraded":"2024-09-16T00:30:36.148977+0000","last_fullsized":"2024-09-16T00:30:36.148977+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:15.929343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766026+0000","last_change":"2024-09-16T00:30:26.080740+0000","last_active":"2024-09-16T00:30:36.766026+0000","last_peered":"2024-09-16T00:30:36.766026+0000","last_clean":"2024-09-16T00:30:36.766026+0000","last_became_active":"2024-09-16T00:30:26.080401+0000","last_became_peered":"2024-09-16T00:30:26.080401+0000","last_unstale":"2024-09-16T00:30:36.766026+0000","last_undegraded":"2024-09-16T00:30:36.766026+0000","last_fullsized":"2024-09-16T00:30:36.766026+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:59:26.563887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766169+0000","last_change":"2024-09-16T00:30:26.076252+0000","last_active":"2024-09-16T00:30:36.766169+0000","last_peered":"2024-09-16T00:30:36.766169+0000","last_clean":"2024-09-16T00:30:36.766169+0000","last_became_active":"2024-09-16T00:30:26.075922+0000","last_became_peered":"2024-09-16T00:30:26.075922+0000","last_unstale":"2024-09-16T00:30:36.766169+0000","last_undegraded":"2024-09-16T00:30:36.766169+0000","last_fullsized":"2024-09-16T00:30:36.766169+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:28:04.822817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.767391+0000","last_change":"2024-09-16T00:30:26.085123+0000","last_active":"2024-09-16T00:30:36.767391+0000","last_peered":"2024-09-16T00:30:36.767391+0000","last_clean":"2024-09-16T00:30:36.767391+0000","last_became_active":"2024-09-16T00:30:26.084782+0000","last_became_peered":"2024-09-16T00:30:26.084782+0000","last_unstale":"2024-09-16T00:30:36.767391+0000","last_undegraded":"2024-09-16T00:30:36.767391+0000","last_fullsized":"2024-09-16T00:30:36.767391+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:09:33.766340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766346+0000","last_change":"2024-09-16T00:30:26.077048+0000","last_active":"2024-09-16T00:30:36.766346+0000","last_peered":"2024-09-16T00:30:36.766346+0000","last_clean":"2024-09-16T00:30:36.766346+0000","last_became_active":"2024-09-16T00:30:26.076719+0000","last_became_peered":"2024-09-16T00:30:26.076719+0000","last_unstale":"2024-09-16T00:30:36.766346+0000","last_undegraded":"2024-09-16T00:30:36.766346+0000","last_fullsized":"2024-09-16T00:30:36.766346+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:58.584246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.149263+0000","last_change":"2024-09-16T00:30:26.078446+0000","last_active":"2024-09-16T00:30:36.149263+0000","last_peered":"2024-09-16T00:30:36.149263+0000","last_clean":"2024-09-16T00:30:36.149263+0000","last_became_active":"2024-09-16T00:30:26.078145+0000","last_became_peered":"2024-09-16T00:30:26.078145+0000","last_unstale":"2024-09-16T00:30:36.149263+0000","last_undegraded":"2024-09-16T00:30:36.149263+0000","last_fullsized":"2024-09-16T00:30:36.149263+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:52:28.739834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765484+0000","last_change":"2024-09-16T00:30:26.075875+0000","last_active":"2024-09-16T00:30:36.765484+0000","last_peered":"2024-09-16T00:30:36.765484+0000","last_clean":"2024-09-16T00:30:36.765484+0000","last_became_active":"2024-09-16T00:30:26.075424+0000","last_became_peered":"2024-09-16T00:30:26.075424+0000","last_unstale":"2024-09-16T00:30:36.765484+0000","last_undegraded":"2024-09-16T00:30:36.765484+0000","last_fullsized":"2024-09-16T00:30:36.765484+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:21:07.958507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766070+0000","last_change":"2024-09-16T00:30:26.077694+0000","last_active":"2024-09-16T00:30:36.766070+0000","last_peered":"2024-09-16T00:30:36.766070+0000","last_clean":"2024-09-16T00:30:36.766070+0000","last_became_active":"2024-09-16T00:30:26.077374+0000","last_became_peered":"2024-09-16T00:30:26.077374+0000","last_unstale":"2024-09-16T00:30:36.766070+0000","last_undegraded":"2024-09-16T00:30:36.766070+0000","last_fullsized":"2024-09-16T00:30:36.766070+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:16:30.822050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766640+0000","last_change":"2024-09-16T00:30:26.085931+0000","last_active":"2024-09-16T00:30:36.766640+0000","last_peered":"2024-09-16T00:30:36.766640+0000","last_clean":"2024-09-16T00:30:36.766640+0000","last_became_active":"2024-09-16T00:30:26.085420+0000","last_became_peered":"2024-09-16T00:30:26.085420+0000","last_unstale":"2024-09-16T00:30:36.766640+0000","last_undegraded":"2024-09-16T00:30:36.766640+0000","last_fullsized":"2024-09-16T00:30:36.766640+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:20:36.019203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.764930+0000","last_change":"2024-09-16T00:30:26.074217+0000","last_active":"2024-09-16T00:30:36.764930+0000","last_peered":"2024-09-16T00:30:36.764930+0000","last_clean":"2024-09-16T00:30:36.764930+0000","last_became_active":"2024-09-16T00:30:26.073856+0000","last_became_peered":"2024-09-16T00:30:26.073856+0000","last_unstale":"2024-09-16T00:30:36.764930+0000","last_undegraded":"2024-09-16T00:30:36.764930+0000","last_fullsized":"2024-09-16T00:30:36.764930+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:53:57.078465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765039+0000","last_change":"2024-09-16T00:30:26.084248+0000","last_active":"2024-09-16T00:30:36.765039+0000","last_peered":"2024-09-16T00:30:36.765039+0000","last_clean":"2024-09-16T00:30:36.765039+0000","last_became_active":"2024-09-16T00:30:26.083530+0000","last_became_peered":"2024-09-16T00:30:26.083530+0000","last_unstale":"2024-09-16T00:30:36.765039+0000","last_undegraded":"2024-09-16T00:30:36.765039+0000","last_fullsized":"2024-09-16T00:30:36.765039+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:16:49.483881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766481+0000","last_change":"2024-09-16T00:30:26.080803+0000","last_active":"2024-09-16T00:30:36.766481+0000","last_peered":"2024-09-16T00:30:36.766481+0000","last_clean":"2024-09-16T00:30:36.766481+0000","last_became_active":"2024-09-16T00:30:26.080482+0000","last_became_peered":"2024-09-16T00:30:26.080482+0000","last_unstale":"2024-09-16T00:30:36.766481+0000","last_undegraded":"2024-09-16T00:30:36.766481+0000","last_fullsized":"2024-09-16T00:30:36.766481+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:35:41.741025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.149107+0000","last_change":"2024-09-16T00:30:26.077680+0000","last_active":"2024-09-16T00:30:36.149107+0000","last_peered":"2024-09-16T00:30:36.149107+0000","last_clean":"2024-09-16T00:30:36.149107+0000","last_became_active":"2024-09-16T00:30:26.077371+0000","last_became_peered":"2024-09-16T00:30:26.077371+0000","last_unstale":"2024-09-16T00:30:36.149107+0000","last_undegraded":"2024-09-16T00:30:36.149107+0000","last_fullsized":"2024-09-16T00:30:36.149107+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:31:34.338617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766418+0000","last_change":"2024-09-16T00:30:26.071531+0000","last_active":"2024-09-16T00:30:36.766418+0000","last_peered":"2024-09-16T00:30:36.766418+0000","last_clean":"2024-09-16T00:30:36.766418+0000","last_became_active":"2024-09-16T00:30:26.071189+0000","last_became_peered":"2024-09-16T00:30:26.071189+0000","last_unstale":"2024-09-16T00:30:36.766418+0000","last_undegraded":"2024-09-16T00:30:36.766418+0000","last_fullsized":"2024-09-16T00:30:36.766418+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:29:18.406272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.153847+0000","last_change":"2024-09-16T00:30:26.085398+0000","last_active":"2024-09-16T00:30:36.153847+0000","last_peered":"2024-09-16T00:30:36.153847+0000","last_clean":"2024-09-16T00:30:36.153847+0000","last_became_active":"2024-09-16T00:30:26.085076+0000","last_became_peered":"2024-09-16T00:30:26.085076+0000","last_unstale":"2024-09-16T00:30:36.153847+0000","last_undegraded":"2024-09-16T00:30:36.153847+0000","last_fullsized":"2024-09-16T00:30:36.153847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:20:20.864660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.149403+0000","last_change":"2024-09-16T00:30:26.080953+0000","last_active":"2024-09-16T00:30:36.149403+0000","last_peered":"2024-09-16T00:30:36.149403+0000","last_clean":"2024-09-16T00:30:36.149403+0000","last_became_active":"2024-09-16T00:30:26.080185+0000","last_became_peered":"2024-09-16T00:30:26.080185+0000","last_unstale":"2024-09-16T00:30:36.149403+0000","last_undegraded":"2024-09-16T00:30:36.149403+0000","last_fullsized":"2024-09-16T00:30:36.149403+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:56.769182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.764374+0000","last_change":"2024-09-16T00:30:26.079320+0000","last_active":"2024-09-16T00:30:36.764374+0000","last_peered":"2024-09-16T00:30:36.764374+0000","last_clean":"2024-09-16T00:30:36.764374+0000","last_became_active":"2024-09-16T00:30:26.078980+0000","last_became_peered":"2024-09-16T00:30:26.078980+0000","last_unstale":"2024-09-16T00:30:36.764374+0000","last_undegraded":"2024-09-16T00:30:36.764374+0000","last_fullsized":"2024-09-16T00:30:36.764374+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:03:15.386657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765410+0000","last_change":"2024-09-16T00:30:26.084948+0000","last_active":"2024-09-16T00:30:36.765410+0000","last_peered":"2024-09-16T00:30:36.765410+0000","last_clean":"2024-09-16T00:30:36.765410+0000","last_became_active":"2024-09-16T00:30:26.084602+0000","last_became_peered":"2024-09-16T00:30:26.084602+0000","last_unstale":"2024-09-16T00:30:36.765410+0000","last_undegraded":"2024-09-16T00:30:36.765410+0000","last_fullsized":"2024-09-16T00:30:36.765410+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:01.709332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766362+0000","last_change":"2024-09-16T00:30:26.080491+0000","last_active":"2024-09-16T00:30:36.766362+0000","last_peered":"2024-09-16T00:30:36.766362+0000","last_clean":"2024-09-16T00:30:36.766362+0000","last_became_active":"2024-09-16T00:30:26.080189+0000","last_became_peered":"2024-09-16T00:30:26.080189+0000","last_unstale":"2024-09-16T00:30:36.766362+0000","last_undegraded":"2024-09-16T00:30:36.766362+0000","last_fullsized":"2024-09-16T00:30:36.766362+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:38.358919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766440+0000","last_change":"2024-09-16T00:30:26.077410+0000","last_active":"2024-09-16T00:30:36.766440+0000","last_peered":"2024-09-16T00:30:36.766440+0000","last_clean":"2024-09-16T00:30:36.766440+0000","last_became_active":"2024-09-16T00:30:26.076498+0000","last_became_peered":"2024-09-16T00:30:26.076498+0000","last_unstale":"2024-09-16T00:30:36.766440+0000","last_undegraded":"2024-09-16T00:30:36.766440+0000","last_fullsized":"2024-09-16T00:30:36.766440+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:54:17.712774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765010+0000","last_change":"2024-09-16T00:30:26.074500+0000","last_active":"2024-09-16T00:30:36.765010+0000","last_peered":"2024-09-16T00:30:36.765010+0000","last_clean":"2024-09-16T00:30:36.765010+0000","last_became_active":"2024-09-16T00:30:26.074177+0000","last_became_peered":"2024-09-16T00:30:26.074177+0000","last_unstale":"2024-09-16T00:30:36.765010+0000","last_undegraded":"2024-09-16T00:30:36.765010+0000","last_fullsized":"2024-09-16T00:30:36.765010+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:43:13.792231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.634914+0000","last_change":"2024-09-16T00:30:26.078358+0000","last_active":"2024-09-16T00:30:36.634914+0000","last_peered":"2024-09-16T00:30:36.634914+0000","last_clean":"2024-09-16T00:30:36.634914+0000","last_became_active":"2024-09-16T00:30:26.077993+0000","last_became_peered":"2024-09-16T00:30:26.077993+0000","last_unstale":"2024-09-16T00:30:36.634914+0000","last_undegraded":"2024-09-16T00:30:36.634914+0000","last_fullsized":"2024-09-16T00:30:36.634914+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:25:10.711394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.154359+0000","last_change":"2024-09-16T00:30:26.083248+0000","last_active":"2024-09-16T00:30:36.154359+0000","last_peered":"2024-09-16T00:30:36.154359+0000","last_clean":"2024-09-16T00:30:36.154359+0000","last_became_active":"2024-09-16T00:30:26.082861+0000","last_became_peered":"2024-09-16T00:30:26.082861+0000","last_unstale":"2024-09-16T00:30:36.154359+0000","last_undegraded":"2024-09-16T00:30:36.154359+0000","last_fullsized":"2024-09-16T00:30:36.154359+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:49:13.883256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765259+0000","last_change":"2024-09-16T00:30:26.086313+0000","last_active":"2024-09-16T00:30:36.765259+0000","last_peered":"2024-09-16T00:30:36.765259+0000","last_clean":"2024-09-16T00:30:36.765259+0000","last_became_active":"2024-09-16T00:30:26.085760+0000","last_became_peered":"2024-09-16T00:30:26.085760+0000","last_unstale":"2024-09-16T00:30:36.765259+0000","last_undegraded":"2024-09-16T00:30:36.765259+0000","last_fullsized":"2024-09-16T00:30:36.765259+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:40:38.236789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.764637+0000","last_change":"2024-09-16T00:30:26.085289+0000","last_active":"2024-09-16T00:30:36.764637+0000","last_peered":"2024-09-16T00:30:36.764637+0000","last_clean":"2024-09-16T00:30:36.764637+0000","last_became_active":"2024-09-16T00:30:26.084995+0000","last_became_peered":"2024-09-16T00:30:26.084995+0000","last_unstale":"2024-09-16T00:30:36.764637+0000","last_undegraded":"2024-09-16T00:30:36.764637+0000","last_fullsized":"2024-09-16T00:30:36.764637+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:45.667892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.634434+0000","last_change":"2024-09-16T00:30:26.088690+0000","last_active":"2024-09-16T00:30:36.634434+0000","last_peered":"2024-09-16T00:30:36.634434+0000","last_clean":"2024-09-16T00:30:36.634434+0000","last_became_active":"2024-09-16T00:30:26.087700+0000","last_became_peered":"2024-09-16T00:30:26.087700+0000","last_unstale":"2024-09-16T00:30:36.634434+0000","last_undegraded":"2024-09-16T00:30:36.634434+0000","last_fullsized":"2024-09-16T00:30:36.634434+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:39:51.445324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.153741+0000","last_change":"2024-09-16T00:30:26.080215+0000","last_active":"2024-09-16T00:30:36.153741+0000","last_peered":"2024-09-16T00:30:36.153741+0000","last_clean":"2024-09-16T00:30:36.153741+0000","last_became_active":"2024-09-16T00:30:26.079038+0000","last_became_peered":"2024-09-16T00:30:26.079038+0000","last_unstale":"2024-09-16T00:30:36.153741+0000","last_undegraded":"2024-09-16T00:30:36.153741+0000","last_fullsized":"2024-09-16T00:30:36.153741+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:01.573960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765084+0000","last_change":"2024-09-16T00:30:26.080966+0000","last_active":"2024-09-16T00:30:36.765084+0000","last_peered":"2024-09-16T00:30:36.765084+0000","last_clean":"2024-09-16T00:30:36.765084+0000","last_became_active":"2024-09-16T00:30:26.080641+0000","last_became_peered":"2024-09-16T00:30:26.080641+0000","last_unstale":"2024-09-16T00:30:36.765084+0000","last_undegraded":"2024-09-16T00:30:36.765084+0000","last_fullsized":"2024-09-16T00:30:36.765084+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:33:01.538169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.764921+0000","last_change":"2024-09-16T00:30:26.072644+0000","last_active":"2024-09-16T00:30:36.764921+0000","last_peered":"2024-09-16T00:30:36.764921+0000","last_clean":"2024-09-16T00:30:36.764921+0000","last_became_active":"2024-09-16T00:30:26.072336+0000","last_became_peered":"2024-09-16T00:30:26.072336+0000","last_unstale":"2024-09-16T00:30:36.764921+0000","last_undegraded":"2024-09-16T00:30:36.764921+0000","last_fullsized":"2024-09-16T00:30:36.764921+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:44:45.760057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.767442+0000","last_change":"2024-09-16T00:30:26.079155+0000","last_active":"2024-09-16T00:30:36.767442+0000","last_peered":"2024-09-16T00:30:36.767442+0000","last_clean":"2024-09-16T00:30:36.767442+0000","last_became_active":"2024-09-16T00:30:26.078820+0000","last_became_peered":"2024-09-16T00:30:26.078820+0000","last_unstale":"2024-09-16T00:30:36.767442+0000","last_undegraded":"2024-09-16T00:30:36.767442+0000","last_fullsized":"2024-09-16T00:30:36.767442+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:35.071143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766830+0000","last_change":"2024-09-16T00:30:26.081537+0000","last_active":"2024-09-16T00:30:36.766830+0000","last_peered":"2024-09-16T00:30:36.766830+0000","last_clean":"2024-09-16T00:30:36.766830+0000","last_became_active":"2024-09-16T00:30:26.080744+0000","last_became_peered":"2024-09-16T00:30:26.080744+0000","last_unstale":"2024-09-16T00:30:36.766830+0000","last_undegraded":"2024-09-16T00:30:36.766830+0000","last_fullsized":"2024-09-16T00:30:36.766830+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:46:38.179888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.149376+0000","last_change":"2024-09-16T00:30:26.081066+0000","last_active":"2024-09-16T00:30:36.149376+0000","last_peered":"2024-09-16T00:30:36.149376+0000","last_clean":"2024-09-16T00:30:36.149376+0000","last_became_active":"2024-09-16T00:30:26.080630+0000","last_became_peered":"2024-09-16T00:30:26.080630+0000","last_unstale":"2024-09-16T00:30:36.149376+0000","last_undegraded":"2024-09-16T00:30:36.149376+0000","last_fullsized":"2024-09-16T00:30:36.149376+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:44:42.644792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":47348,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":89,"num_read_kb":98,"num_write":149,"num_write_kb":91,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":368640,"data_stored":142044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":130,"ondisk_log_size":130,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514519,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":987136,"data_stored":485563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743452,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32804,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712348,"statfs":{"total":95995035648,"available":95961444352,"internally_reserved":0,"allocated":1658880,"data_stored":1116176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906977,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":489895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135911,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1003520,"data_stored":485719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397547,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28584,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716568,"statfs":{"total":95995035648,"available":95965765632,"internally_reserved":0,"allocated":1597440,"data_stored":1081630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561073,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":983040,"data_stored":480975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790005,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28592,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716560,"statfs":{"total":95995035648,"available":95965757440,"internally_reserved":0,"allocated":1605632,"data_stored":1077218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953531,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":995328,"data_stored":485625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182463,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":995328,"data_stored":481169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":525973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":524230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1007616,"data_stored":482363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":43345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":45088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":44923,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:31:19.448 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T00:31:19.703 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:19.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:19 smithi038 ceph-mon[30945]: pgmap v316: 129 pgs: 129 active+clean; 623 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.0 KiB/s wr, 10 op/s 2024-09-16T00:31:19.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:19 smithi122 ceph-mon[35412]: pgmap v316: 129 pgs: 129 active+clean; 623 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.0 KiB/s wr, 10 op/s 2024-09-16T00:31:19.793 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:19 smithi073 ceph-mon[35254]: pgmap v316: 129 pgs: 129 active+clean; 623 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 5.0 KiB/s wr, 10 op/s 2024-09-16T00:31:20.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.188+0000 7fd77b291640 1 -- 172.21.15.38:0/1691012132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd77410c170 msgr2=0x7fd77410e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:20.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.188+0000 7fd77b291640 1 --2- 172.21.15.38:0/1691012132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd77410c170 0x7fd77410e610 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fd76c009ca0 tx=0x7fd76c02a950 comp rx=0 tx=0).stop 2024-09-16T00:31:20.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.188+0000 7fd77b291640 1 -- 172.21.15.38:0/1691012132 shutdown_connections 2024-09-16T00:31:20.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.188+0000 7fd77b291640 1 --2- 172.21.15.38:0/1691012132 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd77410c170 0x7fd77410e610 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.188+0000 7fd77b291640 1 --2- 172.21.15.38:0/1691012132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd774103660 0x7fd77410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.188+0000 7fd77b291640 1 --2- 172.21.15.38:0/1691012132 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd774102d40 0x7fd774103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.188+0000 7fd77b291640 1 -- 172.21.15.38:0/1691012132 >> 172.21.15.38:0/1691012132 conn(0x7fd7740fc7a0 msgr2=0x7fd7740febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:20.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.188+0000 7fd77b291640 1 -- 172.21.15.38:0/1691012132 shutdown_connections 2024-09-16T00:31:20.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.188+0000 7fd77b291640 1 -- 172.21.15.38:0/1691012132 wait complete. 2024-09-16T00:31:20.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.189+0000 7fd77b291640 1 Processor -- start 2024-09-16T00:31:20.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.189+0000 7fd77b291640 1 -- start start 2024-09-16T00:31:20.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.189+0000 7fd77b291640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd774102d40 0x7fd774198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:20.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.189+0000 7fd77b291640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd774103660 0x7fd774198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:20.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.189+0000 7fd77b291640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd77410c170 0x7fd77419cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:20.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.189+0000 7fd77b291640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd774111f70 con 0x7fd77410c170 2024-09-16T00:31:20.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.189+0000 7fd77b291640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd774111df0 con 0x7fd774102d40 2024-09-16T00:31:20.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.189+0000 7fd77b291640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd7741120f0 con 0x7fd774103660 2024-09-16T00:31:20.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.189+0000 7fd77982e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd77410c170 0x7fd77419cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:20.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.189+0000 7fd77982e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd77410c170 0x7fd77419cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44100/0 (socket says 172.21.15.38:44100) 2024-09-16T00:31:20.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.189+0000 7fd77982e640 1 -- 172.21.15.38:0/648407186 learned_addr learned my addr 172.21.15.38:0/648407186 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:31:20.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.189+0000 7fd77882c640 1 --2- 172.21.15.38:0/648407186 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd774103660 0x7fd774198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:20.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.189+0000 7fd77902d640 1 --2- 172.21.15.38:0/648407186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd774102d40 0x7fd774198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:20.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.189+0000 7fd77982e640 1 -- 172.21.15.38:0/648407186 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd774103660 msgr2=0x7fd774198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:20.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.189+0000 7fd77982e640 1 --2- 172.21.15.38:0/648407186 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd774103660 0x7fd774198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.189+0000 7fd77982e640 1 -- 172.21.15.38:0/648407186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd774102d40 msgr2=0x7fd774198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:20.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.189+0000 7fd77982e640 1 --2- 172.21.15.38:0/648407186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd774102d40 0x7fd774198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.189+0000 7fd77982e640 1 -- 172.21.15.38:0/648407186 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd77419d300 con 0x7fd77410c170 2024-09-16T00:31:20.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.189+0000 7fd77902d640 1 --2- 172.21.15.38:0/648407186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd774102d40 0x7fd774198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:31:20.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.190+0000 7fd77982e640 1 --2- 172.21.15.38:0/648407186 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd77410c170 0x7fd77419cc20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd76c006010 tx=0x7fd76c02f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:20.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.190+0000 7fd76a7fc640 1 -- 172.21.15.38:0/648407186 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76c034040 con 0x7fd77410c170 2024-09-16T00:31:20.209 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.190+0000 7fd76a7fc640 1 -- 172.21.15.38:0/648407186 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd76c003e00 con 0x7fd77410c170 2024-09-16T00:31:20.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.190+0000 7fd77b291640 1 -- 172.21.15.38:0/648407186 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd77419d590 con 0x7fd77410c170 2024-09-16T00:31:20.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.190+0000 7fd76a7fc640 1 -- 172.21.15.38:0/648407186 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76c0456e0 con 0x7fd77410c170 2024-09-16T00:31:20.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.190+0000 7fd77b291640 1 -- 172.21.15.38:0/648407186 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7741a4ed0 con 0x7fd77410c170 2024-09-16T00:31:20.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.191+0000 7fd77b291640 1 -- 172.21.15.38:0/648407186 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd77406b070 con 0x7fd77410c170 2024-09-16T00:31:20.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.192+0000 7fd76a7fc640 1 -- 172.21.15.38:0/648407186 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fd76c037020 con 0x7fd77410c170 2024-09-16T00:31:20.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.193+0000 7fd76a7fc640 1 --2- 172.21.15.38:0/648407186 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd7600a32d0 0x7fd7600a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:20.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.193+0000 7fd77902d640 1 --2- 172.21.15.38:0/648407186 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd7600a32d0 0x7fd7600a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:20.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.193+0000 7fd76a7fc640 1 -- 172.21.15.38:0/648407186 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7fd76c0f0d80 con 0x7fd77410c170 2024-09-16T00:31:20.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.193+0000 7fd77902d640 1 --2- 172.21.15.38:0/648407186 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd7600a32d0 0x7fd7600a5790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd7700058c0 tx=0x7fd770005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:20.210 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.196+0000 7fd76a7fc640 1 -- 172.21.15.38:0/648407186 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd76c0bd810 con 0x7fd77410c170 2024-09-16T00:31:20.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.371+0000 7fd77b291640 1 -- 172.21.15.38:0/648407186 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd7740feac0 con 0x7fd7600a32d0 2024-09-16T00:31:20.380 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.380+0000 7fd76a7fc640 1 -- 172.21.15.38:0/648407186 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340341 (secure 0 0 0) 0x7fd7740feac0 con 0x7fd7600a32d0 2024-09-16T00:31:20.384 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:31:20.385 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-16T00:31:20.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.383+0000 7fd77b291640 1 -- 172.21.15.38:0/648407186 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd7600a32d0 msgr2=0x7fd7600a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:20.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.383+0000 7fd77b291640 1 --2- 172.21.15.38:0/648407186 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd7600a32d0 0x7fd7600a5790 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fd7700058c0 tx=0x7fd770005ec0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.383+0000 7fd77b291640 1 -- 172.21.15.38:0/648407186 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd77410c170 msgr2=0x7fd77419cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:20.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.383+0000 7fd77b291640 1 --2- 172.21.15.38:0/648407186 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd77410c170 0x7fd77419cc20 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd76c006010 tx=0x7fd76c02f710 comp rx=0 tx=0).stop 2024-09-16T00:31:20.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.383+0000 7fd77b291640 1 -- 172.21.15.38:0/648407186 shutdown_connections 2024-09-16T00:31:20.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.384+0000 7fd77b291640 1 --2- 172.21.15.38:0/648407186 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd7600a32d0 0x7fd7600a5790 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.384+0000 7fd77b291640 1 --2- 172.21.15.38:0/648407186 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd77410c170 0x7fd77419cc20 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.384+0000 7fd77b291640 1 --2- 172.21.15.38:0/648407186 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd774103660 0x7fd774198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.384+0000 7fd77b291640 1 --2- 172.21.15.38:0/648407186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd774102d40 0x7fd774198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:20.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.384+0000 7fd77b291640 1 -- 172.21.15.38:0/648407186 >> 172.21.15.38:0/648407186 conn(0x7fd7740fc7a0 msgr2=0x7fd7740fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:20.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.384+0000 7fd77b291640 1 -- 172.21.15.38:0/648407186 shutdown_connections 2024-09-16T00:31:20.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:20.384+0000 7fd77b291640 1 -- 172.21.15.38:0/648407186 wait complete. 2024-09-16T00:31:20.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:20 smithi038 ceph-mon[30945]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:31:20.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:20 smithi122 ceph-mon[35412]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:31:20.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:20 smithi073 ceph-mon[35254]: from='client.14835 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:31:21.143 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-09-16T00:31:19.519019+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":651278,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":241,"num_read_kb":231,"num_write":342,"num_write_kb":3047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":276,"ondisk_log_size":276,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379504,"kb_used_data":13616,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124562320,"statfs":{"total":1151940427776,"available":1151551815680,"internally_reserved":0,"allocated":13942784,"data_stored":7729864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":53456,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":106,"num_write":32,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003058"},"pg_stats":[{"pgid":"2.1f","version":"88'9","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:02.316003+0000","last_change":"2024-09-16T00:30:23.933355+0000","last_active":"2024-09-16T00:31:02.316003+0000","last_peered":"2024-09-16T00:31:02.316003+0000","last_clean":"2024-09-16T00:31:02.316003+0000","last_became_active":"2024-09-16T00:30:23.933057+0000","last_became_peered":"2024-09-16T00:30:23.933057+0000","last_unstale":"2024-09-16T00:31:02.316003+0000","last_undegraded":"2024-09-16T00:31:02.316003+0000","last_fullsized":"2024-09-16T00:31:02.316003+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:23:01.713440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765629+0000","last_change":"2024-09-16T00:30:26.084537+0000","last_active":"2024-09-16T00:30:36.765629+0000","last_peered":"2024-09-16T00:30:36.765629+0000","last_clean":"2024-09-16T00:30:36.765629+0000","last_became_active":"2024-09-16T00:30:26.084226+0000","last_became_peered":"2024-09-16T00:30:26.084226+0000","last_unstale":"2024-09-16T00:30:36.765629+0000","last_undegraded":"2024-09-16T00:30:36.765629+0000","last_fullsized":"2024-09-16T00:30:36.765629+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:43:41.628877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765535+0000","last_change":"2024-09-16T00:30:33.154873+0000","last_active":"2024-09-16T00:30:36.765535+0000","last_peered":"2024-09-16T00:30:36.765535+0000","last_clean":"2024-09-16T00:30:36.765535+0000","last_became_active":"2024-09-16T00:30:33.154433+0000","last_became_peered":"2024-09-16T00:30:33.154433+0000","last_unstale":"2024-09-16T00:30:36.765535+0000","last_undegraded":"2024-09-16T00:30:36.765535+0000","last_fullsized":"2024-09-16T00:30:36.765535+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:50.805806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766149+0000","last_change":"2024-09-16T00:30:24.422274+0000","last_active":"2024-09-16T00:30:36.766149+0000","last_peered":"2024-09-16T00:30:36.766149+0000","last_clean":"2024-09-16T00:30:36.766149+0000","last_became_active":"2024-09-16T00:30:24.421902+0000","last_became_peered":"2024-09-16T00:30:24.421902+0000","last_unstale":"2024-09-16T00:30:36.766149+0000","last_undegraded":"2024-09-16T00:30:36.766149+0000","last_fullsized":"2024-09-16T00:30:36.766149+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:01.910187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.767336+0000","last_change":"2024-09-16T00:30:26.088165+0000","last_active":"2024-09-16T00:30:36.767336+0000","last_peered":"2024-09-16T00:30:36.767336+0000","last_clean":"2024-09-16T00:30:36.767336+0000","last_became_active":"2024-09-16T00:30:26.087593+0000","last_became_peered":"2024-09-16T00:30:26.087593+0000","last_unstale":"2024-09-16T00:30:36.767336+0000","last_undegraded":"2024-09-16T00:30:36.767336+0000","last_fullsized":"2024-09-16T00:30:36.767336+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:54.149930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.764467+0000","last_change":"2024-09-16T00:30:33.154586+0000","last_active":"2024-09-16T00:30:36.764467+0000","last_peered":"2024-09-16T00:30:36.764467+0000","last_clean":"2024-09-16T00:30:36.764467+0000","last_became_active":"2024-09-16T00:30:33.154209+0000","last_became_peered":"2024-09-16T00:30:33.154209+0000","last_unstale":"2024-09-16T00:30:36.764467+0000","last_undegraded":"2024-09-16T00:30:36.764467+0000","last_fullsized":"2024-09-16T00:30:36.764467+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:45.776895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:02.314625+0000","last_change":"2024-09-16T00:30:23.932442+0000","last_active":"2024-09-16T00:31:02.314625+0000","last_peered":"2024-09-16T00:31:02.314625+0000","last_clean":"2024-09-16T00:31:02.314625+0000","last_became_active":"2024-09-16T00:30:23.932009+0000","last_became_peered":"2024-09-16T00:30:23.932009+0000","last_unstale":"2024-09-16T00:31:02.314625+0000","last_undegraded":"2024-09-16T00:31:02.314625+0000","last_fullsized":"2024-09-16T00:31:02.314625+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:09:27.078187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765329+0000","last_change":"2024-09-16T00:30:26.087715+0000","last_active":"2024-09-16T00:30:36.765329+0000","last_peered":"2024-09-16T00:30:36.765329+0000","last_clean":"2024-09-16T00:30:36.765329+0000","last_became_active":"2024-09-16T00:30:26.087415+0000","last_became_peered":"2024-09-16T00:30:26.087415+0000","last_unstale":"2024-09-16T00:30:36.765329+0000","last_undegraded":"2024-09-16T00:30:36.765329+0000","last_fullsized":"2024-09-16T00:30:36.765329+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:52.373992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765223+0000","last_change":"2024-09-16T00:30:33.152328+0000","last_active":"2024-09-16T00:30:36.765223+0000","last_peered":"2024-09-16T00:30:36.765223+0000","last_clean":"2024-09-16T00:30:36.765223+0000","last_became_active":"2024-09-16T00:30:33.151954+0000","last_became_peered":"2024-09-16T00:30:33.151954+0000","last_unstale":"2024-09-16T00:30:36.765223+0000","last_undegraded":"2024-09-16T00:30:36.765223+0000","last_fullsized":"2024-09-16T00:30:36.765223+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:36.546729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765938+0000","last_change":"2024-09-16T00:30:23.928988+0000","last_active":"2024-09-16T00:30:36.765938+0000","last_peered":"2024-09-16T00:30:36.765938+0000","last_clean":"2024-09-16T00:30:36.765938+0000","last_became_active":"2024-09-16T00:30:23.928638+0000","last_became_peered":"2024-09-16T00:30:23.928638+0000","last_unstale":"2024-09-16T00:30:36.765938+0000","last_undegraded":"2024-09-16T00:30:36.765938+0000","last_fullsized":"2024-09-16T00:30:36.765938+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:45:58.203455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.767026+0000","last_change":"2024-09-16T00:30:26.086911+0000","last_active":"2024-09-16T00:30:36.767026+0000","last_peered":"2024-09-16T00:30:36.767026+0000","last_clean":"2024-09-16T00:30:36.767026+0000","last_became_active":"2024-09-16T00:30:26.086589+0000","last_became_peered":"2024-09-16T00:30:26.086589+0000","last_unstale":"2024-09-16T00:30:36.767026+0000","last_undegraded":"2024-09-16T00:30:36.767026+0000","last_fullsized":"2024-09-16T00:30:36.767026+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:13:10.389476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.764558+0000","last_change":"2024-09-16T00:30:33.147564+0000","last_active":"2024-09-16T00:30:36.764558+0000","last_peered":"2024-09-16T00:30:36.764558+0000","last_clean":"2024-09-16T00:30:36.764558+0000","last_became_active":"2024-09-16T00:30:33.147119+0000","last_became_peered":"2024-09-16T00:30:33.147119+0000","last_unstale":"2024-09-16T00:30:36.764558+0000","last_undegraded":"2024-09-16T00:30:36.764558+0000","last_fullsized":"2024-09-16T00:30:36.764558+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:00.509122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.634602+0000","last_change":"2024-09-16T00:30:26.080954+0000","last_active":"2024-09-16T00:30:36.634602+0000","last_peered":"2024-09-16T00:30:36.634602+0000","last_clean":"2024-09-16T00:30:36.634602+0000","last_became_active":"2024-09-16T00:30:26.080624+0000","last_became_peered":"2024-09-16T00:30:26.080624+0000","last_unstale":"2024-09-16T00:30:36.634602+0000","last_undegraded":"2024-09-16T00:30:36.634602+0000","last_fullsized":"2024-09-16T00:30:36.634602+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:05:10.440980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:01.303450+0000","last_change":"2024-09-16T00:30:23.933715+0000","last_active":"2024-09-16T00:31:01.303450+0000","last_peered":"2024-09-16T00:31:01.303450+0000","last_clean":"2024-09-16T00:31:01.303450+0000","last_became_active":"2024-09-16T00:30:23.933275+0000","last_became_peered":"2024-09-16T00:30:23.933275+0000","last_unstale":"2024-09-16T00:31:01.303450+0000","last_undegraded":"2024-09-16T00:31:01.303450+0000","last_fullsized":"2024-09-16T00:31:01.303450+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:40:44.846053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.764833+0000","last_change":"2024-09-16T00:30:33.604602+0000","last_active":"2024-09-16T00:30:36.764833+0000","last_peered":"2024-09-16T00:30:36.764833+0000","last_clean":"2024-09-16T00:30:36.764833+0000","last_became_active":"2024-09-16T00:30:33.604107+0000","last_became_peered":"2024-09-16T00:30:33.604107+0000","last_unstale":"2024-09-16T00:30:36.764833+0000","last_undegraded":"2024-09-16T00:30:36.764833+0000","last_fullsized":"2024-09-16T00:30:36.764833+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:41.254445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765248+0000","last_change":"2024-09-16T00:30:26.080612+0000","last_active":"2024-09-16T00:30:36.765248+0000","last_peered":"2024-09-16T00:30:36.765248+0000","last_clean":"2024-09-16T00:30:36.765248+0000","last_became_active":"2024-09-16T00:30:26.080317+0000","last_became_peered":"2024-09-16T00:30:26.080317+0000","last_unstale":"2024-09-16T00:30:36.765248+0000","last_undegraded":"2024-09-16T00:30:36.765248+0000","last_fullsized":"2024-09-16T00:30:36.765248+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:23.814241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766325+0000","last_change":"2024-09-16T00:30:23.936778+0000","last_active":"2024-09-16T00:30:36.766325+0000","last_peered":"2024-09-16T00:30:36.766325+0000","last_clean":"2024-09-16T00:30:36.766325+0000","last_became_active":"2024-09-16T00:30:23.936265+0000","last_became_peered":"2024-09-16T00:30:23.936265+0000","last_unstale":"2024-09-16T00:30:36.766325+0000","last_undegraded":"2024-09-16T00:30:36.766325+0000","last_fullsized":"2024-09-16T00:30:36.766325+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:35:28.288476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765465+0000","last_change":"2024-09-16T00:30:33.148340+0000","last_active":"2024-09-16T00:30:36.765465+0000","last_peered":"2024-09-16T00:30:36.765465+0000","last_clean":"2024-09-16T00:30:36.765465+0000","last_became_active":"2024-09-16T00:30:33.147289+0000","last_became_peered":"2024-09-16T00:30:33.147289+0000","last_unstale":"2024-09-16T00:30:36.765465+0000","last_undegraded":"2024-09-16T00:30:36.765465+0000","last_fullsized":"2024-09-16T00:30:36.765465+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:21.259254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.149061+0000","last_change":"2024-09-16T00:30:26.077325+0000","last_active":"2024-09-16T00:30:36.149061+0000","last_peered":"2024-09-16T00:30:36.149061+0000","last_clean":"2024-09-16T00:30:36.149061+0000","last_became_active":"2024-09-16T00:30:26.077002+0000","last_became_peered":"2024-09-16T00:30:26.077002+0000","last_unstale":"2024-09-16T00:30:36.149061+0000","last_undegraded":"2024-09-16T00:30:36.149061+0000","last_fullsized":"2024-09-16T00:30:36.149061+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:12:38.569247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:01.304089+0000","last_change":"2024-09-16T00:30:23.933025+0000","last_active":"2024-09-16T00:31:01.304089+0000","last_peered":"2024-09-16T00:31:01.304089+0000","last_clean":"2024-09-16T00:31:01.304089+0000","last_became_active":"2024-09-16T00:30:23.932669+0000","last_became_peered":"2024-09-16T00:30:23.932669+0000","last_unstale":"2024-09-16T00:31:01.304089+0000","last_undegraded":"2024-09-16T00:31:01.304089+0000","last_fullsized":"2024-09-16T00:31:01.304089+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:20.593665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.153862+0000","last_change":"2024-09-16T00:30:33.151844+0000","last_active":"2024-09-16T00:30:36.153862+0000","last_peered":"2024-09-16T00:30:36.153862+0000","last_clean":"2024-09-16T00:30:36.153862+0000","last_became_active":"2024-09-16T00:30:33.151311+0000","last_became_peered":"2024-09-16T00:30:33.151311+0000","last_unstale":"2024-09-16T00:30:36.153862+0000","last_undegraded":"2024-09-16T00:30:36.153862+0000","last_fullsized":"2024-09-16T00:30:36.153862+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:51.785233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765023+0000","last_change":"2024-09-16T00:30:26.076104+0000","last_active":"2024-09-16T00:30:36.765023+0000","last_peered":"2024-09-16T00:30:36.765023+0000","last_clean":"2024-09-16T00:30:36.765023+0000","last_became_active":"2024-09-16T00:30:26.075764+0000","last_became_peered":"2024-09-16T00:30:26.075764+0000","last_unstale":"2024-09-16T00:30:36.765023+0000","last_undegraded":"2024-09-16T00:30:36.765023+0000","last_fullsized":"2024-09-16T00:30:36.765023+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:31.228533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:01.301696+0000","last_change":"2024-09-16T00:30:24.422411+0000","last_active":"2024-09-16T00:31:01.301696+0000","last_peered":"2024-09-16T00:31:01.301696+0000","last_clean":"2024-09-16T00:31:01.301696+0000","last_became_active":"2024-09-16T00:30:24.422102+0000","last_became_peered":"2024-09-16T00:30:24.422102+0000","last_unstale":"2024-09-16T00:31:01.301696+0000","last_undegraded":"2024-09-16T00:31:01.301696+0000","last_fullsized":"2024-09-16T00:31:01.301696+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:06:00.572070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765253+0000","last_change":"2024-09-16T00:30:33.605397+0000","last_active":"2024-09-16T00:30:36.765253+0000","last_peered":"2024-09-16T00:30:36.765253+0000","last_clean":"2024-09-16T00:30:36.765253+0000","last_became_active":"2024-09-16T00:30:33.604978+0000","last_became_peered":"2024-09-16T00:30:33.604978+0000","last_unstale":"2024-09-16T00:30:36.765253+0000","last_undegraded":"2024-09-16T00:30:36.765253+0000","last_fullsized":"2024-09-16T00:30:36.765253+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:35:16.766056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766891+0000","last_change":"2024-09-16T00:30:26.082684+0000","last_active":"2024-09-16T00:30:36.766891+0000","last_peered":"2024-09-16T00:30:36.766891+0000","last_clean":"2024-09-16T00:30:36.766891+0000","last_became_active":"2024-09-16T00:30:26.082335+0000","last_became_peered":"2024-09-16T00:30:26.082335+0000","last_unstale":"2024-09-16T00:30:36.766891+0000","last_undegraded":"2024-09-16T00:30:36.766891+0000","last_fullsized":"2024-09-16T00:30:36.766891+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:51:21.389627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:01.302667+0000","last_change":"2024-09-16T00:30:24.587840+0000","last_active":"2024-09-16T00:31:01.302667+0000","last_peered":"2024-09-16T00:31:01.302667+0000","last_clean":"2024-09-16T00:31:01.302667+0000","last_became_active":"2024-09-16T00:30:24.587436+0000","last_became_peered":"2024-09-16T00:30:24.587436+0000","last_unstale":"2024-09-16T00:31:01.302667+0000","last_undegraded":"2024-09-16T00:31:01.302667+0000","last_fullsized":"2024-09-16T00:31:01.302667+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:06.758585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765780+0000","last_change":"2024-09-16T00:30:33.145051+0000","last_active":"2024-09-16T00:30:36.765780+0000","last_peered":"2024-09-16T00:30:36.765780+0000","last_clean":"2024-09-16T00:30:36.765780+0000","last_became_active":"2024-09-16T00:30:33.144585+0000","last_became_peered":"2024-09-16T00:30:33.144585+0000","last_unstale":"2024-09-16T00:30:36.765780+0000","last_undegraded":"2024-09-16T00:30:36.765780+0000","last_fullsized":"2024-09-16T00:30:36.765780+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:48:23.588495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766177+0000","last_change":"2024-09-16T00:30:26.078830+0000","last_active":"2024-09-16T00:30:36.766177+0000","last_peered":"2024-09-16T00:30:36.766177+0000","last_clean":"2024-09-16T00:30:36.766177+0000","last_became_active":"2024-09-16T00:30:26.078520+0000","last_became_peered":"2024-09-16T00:30:26.078520+0000","last_unstale":"2024-09-16T00:30:36.766177+0000","last_undegraded":"2024-09-16T00:30:36.766177+0000","last_fullsized":"2024-09-16T00:30:36.766177+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:26:07.790519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'9","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:17.076353+0000","last_change":"2024-09-16T00:30:23.933284+0000","last_active":"2024-09-16T00:31:17.076353+0000","last_peered":"2024-09-16T00:31:17.076353+0000","last_clean":"2024-09-16T00:31:17.076353+0000","last_became_active":"2024-09-16T00:30:23.932920+0000","last_became_peered":"2024-09-16T00:30:23.932920+0000","last_unstale":"2024-09-16T00:31:17.076353+0000","last_undegraded":"2024-09-16T00:31:17.076353+0000","last_fullsized":"2024-09-16T00:31:17.076353+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:38.199391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.764616+0000","last_change":"2024-09-16T00:30:33.604308+0000","last_active":"2024-09-16T00:30:36.764616+0000","last_peered":"2024-09-16T00:30:36.764616+0000","last_clean":"2024-09-16T00:30:36.764616+0000","last_became_active":"2024-09-16T00:30:33.603889+0000","last_became_peered":"2024-09-16T00:30:33.603889+0000","last_unstale":"2024-09-16T00:30:36.764616+0000","last_undegraded":"2024-09-16T00:30:36.764616+0000","last_fullsized":"2024-09-16T00:30:36.764616+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:32:55.173063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766489+0000","last_change":"2024-09-16T00:30:26.078515+0000","last_active":"2024-09-16T00:30:36.766489+0000","last_peered":"2024-09-16T00:30:36.766489+0000","last_clean":"2024-09-16T00:30:36.766489+0000","last_became_active":"2024-09-16T00:30:26.078173+0000","last_became_peered":"2024-09-16T00:30:26.078173+0000","last_unstale":"2024-09-16T00:30:36.766489+0000","last_undegraded":"2024-09-16T00:30:36.766489+0000","last_fullsized":"2024-09-16T00:30:36.766489+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:40:31.471154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:01.300620+0000","last_change":"2024-09-16T00:30:24.423209+0000","last_active":"2024-09-16T00:31:01.300620+0000","last_peered":"2024-09-16T00:31:01.300620+0000","last_clean":"2024-09-16T00:31:01.300620+0000","last_became_active":"2024-09-16T00:30:24.422157+0000","last_became_peered":"2024-09-16T00:30:24.422157+0000","last_unstale":"2024-09-16T00:31:01.300620+0000","last_undegraded":"2024-09-16T00:31:01.300620+0000","last_fullsized":"2024-09-16T00:31:01.300620+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:28.262392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.149166+0000","last_change":"2024-09-16T00:30:33.146204+0000","last_active":"2024-09-16T00:30:36.149166+0000","last_peered":"2024-09-16T00:30:36.149166+0000","last_clean":"2024-09-16T00:30:36.149166+0000","last_became_active":"2024-09-16T00:30:33.145553+0000","last_became_peered":"2024-09-16T00:30:33.145553+0000","last_unstale":"2024-09-16T00:30:36.149166+0000","last_undegraded":"2024-09-16T00:30:36.149166+0000","last_fullsized":"2024-09-16T00:30:36.149166+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:22.096050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.153876+0000","last_change":"2024-09-16T00:30:26.070153+0000","last_active":"2024-09-16T00:30:36.153876+0000","last_peered":"2024-09-16T00:30:36.153876+0000","last_clean":"2024-09-16T00:30:36.153876+0000","last_became_active":"2024-09-16T00:30:26.069792+0000","last_became_peered":"2024-09-16T00:30:26.069792+0000","last_unstale":"2024-09-16T00:30:36.153876+0000","last_undegraded":"2024-09-16T00:30:36.153876+0000","last_fullsized":"2024-09-16T00:30:36.153876+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:28.164242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":30,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.153735+0000","last_change":"2024-09-16T00:30:24.423261+0000","last_active":"2024-09-16T00:30:36.153735+0000","last_peered":"2024-09-16T00:30:36.153735+0000","last_clean":"2024-09-16T00:30:36.153735+0000","last_became_active":"2024-09-16T00:30:24.422650+0000","last_became_peered":"2024-09-16T00:30:24.422650+0000","last_unstale":"2024-09-16T00:30:36.153735+0000","last_undegraded":"2024-09-16T00:30:36.153735+0000","last_fullsized":"2024-09-16T00:30:36.153735+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:00.211744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765204+0000","last_change":"2024-09-16T00:30:33.604492+0000","last_active":"2024-09-16T00:30:36.765204+0000","last_peered":"2024-09-16T00:30:36.765204+0000","last_clean":"2024-09-16T00:30:36.765204+0000","last_became_active":"2024-09-16T00:30:33.604015+0000","last_became_peered":"2024-09-16T00:30:33.604015+0000","last_unstale":"2024-09-16T00:30:36.765204+0000","last_undegraded":"2024-09-16T00:30:36.765204+0000","last_fullsized":"2024-09-16T00:30:36.765204+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:40:22.516583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.153854+0000","last_change":"2024-09-16T00:30:26.075517+0000","last_active":"2024-09-16T00:30:36.153854+0000","last_peered":"2024-09-16T00:30:36.153854+0000","last_clean":"2024-09-16T00:30:36.153854+0000","last_became_active":"2024-09-16T00:30:26.075113+0000","last_became_peered":"2024-09-16T00:30:26.075113+0000","last_unstale":"2024-09-16T00:30:36.153854+0000","last_undegraded":"2024-09-16T00:30:36.153854+0000","last_fullsized":"2024-09-16T00:30:36.153854+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:11.427010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.634735+0000","last_change":"2024-09-16T00:30:23.933639+0000","last_active":"2024-09-16T00:30:36.634735+0000","last_peered":"2024-09-16T00:30:36.634735+0000","last_clean":"2024-09-16T00:30:36.634735+0000","last_became_active":"2024-09-16T00:30:23.933322+0000","last_became_peered":"2024-09-16T00:30:23.933322+0000","last_unstale":"2024-09-16T00:30:36.634735+0000","last_undegraded":"2024-09-16T00:30:36.634735+0000","last_fullsized":"2024-09-16T00:30:36.634735+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:59:59.082702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766116+0000","last_change":"2024-09-16T00:30:33.148508+0000","last_active":"2024-09-16T00:30:36.766116+0000","last_peered":"2024-09-16T00:30:36.766116+0000","last_clean":"2024-09-16T00:30:36.766116+0000","last_became_active":"2024-09-16T00:30:33.148091+0000","last_became_peered":"2024-09-16T00:30:33.148091+0000","last_unstale":"2024-09-16T00:30:36.766116+0000","last_undegraded":"2024-09-16T00:30:36.766116+0000","last_fullsized":"2024-09-16T00:30:36.766116+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:50.052526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766765+0000","last_change":"2024-09-16T00:30:26.080374+0000","last_active":"2024-09-16T00:30:36.766765+0000","last_peered":"2024-09-16T00:30:36.766765+0000","last_clean":"2024-09-16T00:30:36.766765+0000","last_became_active":"2024-09-16T00:30:26.080044+0000","last_became_peered":"2024-09-16T00:30:26.080044+0000","last_unstale":"2024-09-16T00:30:36.766765+0000","last_undegraded":"2024-09-16T00:30:36.766765+0000","last_fullsized":"2024-09-16T00:30:36.766765+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:38:36.034903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:13.345304+0000","last_change":"2024-09-16T00:30:24.422137+0000","last_active":"2024-09-16T00:31:13.345304+0000","last_peered":"2024-09-16T00:31:13.345304+0000","last_clean":"2024-09-16T00:31:13.345304+0000","last_became_active":"2024-09-16T00:30:24.421778+0000","last_became_peered":"2024-09-16T00:30:24.421778+0000","last_unstale":"2024-09-16T00:31:13.345304+0000","last_undegraded":"2024-09-16T00:31:13.345304+0000","last_fullsized":"2024-09-16T00:31:13.345304+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:54:21.964099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":12,"num_read_kb":12,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766397+0000","last_change":"2024-09-16T00:30:33.604263+0000","last_active":"2024-09-16T00:30:36.766397+0000","last_peered":"2024-09-16T00:30:36.766397+0000","last_clean":"2024-09-16T00:30:36.766397+0000","last_became_active":"2024-09-16T00:30:33.603804+0000","last_became_peered":"2024-09-16T00:30:33.603804+0000","last_unstale":"2024-09-16T00:30:36.766397+0000","last_undegraded":"2024-09-16T00:30:36.766397+0000","last_fullsized":"2024-09-16T00:30:36.766397+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:08:37.590827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765524+0000","last_change":"2024-09-16T00:30:26.082916+0000","last_active":"2024-09-16T00:30:36.765524+0000","last_peered":"2024-09-16T00:30:36.765524+0000","last_clean":"2024-09-16T00:30:36.765524+0000","last_became_active":"2024-09-16T00:30:26.082606+0000","last_became_peered":"2024-09-16T00:30:26.082606+0000","last_unstale":"2024-09-16T00:30:36.765524+0000","last_undegraded":"2024-09-16T00:30:36.765524+0000","last_fullsized":"2024-09-16T00:30:36.765524+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:56.777640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:02.314276+0000","last_change":"2024-09-16T00:30:23.937602+0000","last_active":"2024-09-16T00:31:02.314276+0000","last_peered":"2024-09-16T00:31:02.314276+0000","last_clean":"2024-09-16T00:31:02.314276+0000","last_became_active":"2024-09-16T00:30:23.937261+0000","last_became_peered":"2024-09-16T00:30:23.937261+0000","last_unstale":"2024-09-16T00:31:02.314276+0000","last_undegraded":"2024-09-16T00:31:02.314276+0000","last_fullsized":"2024-09-16T00:31:02.314276+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:24.698758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765812+0000","last_change":"2024-09-16T00:30:33.605536+0000","last_active":"2024-09-16T00:30:36.765812+0000","last_peered":"2024-09-16T00:30:36.765812+0000","last_clean":"2024-09-16T00:30:36.765812+0000","last_became_active":"2024-09-16T00:30:33.605073+0000","last_became_peered":"2024-09-16T00:30:33.605073+0000","last_unstale":"2024-09-16T00:30:36.765812+0000","last_undegraded":"2024-09-16T00:30:36.765812+0000","last_fullsized":"2024-09-16T00:30:36.765812+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:11.143052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765181+0000","last_change":"2024-09-16T00:30:26.084507+0000","last_active":"2024-09-16T00:30:36.765181+0000","last_peered":"2024-09-16T00:30:36.765181+0000","last_clean":"2024-09-16T00:30:36.765181+0000","last_became_active":"2024-09-16T00:30:26.084152+0000","last_became_peered":"2024-09-16T00:30:26.084152+0000","last_unstale":"2024-09-16T00:30:36.765181+0000","last_undegraded":"2024-09-16T00:30:36.765181+0000","last_fullsized":"2024-09-16T00:30:36.765181+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:27:21.557409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:03.327725+0000","last_change":"2024-09-16T00:30:23.932686+0000","last_active":"2024-09-16T00:31:03.327725+0000","last_peered":"2024-09-16T00:31:03.327725+0000","last_clean":"2024-09-16T00:31:03.327725+0000","last_became_active":"2024-09-16T00:30:23.932343+0000","last_became_peered":"2024-09-16T00:30:23.932343+0000","last_unstale":"2024-09-16T00:31:03.327725+0000","last_undegraded":"2024-09-16T00:31:03.327725+0000","last_fullsized":"2024-09-16T00:31:03.327725+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:13:44.185712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":342,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.634509+0000","last_change":"2024-09-16T00:29:11.143642+0000","last_active":"2024-09-16T00:30:36.634509+0000","last_peered":"2024-09-16T00:30:36.634509+0000","last_clean":"2024-09-16T00:30:36.634509+0000","last_became_active":"2024-09-16T00:29:11.143227+0000","last_became_peered":"2024-09-16T00:29:11.143227+0000","last_unstale":"2024-09-16T00:30:36.634509+0000","last_undegraded":"2024-09-16T00:30:36.634509+0000","last_fullsized":"2024-09-16T00:30:36.634509+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:25:42.361316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:25:42.361316+0000","last_clean_scrub_stamp":"2024-09-16T00:25:42.361316+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:29:12.740474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":18,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.634923+0000","last_change":"2024-09-16T00:30:33.155555+0000","last_active":"2024-09-16T00:30:36.634923+0000","last_peered":"2024-09-16T00:30:36.634923+0000","last_clean":"2024-09-16T00:30:36.634923+0000","last_became_active":"2024-09-16T00:30:33.155170+0000","last_became_peered":"2024-09-16T00:30:33.155170+0000","last_unstale":"2024-09-16T00:30:36.634923+0000","last_undegraded":"2024-09-16T00:30:36.634923+0000","last_fullsized":"2024-09-16T00:30:36.634923+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:34:46.214422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.634987+0000","last_change":"2024-09-16T00:30:26.076003+0000","last_active":"2024-09-16T00:30:36.634987+0000","last_peered":"2024-09-16T00:30:36.634987+0000","last_clean":"2024-09-16T00:30:36.634987+0000","last_became_active":"2024-09-16T00:30:26.075666+0000","last_became_peered":"2024-09-16T00:30:26.075666+0000","last_unstale":"2024-09-16T00:30:36.634987+0000","last_undegraded":"2024-09-16T00:30:36.634987+0000","last_fullsized":"2024-09-16T00:30:36.634987+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:30.113380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:02.315587+0000","last_change":"2024-09-16T00:30:24.587469+0000","last_active":"2024-09-16T00:31:02.315587+0000","last_peered":"2024-09-16T00:31:02.315587+0000","last_clean":"2024-09-16T00:31:02.315587+0000","last_became_active":"2024-09-16T00:30:24.587049+0000","last_became_peered":"2024-09-16T00:30:24.587049+0000","last_unstale":"2024-09-16T00:31:02.315587+0000","last_undegraded":"2024-09-16T00:31:02.315587+0000","last_fullsized":"2024-09-16T00:31:02.315587+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:17:45.934505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.764736+0000","last_change":"2024-09-16T00:30:33.145921+0000","last_active":"2024-09-16T00:30:36.764736+0000","last_peered":"2024-09-16T00:30:36.764736+0000","last_clean":"2024-09-16T00:30:36.764736+0000","last_became_active":"2024-09-16T00:30:33.145450+0000","last_became_peered":"2024-09-16T00:30:33.145450+0000","last_unstale":"2024-09-16T00:30:36.764736+0000","last_undegraded":"2024-09-16T00:30:36.764736+0000","last_fullsized":"2024-09-16T00:30:36.764736+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:53:51.698686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.634824+0000","last_change":"2024-09-16T00:30:26.087940+0000","last_active":"2024-09-16T00:30:36.634824+0000","last_peered":"2024-09-16T00:30:36.634824+0000","last_clean":"2024-09-16T00:30:36.634824+0000","last_became_active":"2024-09-16T00:30:26.087599+0000","last_became_peered":"2024-09-16T00:30:26.087599+0000","last_unstale":"2024-09-16T00:30:36.634824+0000","last_undegraded":"2024-09-16T00:30:36.634824+0000","last_fullsized":"2024-09-16T00:30:36.634824+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:43.934839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.767180+0000","last_change":"2024-09-16T00:30:23.930986+0000","last_active":"2024-09-16T00:30:36.767180+0000","last_peered":"2024-09-16T00:30:36.767180+0000","last_clean":"2024-09-16T00:30:36.767180+0000","last_became_active":"2024-09-16T00:30:23.930588+0000","last_became_peered":"2024-09-16T00:30:23.930588+0000","last_unstale":"2024-09-16T00:30:36.767180+0000","last_undegraded":"2024-09-16T00:30:36.767180+0000","last_fullsized":"2024-09-16T00:30:36.767180+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:04:41.902358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.767280+0000","last_change":"2024-09-16T00:30:33.154894+0000","last_active":"2024-09-16T00:30:36.767280+0000","last_peered":"2024-09-16T00:30:36.767280+0000","last_clean":"2024-09-16T00:30:36.767280+0000","last_became_active":"2024-09-16T00:30:33.154490+0000","last_became_peered":"2024-09-16T00:30:33.154490+0000","last_unstale":"2024-09-16T00:30:36.767280+0000","last_undegraded":"2024-09-16T00:30:36.767280+0000","last_fullsized":"2024-09-16T00:30:36.767280+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:59:53.854212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.634981+0000","last_change":"2024-09-16T00:30:26.077069+0000","last_active":"2024-09-16T00:30:36.634981+0000","last_peered":"2024-09-16T00:30:36.634981+0000","last_clean":"2024-09-16T00:30:36.634981+0000","last_became_active":"2024-09-16T00:30:26.076718+0000","last_became_peered":"2024-09-16T00:30:26.076718+0000","last_unstale":"2024-09-16T00:30:36.634981+0000","last_undegraded":"2024-09-16T00:30:36.634981+0000","last_fullsized":"2024-09-16T00:30:36.634981+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:25:37.541704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766245+0000","last_change":"2024-09-16T00:30:23.932081+0000","last_active":"2024-09-16T00:30:36.766245+0000","last_peered":"2024-09-16T00:30:36.766245+0000","last_clean":"2024-09-16T00:30:36.766245+0000","last_became_active":"2024-09-16T00:30:23.931729+0000","last_became_peered":"2024-09-16T00:30:23.931729+0000","last_unstale":"2024-09-16T00:30:36.766245+0000","last_undegraded":"2024-09-16T00:30:36.766245+0000","last_fullsized":"2024-09-16T00:30:36.766245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:20:05.178580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766427+0000","last_change":"2024-09-16T00:30:33.147738+0000","last_active":"2024-09-16T00:30:36.766427+0000","last_peered":"2024-09-16T00:30:36.766427+0000","last_clean":"2024-09-16T00:30:36.766427+0000","last_became_active":"2024-09-16T00:30:33.147326+0000","last_became_peered":"2024-09-16T00:30:33.147326+0000","last_unstale":"2024-09-16T00:30:36.766427+0000","last_undegraded":"2024-09-16T00:30:36.766427+0000","last_fullsized":"2024-09-16T00:30:36.766427+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:44:55.123980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.149267+0000","last_change":"2024-09-16T00:30:26.079312+0000","last_active":"2024-09-16T00:30:36.149267+0000","last_peered":"2024-09-16T00:30:36.149267+0000","last_clean":"2024-09-16T00:30:36.149267+0000","last_became_active":"2024-09-16T00:30:26.078986+0000","last_became_peered":"2024-09-16T00:30:26.078986+0000","last_unstale":"2024-09-16T00:30:36.149267+0000","last_undegraded":"2024-09-16T00:30:36.149267+0000","last_fullsized":"2024-09-16T00:30:36.149267+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:02:06.602850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:03.328650+0000","last_change":"2024-09-16T00:30:23.931313+0000","last_active":"2024-09-16T00:31:03.328650+0000","last_peered":"2024-09-16T00:31:03.328650+0000","last_clean":"2024-09-16T00:31:03.328650+0000","last_became_active":"2024-09-16T00:30:23.930296+0000","last_became_peered":"2024-09-16T00:30:23.930296+0000","last_unstale":"2024-09-16T00:31:03.328650+0000","last_undegraded":"2024-09-16T00:31:03.328650+0000","last_fullsized":"2024-09-16T00:31:03.328650+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:21:30.006861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766732+0000","last_change":"2024-09-16T00:30:33.149154+0000","last_active":"2024-09-16T00:30:36.766732+0000","last_peered":"2024-09-16T00:30:36.766732+0000","last_clean":"2024-09-16T00:30:36.766732+0000","last_became_active":"2024-09-16T00:30:33.148763+0000","last_became_peered":"2024-09-16T00:30:33.148763+0000","last_unstale":"2024-09-16T00:30:36.766732+0000","last_undegraded":"2024-09-16T00:30:36.766732+0000","last_fullsized":"2024-09-16T00:30:36.766732+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:58:05.884919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766775+0000","last_change":"2024-09-16T00:30:26.086905+0000","last_active":"2024-09-16T00:30:36.766775+0000","last_peered":"2024-09-16T00:30:36.766775+0000","last_clean":"2024-09-16T00:30:36.766775+0000","last_became_active":"2024-09-16T00:30:26.086519+0000","last_became_peered":"2024-09-16T00:30:26.086519+0000","last_unstale":"2024-09-16T00:30:36.766775+0000","last_undegraded":"2024-09-16T00:30:36.766775+0000","last_fullsized":"2024-09-16T00:30:36.766775+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:51:08.787679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:02.315689+0000","last_change":"2024-09-16T00:30:24.422893+0000","last_active":"2024-09-16T00:31:02.315689+0000","last_peered":"2024-09-16T00:31:02.315689+0000","last_clean":"2024-09-16T00:31:02.315689+0000","last_became_active":"2024-09-16T00:30:24.422073+0000","last_became_peered":"2024-09-16T00:30:24.422073+0000","last_unstale":"2024-09-16T00:31:02.315689+0000","last_undegraded":"2024-09-16T00:31:02.315689+0000","last_fullsized":"2024-09-16T00:31:02.315689+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:15:11.375957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765949+0000","last_change":"2024-09-16T00:30:33.152582+0000","last_active":"2024-09-16T00:30:36.765949+0000","last_peered":"2024-09-16T00:30:36.765949+0000","last_clean":"2024-09-16T00:30:36.765949+0000","last_became_active":"2024-09-16T00:30:33.152180+0000","last_became_peered":"2024-09-16T00:30:33.152180+0000","last_unstale":"2024-09-16T00:30:36.765949+0000","last_undegraded":"2024-09-16T00:30:36.765949+0000","last_fullsized":"2024-09-16T00:30:36.765949+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:41:10.499391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.634299+0000","last_change":"2024-09-16T00:30:26.086052+0000","last_active":"2024-09-16T00:30:36.634299+0000","last_peered":"2024-09-16T00:30:36.634299+0000","last_clean":"2024-09-16T00:30:36.634299+0000","last_became_active":"2024-09-16T00:30:26.085756+0000","last_became_peered":"2024-09-16T00:30:26.085756+0000","last_unstale":"2024-09-16T00:30:36.634299+0000","last_undegraded":"2024-09-16T00:30:36.634299+0000","last_fullsized":"2024-09-16T00:30:36.634299+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:52.582166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:16.350248+0000","last_change":"2024-09-16T00:30:23.933528+0000","last_active":"2024-09-16T00:31:16.350248+0000","last_peered":"2024-09-16T00:31:16.350248+0000","last_clean":"2024-09-16T00:31:16.350248+0000","last_became_active":"2024-09-16T00:30:23.933186+0000","last_became_peered":"2024-09-16T00:30:23.933186+0000","last_unstale":"2024-09-16T00:31:16.350248+0000","last_undegraded":"2024-09-16T00:31:16.350248+0000","last_fullsized":"2024-09-16T00:31:16.350248+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:11:00.489676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.153953+0000","last_change":"2024-09-16T00:30:33.154711+0000","last_active":"2024-09-16T00:30:36.153953+0000","last_peered":"2024-09-16T00:30:36.153953+0000","last_clean":"2024-09-16T00:30:36.153953+0000","last_became_active":"2024-09-16T00:30:33.154112+0000","last_became_peered":"2024-09-16T00:30:33.154112+0000","last_unstale":"2024-09-16T00:30:36.153953+0000","last_undegraded":"2024-09-16T00:30:36.153953+0000","last_fullsized":"2024-09-16T00:30:36.153953+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:54:50.767406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.634689+0000","last_change":"2024-09-16T00:30:26.084509+0000","last_active":"2024-09-16T00:30:36.634689+0000","last_peered":"2024-09-16T00:30:36.634689+0000","last_clean":"2024-09-16T00:30:36.634689+0000","last_became_active":"2024-09-16T00:30:26.084188+0000","last_became_peered":"2024-09-16T00:30:26.084188+0000","last_unstale":"2024-09-16T00:30:36.634689+0000","last_undegraded":"2024-09-16T00:30:36.634689+0000","last_fullsized":"2024-09-16T00:30:36.634689+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:21:48.981277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:01.303518+0000","last_change":"2024-09-16T00:30:23.933499+0000","last_active":"2024-09-16T00:31:01.303518+0000","last_peered":"2024-09-16T00:31:01.303518+0000","last_clean":"2024-09-16T00:31:01.303518+0000","last_became_active":"2024-09-16T00:30:23.933082+0000","last_became_peered":"2024-09-16T00:30:23.933082+0000","last_unstale":"2024-09-16T00:31:01.303518+0000","last_undegraded":"2024-09-16T00:31:01.303518+0000","last_fullsized":"2024-09-16T00:31:01.303518+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:42:18.998433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.149255+0000","last_change":"2024-09-16T00:30:33.154748+0000","last_active":"2024-09-16T00:30:36.149255+0000","last_peered":"2024-09-16T00:30:36.149255+0000","last_clean":"2024-09-16T00:30:36.149255+0000","last_became_active":"2024-09-16T00:30:33.154305+0000","last_became_peered":"2024-09-16T00:30:33.154305+0000","last_unstale":"2024-09-16T00:30:36.149255+0000","last_undegraded":"2024-09-16T00:30:36.149255+0000","last_fullsized":"2024-09-16T00:30:36.149255+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:57:34.651844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765961+0000","last_change":"2024-09-16T00:30:26.080517+0000","last_active":"2024-09-16T00:30:36.765961+0000","last_peered":"2024-09-16T00:30:36.765961+0000","last_clean":"2024-09-16T00:30:36.765961+0000","last_became_active":"2024-09-16T00:30:26.080197+0000","last_became_peered":"2024-09-16T00:30:26.080197+0000","last_unstale":"2024-09-16T00:30:36.765961+0000","last_undegraded":"2024-09-16T00:30:36.765961+0000","last_fullsized":"2024-09-16T00:30:36.765961+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:15.508562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":37,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:02.314135+0000","last_change":"2024-09-16T00:30:23.933275+0000","last_active":"2024-09-16T00:31:02.314135+0000","last_peered":"2024-09-16T00:31:02.314135+0000","last_clean":"2024-09-16T00:31:02.314135+0000","last_became_active":"2024-09-16T00:30:23.932609+0000","last_became_peered":"2024-09-16T00:30:23.932609+0000","last_unstale":"2024-09-16T00:31:02.314135+0000","last_undegraded":"2024-09-16T00:31:02.314135+0000","last_fullsized":"2024-09-16T00:31:02.314135+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:59:19.896758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766830+0000","last_change":"2024-09-16T00:30:33.145803+0000","last_active":"2024-09-16T00:30:36.766830+0000","last_peered":"2024-09-16T00:30:36.766830+0000","last_clean":"2024-09-16T00:30:36.766830+0000","last_became_active":"2024-09-16T00:30:33.145366+0000","last_became_peered":"2024-09-16T00:30:33.145366+0000","last_unstale":"2024-09-16T00:30:36.766830+0000","last_undegraded":"2024-09-16T00:30:36.766830+0000","last_fullsized":"2024-09-16T00:30:36.766830+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:47:28.689305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765528+0000","last_change":"2024-09-16T00:30:26.079415+0000","last_active":"2024-09-16T00:30:36.765528+0000","last_peered":"2024-09-16T00:30:36.765528+0000","last_clean":"2024-09-16T00:30:36.765528+0000","last_became_active":"2024-09-16T00:30:26.079075+0000","last_became_peered":"2024-09-16T00:30:26.079075+0000","last_unstale":"2024-09-16T00:30:36.765528+0000","last_undegraded":"2024-09-16T00:30:36.765528+0000","last_fullsized":"2024-09-16T00:30:36.765528+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:18:18.236342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"88'29","reported_seq":97,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:18.353991+0000","last_change":"2024-09-16T00:30:23.933172+0000","last_active":"2024-09-16T00:31:18.353991+0000","last_peered":"2024-09-16T00:31:18.353991+0000","last_clean":"2024-09-16T00:31:18.353991+0000","last_became_active":"2024-09-16T00:30:23.932492+0000","last_became_peered":"2024-09-16T00:30:23.932492+0000","last_unstale":"2024-09-16T00:31:18.353991+0000","last_undegraded":"2024-09-16T00:31:18.353991+0000","last_fullsized":"2024-09-16T00:31:18.353991+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:29.169448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":56673,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":75,"num_write":30,"num_write_kb":67,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.153383+0000","last_change":"2024-09-16T00:30:33.151089+0000","last_active":"2024-09-16T00:30:36.153383+0000","last_peered":"2024-09-16T00:30:36.153383+0000","last_clean":"2024-09-16T00:30:36.153383+0000","last_became_active":"2024-09-16T00:30:33.150642+0000","last_became_peered":"2024-09-16T00:30:33.150642+0000","last_unstale":"2024-09-16T00:30:36.153383+0000","last_undegraded":"2024-09-16T00:30:36.153383+0000","last_fullsized":"2024-09-16T00:30:36.153383+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:24:31.022949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765334+0000","last_change":"2024-09-16T00:30:26.085419+0000","last_active":"2024-09-16T00:30:36.765334+0000","last_peered":"2024-09-16T00:30:36.765334+0000","last_clean":"2024-09-16T00:30:36.765334+0000","last_became_active":"2024-09-16T00:30:26.085103+0000","last_became_peered":"2024-09-16T00:30:26.085103+0000","last_unstale":"2024-09-16T00:30:36.765334+0000","last_undegraded":"2024-09-16T00:30:36.765334+0000","last_fullsized":"2024-09-16T00:30:36.765334+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:56:26.352779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":35,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:01.304909+0000","last_change":"2024-09-16T00:30:24.422646+0000","last_active":"2024-09-16T00:31:01.304909+0000","last_peered":"2024-09-16T00:31:01.304909+0000","last_clean":"2024-09-16T00:31:01.304909+0000","last_became_active":"2024-09-16T00:30:24.422065+0000","last_became_peered":"2024-09-16T00:30:24.422065+0000","last_unstale":"2024-09-16T00:31:01.304909+0000","last_undegraded":"2024-09-16T00:31:01.304909+0000","last_fullsized":"2024-09-16T00:31:01.304909+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:53:12.401613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.764965+0000","last_change":"2024-09-16T00:30:33.147692+0000","last_active":"2024-09-16T00:30:36.764965+0000","last_peered":"2024-09-16T00:30:36.764965+0000","last_clean":"2024-09-16T00:30:36.764965+0000","last_became_active":"2024-09-16T00:30:33.147115+0000","last_became_peered":"2024-09-16T00:30:33.147115+0000","last_unstale":"2024-09-16T00:30:36.764965+0000","last_undegraded":"2024-09-16T00:30:36.764965+0000","last_fullsized":"2024-09-16T00:30:36.764965+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:44:04.536894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.767413+0000","last_change":"2024-09-16T00:30:26.088427+0000","last_active":"2024-09-16T00:30:36.767413+0000","last_peered":"2024-09-16T00:30:36.767413+0000","last_clean":"2024-09-16T00:30:36.767413+0000","last_became_active":"2024-09-16T00:30:26.086454+0000","last_became_peered":"2024-09-16T00:30:26.086454+0000","last_unstale":"2024-09-16T00:30:36.767413+0000","last_undegraded":"2024-09-16T00:30:36.767413+0000","last_fullsized":"2024-09-16T00:30:36.767413+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:54:29.679959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:01.302823+0000","last_change":"2024-09-16T00:30:23.936793+0000","last_active":"2024-09-16T00:31:01.302823+0000","last_peered":"2024-09-16T00:31:01.302823+0000","last_clean":"2024-09-16T00:31:01.302823+0000","last_became_active":"2024-09-16T00:30:23.936445+0000","last_became_peered":"2024-09-16T00:30:23.936445+0000","last_unstale":"2024-09-16T00:31:01.302823+0000","last_undegraded":"2024-09-16T00:31:01.302823+0000","last_fullsized":"2024-09-16T00:31:01.302823+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:33:06.645927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.153293+0000","last_change":"2024-09-16T00:30:33.156510+0000","last_active":"2024-09-16T00:30:36.153293+0000","last_peered":"2024-09-16T00:30:36.153293+0000","last_clean":"2024-09-16T00:30:36.153293+0000","last_became_active":"2024-09-16T00:30:33.156088+0000","last_became_peered":"2024-09-16T00:30:33.156088+0000","last_unstale":"2024-09-16T00:30:36.153293+0000","last_undegraded":"2024-09-16T00:30:36.153293+0000","last_fullsized":"2024-09-16T00:30:36.153293+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:34.230901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765614+0000","last_change":"2024-09-16T00:30:26.082598+0000","last_active":"2024-09-16T00:30:36.765614+0000","last_peered":"2024-09-16T00:30:36.765614+0000","last_clean":"2024-09-16T00:30:36.765614+0000","last_became_active":"2024-09-16T00:30:26.082286+0000","last_became_peered":"2024-09-16T00:30:26.082286+0000","last_unstale":"2024-09-16T00:30:36.765614+0000","last_undegraded":"2024-09-16T00:30:36.765614+0000","last_fullsized":"2024-09-16T00:30:36.765614+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:22:18.579149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":49,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:17.351638+0000","last_change":"2024-09-16T00:30:23.931097+0000","last_active":"2024-09-16T00:31:17.351638+0000","last_peered":"2024-09-16T00:31:17.351638+0000","last_clean":"2024-09-16T00:31:17.351638+0000","last_became_active":"2024-09-16T00:30:23.930719+0000","last_became_peered":"2024-09-16T00:30:23.930719+0000","last_unstale":"2024-09-16T00:31:17.351638+0000","last_undegraded":"2024-09-16T00:31:17.351638+0000","last_fullsized":"2024-09-16T00:31:17.351638+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:13:47.887754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.764905+0000","last_change":"2024-09-16T00:30:33.142581+0000","last_active":"2024-09-16T00:30:36.764905+0000","last_peered":"2024-09-16T00:30:36.764905+0000","last_clean":"2024-09-16T00:30:36.764905+0000","last_became_active":"2024-09-16T00:30:33.142263+0000","last_became_peered":"2024-09-16T00:30:33.142263+0000","last_unstale":"2024-09-16T00:30:36.764905+0000","last_undegraded":"2024-09-16T00:30:36.764905+0000","last_fullsized":"2024-09-16T00:30:36.764905+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:10:06.456011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766715+0000","last_change":"2024-09-16T00:30:33.152084+0000","last_active":"2024-09-16T00:30:36.766715+0000","last_peered":"2024-09-16T00:30:36.766715+0000","last_clean":"2024-09-16T00:30:36.766715+0000","last_became_active":"2024-09-16T00:30:33.151681+0000","last_became_peered":"2024-09-16T00:30:33.151681+0000","last_unstale":"2024-09-16T00:30:36.766715+0000","last_undegraded":"2024-09-16T00:30:36.766715+0000","last_fullsized":"2024-09-16T00:30:36.766715+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:36.966498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.153316+0000","last_change":"2024-09-16T00:30:26.084200+0000","last_active":"2024-09-16T00:30:36.153316+0000","last_peered":"2024-09-16T00:30:36.153316+0000","last_clean":"2024-09-16T00:30:36.153316+0000","last_became_active":"2024-09-16T00:30:26.083858+0000","last_became_peered":"2024-09-16T00:30:26.083858+0000","last_unstale":"2024-09-16T00:30:36.153316+0000","last_undegraded":"2024-09-16T00:30:36.153316+0000","last_fullsized":"2024-09-16T00:30:36.153316+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:42.712624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.767012+0000","last_change":"2024-09-16T00:30:23.932399+0000","last_active":"2024-09-16T00:30:36.767012+0000","last_peered":"2024-09-16T00:30:36.767012+0000","last_clean":"2024-09-16T00:30:36.767012+0000","last_became_active":"2024-09-16T00:30:23.931996+0000","last_became_peered":"2024-09-16T00:30:23.931996+0000","last_unstale":"2024-09-16T00:30:36.767012+0000","last_undegraded":"2024-09-16T00:30:36.767012+0000","last_fullsized":"2024-09-16T00:30:36.767012+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:47:09.634507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.149263+0000","last_change":"2024-09-16T00:30:33.154563+0000","last_active":"2024-09-16T00:30:36.149263+0000","last_peered":"2024-09-16T00:30:36.149263+0000","last_clean":"2024-09-16T00:30:36.149263+0000","last_became_active":"2024-09-16T00:30:33.154105+0000","last_became_peered":"2024-09-16T00:30:33.154105+0000","last_unstale":"2024-09-16T00:30:36.149263+0000","last_undegraded":"2024-09-16T00:30:36.149263+0000","last_fullsized":"2024-09-16T00:30:36.149263+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:59.966255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.764688+0000","last_change":"2024-09-16T00:30:26.075126+0000","last_active":"2024-09-16T00:30:36.764688+0000","last_peered":"2024-09-16T00:30:36.764688+0000","last_clean":"2024-09-16T00:30:36.764688+0000","last_became_active":"2024-09-16T00:30:26.074819+0000","last_became_peered":"2024-09-16T00:30:26.074819+0000","last_unstale":"2024-09-16T00:30:36.764688+0000","last_undegraded":"2024-09-16T00:30:36.764688+0000","last_fullsized":"2024-09-16T00:30:36.764688+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:36.996346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:02.315266+0000","last_change":"2024-09-16T00:30:23.929943+0000","last_active":"2024-09-16T00:31:02.315266+0000","last_peered":"2024-09-16T00:31:02.315266+0000","last_clean":"2024-09-16T00:31:02.315266+0000","last_became_active":"2024-09-16T00:30:23.929578+0000","last_became_peered":"2024-09-16T00:30:23.929578+0000","last_unstale":"2024-09-16T00:31:02.315266+0000","last_undegraded":"2024-09-16T00:31:02.315266+0000","last_fullsized":"2024-09-16T00:31:02.315266+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:05:12.505543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:02.313911+0000","last_change":"2024-09-16T00:30:23.935758+0000","last_active":"2024-09-16T00:31:02.313911+0000","last_peered":"2024-09-16T00:31:02.313911+0000","last_clean":"2024-09-16T00:31:02.313911+0000","last_became_active":"2024-09-16T00:30:23.935321+0000","last_became_peered":"2024-09-16T00:30:23.935321+0000","last_unstale":"2024-09-16T00:31:02.313911+0000","last_undegraded":"2024-09-16T00:31:02.313911+0000","last_fullsized":"2024-09-16T00:31:02.313911+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:35:40.513435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765532+0000","last_change":"2024-09-16T00:30:26.086316+0000","last_active":"2024-09-16T00:30:36.765532+0000","last_peered":"2024-09-16T00:30:36.765532+0000","last_clean":"2024-09-16T00:30:36.765532+0000","last_became_active":"2024-09-16T00:30:26.085800+0000","last_became_peered":"2024-09-16T00:30:26.085800+0000","last_unstale":"2024-09-16T00:30:36.765532+0000","last_undegraded":"2024-09-16T00:30:36.765532+0000","last_fullsized":"2024-09-16T00:30:36.765532+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:05:36.277127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766520+0000","last_change":"2024-09-16T00:30:33.155665+0000","last_active":"2024-09-16T00:30:36.766520+0000","last_peered":"2024-09-16T00:30:36.766520+0000","last_clean":"2024-09-16T00:30:36.766520+0000","last_became_active":"2024-09-16T00:30:33.155253+0000","last_became_peered":"2024-09-16T00:30:33.155253+0000","last_unstale":"2024-09-16T00:30:36.766520+0000","last_undegraded":"2024-09-16T00:30:36.766520+0000","last_fullsized":"2024-09-16T00:30:36.766520+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:37.545232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":54,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:31:16.350933+0000","last_change":"2024-09-16T00:30:23.934487+0000","last_active":"2024-09-16T00:31:16.350933+0000","last_peered":"2024-09-16T00:31:16.350933+0000","last_clean":"2024-09-16T00:31:16.350933+0000","last_became_active":"2024-09-16T00:30:23.934137+0000","last_became_peered":"2024-09-16T00:30:23.934137+0000","last_unstale":"2024-09-16T00:31:16.350933+0000","last_undegraded":"2024-09-16T00:31:16.350933+0000","last_fullsized":"2024-09-16T00:31:16.350933+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:14:33.088776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":18,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765355+0000","last_change":"2024-09-16T00:30:26.077533+0000","last_active":"2024-09-16T00:30:36.765355+0000","last_peered":"2024-09-16T00:30:36.765355+0000","last_clean":"2024-09-16T00:30:36.765355+0000","last_became_active":"2024-09-16T00:30:26.076443+0000","last_became_peered":"2024-09-16T00:30:26.076443+0000","last_unstale":"2024-09-16T00:30:36.765355+0000","last_undegraded":"2024-09-16T00:30:36.765355+0000","last_fullsized":"2024-09-16T00:30:36.765355+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:39:17.495521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766308+0000","last_change":"2024-09-16T00:30:33.146123+0000","last_active":"2024-09-16T00:30:36.766308+0000","last_peered":"2024-09-16T00:30:36.766308+0000","last_clean":"2024-09-16T00:30:36.766308+0000","last_became_active":"2024-09-16T00:30:33.145716+0000","last_became_peered":"2024-09-16T00:30:33.145716+0000","last_unstale":"2024-09-16T00:30:36.766308+0000","last_undegraded":"2024-09-16T00:30:36.766308+0000","last_fullsized":"2024-09-16T00:30:36.766308+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:14:27.281367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.148977+0000","last_change":"2024-09-16T00:30:26.083155+0000","last_active":"2024-09-16T00:30:36.148977+0000","last_peered":"2024-09-16T00:30:36.148977+0000","last_clean":"2024-09-16T00:30:36.148977+0000","last_became_active":"2024-09-16T00:30:26.082782+0000","last_became_peered":"2024-09-16T00:30:26.082782+0000","last_unstale":"2024-09-16T00:30:36.148977+0000","last_undegraded":"2024-09-16T00:30:36.148977+0000","last_fullsized":"2024-09-16T00:30:36.148977+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:15.929343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766026+0000","last_change":"2024-09-16T00:30:26.080740+0000","last_active":"2024-09-16T00:30:36.766026+0000","last_peered":"2024-09-16T00:30:36.766026+0000","last_clean":"2024-09-16T00:30:36.766026+0000","last_became_active":"2024-09-16T00:30:26.080401+0000","last_became_peered":"2024-09-16T00:30:26.080401+0000","last_unstale":"2024-09-16T00:30:36.766026+0000","last_undegraded":"2024-09-16T00:30:36.766026+0000","last_fullsized":"2024-09-16T00:30:36.766026+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:59:26.563887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766169+0000","last_change":"2024-09-16T00:30:26.076252+0000","last_active":"2024-09-16T00:30:36.766169+0000","last_peered":"2024-09-16T00:30:36.766169+0000","last_clean":"2024-09-16T00:30:36.766169+0000","last_became_active":"2024-09-16T00:30:26.075922+0000","last_became_peered":"2024-09-16T00:30:26.075922+0000","last_unstale":"2024-09-16T00:30:36.766169+0000","last_undegraded":"2024-09-16T00:30:36.766169+0000","last_fullsized":"2024-09-16T00:30:36.766169+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:28:04.822817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.767391+0000","last_change":"2024-09-16T00:30:26.085123+0000","last_active":"2024-09-16T00:30:36.767391+0000","last_peered":"2024-09-16T00:30:36.767391+0000","last_clean":"2024-09-16T00:30:36.767391+0000","last_became_active":"2024-09-16T00:30:26.084782+0000","last_became_peered":"2024-09-16T00:30:26.084782+0000","last_unstale":"2024-09-16T00:30:36.767391+0000","last_undegraded":"2024-09-16T00:30:36.767391+0000","last_fullsized":"2024-09-16T00:30:36.767391+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:09:33.766340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766346+0000","last_change":"2024-09-16T00:30:26.077048+0000","last_active":"2024-09-16T00:30:36.766346+0000","last_peered":"2024-09-16T00:30:36.766346+0000","last_clean":"2024-09-16T00:30:36.766346+0000","last_became_active":"2024-09-16T00:30:26.076719+0000","last_became_peered":"2024-09-16T00:30:26.076719+0000","last_unstale":"2024-09-16T00:30:36.766346+0000","last_undegraded":"2024-09-16T00:30:36.766346+0000","last_fullsized":"2024-09-16T00:30:36.766346+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:58.584246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.149263+0000","last_change":"2024-09-16T00:30:26.078446+0000","last_active":"2024-09-16T00:30:36.149263+0000","last_peered":"2024-09-16T00:30:36.149263+0000","last_clean":"2024-09-16T00:30:36.149263+0000","last_became_active":"2024-09-16T00:30:26.078145+0000","last_became_peered":"2024-09-16T00:30:26.078145+0000","last_unstale":"2024-09-16T00:30:36.149263+0000","last_undegraded":"2024-09-16T00:30:36.149263+0000","last_fullsized":"2024-09-16T00:30:36.149263+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:52:28.739834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765484+0000","last_change":"2024-09-16T00:30:26.075875+0000","last_active":"2024-09-16T00:30:36.765484+0000","last_peered":"2024-09-16T00:30:36.765484+0000","last_clean":"2024-09-16T00:30:36.765484+0000","last_became_active":"2024-09-16T00:30:26.075424+0000","last_became_peered":"2024-09-16T00:30:26.075424+0000","last_unstale":"2024-09-16T00:30:36.765484+0000","last_undegraded":"2024-09-16T00:30:36.765484+0000","last_fullsized":"2024-09-16T00:30:36.765484+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:21:07.958507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766070+0000","last_change":"2024-09-16T00:30:26.077694+0000","last_active":"2024-09-16T00:30:36.766070+0000","last_peered":"2024-09-16T00:30:36.766070+0000","last_clean":"2024-09-16T00:30:36.766070+0000","last_became_active":"2024-09-16T00:30:26.077374+0000","last_became_peered":"2024-09-16T00:30:26.077374+0000","last_unstale":"2024-09-16T00:30:36.766070+0000","last_undegraded":"2024-09-16T00:30:36.766070+0000","last_fullsized":"2024-09-16T00:30:36.766070+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:16:30.822050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766640+0000","last_change":"2024-09-16T00:30:26.085931+0000","last_active":"2024-09-16T00:30:36.766640+0000","last_peered":"2024-09-16T00:30:36.766640+0000","last_clean":"2024-09-16T00:30:36.766640+0000","last_became_active":"2024-09-16T00:30:26.085420+0000","last_became_peered":"2024-09-16T00:30:26.085420+0000","last_unstale":"2024-09-16T00:30:36.766640+0000","last_undegraded":"2024-09-16T00:30:36.766640+0000","last_fullsized":"2024-09-16T00:30:36.766640+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:20:36.019203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.764930+0000","last_change":"2024-09-16T00:30:26.074217+0000","last_active":"2024-09-16T00:30:36.764930+0000","last_peered":"2024-09-16T00:30:36.764930+0000","last_clean":"2024-09-16T00:30:36.764930+0000","last_became_active":"2024-09-16T00:30:26.073856+0000","last_became_peered":"2024-09-16T00:30:26.073856+0000","last_unstale":"2024-09-16T00:30:36.764930+0000","last_undegraded":"2024-09-16T00:30:36.764930+0000","last_fullsized":"2024-09-16T00:30:36.764930+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:53:57.078465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765039+0000","last_change":"2024-09-16T00:30:26.084248+0000","last_active":"2024-09-16T00:30:36.765039+0000","last_peered":"2024-09-16T00:30:36.765039+0000","last_clean":"2024-09-16T00:30:36.765039+0000","last_became_active":"2024-09-16T00:30:26.083530+0000","last_became_peered":"2024-09-16T00:30:26.083530+0000","last_unstale":"2024-09-16T00:30:36.765039+0000","last_undegraded":"2024-09-16T00:30:36.765039+0000","last_fullsized":"2024-09-16T00:30:36.765039+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:16:49.483881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766481+0000","last_change":"2024-09-16T00:30:26.080803+0000","last_active":"2024-09-16T00:30:36.766481+0000","last_peered":"2024-09-16T00:30:36.766481+0000","last_clean":"2024-09-16T00:30:36.766481+0000","last_became_active":"2024-09-16T00:30:26.080482+0000","last_became_peered":"2024-09-16T00:30:26.080482+0000","last_unstale":"2024-09-16T00:30:36.766481+0000","last_undegraded":"2024-09-16T00:30:36.766481+0000","last_fullsized":"2024-09-16T00:30:36.766481+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:35:41.741025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.149107+0000","last_change":"2024-09-16T00:30:26.077680+0000","last_active":"2024-09-16T00:30:36.149107+0000","last_peered":"2024-09-16T00:30:36.149107+0000","last_clean":"2024-09-16T00:30:36.149107+0000","last_became_active":"2024-09-16T00:30:26.077371+0000","last_became_peered":"2024-09-16T00:30:26.077371+0000","last_unstale":"2024-09-16T00:30:36.149107+0000","last_undegraded":"2024-09-16T00:30:36.149107+0000","last_fullsized":"2024-09-16T00:30:36.149107+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:31:34.338617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766418+0000","last_change":"2024-09-16T00:30:26.071531+0000","last_active":"2024-09-16T00:30:36.766418+0000","last_peered":"2024-09-16T00:30:36.766418+0000","last_clean":"2024-09-16T00:30:36.766418+0000","last_became_active":"2024-09-16T00:30:26.071189+0000","last_became_peered":"2024-09-16T00:30:26.071189+0000","last_unstale":"2024-09-16T00:30:36.766418+0000","last_undegraded":"2024-09-16T00:30:36.766418+0000","last_fullsized":"2024-09-16T00:30:36.766418+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:29:18.406272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.153847+0000","last_change":"2024-09-16T00:30:26.085398+0000","last_active":"2024-09-16T00:30:36.153847+0000","last_peered":"2024-09-16T00:30:36.153847+0000","last_clean":"2024-09-16T00:30:36.153847+0000","last_became_active":"2024-09-16T00:30:26.085076+0000","last_became_peered":"2024-09-16T00:30:26.085076+0000","last_unstale":"2024-09-16T00:30:36.153847+0000","last_undegraded":"2024-09-16T00:30:36.153847+0000","last_fullsized":"2024-09-16T00:30:36.153847+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:20:20.864660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.149403+0000","last_change":"2024-09-16T00:30:26.080953+0000","last_active":"2024-09-16T00:30:36.149403+0000","last_peered":"2024-09-16T00:30:36.149403+0000","last_clean":"2024-09-16T00:30:36.149403+0000","last_became_active":"2024-09-16T00:30:26.080185+0000","last_became_peered":"2024-09-16T00:30:26.080185+0000","last_unstale":"2024-09-16T00:30:36.149403+0000","last_undegraded":"2024-09-16T00:30:36.149403+0000","last_fullsized":"2024-09-16T00:30:36.149403+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:56.769182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.764374+0000","last_change":"2024-09-16T00:30:26.079320+0000","last_active":"2024-09-16T00:30:36.764374+0000","last_peered":"2024-09-16T00:30:36.764374+0000","last_clean":"2024-09-16T00:30:36.764374+0000","last_became_active":"2024-09-16T00:30:26.078980+0000","last_became_peered":"2024-09-16T00:30:26.078980+0000","last_unstale":"2024-09-16T00:30:36.764374+0000","last_undegraded":"2024-09-16T00:30:36.764374+0000","last_fullsized":"2024-09-16T00:30:36.764374+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:03:15.386657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765410+0000","last_change":"2024-09-16T00:30:26.084948+0000","last_active":"2024-09-16T00:30:36.765410+0000","last_peered":"2024-09-16T00:30:36.765410+0000","last_clean":"2024-09-16T00:30:36.765410+0000","last_became_active":"2024-09-16T00:30:26.084602+0000","last_became_peered":"2024-09-16T00:30:26.084602+0000","last_unstale":"2024-09-16T00:30:36.765410+0000","last_undegraded":"2024-09-16T00:30:36.765410+0000","last_fullsized":"2024-09-16T00:30:36.765410+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:01.709332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766362+0000","last_change":"2024-09-16T00:30:26.080491+0000","last_active":"2024-09-16T00:30:36.766362+0000","last_peered":"2024-09-16T00:30:36.766362+0000","last_clean":"2024-09-16T00:30:36.766362+0000","last_became_active":"2024-09-16T00:30:26.080189+0000","last_became_peered":"2024-09-16T00:30:26.080189+0000","last_unstale":"2024-09-16T00:30:36.766362+0000","last_undegraded":"2024-09-16T00:30:36.766362+0000","last_fullsized":"2024-09-16T00:30:36.766362+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:38.358919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766440+0000","last_change":"2024-09-16T00:30:26.077410+0000","last_active":"2024-09-16T00:30:36.766440+0000","last_peered":"2024-09-16T00:30:36.766440+0000","last_clean":"2024-09-16T00:30:36.766440+0000","last_became_active":"2024-09-16T00:30:26.076498+0000","last_became_peered":"2024-09-16T00:30:26.076498+0000","last_unstale":"2024-09-16T00:30:36.766440+0000","last_undegraded":"2024-09-16T00:30:36.766440+0000","last_fullsized":"2024-09-16T00:30:36.766440+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:54:17.712774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765010+0000","last_change":"2024-09-16T00:30:26.074500+0000","last_active":"2024-09-16T00:30:36.765010+0000","last_peered":"2024-09-16T00:30:36.765010+0000","last_clean":"2024-09-16T00:30:36.765010+0000","last_became_active":"2024-09-16T00:30:26.074177+0000","last_became_peered":"2024-09-16T00:30:26.074177+0000","last_unstale":"2024-09-16T00:30:36.765010+0000","last_undegraded":"2024-09-16T00:30:36.765010+0000","last_fullsized":"2024-09-16T00:30:36.765010+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:43:13.792231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.634914+0000","last_change":"2024-09-16T00:30:26.078358+0000","last_active":"2024-09-16T00:30:36.634914+0000","last_peered":"2024-09-16T00:30:36.634914+0000","last_clean":"2024-09-16T00:30:36.634914+0000","last_became_active":"2024-09-16T00:30:26.077993+0000","last_became_peered":"2024-09-16T00:30:26.077993+0000","last_unstale":"2024-09-16T00:30:36.634914+0000","last_undegraded":"2024-09-16T00:30:36.634914+0000","last_fullsized":"2024-09-16T00:30:36.634914+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:25:10.711394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.154359+0000","last_change":"2024-09-16T00:30:26.083248+0000","last_active":"2024-09-16T00:30:36.154359+0000","last_peered":"2024-09-16T00:30:36.154359+0000","last_clean":"2024-09-16T00:30:36.154359+0000","last_became_active":"2024-09-16T00:30:26.082861+0000","last_became_peered":"2024-09-16T00:30:26.082861+0000","last_unstale":"2024-09-16T00:30:36.154359+0000","last_undegraded":"2024-09-16T00:30:36.154359+0000","last_fullsized":"2024-09-16T00:30:36.154359+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:49:13.883256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765259+0000","last_change":"2024-09-16T00:30:26.086313+0000","last_active":"2024-09-16T00:30:36.765259+0000","last_peered":"2024-09-16T00:30:36.765259+0000","last_clean":"2024-09-16T00:30:36.765259+0000","last_became_active":"2024-09-16T00:30:26.085760+0000","last_became_peered":"2024-09-16T00:30:26.085760+0000","last_unstale":"2024-09-16T00:30:36.765259+0000","last_undegraded":"2024-09-16T00:30:36.765259+0000","last_fullsized":"2024-09-16T00:30:36.765259+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:40:38.236789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.764637+0000","last_change":"2024-09-16T00:30:26.085289+0000","last_active":"2024-09-16T00:30:36.764637+0000","last_peered":"2024-09-16T00:30:36.764637+0000","last_clean":"2024-09-16T00:30:36.764637+0000","last_became_active":"2024-09-16T00:30:26.084995+0000","last_became_peered":"2024-09-16T00:30:26.084995+0000","last_unstale":"2024-09-16T00:30:36.764637+0000","last_undegraded":"2024-09-16T00:30:36.764637+0000","last_fullsized":"2024-09-16T00:30:36.764637+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:45.667892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.634434+0000","last_change":"2024-09-16T00:30:26.088690+0000","last_active":"2024-09-16T00:30:36.634434+0000","last_peered":"2024-09-16T00:30:36.634434+0000","last_clean":"2024-09-16T00:30:36.634434+0000","last_became_active":"2024-09-16T00:30:26.087700+0000","last_became_peered":"2024-09-16T00:30:26.087700+0000","last_unstale":"2024-09-16T00:30:36.634434+0000","last_undegraded":"2024-09-16T00:30:36.634434+0000","last_fullsized":"2024-09-16T00:30:36.634434+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:39:51.445324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.153741+0000","last_change":"2024-09-16T00:30:26.080215+0000","last_active":"2024-09-16T00:30:36.153741+0000","last_peered":"2024-09-16T00:30:36.153741+0000","last_clean":"2024-09-16T00:30:36.153741+0000","last_became_active":"2024-09-16T00:30:26.079038+0000","last_became_peered":"2024-09-16T00:30:26.079038+0000","last_unstale":"2024-09-16T00:30:36.153741+0000","last_undegraded":"2024-09-16T00:30:36.153741+0000","last_fullsized":"2024-09-16T00:30:36.153741+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:01.573960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.765084+0000","last_change":"2024-09-16T00:30:26.080966+0000","last_active":"2024-09-16T00:30:36.765084+0000","last_peered":"2024-09-16T00:30:36.765084+0000","last_clean":"2024-09-16T00:30:36.765084+0000","last_became_active":"2024-09-16T00:30:26.080641+0000","last_became_peered":"2024-09-16T00:30:26.080641+0000","last_unstale":"2024-09-16T00:30:36.765084+0000","last_undegraded":"2024-09-16T00:30:36.765084+0000","last_fullsized":"2024-09-16T00:30:36.765084+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:33:01.538169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.764921+0000","last_change":"2024-09-16T00:30:26.072644+0000","last_active":"2024-09-16T00:30:36.764921+0000","last_peered":"2024-09-16T00:30:36.764921+0000","last_clean":"2024-09-16T00:30:36.764921+0000","last_became_active":"2024-09-16T00:30:26.072336+0000","last_became_peered":"2024-09-16T00:30:26.072336+0000","last_unstale":"2024-09-16T00:30:36.764921+0000","last_undegraded":"2024-09-16T00:30:36.764921+0000","last_fullsized":"2024-09-16T00:30:36.764921+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:44:45.760057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.767442+0000","last_change":"2024-09-16T00:30:26.079155+0000","last_active":"2024-09-16T00:30:36.767442+0000","last_peered":"2024-09-16T00:30:36.767442+0000","last_clean":"2024-09-16T00:30:36.767442+0000","last_became_active":"2024-09-16T00:30:26.078820+0000","last_became_peered":"2024-09-16T00:30:26.078820+0000","last_unstale":"2024-09-16T00:30:36.767442+0000","last_undegraded":"2024-09-16T00:30:36.767442+0000","last_fullsized":"2024-09-16T00:30:36.767442+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:35.071143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.766830+0000","last_change":"2024-09-16T00:30:26.081537+0000","last_active":"2024-09-16T00:30:36.766830+0000","last_peered":"2024-09-16T00:30:36.766830+0000","last_clean":"2024-09-16T00:30:36.766830+0000","last_became_active":"2024-09-16T00:30:26.080744+0000","last_became_peered":"2024-09-16T00:30:26.080744+0000","last_unstale":"2024-09-16T00:30:36.766830+0000","last_undegraded":"2024-09-16T00:30:36.766830+0000","last_fullsized":"2024-09-16T00:30:36.766830+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:46:38.179888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-09-16T00:30:36.149376+0000","last_change":"2024-09-16T00:30:26.081066+0000","last_active":"2024-09-16T00:30:36.149376+0000","last_peered":"2024-09-16T00:30:36.149376+0000","last_clean":"2024-09-16T00:30:36.149376+0000","last_became_active":"2024-09-16T00:30:26.080630+0000","last_became_peered":"2024-09-16T00:30:26.080630+0000","last_unstale":"2024-09-16T00:30:36.149376+0000","last_undegraded":"2024-09-16T00:30:36.149376+0000","last_fullsized":"2024-09-16T00:30:36.149376+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:44:42.644792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":60676,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":115,"num_read_kb":122,"num_write":159,"num_write_kb":107,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":380928,"data_stored":155372,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514519,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36308,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708844,"statfs":{"total":95995035648,"available":95957856256,"internally_reserved":0,"allocated":987136,"data_stored":485563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743453,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32816,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712336,"statfs":{"total":95995035648,"available":95961432064,"internally_reserved":0,"allocated":1671168,"data_stored":1129504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906978,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1007616,"data_stored":489895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135911,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36324,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708828,"statfs":{"total":95995035648,"available":95957839872,"internally_reserved":0,"allocated":1003520,"data_stored":485719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397547,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28584,"kb_used_data":1560,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716568,"statfs":{"total":95995035648,"available":95965765632,"internally_reserved":0,"allocated":1597440,"data_stored":1081630,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561073,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":983040,"data_stored":480975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790006,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28592,"kb_used_data":1568,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716560,"statfs":{"total":95995035648,"available":95965757440,"internally_reserved":0,"allocated":1605632,"data_stored":1077218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953532,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36316,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708836,"statfs":{"total":95995035648,"available":95957848064,"internally_reserved":0,"allocated":995328,"data_stored":485625,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182463,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32156,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712996,"statfs":{"total":95995035648,"available":95962107904,"internally_reserved":0,"allocated":995328,"data_stored":481169,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411398,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1056768,"data_stored":525973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574923,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1032192,"data_stored":524230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738448,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1007616,"data_stored":482363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":43345,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":45088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":58251,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:31:21.148 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-16T00:31:21.149 INFO:tasks.cephadm:Setting up client nodes... 2024-09-16T00:31:21.149 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-16T00:31:21.409 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:21.679 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:21 smithi038 ceph-mon[30945]: pgmap v317: 129 pgs: 129 active+clean; 636 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.5 KiB/s wr, 11 op/s 2024-09-16T00:31:21.679 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:21 smithi038 ceph-mon[30945]: from='client.14841 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:31:21.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:21 smithi122 ceph-mon[35412]: pgmap v317: 129 pgs: 129 active+clean; 636 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.5 KiB/s wr, 11 op/s 2024-09-16T00:31:21.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:21 smithi122 ceph-mon[35412]: from='client.14841 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:31:21.793 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:21 smithi073 ceph-mon[35254]: pgmap v317: 129 pgs: 129 active+clean; 636 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 5.5 KiB/s wr, 11 op/s 2024-09-16T00:31:21.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:21 smithi073 ceph-mon[35254]: from='client.14841 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:31:21.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:21.898+0000 7f7fe3936640 1 -- 172.21.15.38:0/3016956524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fdc105070 msgr2=0x7f7fdc107460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:21.908 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:21.898+0000 7f7fe3936640 1 --2- 172.21.15.38:0/3016956524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fdc105070 0x7f7fdc107460 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0009900 tx=0x7f7fd002a410 comp rx=0 tx=0).stop 2024-09-16T00:31:21.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:21.898+0000 7f7fe3936640 1 -- 172.21.15.38:0/3016956524 shutdown_connections 2024-09-16T00:31:21.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:21.898+0000 7f7fe3936640 1 --2- 172.21.15.38:0/3016956524 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fdc1079a0 0x7f7fdc109d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:21.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:21.898+0000 7f7fe3936640 1 --2- 172.21.15.38:0/3016956524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fdc105070 0x7f7fdc107460 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:21.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:21.898+0000 7f7fe3936640 1 --2- 172.21.15.38:0/3016956524 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fdc102740 0x7f7fdc104b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:21.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:21.898+0000 7f7fe3936640 1 -- 172.21.15.38:0/3016956524 >> 172.21.15.38:0/3016956524 conn(0x7f7fdc0fc2a0 msgr2=0x7f7fdc0fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:21.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:21.898+0000 7f7fe3936640 1 -- 172.21.15.38:0/3016956524 shutdown_connections 2024-09-16T00:31:21.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:21.898+0000 7f7fe3936640 1 -- 172.21.15.38:0/3016956524 wait complete. 2024-09-16T00:31:21.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:21.899+0000 7f7fe3936640 1 Processor -- start 2024-09-16T00:31:21.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:21.899+0000 7f7fe3936640 1 -- start start 2024-09-16T00:31:21.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:21.899+0000 7f7fe3936640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fdc102740 0x7f7fdc197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:21.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:21.899+0000 7f7fe3936640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fdc105070 0x7f7fdc198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:21.909 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:21.899+0000 7f7fe3936640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fdc1079a0 0x7f7fdc19f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:21.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:21.899+0000 7f7fe3936640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7fdc10d870 con 0x7f7fdc105070 2024-09-16T00:31:21.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:21.899+0000 7f7fe3936640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7fdc10d6f0 con 0x7f7fdc102740 2024-09-16T00:31:21.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:21.899+0000 7f7fe3936640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7fdc10d9f0 con 0x7f7fdc1079a0 2024-09-16T00:31:21.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:21.899+0000 7f7fe0ed1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fdc105070 0x7f7fdc198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:21.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:21.899+0000 7f7fe0ed1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fdc105070 0x7f7fdc198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44132/0 (socket says 172.21.15.38:44132) 2024-09-16T00:31:21.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:21.899+0000 7f7fe0ed1640 1 -- 172.21.15.38:0/2631439455 learned_addr learned my addr 172.21.15.38:0/2631439455 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:31:21.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:21.899+0000 7f7fe16d2640 1 --2- 172.21.15.38:0/2631439455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fdc102740 0x7f7fdc197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:21.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:21.899+0000 7f7fe1ed3640 1 --2- 172.21.15.38:0/2631439455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fdc1079a0 0x7f7fdc19f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:21.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:21.900+0000 7f7fe0ed1640 1 -- 172.21.15.38:0/2631439455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fdc1079a0 msgr2=0x7f7fdc19f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:21.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:21.900+0000 7f7fe0ed1640 1 --2- 172.21.15.38:0/2631439455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fdc1079a0 0x7f7fdc19f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:21.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:21.900+0000 7f7fe0ed1640 1 -- 172.21.15.38:0/2631439455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fdc102740 msgr2=0x7f7fdc197f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:21.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:21.900+0000 7f7fe0ed1640 1 --2- 172.21.15.38:0/2631439455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fdc102740 0x7f7fdc197f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:21.910 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:21.900+0000 7f7fe0ed1640 1 -- 172.21.15.38:0/2631439455 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fdc19fb50 con 0x7f7fdc105070 2024-09-16T00:31:21.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:21.900+0000 7f7fe0ed1640 1 --2- 172.21.15.38:0/2631439455 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fdc105070 0x7f7fdc198460 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0031610 tx=0x7f7fd00043b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:21.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:21.900+0000 7f7fce7fc640 1 -- 172.21.15.38:0/2631439455 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd003f030 con 0x7f7fdc105070 2024-09-16T00:31:21.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:21.900+0000 7f7fce7fc640 1 -- 172.21.15.38:0/2631439455 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7fd0003ce0 con 0x7f7fdc105070 2024-09-16T00:31:21.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:21.900+0000 7f7fe3936640 1 -- 172.21.15.38:0/2631439455 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fdc19fde0 con 0x7f7fdc105070 2024-09-16T00:31:21.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:21.900+0000 7f7fce7fc640 1 -- 172.21.15.38:0/2631439455 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd0035070 con 0x7f7fdc105070 2024-09-16T00:31:21.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:21.900+0000 7f7fe3936640 1 -- 172.21.15.38:0/2631439455 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fdc1a0260 con 0x7f7fdc105070 2024-09-16T00:31:21.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:21.901+0000 7f7fe3936640 1 -- 172.21.15.38:0/2631439455 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fdc0fdd90 con 0x7f7fdc105070 2024-09-16T00:31:21.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:21.902+0000 7f7fce7fc640 1 -- 172.21.15.38:0/2631439455 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f7fd0042030 con 0x7f7fdc105070 2024-09-16T00:31:21.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:21.903+0000 7f7fce7fc640 1 --2- 172.21.15.38:0/2631439455 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f7fc80a3300 0x7f7fc80a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:21.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:21.903+0000 7f7fce7fc640 1 -- 172.21.15.38:0/2631439455 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8802+0+0 (secure 0 0 0) 0x7f7fd00f0180 con 0x7f7fdc105070 2024-09-16T00:31:21.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:21.903+0000 7f7fe16d2640 1 --2- 172.21.15.38:0/2631439455 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f7fc80a3300 0x7f7fc80a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:21.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:21.903+0000 7f7fe16d2640 1 --2- 172.21.15.38:0/2631439455 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f7fc80a3300 0x7f7fc80a57c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7fd800bbe0 tx=0x7f7fd800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:21.911 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:21.907+0000 7f7fce7fc640 1 -- 172.21.15.38:0/2631439455 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7fd00bc4b0 con 0x7f7fdc105070 2024-09-16T00:31:22.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:22.155+0000 7f7fe3936640 1 -- 172.21.15.38:0/2631439455 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f7fdc02d7c0 con 0x7f7fdc105070 2024-09-16T00:31:22.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:22.157+0000 7f7fce7fc640 1 -- 172.21.15.38:0/2631439455 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f7fd00c07b0 con 0x7f7fdc105070 2024-09-16T00:31:22.158 INFO:teuthology.orchestra.run.smithi038.stdout:[client.0] 2024-09-16T00:31:22.158 INFO:teuthology.orchestra.run.smithi038.stdout: key = AQBafOdmidFUCRAALyica/wTMjlPyvRi9u40cw== 2024-09-16T00:31:22.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:22.159+0000 7f7fe3936640 1 -- 172.21.15.38:0/2631439455 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f7fc80a3300 msgr2=0x7f7fc80a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:22.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:22.159+0000 7f7fe3936640 1 --2- 172.21.15.38:0/2631439455 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f7fc80a3300 0x7f7fc80a57c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f7fd800bbe0 tx=0x7f7fd800b390 comp rx=0 tx=0).stop 2024-09-16T00:31:22.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:22.159+0000 7f7fe3936640 1 -- 172.21.15.38:0/2631439455 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fdc105070 msgr2=0x7f7fdc198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:22.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:22.159+0000 7f7fe3936640 1 --2- 172.21.15.38:0/2631439455 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fdc105070 0x7f7fdc198460 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0031610 tx=0x7f7fd00043b0 comp rx=0 tx=0).stop 2024-09-16T00:31:22.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:22.159+0000 7f7fe3936640 1 -- 172.21.15.38:0/2631439455 shutdown_connections 2024-09-16T00:31:22.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:22.159+0000 7f7fe3936640 1 --2- 172.21.15.38:0/2631439455 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f7fc80a3300 0x7f7fc80a57c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:22.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:22.160+0000 7f7fe3936640 1 --2- 172.21.15.38:0/2631439455 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7fdc1079a0 0x7f7fdc19f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:22.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:22.160+0000 7f7fe3936640 1 --2- 172.21.15.38:0/2631439455 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7fdc105070 0x7f7fdc198460 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:22.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:22.160+0000 7f7fe3936640 1 --2- 172.21.15.38:0/2631439455 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fdc102740 0x7f7fdc197f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:22.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:22.160+0000 7f7fe3936640 1 -- 172.21.15.38:0/2631439455 >> 172.21.15.38:0/2631439455 conn(0x7f7fdc0fc2a0 msgr2=0x7f7fdc1030c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:22.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:22.160+0000 7f7fe3936640 1 -- 172.21.15.38:0/2631439455 shutdown_connections 2024-09-16T00:31:22.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:22.160+0000 7f7fe3936640 1 -- 172.21.15.38:0/2631439455 wait complete. 2024-09-16T00:31:22.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:22 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-16T00:31:22.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:22 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-09-16T00:31:22.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:22 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-16T00:31:22.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:22 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]: dispatch 2024-09-16T00:31:22.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:22 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8, 5, 9]}]: dispatch 2024-09-16T00:31:22.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:22 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-09-16T00:31:22.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:22 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-09-16T00:31:22.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:22 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-09-16T00:31:22.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:22 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-09-16T00:31:22.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:22 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/2631439455' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T00:31:22.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:22 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/2631439455' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T00:31:22.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:22 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-16T00:31:22.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:22 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-09-16T00:31:22.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:22 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-16T00:31:22.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:22 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]: dispatch 2024-09-16T00:31:22.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:22 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8, 5, 9]}]: dispatch 2024-09-16T00:31:22.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:22 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-09-16T00:31:22.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:22 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-09-16T00:31:22.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:22 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-09-16T00:31:22.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:22 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-09-16T00:31:22.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:22 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/2631439455' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T00:31:22.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:22 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/2631439455' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T00:31:22.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:22 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-09-16T00:31:22.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:22 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]: dispatch 2024-09-16T00:31:22.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:22 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]: dispatch 2024-09-16T00:31:22.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:22 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]: dispatch 2024-09-16T00:31:22.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:22 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8, 5, 9]}]: dispatch 2024-09-16T00:31:22.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:22 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]: dispatch 2024-09-16T00:31:22.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:22 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-09-16T00:31:22.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:22 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-09-16T00:31:22.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:22 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]: dispatch 2024-09-16T00:31:22.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:22 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/2631439455' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-16T00:31:22.795 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:22 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/2631439455' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-16T00:31:22.870 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-16T00:31:22.870 DEBUG:teuthology.orchestra.run.smithi038:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-16T00:31:22.870 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-16T00:31:22.911 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-16T00:31:22.911 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-16T00:31:22.911 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph mgr dump --format=json 2024-09-16T00:31:23.214 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:23.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:23 smithi038 ceph-mon[30945]: pgmap v318: 129 pgs: 129 active+clean; 636 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.6 KiB/s wr, 7 op/s 2024-09-16T00:31:23.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:23 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-16T00:31:23.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:23 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-09-16T00:31:23.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:23 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-16T00:31:23.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:23 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-09-16T00:31:23.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:23 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8, 5, 9]}]': finished 2024-09-16T00:31:23.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:23 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-09-16T00:31:23.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:23 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-16T00:31:23.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:23 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-09-16T00:31:23.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:23 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-09-16T00:31:23.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:23 smithi038 ceph-mon[30945]: osdmap e89: 12 total, 12 up, 12 in 2024-09-16T00:31:23.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:23 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:31:23.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:23 smithi122 ceph-mon[35412]: pgmap v318: 129 pgs: 129 active+clean; 636 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.6 KiB/s wr, 7 op/s 2024-09-16T00:31:23.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:23 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-16T00:31:23.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:23 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-09-16T00:31:23.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:23 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-16T00:31:23.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:23 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-09-16T00:31:23.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:23 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8, 5, 9]}]': finished 2024-09-16T00:31:23.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:23 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-09-16T00:31:23.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:23 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-16T00:31:23.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:23 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-09-16T00:31:23.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:23 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-09-16T00:31:23.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:23 smithi122 ceph-mon[35412]: osdmap e89: 12 total, 12 up, 12 in 2024-09-16T00:31:23.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:23 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:31:23.768 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:23 smithi073 ceph-mon[35254]: pgmap v318: 129 pgs: 129 active+clean; 636 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.6 KiB/s wr, 7 op/s 2024-09-16T00:31:23.768 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:23 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-09-16T00:31:23.768 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:23 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.13", "id": [2, 11]}]': finished 2024-09-16T00:31:23.768 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:23 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.16", "id": [2, 4]}]': finished 2024-09-16T00:31:23.768 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:23 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.7", "id": [3, 9]}]': finished 2024-09-16T00:31:23.768 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:23 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8, 5, 9]}]': finished 2024-09-16T00:31:23.768 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:23 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [4, 6]}]': finished 2024-09-16T00:31:23.768 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:23 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-09-16T00:31:23.768 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:23 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-09-16T00:31:23.769 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:23 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [9, 8]}]': finished 2024-09-16T00:31:23.769 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:23 smithi073 ceph-mon[35254]: osdmap e89: 12 total, 12 up, 12 in 2024-09-16T00:31:23.769 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:23 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:31:23.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:23.858+0000 7fd66fbb9640 1 -- 172.21.15.38:0/3662322520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd668101bc0 msgr2=0x7fd66810d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:23.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:23.858+0000 7fd66fbb9640 1 --2- 172.21.15.38:0/3662322520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd668101bc0 0x7fd66810d910 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fd660009ca0 tx=0x7fd66002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:31:23.876 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:23.858+0000 7fd66fbb9640 1 -- 172.21.15.38:0/3662322520 shutdown_connections 2024-09-16T00:31:23.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:23.858+0000 7fd66fbb9640 1 --2- 172.21.15.38:0/3662322520 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd668101bc0 0x7fd66810d910 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:23.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:23.858+0000 7fd66fbb9640 1 --2- 172.21.15.38:0/3662322520 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd668101240 0x7fd668101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:23.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:23.858+0000 7fd66fbb9640 1 --2- 172.21.15.38:0/3662322520 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd668105a90 0x7fd668105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:23.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:23.858+0000 7fd66fbb9640 1 -- 172.21.15.38:0/3662322520 >> 172.21.15.38:0/3662322520 conn(0x7fd668077210 msgr2=0x7fd6680fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:23.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:23.858+0000 7fd66fbb9640 1 -- 172.21.15.38:0/3662322520 shutdown_connections 2024-09-16T00:31:23.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:23.858+0000 7fd66fbb9640 1 -- 172.21.15.38:0/3662322520 wait complete. 2024-09-16T00:31:23.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:23.859+0000 7fd66fbb9640 1 Processor -- start 2024-09-16T00:31:23.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:23.859+0000 7fd66fbb9640 1 -- start start 2024-09-16T00:31:23.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:23.859+0000 7fd66fbb9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd668101240 0x7fd668199200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:23.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:23.859+0000 7fd66fbb9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd668101bc0 0x7fd668199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:23.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:23.859+0000 7fd66fbb9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd668105a90 0x7fd6681a4280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:23.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:23.859+0000 7fd66fbb9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd668110f70 con 0x7fd668101240 2024-09-16T00:31:23.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:23.859+0000 7fd66fbb9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd668110df0 con 0x7fd668105a90 2024-09-16T00:31:23.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:23.859+0000 7fd66fbb9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd6681110f0 con 0x7fd668101bc0 2024-09-16T00:31:23.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:23.859+0000 7fd66d955640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd668101240 0x7fd668199200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:23.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:23.859+0000 7fd66d955640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd668101240 0x7fd668199200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:44146/0 (socket says 172.21.15.38:44146) 2024-09-16T00:31:23.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:23.859+0000 7fd66d955640 1 -- 172.21.15.38:0/2995884601 learned_addr learned my addr 172.21.15.38:0/2995884601 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:31:23.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:23.859+0000 7fd66e156640 1 --2- 172.21.15.38:0/2995884601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd668105a90 0x7fd6681a4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:23.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:23.859+0000 7fd66d154640 1 --2- 172.21.15.38:0/2995884601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd668101bc0 0x7fd668199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:23.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:23.859+0000 7fd66d955640 1 -- 172.21.15.38:0/2995884601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd668101bc0 msgr2=0x7fd668199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:23.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:23.859+0000 7fd66d955640 1 --2- 172.21.15.38:0/2995884601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd668101bc0 0x7fd668199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:23.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:23.859+0000 7fd66d955640 1 -- 172.21.15.38:0/2995884601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd668105a90 msgr2=0x7fd6681a4280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:23.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:23.859+0000 7fd66d955640 1 --2- 172.21.15.38:0/2995884601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd668105a90 0x7fd6681a4280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:23.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:23.859+0000 7fd66d955640 1 -- 172.21.15.38:0/2995884601 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6681a47c0 con 0x7fd668101240 2024-09-16T00:31:23.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:23.859+0000 7fd66d154640 1 --2- 172.21.15.38:0/2995884601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd668101bc0 0x7fd668199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:31:23.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:23.860+0000 7fd66d955640 1 --2- 172.21.15.38:0/2995884601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd668101240 0x7fd668199200 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd66400d950 tx=0x7fd66400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:23.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:23.860+0000 7fd65effd640 1 -- 172.21.15.38:0/2995884601 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd664017070 con 0x7fd668101240 2024-09-16T00:31:23.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:23.860+0000 7fd65effd640 1 -- 172.21.15.38:0/2995884601 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd664009c30 con 0x7fd668101240 2024-09-16T00:31:23.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:23.860+0000 7fd66fbb9640 1 -- 172.21.15.38:0/2995884601 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6681a4990 con 0x7fd668101240 2024-09-16T00:31:23.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:23.860+0000 7fd65effd640 1 -- 172.21.15.38:0/2995884601 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd66400f6b0 con 0x7fd668101240 2024-09-16T00:31:23.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:23.860+0000 7fd66fbb9640 1 -- 172.21.15.38:0/2995884601 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6681a4ed0 con 0x7fd668101240 2024-09-16T00:31:23.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:23.861+0000 7fd66fbb9640 1 -- 172.21.15.38:0/2995884601 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd668101680 con 0x7fd668101240 2024-09-16T00:31:23.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:23.862+0000 7fd65effd640 1 -- 172.21.15.38:0/2995884601 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fd6640109d0 con 0x7fd668101240 2024-09-16T00:31:23.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:23.863+0000 7fd65effd640 1 --2- 172.21.15.38:0/2995884601 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd6540a32d0 0x7fd6540a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:23.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:23.863+0000 7fd65effd640 1 -- 172.21.15.38:0/2995884601 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7fd6640cf970 con 0x7fd668101240 2024-09-16T00:31:23.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:23.863+0000 7fd66d154640 1 --2- 172.21.15.38:0/2995884601 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd6540a32d0 0x7fd6540a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:23.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:23.863+0000 7fd66d154640 1 --2- 172.21.15.38:0/2995884601 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd6540a32d0 0x7fd6540a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd668100cc0 tx=0x7fd658005d90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:23.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:23.867+0000 7fd65effd640 1 -- 172.21.15.38:0/2995884601 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd664009de0 con 0x7fd668101240 2024-09-16T00:31:24.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:24.090+0000 7fd66fbb9640 1 -- 172.21.15.38:0/2995884601 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fd6681a5100 con 0x7fd668101240 2024-09-16T00:31:24.095 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:24.095+0000 7fd65effd640 1 -- 172.21.15.38:0/2995884601 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264049 (secure 0 0 0) 0x7fd66409c1f0 con 0x7fd668101240 2024-09-16T00:31:24.099 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:31:24.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:24.099+0000 7fd66fbb9640 1 -- 172.21.15.38:0/2995884601 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd6540a32d0 msgr2=0x7fd6540a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:24.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:24.099+0000 7fd66fbb9640 1 --2- 172.21.15.38:0/2995884601 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd6540a32d0 0x7fd6540a5790 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fd668100cc0 tx=0x7fd658005d90 comp rx=0 tx=0).stop 2024-09-16T00:31:24.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:24.099+0000 7fd66fbb9640 1 -- 172.21.15.38:0/2995884601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd668101240 msgr2=0x7fd668199200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:24.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:24.099+0000 7fd66fbb9640 1 --2- 172.21.15.38:0/2995884601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd668101240 0x7fd668199200 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd66400d950 tx=0x7fd66400de10 comp rx=0 tx=0).stop 2024-09-16T00:31:24.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:24.099+0000 7fd66fbb9640 1 -- 172.21.15.38:0/2995884601 shutdown_connections 2024-09-16T00:31:24.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:24.099+0000 7fd66fbb9640 1 --2- 172.21.15.38:0/2995884601 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd6540a32d0 0x7fd6540a5790 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:24.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:24.099+0000 7fd66fbb9640 1 --2- 172.21.15.38:0/2995884601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd668105a90 0x7fd6681a4280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:24.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:24.099+0000 7fd66fbb9640 1 --2- 172.21.15.38:0/2995884601 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd668101bc0 0x7fd668199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:24.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:24.099+0000 7fd66fbb9640 1 --2- 172.21.15.38:0/2995884601 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd668101240 0x7fd668199200 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:24.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:24.099+0000 7fd66fbb9640 1 -- 172.21.15.38:0/2995884601 >> 172.21.15.38:0/2995884601 conn(0x7fd668077210 msgr2=0x7fd668103850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:24.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:24.099+0000 7fd66fbb9640 1 -- 172.21.15.38:0/2995884601 shutdown_connections 2024-09-16T00:31:24.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:24.099+0000 7fd66fbb9640 1 -- 172.21.15.38:0/2995884601 wait complete. 2024-09-16T00:31:24.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:24 smithi038 ceph-mon[30945]: osdmap e90: 12 total, 12 up, 12 in 2024-09-16T00:31:24.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:24 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/2995884601' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-16T00:31:24.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:24 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:31:24.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:24 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:31:24.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:24 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:31:24.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:24 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:31:24.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:24 smithi122 ceph-mon[35412]: osdmap e90: 12 total, 12 up, 12 in 2024-09-16T00:31:24.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:24 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/2995884601' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-16T00:31:24.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:24 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:31:24.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:24 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:31:24.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:24 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:31:24.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:24 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:31:24.793 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:24 smithi073 ceph-mon[35254]: osdmap e90: 12 total, 12 up, 12 in 2024-09-16T00:31:24.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:24 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/2995884601' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-16T00:31:24.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:24 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:31:24.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:24 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:31:24.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:24 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:31:24.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:24 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:31:25.408 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":17,"flags":0,"active_gid":14150,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6800","nonce":1461245815},{"type":"v1","addr":"172.21.15.38:6801","nonce":1461245815}]},"active_addr":"172.21.15.38:6801/1461245815","active_change":"2024-09-16T00:23:21.417599+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.5","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.38:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.38:0","nonce":2896512449}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.38:0","nonce":3857151775}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.38:0","nonce":3842703753}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.38:0","nonce":1658697649}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.38:0","nonce":943300535}]}]} 2024-09-16T00:31:25.413 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-16T00:31:25.413 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-16T00:31:25.413 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:31:25.707 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:25.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:25 smithi038 ceph-mon[30945]: pgmap v321: 129 pgs: 2 peering, 127 active+clean; 636 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-09-16T00:31:25.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:25 smithi122 ceph-mon[35412]: pgmap v321: 129 pgs: 2 peering, 127 active+clean; 636 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-09-16T00:31:25.795 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:25 smithi073 ceph-mon[35254]: pgmap v321: 129 pgs: 2 peering, 127 active+clean; 636 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 2.0 KiB/s wr, 6 op/s 2024-09-16T00:31:26.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.175+0000 7f664eb7e640 1 -- 172.21.15.38:0/1667962902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6648102df0 msgr2=0x7f66481031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:26.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.175+0000 7f664eb7e640 1 --2- 172.21.15.38:0/1667962902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6648102df0 0x7f66481031d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f663c009900 tx=0x7f663c02a410 comp rx=0 tx=0).stop 2024-09-16T00:31:26.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.176+0000 7f664eb7e640 1 -- 172.21.15.38:0/1667962902 shutdown_connections 2024-09-16T00:31:26.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.176+0000 7f664eb7e640 1 --2- 172.21.15.38:0/1667962902 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f664810c2f0 0x7f664810e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:26.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.176+0000 7f664eb7e640 1 --2- 172.21.15.38:0/1667962902 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66481037a0 0x7f664810bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:26.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.176+0000 7f664eb7e640 1 --2- 172.21.15.38:0/1667962902 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6648102df0 0x7f66481031d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:26.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.176+0000 7f664eb7e640 1 -- 172.21.15.38:0/1667962902 >> 172.21.15.38:0/1667962902 conn(0x7f66480fc7d0 msgr2=0x7f66480febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:26.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.176+0000 7f664eb7e640 1 -- 172.21.15.38:0/1667962902 shutdown_connections 2024-09-16T00:31:26.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.176+0000 7f664eb7e640 1 -- 172.21.15.38:0/1667962902 wait complete. 2024-09-16T00:31:26.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.176+0000 7f664eb7e640 1 Processor -- start 2024-09-16T00:31:26.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.176+0000 7f664eb7e640 1 -- start start 2024-09-16T00:31:26.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.177+0000 7f664eb7e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6648102df0 0x7f6648198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:26.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.177+0000 7f664eb7e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66481037a0 0x7f6648198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:26.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.177+0000 7f664eb7e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f664810c2f0 0x7f664819cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:26.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.177+0000 7f664eb7e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6648112110 con 0x7f6648102df0 2024-09-16T00:31:26.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.177+0000 7f664eb7e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6648111f90 con 0x7f664810c2f0 2024-09-16T00:31:26.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.177+0000 7f664eb7e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6648112290 con 0x7f66481037a0 2024-09-16T00:31:26.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.177+0000 7f664c91a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6648102df0 0x7f6648198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:26.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.177+0000 7f664c91a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6648102df0 0x7f6648198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38706/0 (socket says 172.21.15.38:38706) 2024-09-16T00:31:26.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.177+0000 7f664c91a640 1 -- 172.21.15.38:0/996136047 learned_addr learned my addr 172.21.15.38:0/996136047 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:31:26.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.177+0000 7f6647fff640 1 --2- 172.21.15.38:0/996136047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66481037a0 0x7f6648198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:26.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.177+0000 7f664d11b640 1 --2- 172.21.15.38:0/996136047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f664810c2f0 0x7f664819cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:26.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.177+0000 7f664c91a640 1 -- 172.21.15.38:0/996136047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66481037a0 msgr2=0x7f6648198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:26.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.177+0000 7f664c91a640 1 --2- 172.21.15.38:0/996136047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66481037a0 0x7f6648198bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:26.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.177+0000 7f664c91a640 1 -- 172.21.15.38:0/996136047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f664810c2f0 msgr2=0x7f664819cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:26.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.177+0000 7f664c91a640 1 --2- 172.21.15.38:0/996136047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f664810c2f0 0x7f664819cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:26.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.177+0000 7f664c91a640 1 -- 172.21.15.38:0/996136047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f664819d640 con 0x7f6648102df0 2024-09-16T00:31:26.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.177+0000 7f6647fff640 1 --2- 172.21.15.38:0/996136047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66481037a0 0x7f6648198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:31:26.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.177+0000 7f664d11b640 1 --2- 172.21.15.38:0/996136047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f664810c2f0 0x7f664819cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:31:26.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.177+0000 7f664c91a640 1 --2- 172.21.15.38:0/996136047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6648102df0 0x7f6648198690 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f663c006010 tx=0x7f663c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:26.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.177+0000 7f6645ffb640 1 -- 172.21.15.38:0/996136047 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c03f030 con 0x7f6648102df0 2024-09-16T00:31:26.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.177+0000 7f664eb7e640 1 -- 172.21.15.38:0/996136047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f664819d8d0 con 0x7f6648102df0 2024-09-16T00:31:26.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.177+0000 7f6645ffb640 1 -- 172.21.15.38:0/996136047 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f663c003ce0 con 0x7f6648102df0 2024-09-16T00:31:26.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.177+0000 7f6645ffb640 1 -- 172.21.15.38:0/996136047 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f663c035070 con 0x7f6648102df0 2024-09-16T00:31:26.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.178+0000 7f664eb7e640 1 -- 172.21.15.38:0/996136047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66481a5210 con 0x7f6648102df0 2024-09-16T00:31:26.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.179+0000 7f664eb7e640 1 -- 172.21.15.38:0/996136047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f664806a790 con 0x7f6648102df0 2024-09-16T00:31:26.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.181+0000 7f6645ffb640 1 -- 172.21.15.38:0/996136047 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f663c042030 con 0x7f6648102df0 2024-09-16T00:31:26.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.181+0000 7f6645ffb640 1 --2- 172.21.15.38:0/996136047 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f66380a32d0 0x7f66380a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:26.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.182+0000 7f6645ffb640 1 -- 172.21.15.38:0/996136047 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f663c0f06a0 con 0x7f6648102df0 2024-09-16T00:31:26.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.182+0000 7f6647fff640 1 --2- 172.21.15.38:0/996136047 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f66380a32d0 0x7f66380a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:26.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.182+0000 7f6647fff640 1 --2- 172.21.15.38:0/996136047 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f66380a32d0 0x7f66380a5790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f664810d6e0 tx=0x7f6634005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:26.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.185+0000 7f6645ffb640 1 -- 172.21.15.38:0/996136047 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f663c0bcf80 con 0x7f6648102df0 2024-09-16T00:31:26.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.365+0000 7f664eb7e640 1 -- 172.21.15.38:0/996136047 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f66481031d0 con 0x7f6648102df0 2024-09-16T00:31:26.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.367+0000 7f6645ffb640 1 -- 172.21.15.38:0/996136047 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24674 (secure 0 0 0) 0x7f663c0bc880 con 0x7f6648102df0 2024-09-16T00:31:26.368 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:31:26.368 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":90,"fsid":"983b2072-73c1-11ef-bceb-c7b262605968","created":"2024-09-16T00:22:34.897053+0000","modified":"2024-09-16T00:31:23.431474+0000","last_up_change":"2024-09-16T00:29:34.503134+0000","last_in_change":"2024-09-16T00:29:10.135656+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:25:41.931977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:30:22.741889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:30:25.042590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:30:31.554350+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3afc20e3-869f-43a9-a911-b0e8d07de86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6803","nonce":1622028921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6805","nonce":1622028921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6809","nonce":1622028921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6807","nonce":1622028921}]},"public_addr":"172.21.15.38:6803/1622028921","cluster_addr":"172.21.15.38:6805/1622028921","heartbeat_back_addr":"172.21.15.38:6809/1622028921","heartbeat_front_addr":"172.21.15.38:6807/1622028921","state":["exists","up"]},{"osd":1,"uuid":"6d9d7acb-45ed-4543-8afc-6433fc7f096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6801","nonce":3811690336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6803","nonce":3811690336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6807","nonce":3811690336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6805","nonce":3811690336}]},"public_addr":"172.21.15.73:6801/3811690336","cluster_addr":"172.21.15.73:6803/3811690336","heartbeat_back_addr":"172.21.15.73:6807/3811690336","heartbeat_front_addr":"172.21.15.73:6805/3811690336","state":["exists","up"]},{"osd":2,"uuid":"48690f35-99e9-4f45-9bee-487857caa8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6801","nonce":1754741655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6803","nonce":1754741655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6807","nonce":1754741655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6805","nonce":1754741655}]},"public_addr":"172.21.15.122:6801/1754741655","cluster_addr":"172.21.15.122:6803/1754741655","heartbeat_back_addr":"172.21.15.122:6807/1754741655","heartbeat_front_addr":"172.21.15.122:6805/1754741655","state":["exists","up"]},{"osd":3,"uuid":"28ebb4a0-92e7-4f57-82a5-b9b761eca269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6811","nonce":3523190079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6813","nonce":3523190079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6817","nonce":3523190079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6815","nonce":3523190079}]},"public_addr":"172.21.15.38:6811/3523190079","cluster_addr":"172.21.15.38:6813/3523190079","heartbeat_back_addr":"172.21.15.38:6817/3523190079","heartbeat_front_addr":"172.21.15.38:6815/3523190079","state":["exists","up"]},{"osd":4,"uuid":"5ea363f7-7ad8-45ed-b7e4-4b010fe7f8cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6809","nonce":1469927628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6811","nonce":1469927628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6815","nonce":1469927628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6813","nonce":1469927628}]},"public_addr":"172.21.15.73:6809/1469927628","cluster_addr":"172.21.15.73:6811/1469927628","heartbeat_back_addr":"172.21.15.73:6815/1469927628","heartbeat_front_addr":"172.21.15.73:6813/1469927628","state":["exists","up"]},{"osd":5,"uuid":"e3c66d43-dd28-4bb1-a886-62a869be82cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6809","nonce":2954413058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6811","nonce":2954413058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6815","nonce":2954413058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6813","nonce":2954413058}]},"public_addr":"172.21.15.122:6809/2954413058","cluster_addr":"172.21.15.122:6811/2954413058","heartbeat_back_addr":"172.21.15.122:6815/2954413058","heartbeat_front_addr":"172.21.15.122:6813/2954413058","state":["exists","up"]},{"osd":6,"uuid":"211497ac-7c48-4095-bfdb-1d4df0149f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6819","nonce":1415719550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6821","nonce":1415719550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6825","nonce":1415719550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6823","nonce":1415719550}]},"public_addr":"172.21.15.38:6819/1415719550","cluster_addr":"172.21.15.38:6821/1415719550","heartbeat_back_addr":"172.21.15.38:6825/1415719550","heartbeat_front_addr":"172.21.15.38:6823/1415719550","state":["exists","up"]},{"osd":7,"uuid":"06512315-3f5e-44f8-976d-d3f5f1ba00e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6817","nonce":2634361473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6819","nonce":2634361473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6823","nonce":2634361473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6821","nonce":2634361473}]},"public_addr":"172.21.15.73:6817/2634361473","cluster_addr":"172.21.15.73:6819/2634361473","heartbeat_back_addr":"172.21.15.73:6823/2634361473","heartbeat_front_addr":"172.21.15.73:6821/2634361473","state":["exists","up"]},{"osd":8,"uuid":"db703c0a-1f87-46f0-be5f-f826f7f9daac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6817","nonce":1800567633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6819","nonce":1800567633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6823","nonce":1800567633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6821","nonce":1800567633}]},"public_addr":"172.21.15.122:6817/1800567633","cluster_addr":"172.21.15.122:6819/1800567633","heartbeat_back_addr":"172.21.15.122:6823/1800567633","heartbeat_front_addr":"172.21.15.122:6821/1800567633","state":["exists","up"]},{"osd":9,"uuid":"f5384200-b3b4-4936-85e1-fc57a9fe2bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6827","nonce":445727833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6829","nonce":445727833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6833","nonce":445727833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6831","nonce":445727833}]},"public_addr":"172.21.15.38:6827/445727833","cluster_addr":"172.21.15.38:6829/445727833","heartbeat_back_addr":"172.21.15.38:6833/445727833","heartbeat_front_addr":"172.21.15.38:6831/445727833","state":["exists","up"]},{"osd":10,"uuid":"c22883d9-6672-4796-a961-963d21eb37fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6825","nonce":43297139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6827","nonce":43297139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6831","nonce":43297139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6829","nonce":43297139}]},"public_addr":"172.21.15.73:6825/43297139","cluster_addr":"172.21.15.73:6827/43297139","heartbeat_back_addr":"172.21.15.73:6831/43297139","heartbeat_front_addr":"172.21.15.73:6829/43297139","state":["exists","up"]},{"osd":11,"uuid":"9f98e8cb-1bff-4b8b-aa19-56b691387383","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6825","nonce":1886446061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6827","nonce":1886446061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6831","nonce":1886446061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6829","nonce":1886446061}]},"public_addr":"172.21.15.122:6825/1886446061","cluster_addr":"172.21.15.122:6827/1886446061","heartbeat_back_addr":"172.21.15.122:6831/1886446061","heartbeat_front_addr":"172.21.15.122:6829/1886446061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:47.384012+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:12.834488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:38.770055+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:04.800466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:30.808076+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:56.646967+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:27:22.721270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:27:48.510011+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:28:13.922944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:28:40.514947+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:29:06.031052+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:29:31.882062+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6801/1779324916":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/924150447":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/141237849":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/2302081750":"2024-09-17T00:22:56.844212+0000","172.21.15.38:0/2245468594":"2024-09-17T00:22:56.844212+0000","172.21.15.38:6801/583006373":"2024-09-17T00:22:56.844212+0000","172.21.15.38:0/2435492667":"2024-09-17T00:23:21.417483+0000","172.21.15.38:6800/583006373":"2024-09-17T00:22:56.844212+0000","172.21.15.38:6800/1779324916":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/1103516837":"2024-09-17T00:22:56.844212+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:31:26.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.370+0000 7f664eb7e640 1 -- 172.21.15.38:0/996136047 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f66380a32d0 msgr2=0x7f66380a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:26.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.370+0000 7f664eb7e640 1 --2- 172.21.15.38:0/996136047 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f66380a32d0 0x7f66380a5790 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f664810d6e0 tx=0x7f6634005f90 comp rx=0 tx=0).stop 2024-09-16T00:31:26.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.370+0000 7f664eb7e640 1 -- 172.21.15.38:0/996136047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6648102df0 msgr2=0x7f6648198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:26.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.370+0000 7f664eb7e640 1 --2- 172.21.15.38:0/996136047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6648102df0 0x7f6648198690 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f663c006010 tx=0x7f663c0042f0 comp rx=0 tx=0).stop 2024-09-16T00:31:26.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.370+0000 7f664eb7e640 1 -- 172.21.15.38:0/996136047 shutdown_connections 2024-09-16T00:31:26.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.370+0000 7f664eb7e640 1 --2- 172.21.15.38:0/996136047 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f66380a32d0 0x7f66380a5790 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:26.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.370+0000 7f664eb7e640 1 --2- 172.21.15.38:0/996136047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f664810c2f0 0x7f664819cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:26.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.370+0000 7f664eb7e640 1 --2- 172.21.15.38:0/996136047 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f66481037a0 0x7f6648198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:26.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.370+0000 7f664eb7e640 1 --2- 172.21.15.38:0/996136047 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6648102df0 0x7f6648198690 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:26.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.370+0000 7f664eb7e640 1 -- 172.21.15.38:0/996136047 >> 172.21.15.38:0/996136047 conn(0x7f66480fc7d0 msgr2=0x7f66480fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:26.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.370+0000 7f664eb7e640 1 -- 172.21.15.38:0/996136047 shutdown_connections 2024-09-16T00:31:26.372 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:26.370+0000 7f664eb7e640 1 -- 172.21.15.38:0/996136047 wait complete. 2024-09-16T00:31:27.053 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-16T00:31:27.053 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:31:27.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:27 smithi073 ceph-mon[35254]: pgmap v322: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.6 KiB/s wr, 9 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-09-16T00:31:27.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:27 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:31:27.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:27 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:31:27.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:27 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/996136047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:31:27.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:27 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:31:27.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:27 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:31:27.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:27 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:31:27.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:27 smithi073 ceph-mon[35254]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-16T00:31:27.311 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:27.433 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:27 smithi038 ceph-mon[30945]: pgmap v322: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.6 KiB/s wr, 9 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-09-16T00:31:27.433 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:27 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:31:27.433 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:27 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:31:27.433 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:27 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/996136047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:31:27.433 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:27 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:31:27.433 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:27 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:31:27.433 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:27 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:31:27.433 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:27 smithi038 ceph-mon[30945]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-16T00:31:27.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:27 smithi122 ceph-mon[35412]: pgmap v322: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 2.6 KiB/s wr, 9 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-09-16T00:31:27.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:27 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:31:27.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:27 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:31:27.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:27 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/996136047' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:31:27.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:27 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:31:27.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:27 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:31:27.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:27 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:31:27.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:27 smithi122 ceph-mon[35412]: Health check failed: Reduced data availability: 1 pg inactive, 1 pg peering (PG_AVAILABILITY) 2024-09-16T00:31:27.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.816+0000 7fd291d3b640 1 -- 172.21.15.38:0/2765474574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd28c101de0 msgr2=0x7fd28c1021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:27.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.816+0000 7fd291d3b640 1 --2- 172.21.15.38:0/2765474574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd28c101de0 0x7fd28c1021c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd27c009900 tx=0x7fd27c02a410 comp rx=0 tx=0).stop 2024-09-16T00:31:27.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.816+0000 7fd291d3b640 1 -- 172.21.15.38:0/2765474574 shutdown_connections 2024-09-16T00:31:27.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.816+0000 7fd291d3b640 1 --2- 172.21.15.38:0/2765474574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd28c106f10 0x7fd28c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:27.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.816+0000 7fd291d3b640 1 --2- 172.21.15.38:0/2765474574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd28c102790 0x7fd28c1067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:27.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.816+0000 7fd291d3b640 1 --2- 172.21.15.38:0/2765474574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd28c101de0 0x7fd28c1021c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:27.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.816+0000 7fd291d3b640 1 -- 172.21.15.38:0/2765474574 >> 172.21.15.38:0/2765474574 conn(0x7fd28c0fd8c0 msgr2=0x7fd28c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:27.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.816+0000 7fd291d3b640 1 -- 172.21.15.38:0/2765474574 shutdown_connections 2024-09-16T00:31:27.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.816+0000 7fd291d3b640 1 -- 172.21.15.38:0/2765474574 wait complete. 2024-09-16T00:31:27.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.816+0000 7fd291d3b640 1 Processor -- start 2024-09-16T00:31:27.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.817+0000 7fd291d3b640 1 -- start start 2024-09-16T00:31:27.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.817+0000 7fd291d3b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd28c101de0 0x7fd28c0744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:27.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.817+0000 7fd291d3b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd28c102790 0x7fd28c074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:27.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.817+0000 7fd291d3b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd28c106f10 0x7fd28c074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:27.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.817+0000 7fd291d3b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c111f70 con 0x7fd28c101de0 2024-09-16T00:31:27.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.817+0000 7fd291d3b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c111df0 con 0x7fd28c106f10 2024-09-16T00:31:27.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.817+0000 7fd291d3b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd28c1120f0 con 0x7fd28c102790 2024-09-16T00:31:27.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.817+0000 7fd28b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd28c101de0 0x7fd28c0744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:27.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.817+0000 7fd28b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd28c101de0 0x7fd28c0744f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38720/0 (socket says 172.21.15.38:38720) 2024-09-16T00:31:27.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.817+0000 7fd28b7fe640 1 -- 172.21.15.38:0/2095536723 learned_addr learned my addr 172.21.15.38:0/2095536723 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:31:27.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.817+0000 7fd28affd640 1 --2- 172.21.15.38:0/2095536723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd28c102790 0x7fd28c074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:27.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.817+0000 7fd28bfff640 1 --2- 172.21.15.38:0/2095536723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd28c106f10 0x7fd28c074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:27.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.817+0000 7fd28b7fe640 1 -- 172.21.15.38:0/2095536723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd28c102790 msgr2=0x7fd28c074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:27.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.817+0000 7fd28b7fe640 1 --2- 172.21.15.38:0/2095536723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd28c102790 0x7fd28c074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:27.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.817+0000 7fd28b7fe640 1 -- 172.21.15.38:0/2095536723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd28c106f10 msgr2=0x7fd28c074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:27.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.817+0000 7fd28b7fe640 1 --2- 172.21.15.38:0/2095536723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd28c106f10 0x7fd28c074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:27.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.817+0000 7fd28b7fe640 1 -- 172.21.15.38:0/2095536723 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd28c1a6990 con 0x7fd28c101de0 2024-09-16T00:31:27.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.817+0000 7fd28affd640 1 --2- 172.21.15.38:0/2095536723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd28c102790 0x7fd28c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:31:27.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.817+0000 7fd28bfff640 1 --2- 172.21.15.38:0/2095536723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd28c106f10 0x7fd28c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:31:27.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.817+0000 7fd28b7fe640 1 --2- 172.21.15.38:0/2095536723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd28c101de0 0x7fd28c0744f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd27c006010 tx=0x7fd27c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:27.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.818+0000 7fd288ff9640 1 -- 172.21.15.38:0/2095536723 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27c03f030 con 0x7fd28c101de0 2024-09-16T00:31:27.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.818+0000 7fd291d3b640 1 -- 172.21.15.38:0/2095536723 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd28c1a6c20 con 0x7fd28c101de0 2024-09-16T00:31:27.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.818+0000 7fd288ff9640 1 -- 172.21.15.38:0/2095536723 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd27c003ce0 con 0x7fd28c101de0 2024-09-16T00:31:27.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.818+0000 7fd288ff9640 1 -- 172.21.15.38:0/2095536723 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd27c035070 con 0x7fd28c101de0 2024-09-16T00:31:27.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.818+0000 7fd291d3b640 1 -- 172.21.15.38:0/2095536723 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd28c1a7150 con 0x7fd28c101de0 2024-09-16T00:31:27.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.819+0000 7fd291d3b640 1 -- 172.21.15.38:0/2095536723 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd28c1021c0 con 0x7fd28c101de0 2024-09-16T00:31:27.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.820+0000 7fd288ff9640 1 -- 172.21.15.38:0/2095536723 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fd27c042030 con 0x7fd28c101de0 2024-09-16T00:31:27.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.821+0000 7fd288ff9640 1 --2- 172.21.15.38:0/2095536723 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd2740a32d0 0x7fd2740a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:27.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.821+0000 7fd28affd640 1 --2- 172.21.15.38:0/2095536723 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd2740a32d0 0x7fd2740a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:27.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.821+0000 7fd288ff9640 1 -- 172.21.15.38:0/2095536723 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7fd27c0f0330 con 0x7fd28c101de0 2024-09-16T00:31:27.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.822+0000 7fd28affd640 1 --2- 172.21.15.38:0/2095536723 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd2740a32d0 0x7fd2740a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd2800058c0 tx=0x7fd280005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:27.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:27.825+0000 7fd288ff9640 1 -- 172.21.15.38:0/2095536723 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd27c0bcd10 con 0x7fd28c101de0 2024-09-16T00:31:28.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:28.003+0000 7fd291d3b640 1 -- 172.21.15.38:0/2095536723 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd28c10a0f0 con 0x7fd28c101de0 2024-09-16T00:31:28.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:28.005+0000 7fd288ff9640 1 -- 172.21.15.38:0/2095536723 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24674 (secure 0 0 0) 0x7fd27c0bc610 con 0x7fd28c101de0 2024-09-16T00:31:28.006 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:31:28.006 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":90,"fsid":"983b2072-73c1-11ef-bceb-c7b262605968","created":"2024-09-16T00:22:34.897053+0000","modified":"2024-09-16T00:31:23.431474+0000","last_up_change":"2024-09-16T00:29:34.503134+0000","last_in_change":"2024-09-16T00:29:10.135656+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:25:41.931977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:30:22.741889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:30:25.042590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:30:31.554350+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3afc20e3-869f-43a9-a911-b0e8d07de86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6803","nonce":1622028921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6805","nonce":1622028921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6809","nonce":1622028921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6807","nonce":1622028921}]},"public_addr":"172.21.15.38:6803/1622028921","cluster_addr":"172.21.15.38:6805/1622028921","heartbeat_back_addr":"172.21.15.38:6809/1622028921","heartbeat_front_addr":"172.21.15.38:6807/1622028921","state":["exists","up"]},{"osd":1,"uuid":"6d9d7acb-45ed-4543-8afc-6433fc7f096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6801","nonce":3811690336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6803","nonce":3811690336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6807","nonce":3811690336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6805","nonce":3811690336}]},"public_addr":"172.21.15.73:6801/3811690336","cluster_addr":"172.21.15.73:6803/3811690336","heartbeat_back_addr":"172.21.15.73:6807/3811690336","heartbeat_front_addr":"172.21.15.73:6805/3811690336","state":["exists","up"]},{"osd":2,"uuid":"48690f35-99e9-4f45-9bee-487857caa8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6801","nonce":1754741655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6803","nonce":1754741655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6807","nonce":1754741655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6805","nonce":1754741655}]},"public_addr":"172.21.15.122:6801/1754741655","cluster_addr":"172.21.15.122:6803/1754741655","heartbeat_back_addr":"172.21.15.122:6807/1754741655","heartbeat_front_addr":"172.21.15.122:6805/1754741655","state":["exists","up"]},{"osd":3,"uuid":"28ebb4a0-92e7-4f57-82a5-b9b761eca269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6811","nonce":3523190079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6813","nonce":3523190079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6817","nonce":3523190079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6815","nonce":3523190079}]},"public_addr":"172.21.15.38:6811/3523190079","cluster_addr":"172.21.15.38:6813/3523190079","heartbeat_back_addr":"172.21.15.38:6817/3523190079","heartbeat_front_addr":"172.21.15.38:6815/3523190079","state":["exists","up"]},{"osd":4,"uuid":"5ea363f7-7ad8-45ed-b7e4-4b010fe7f8cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6809","nonce":1469927628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6811","nonce":1469927628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6815","nonce":1469927628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6813","nonce":1469927628}]},"public_addr":"172.21.15.73:6809/1469927628","cluster_addr":"172.21.15.73:6811/1469927628","heartbeat_back_addr":"172.21.15.73:6815/1469927628","heartbeat_front_addr":"172.21.15.73:6813/1469927628","state":["exists","up"]},{"osd":5,"uuid":"e3c66d43-dd28-4bb1-a886-62a869be82cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6809","nonce":2954413058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6811","nonce":2954413058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6815","nonce":2954413058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6813","nonce":2954413058}]},"public_addr":"172.21.15.122:6809/2954413058","cluster_addr":"172.21.15.122:6811/2954413058","heartbeat_back_addr":"172.21.15.122:6815/2954413058","heartbeat_front_addr":"172.21.15.122:6813/2954413058","state":["exists","up"]},{"osd":6,"uuid":"211497ac-7c48-4095-bfdb-1d4df0149f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6819","nonce":1415719550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6821","nonce":1415719550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6825","nonce":1415719550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6823","nonce":1415719550}]},"public_addr":"172.21.15.38:6819/1415719550","cluster_addr":"172.21.15.38:6821/1415719550","heartbeat_back_addr":"172.21.15.38:6825/1415719550","heartbeat_front_addr":"172.21.15.38:6823/1415719550","state":["exists","up"]},{"osd":7,"uuid":"06512315-3f5e-44f8-976d-d3f5f1ba00e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6817","nonce":2634361473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6819","nonce":2634361473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6823","nonce":2634361473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6821","nonce":2634361473}]},"public_addr":"172.21.15.73:6817/2634361473","cluster_addr":"172.21.15.73:6819/2634361473","heartbeat_back_addr":"172.21.15.73:6823/2634361473","heartbeat_front_addr":"172.21.15.73:6821/2634361473","state":["exists","up"]},{"osd":8,"uuid":"db703c0a-1f87-46f0-be5f-f826f7f9daac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6817","nonce":1800567633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6819","nonce":1800567633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6823","nonce":1800567633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6821","nonce":1800567633}]},"public_addr":"172.21.15.122:6817/1800567633","cluster_addr":"172.21.15.122:6819/1800567633","heartbeat_back_addr":"172.21.15.122:6823/1800567633","heartbeat_front_addr":"172.21.15.122:6821/1800567633","state":["exists","up"]},{"osd":9,"uuid":"f5384200-b3b4-4936-85e1-fc57a9fe2bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6827","nonce":445727833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6829","nonce":445727833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6833","nonce":445727833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6831","nonce":445727833}]},"public_addr":"172.21.15.38:6827/445727833","cluster_addr":"172.21.15.38:6829/445727833","heartbeat_back_addr":"172.21.15.38:6833/445727833","heartbeat_front_addr":"172.21.15.38:6831/445727833","state":["exists","up"]},{"osd":10,"uuid":"c22883d9-6672-4796-a961-963d21eb37fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6825","nonce":43297139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6827","nonce":43297139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6831","nonce":43297139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6829","nonce":43297139}]},"public_addr":"172.21.15.73:6825/43297139","cluster_addr":"172.21.15.73:6827/43297139","heartbeat_back_addr":"172.21.15.73:6831/43297139","heartbeat_front_addr":"172.21.15.73:6829/43297139","state":["exists","up"]},{"osd":11,"uuid":"9f98e8cb-1bff-4b8b-aa19-56b691387383","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6825","nonce":1886446061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6827","nonce":1886446061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6831","nonce":1886446061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6829","nonce":1886446061}]},"public_addr":"172.21.15.122:6825/1886446061","cluster_addr":"172.21.15.122:6827/1886446061","heartbeat_back_addr":"172.21.15.122:6831/1886446061","heartbeat_front_addr":"172.21.15.122:6829/1886446061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:47.384012+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:12.834488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:38.770055+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:04.800466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:30.808076+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:56.646967+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:27:22.721270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:27:48.510011+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:28:13.922944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:28:40.514947+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:29:06.031052+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:29:31.882062+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6801/1779324916":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/924150447":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/141237849":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/2302081750":"2024-09-17T00:22:56.844212+0000","172.21.15.38:0/2245468594":"2024-09-17T00:22:56.844212+0000","172.21.15.38:6801/583006373":"2024-09-17T00:22:56.844212+0000","172.21.15.38:0/2435492667":"2024-09-17T00:23:21.417483+0000","172.21.15.38:6800/583006373":"2024-09-17T00:22:56.844212+0000","172.21.15.38:6800/1779324916":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/1103516837":"2024-09-17T00:22:56.844212+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:31:28.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:28.007+0000 7fd291d3b640 1 -- 172.21.15.38:0/2095536723 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd2740a32d0 msgr2=0x7fd2740a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:28.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:28.007+0000 7fd291d3b640 1 --2- 172.21.15.38:0/2095536723 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd2740a32d0 0x7fd2740a5790 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fd2800058c0 tx=0x7fd280005850 comp rx=0 tx=0).stop 2024-09-16T00:31:28.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:28.007+0000 7fd291d3b640 1 -- 172.21.15.38:0/2095536723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd28c101de0 msgr2=0x7fd28c0744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:28.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:28.007+0000 7fd291d3b640 1 --2- 172.21.15.38:0/2095536723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd28c101de0 0x7fd28c0744f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd27c006010 tx=0x7fd27c0042f0 comp rx=0 tx=0).stop 2024-09-16T00:31:28.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:28.007+0000 7fd291d3b640 1 -- 172.21.15.38:0/2095536723 shutdown_connections 2024-09-16T00:31:28.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:28.007+0000 7fd291d3b640 1 --2- 172.21.15.38:0/2095536723 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd2740a32d0 0x7fd2740a5790 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:28.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:28.007+0000 7fd291d3b640 1 --2- 172.21.15.38:0/2095536723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd28c106f10 0x7fd28c074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:28.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:28.007+0000 7fd291d3b640 1 --2- 172.21.15.38:0/2095536723 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd28c102790 0x7fd28c074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:28.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:28.007+0000 7fd291d3b640 1 --2- 172.21.15.38:0/2095536723 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd28c101de0 0x7fd28c0744f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:28.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:28.007+0000 7fd291d3b640 1 -- 172.21.15.38:0/2095536723 >> 172.21.15.38:0/2095536723 conn(0x7fd28c0fd8c0 msgr2=0x7fd28c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:28.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:28.007+0000 7fd291d3b640 1 -- 172.21.15.38:0/2095536723 shutdown_connections 2024-09-16T00:31:28.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:28.007+0000 7fd291d3b640 1 -- 172.21.15.38:0/2095536723 wait complete. 2024-09-16T00:31:28.774 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-16T00:31:28.774 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-16T00:31:28.774 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-16T00:31:28.775 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-16T00:31:28.775 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-16T00:31:28.776 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-16T00:31:28.776 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-16T00:31:28.776 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-16T00:31:28.777 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-09-16T00:31:28.778 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-09-16T00:31:28.778 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-09-16T00:31:28.779 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-09-16T00:31:29.010 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:29 smithi038 ceph-mon[30945]: pgmap v323: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 639 B/s wr, 5 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-09-16T00:31:29.010 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:29 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/2095536723' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:31:29.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:29 smithi073 ceph-mon[35254]: pgmap v323: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 639 B/s wr, 5 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-09-16T00:31:29.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:29 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/2095536723' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:31:29.439 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:29.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:29 smithi122 ceph-mon[35412]: pgmap v323: 129 pgs: 2 peering, 127 active+clean; 640 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 639 B/s wr, 5 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-09-16T00:31:29.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:29 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/2095536723' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:31:29.496 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:29.496 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:30.214 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:30 smithi038 ceph-mon[30945]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-16T00:31:30.214 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:30 smithi038 ceph-mon[30945]: Cluster is now healthy 2024-09-16T00:31:30.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.213+0000 7f15c3207640 1 -- 172.21.15.38:0/1446998035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15bc101420 msgr2=0x7f15bc10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:30.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.213+0000 7f15c3207640 1 --2- 172.21.15.38:0/1446998035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15bc101420 0x7f15bc10d950 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f15b4009cd0 tx=0x7f15b402a8e0 comp rx=0 tx=0).stop 2024-09-16T00:31:30.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.213+0000 7f15c3207640 1 -- 172.21.15.38:0/1446998035 shutdown_connections 2024-09-16T00:31:30.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.213+0000 7f15c3207640 1 --2- 172.21.15.38:0/1446998035 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15bc101420 0x7f15bc10d950 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:30.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.213+0000 7f15c3207640 1 --2- 172.21.15.38:0/1446998035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15bc100a80 0x7f15bc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:30.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.213+0000 7f15c3207640 1 --2- 172.21.15.38:0/1446998035 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15bc106a80 0x7f15bc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:30.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.213+0000 7f15c3207640 1 -- 172.21.15.38:0/1446998035 >> 172.21.15.38:0/1446998035 conn(0x7f15bc0fc7a0 msgr2=0x7f15bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:30.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.213+0000 7f15c3207640 1 -- 172.21.15.38:0/1446998035 shutdown_connections 2024-09-16T00:31:30.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.213+0000 7f15c3207640 1 -- 172.21.15.38:0/1446998035 wait complete. 2024-09-16T00:31:30.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.214+0000 7f15c3207640 1 Processor -- start 2024-09-16T00:31:30.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.214+0000 7f15c3207640 1 -- start start 2024-09-16T00:31:30.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.214+0000 7f15c3207640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15bc100a80 0x7f15bc1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:30.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.214+0000 7f15c3207640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15bc101420 0x7f15bc198900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:30.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.214+0000 7f15c3207640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15bc106a80 0x7f15bc19cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:30.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.214+0000 7f15c3207640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc111fa0 con 0x7f15bc101420 2024-09-16T00:31:30.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.214+0000 7f15c3207640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc111e20 con 0x7f15bc100a80 2024-09-16T00:31:30.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.214+0000 7f15c3207640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc112120 con 0x7f15bc106a80 2024-09-16T00:31:30.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.214+0000 7f15bbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15bc101420 0x7f15bc198900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:30.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.214+0000 7f15bbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15bc101420 0x7f15bc198900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38726/0 (socket says 172.21.15.38:38726) 2024-09-16T00:31:30.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.214+0000 7f15bbfff640 1 -- 172.21.15.38:0/1971654210 learned_addr learned my addr 172.21.15.38:0/1971654210 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:31:30.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.214+0000 7f15c0fa3640 1 --2- 172.21.15.38:0/1971654210 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15bc100a80 0x7f15bc1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:30.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.214+0000 7f15c17a4640 1 --2- 172.21.15.38:0/1971654210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15bc106a80 0x7f15bc19cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:30.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.215+0000 7f15bbfff640 1 -- 172.21.15.38:0/1971654210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15bc106a80 msgr2=0x7f15bc19cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:30.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.215+0000 7f15bbfff640 1 --2- 172.21.15.38:0/1971654210 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15bc106a80 0x7f15bc19cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:30.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.215+0000 7f15bbfff640 1 -- 172.21.15.38:0/1971654210 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15bc100a80 msgr2=0x7f15bc1983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:30.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.215+0000 7f15bbfff640 1 --2- 172.21.15.38:0/1971654210 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15bc100a80 0x7f15bc1983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:30.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.215+0000 7f15bbfff640 1 -- 172.21.15.38:0/1971654210 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15bc19d370 con 0x7f15bc101420 2024-09-16T00:31:30.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.215+0000 7f15bbfff640 1 --2- 172.21.15.38:0/1971654210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15bc101420 0x7f15bc198900 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f15b000d950 tx=0x7f15b000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:30.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.215+0000 7f15b9ffb640 1 -- 172.21.15.38:0/1971654210 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b0017070 con 0x7f15bc101420 2024-09-16T00:31:30.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.215+0000 7f15b9ffb640 1 -- 172.21.15.38:0/1971654210 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15b0009c30 con 0x7f15bc101420 2024-09-16T00:31:30.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.215+0000 7f15c3207640 1 -- 172.21.15.38:0/1971654210 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15bc19d660 con 0x7f15bc101420 2024-09-16T00:31:30.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.215+0000 7f15b9ffb640 1 -- 172.21.15.38:0/1971654210 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b000f6b0 con 0x7f15bc101420 2024-09-16T00:31:30.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.215+0000 7f15c3207640 1 -- 172.21.15.38:0/1971654210 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15bc1a4f40 con 0x7f15bc101420 2024-09-16T00:31:30.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.216+0000 7f15c3207640 1 -- 172.21.15.38:0/1971654210 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f15bc0fe960 con 0x7f15bc101420 2024-09-16T00:31:30.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.217+0000 7f15b9ffb640 1 -- 172.21.15.38:0/1971654210 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f15b00109d0 con 0x7f15bc101420 2024-09-16T00:31:30.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.218+0000 7f15b9ffb640 1 --2- 172.21.15.38:0/1971654210 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f15a40a3350 0x7f15a40a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:30.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.218+0000 7f15c0fa3640 1 --2- 172.21.15.38:0/1971654210 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f15a40a3350 0x7f15a40a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:30.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.218+0000 7f15b9ffb640 1 -- 172.21.15.38:0/1971654210 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f15b00cfe10 con 0x7f15bc101420 2024-09-16T00:31:30.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.218+0000 7f15b9ffb640 1 --2- 172.21.15.38:0/1971654210 >> [v2:172.21.15.122:6816/1800567633,v1:172.21.15.122:6817/1800567633] conn(0x7f15a40ae4a0 0x7f15a40b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:30.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.218+0000 7f15b9ffb640 1 -- 172.21.15.38:0/1971654210 --> [v2:172.21.15.122:6816/1800567633,v1:172.21.15.122:6817/1800567633] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f15b001de50 con 0x7f15a40ae4a0 2024-09-16T00:31:30.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.218+0000 7f15b9ffb640 1 -- 172.21.15.38:0/1971654210 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f15b00d01c0 con 0x7f15bc101420 2024-09-16T00:31:30.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.218+0000 7f15c0fa3640 1 --2- 172.21.15.38:0/1971654210 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f15a40a3350 0x7f15a40a5810 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00bb60 tx=0x7f15ac00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:30.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.218+0000 7f15c17a4640 1 --2- 172.21.15.38:0/1971654210 >> [v2:172.21.15.122:6816/1800567633,v1:172.21.15.122:6817/1800567633] conn(0x7f15a40ae4a0 0x7f15a40b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:30.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.219+0000 7f15c17a4640 1 --2- 172.21.15.38:0/1971654210 >> [v2:172.21.15.122:6816/1800567633,v1:172.21.15.122:6817/1800567633] conn(0x7f15a40ae4a0 0x7f15a40b0940 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:30.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.220+0000 7f15b9ffb640 1 -- 172.21.15.38:0/1971654210 <== osd.8 v2:172.21.15.122:6816/1800567633 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f15b001de50 con 0x7f15a40ae4a0 2024-09-16T00:31:30.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.234+0000 7f15c3207640 1 -- 172.21.15.38:0/1971654210 --> [v2:172.21.15.122:6816/1800567633,v1:172.21.15.122:6817/1800567633] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f15bc1021c0 con 0x7f15a40ae4a0 2024-09-16T00:31:30.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.235+0000 7f15b9ffb640 1 -- 172.21.15.38:0/1971654210 <== osd.8 v2:172.21.15.122:6816/1800567633 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f15bc1021c0 con 0x7f15a40ae4a0 2024-09-16T00:31:30.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.235+0000 7f15c3207640 1 -- 172.21.15.38:0/1971654210 >> [v2:172.21.15.122:6816/1800567633,v1:172.21.15.122:6817/1800567633] conn(0x7f15a40ae4a0 msgr2=0x7f15a40b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:30.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.235+0000 7f15c3207640 1 --2- 172.21.15.38:0/1971654210 >> [v2:172.21.15.122:6816/1800567633,v1:172.21.15.122:6817/1800567633] conn(0x7f15a40ae4a0 0x7f15a40b0940 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:30.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.235+0000 7f15c3207640 1 -- 172.21.15.38:0/1971654210 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f15a40a3350 msgr2=0x7f15a40a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:30.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.235+0000 7f15c3207640 1 --2- 172.21.15.38:0/1971654210 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f15a40a3350 0x7f15a40a5810 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f15ac00bb60 tx=0x7f15ac00b390 comp rx=0 tx=0).stop 2024-09-16T00:31:30.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.235+0000 7f15c3207640 1 -- 172.21.15.38:0/1971654210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15bc101420 msgr2=0x7f15bc198900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:30.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.235+0000 7f15c3207640 1 --2- 172.21.15.38:0/1971654210 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15bc101420 0x7f15bc198900 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f15b000d950 tx=0x7f15b000de10 comp rx=0 tx=0).stop 2024-09-16T00:31:30.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.235+0000 7f15c17a4640 1 -- 172.21.15.38:0/1971654210 reap_dead start 2024-09-16T00:31:30.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.235+0000 7f15c3207640 1 -- 172.21.15.38:0/1971654210 shutdown_connections 2024-09-16T00:31:30.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.235+0000 7f15c3207640 1 -- 172.21.15.38:0/1971654210 >> 172.21.15.38:0/1971654210 conn(0x7f15bc0fc7a0 msgr2=0x7f15bc0fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:30.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.236+0000 7f15c3207640 1 -- 172.21.15.38:0/1971654210 shutdown_connections 2024-09-16T00:31:30.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.236+0000 7f15c3207640 1 -- 172.21.15.38:0/1971654210 wait complete. 2024-09-16T00:31:30.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:30 smithi122 ceph-mon[35412]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-16T00:31:30.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:30 smithi122 ceph-mon[35412]: Cluster is now healthy 2024-09-16T00:31:30.487 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:30.489 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:30.489 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:30.490 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:30.490 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:30.490 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:30.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:30 smithi073 ceph-mon[35254]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg inactive, 1 pg peering) 2024-09-16T00:31:30.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:30 smithi073 ceph-mon[35254]: Cluster is now healthy 2024-09-16T00:31:30.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.688+0000 7feaef30a640 1 -- 172.21.15.38:0/2855383057 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feae810c170 msgr2=0x7feae810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:30.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.688+0000 7feaef30a640 1 --2- 172.21.15.38:0/2855383057 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feae810c170 0x7feae810e610 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fead8009ca0 tx=0x7fead802a950 comp rx=0 tx=0).stop 2024-09-16T00:31:30.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.688+0000 7feaef30a640 1 -- 172.21.15.38:0/2855383057 shutdown_connections 2024-09-16T00:31:30.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.688+0000 7feaef30a640 1 --2- 172.21.15.38:0/2855383057 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feae810c170 0x7feae810e610 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:30.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.688+0000 7feaef30a640 1 --2- 172.21.15.38:0/2855383057 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feae8103660 0x7feae810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:30.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.688+0000 7feaef30a640 1 --2- 172.21.15.38:0/2855383057 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feae8102d40 0x7feae8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:30.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.688+0000 7feaef30a640 1 -- 172.21.15.38:0/2855383057 >> 172.21.15.38:0/2855383057 conn(0x7feae80fc7a0 msgr2=0x7feae80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:30.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.689+0000 7feaef30a640 1 -- 172.21.15.38:0/2855383057 shutdown_connections 2024-09-16T00:31:30.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.689+0000 7feaef30a640 1 -- 172.21.15.38:0/2855383057 wait complete. 2024-09-16T00:31:30.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.689+0000 7feaef30a640 1 Processor -- start 2024-09-16T00:31:30.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.689+0000 7feaef30a640 1 -- start start 2024-09-16T00:31:30.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.689+0000 7feaef30a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feae8102d40 0x7feae81009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:30.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.689+0000 7feaef30a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feae8103660 0x7feae8100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:30.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.689+0000 7feaef30a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feae810c170 0x7feae8101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:30.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.689+0000 7feaef30a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7feae8111f30 con 0x7feae8102d40 2024-09-16T00:31:30.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.689+0000 7feaef30a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7feae8111db0 con 0x7feae8103660 2024-09-16T00:31:30.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.689+0000 7feaef30a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feae81120b0 con 0x7feae810c170 2024-09-16T00:31:30.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.690+0000 7feaed0a6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feae8102d40 0x7feae81009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:30.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.690+0000 7feaed0a6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feae8102d40 0x7feae81009a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38736/0 (socket says 172.21.15.38:38736) 2024-09-16T00:31:30.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.690+0000 7feaed0a6640 1 -- 172.21.15.38:0/946275995 learned_addr learned my addr 172.21.15.38:0/946275995 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:31:30.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.690+0000 7feaed8a7640 1 --2- 172.21.15.38:0/946275995 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feae810c170 0x7feae8101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:30.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.690+0000 7feaec8a5640 1 --2- 172.21.15.38:0/946275995 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feae8103660 0x7feae8100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:30.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.690+0000 7feaed0a6640 1 -- 172.21.15.38:0/946275995 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feae810c170 msgr2=0x7feae8101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:30.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.690+0000 7feaed0a6640 1 --2- 172.21.15.38:0/946275995 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feae810c170 0x7feae8101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:30.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.690+0000 7feaed0a6640 1 -- 172.21.15.38:0/946275995 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feae8103660 msgr2=0x7feae8100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:30.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.690+0000 7feaed0a6640 1 --2- 172.21.15.38:0/946275995 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feae8103660 0x7feae8100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:30.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.690+0000 7feaed0a6640 1 -- 172.21.15.38:0/946275995 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feae81a68b0 con 0x7feae8102d40 2024-09-16T00:31:30.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.690+0000 7feaed0a6640 1 --2- 172.21.15.38:0/946275995 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feae8102d40 0x7feae81009a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7feae000d950 tx=0x7feae000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:30.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.690+0000 7feae67fc640 1 -- 172.21.15.38:0/946275995 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae0017070 con 0x7feae8102d40 2024-09-16T00:31:30.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.690+0000 7feae67fc640 1 -- 172.21.15.38:0/946275995 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7feae0009c30 con 0x7feae8102d40 2024-09-16T00:31:30.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.690+0000 7feaef30a640 1 -- 172.21.15.38:0/946275995 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feae81a6ba0 con 0x7feae8102d40 2024-09-16T00:31:30.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.690+0000 7feae67fc640 1 -- 172.21.15.38:0/946275995 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feae000f6b0 con 0x7feae8102d40 2024-09-16T00:31:30.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.690+0000 7feaef30a640 1 -- 172.21.15.38:0/946275995 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feae81a7130 con 0x7feae8102d40 2024-09-16T00:31:30.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.691+0000 7feaef30a640 1 -- 172.21.15.38:0/946275995 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7feae80628c0 con 0x7feae8102d40 2024-09-16T00:31:30.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.693+0000 7feae67fc640 1 -- 172.21.15.38:0/946275995 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7feae00109d0 con 0x7feae8102d40 2024-09-16T00:31:30.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.694+0000 7feae67fc640 1 --2- 172.21.15.38:0/946275995 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fead00a3350 0x7fead00a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:30.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.694+0000 7feaec8a5640 1 --2- 172.21.15.38:0/946275995 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fead00a3350 0x7fead00a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:30.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.694+0000 7feae67fc640 1 -- 172.21.15.38:0/946275995 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7feae00cfd00 con 0x7feae8102d40 2024-09-16T00:31:30.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.694+0000 7feae67fc640 1 --2- 172.21.15.38:0/946275995 >> [v2:172.21.15.38:6810/3523190079,v1:172.21.15.38:6811/3523190079] conn(0x7fead00ae4a0 0x7fead00b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:30.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.694+0000 7feae67fc640 1 -- 172.21.15.38:0/946275995 --> [v2:172.21.15.38:6810/3523190079,v1:172.21.15.38:6811/3523190079] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7feae001ce50 con 0x7fead00ae4a0 2024-09-16T00:31:30.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.694+0000 7feae67fc640 1 -- 172.21.15.38:0/946275995 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7feae00d0150 con 0x7feae8102d40 2024-09-16T00:31:30.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.694+0000 7feaed8a7640 1 --2- 172.21.15.38:0/946275995 >> [v2:172.21.15.38:6810/3523190079,v1:172.21.15.38:6811/3523190079] conn(0x7fead00ae4a0 0x7fead00b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:30.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.694+0000 7feaec8a5640 1 --2- 172.21.15.38:0/946275995 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fead00a3350 0x7fead00a5810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7feae8102430 tx=0x7feadc00b2e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:30.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.695+0000 7feaed8a7640 1 --2- 172.21.15.38:0/946275995 >> [v2:172.21.15.38:6810/3523190079,v1:172.21.15.38:6811/3523190079] conn(0x7fead00ae4a0 0x7fead00b0940 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:30.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.695+0000 7feae67fc640 1 -- 172.21.15.38:0/946275995 <== osd.3 v2:172.21.15.38:6810/3523190079 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7feae001ce50 con 0x7fead00ae4a0 2024-09-16T00:31:30.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.709+0000 7feaef30a640 1 -- 172.21.15.38:0/946275995 --> [v2:172.21.15.38:6810/3523190079,v1:172.21.15.38:6811/3523190079] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7feae8062cc0 con 0x7fead00ae4a0 2024-09-16T00:31:30.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.710+0000 7feae67fc640 1 -- 172.21.15.38:0/946275995 <== osd.3 v2:172.21.15.38:6810/3523190079 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7feae8062cc0 con 0x7fead00ae4a0 2024-09-16T00:31:30.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.710+0000 7feaef30a640 1 -- 172.21.15.38:0/946275995 >> [v2:172.21.15.38:6810/3523190079,v1:172.21.15.38:6811/3523190079] conn(0x7fead00ae4a0 msgr2=0x7fead00b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:30.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.710+0000 7feaef30a640 1 --2- 172.21.15.38:0/946275995 >> [v2:172.21.15.38:6810/3523190079,v1:172.21.15.38:6811/3523190079] conn(0x7fead00ae4a0 0x7fead00b0940 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:30.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.710+0000 7feaef30a640 1 -- 172.21.15.38:0/946275995 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fead00a3350 msgr2=0x7fead00a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:30.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.710+0000 7feaef30a640 1 --2- 172.21.15.38:0/946275995 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fead00a3350 0x7fead00a5810 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7feae8102430 tx=0x7feadc00b2e0 comp rx=0 tx=0).stop 2024-09-16T00:31:30.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.710+0000 7feaef30a640 1 -- 172.21.15.38:0/946275995 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feae8102d40 msgr2=0x7feae81009a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:30.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.710+0000 7feaef30a640 1 --2- 172.21.15.38:0/946275995 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feae8102d40 0x7feae81009a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7feae000d950 tx=0x7feae000de10 comp rx=0 tx=0).stop 2024-09-16T00:31:30.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.710+0000 7feaed8a7640 1 -- 172.21.15.38:0/946275995 reap_dead start 2024-09-16T00:31:30.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.710+0000 7feaef30a640 1 -- 172.21.15.38:0/946275995 shutdown_connections 2024-09-16T00:31:30.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.710+0000 7feaef30a640 1 -- 172.21.15.38:0/946275995 >> 172.21.15.38:0/946275995 conn(0x7feae80fc7a0 msgr2=0x7feae80fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:30.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.710+0000 7feaef30a640 1 -- 172.21.15.38:0/946275995 shutdown_connections 2024-09-16T00:31:30.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:30.710+0000 7feaef30a640 1 -- 172.21.15.38:0/946275995 wait complete. 2024-09-16T00:31:30.952 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:30.978 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:31.349 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:31 smithi038 ceph-mon[30945]: pgmap v324: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1023 B/s wr, 7 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-09-16T00:31:31.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:31 smithi122 ceph-mon[35412]: pgmap v324: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1023 B/s wr, 7 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-09-16T00:31:31.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:31 smithi073 ceph-mon[35254]: pgmap v324: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1023 B/s wr, 7 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-09-16T00:31:32.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.417+0000 7f20ba22b640 1 -- 172.21.15.38:0/3510352650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b4102d40 msgr2=0x7f20b4103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:32.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.417+0000 7f20ba22b640 1 --2- 172.21.15.38:0/3510352650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b4102d40 0x7f20b4103120 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f20a4009900 tx=0x7f20a402a480 comp rx=0 tx=0).stop 2024-09-16T00:31:32.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.417+0000 7f20ba22b640 1 -- 172.21.15.38:0/3510352650 shutdown_connections 2024-09-16T00:31:32.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.417+0000 7f20ba22b640 1 --2- 172.21.15.38:0/3510352650 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20b410c170 0x7f20b410e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:32.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.417+0000 7f20ba22b640 1 --2- 172.21.15.38:0/3510352650 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20b4103660 0x7f20b410bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:32.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.417+0000 7f20ba22b640 1 --2- 172.21.15.38:0/3510352650 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b4102d40 0x7f20b4103120 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:32.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.417+0000 7f20ba22b640 1 -- 172.21.15.38:0/3510352650 >> 172.21.15.38:0/3510352650 conn(0x7f20b40fc7a0 msgr2=0x7f20b40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:32.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.417+0000 7f20ba22b640 1 -- 172.21.15.38:0/3510352650 shutdown_connections 2024-09-16T00:31:32.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.418+0000 7f20ba22b640 1 -- 172.21.15.38:0/3510352650 wait complete. 2024-09-16T00:31:32.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.418+0000 7f20ba22b640 1 Processor -- start 2024-09-16T00:31:32.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.418+0000 7f20ba22b640 1 -- start start 2024-09-16T00:31:32.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.419+0000 7f20ba22b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20b4102d40 0x7f20b41009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:32.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.419+0000 7f20ba22b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b4103660 0x7f20b4100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:32.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.419+0000 7f20ba22b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20b410c170 0x7f20b4101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:32.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.419+0000 7f20ba22b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f20b4111f70 con 0x7f20b4103660 2024-09-16T00:31:32.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.419+0000 7f20ba22b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f20b4111df0 con 0x7f20b410c170 2024-09-16T00:31:32.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.419+0000 7f20ba22b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f20b41120f0 con 0x7f20b4102d40 2024-09-16T00:31:32.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.419+0000 7f20b2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b4103660 0x7f20b4100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:32.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.419+0000 7f20b2ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b4103660 0x7f20b4100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38758/0 (socket says 172.21.15.38:38758) 2024-09-16T00:31:32.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.419+0000 7f20b2ffd640 1 -- 172.21.15.38:0/1690840306 learned_addr learned my addr 172.21.15.38:0/1690840306 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:31:32.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.419+0000 7f20b37fe640 1 --2- 172.21.15.38:0/1690840306 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20b4102d40 0x7f20b41009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:32.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.419+0000 7f20b3fff640 1 --2- 172.21.15.38:0/1690840306 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20b410c170 0x7f20b4101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:32.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.419+0000 7f20b2ffd640 1 -- 172.21.15.38:0/1690840306 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20b4102d40 msgr2=0x7f20b41009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:32.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.419+0000 7f20b2ffd640 1 --2- 172.21.15.38:0/1690840306 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20b4102d40 0x7f20b41009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:32.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.419+0000 7f20b2ffd640 1 -- 172.21.15.38:0/1690840306 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20b410c170 msgr2=0x7f20b4101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:32.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.419+0000 7f20b2ffd640 1 --2- 172.21.15.38:0/1690840306 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20b410c170 0x7f20b4101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:32.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.419+0000 7f20b2ffd640 1 -- 172.21.15.38:0/1690840306 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b41a6910 con 0x7f20b4103660 2024-09-16T00:31:32.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.419+0000 7f20b2ffd640 1 --2- 172.21.15.38:0/1690840306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b4103660 0x7f20b4100ee0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f20a800d750 tx=0x7f20a800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:32.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.419+0000 7f20b0ff9640 1 -- 172.21.15.38:0/1690840306 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a8017070 con 0x7f20b4103660 2024-09-16T00:31:32.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.419+0000 7f20b0ff9640 1 -- 172.21.15.38:0/1690840306 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20a8005020 con 0x7f20b4103660 2024-09-16T00:31:32.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.419+0000 7f20ba22b640 1 -- 172.21.15.38:0/1690840306 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20b41a6c00 con 0x7f20b4103660 2024-09-16T00:31:32.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.419+0000 7f20b0ff9640 1 -- 172.21.15.38:0/1690840306 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a8004d00 con 0x7f20b4103660 2024-09-16T00:31:32.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.419+0000 7f20ba22b640 1 -- 172.21.15.38:0/1690840306 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20b41a70f0 con 0x7f20b4103660 2024-09-16T00:31:32.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.420+0000 7f20ba22b640 1 -- 172.21.15.38:0/1690840306 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f20b40628c0 con 0x7f20b4103660 2024-09-16T00:31:32.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.422+0000 7f20b0ff9640 1 -- 172.21.15.38:0/1690840306 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f20a8005560 con 0x7f20b4103660 2024-09-16T00:31:32.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.422+0000 7f20b0ff9640 1 --2- 172.21.15.38:0/1690840306 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f209c0a32d0 0x7f209c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:32.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.423+0000 7f20b37fe640 1 --2- 172.21.15.38:0/1690840306 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f209c0a32d0 0x7f209c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:32.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.423+0000 7f20b0ff9640 1 -- 172.21.15.38:0/1690840306 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f20a80cfb60 con 0x7f20b4103660 2024-09-16T00:31:32.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.423+0000 7f20b0ff9640 1 --2- 172.21.15.38:0/1690840306 >> [v2:172.21.15.73:6816/2634361473,v1:172.21.15.73:6817/2634361473] conn(0x7f209c0ae420 0x7f209c0b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:32.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.423+0000 7f20b0ff9640 1 -- 172.21.15.38:0/1690840306 --> [v2:172.21.15.73:6816/2634361473,v1:172.21.15.73:6817/2634361473] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f20a8022e50 con 0x7f209c0ae420 2024-09-16T00:31:32.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.423+0000 7f20b0ff9640 1 -- 172.21.15.38:0/1690840306 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f20a80cffb0 con 0x7f20b4103660 2024-09-16T00:31:32.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.423+0000 7f20b37fe640 1 --2- 172.21.15.38:0/1690840306 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f209c0a32d0 0x7f209c0a5790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f20a4009230 tx=0x7f20a40058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:32.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.423+0000 7f20b3fff640 1 --2- 172.21.15.38:0/1690840306 >> [v2:172.21.15.73:6816/2634361473,v1:172.21.15.73:6817/2634361473] conn(0x7f209c0ae420 0x7f209c0b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:32.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.424+0000 7f20b3fff640 1 --2- 172.21.15.38:0/1690840306 >> [v2:172.21.15.73:6816/2634361473,v1:172.21.15.73:6817/2634361473] conn(0x7f209c0ae420 0x7f209c0b08c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:32.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.424+0000 7f20b0ff9640 1 -- 172.21.15.38:0/1690840306 <== osd.7 v2:172.21.15.73:6816/2634361473 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f20a8022e50 con 0x7f209c0ae420 2024-09-16T00:31:32.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.438+0000 7f20ba22b640 1 -- 172.21.15.38:0/1690840306 --> [v2:172.21.15.73:6816/2634361473,v1:172.21.15.73:6817/2634361473] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f20b4062d40 con 0x7f209c0ae420 2024-09-16T00:31:32.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.439+0000 7f20b0ff9640 1 -- 172.21.15.38:0/1690840306 <== osd.7 v2:172.21.15.73:6816/2634361473 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f20b4062d40 con 0x7f209c0ae420 2024-09-16T00:31:32.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.439+0000 7f20ba22b640 1 -- 172.21.15.38:0/1690840306 >> [v2:172.21.15.73:6816/2634361473,v1:172.21.15.73:6817/2634361473] conn(0x7f209c0ae420 msgr2=0x7f209c0b08c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:32.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.439+0000 7f20ba22b640 1 --2- 172.21.15.38:0/1690840306 >> [v2:172.21.15.73:6816/2634361473,v1:172.21.15.73:6817/2634361473] conn(0x7f209c0ae420 0x7f209c0b08c0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:32.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.439+0000 7f20ba22b640 1 -- 172.21.15.38:0/1690840306 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f209c0a32d0 msgr2=0x7f209c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:32.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.439+0000 7f20ba22b640 1 --2- 172.21.15.38:0/1690840306 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f209c0a32d0 0x7f209c0a5790 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f20a4009230 tx=0x7f20a40058d0 comp rx=0 tx=0).stop 2024-09-16T00:31:32.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.439+0000 7f20ba22b640 1 -- 172.21.15.38:0/1690840306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b4103660 msgr2=0x7f20b4100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:32.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.439+0000 7f20ba22b640 1 --2- 172.21.15.38:0/1690840306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20b4103660 0x7f20b4100ee0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f20a800d750 tx=0x7f20a800dc10 comp rx=0 tx=0).stop 2024-09-16T00:31:32.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.439+0000 7f20b3fff640 1 -- 172.21.15.38:0/1690840306 reap_dead start 2024-09-16T00:31:32.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.439+0000 7f20ba22b640 1 -- 172.21.15.38:0/1690840306 shutdown_connections 2024-09-16T00:31:32.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.439+0000 7f20ba22b640 1 -- 172.21.15.38:0/1690840306 >> 172.21.15.38:0/1690840306 conn(0x7f20b40fc7a0 msgr2=0x7f20b40fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:32.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.440+0000 7f20ba22b640 1 -- 172.21.15.38:0/1690840306 shutdown_connections 2024-09-16T00:31:32.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.440+0000 7f20ba22b640 1 -- 172.21.15.38:0/1690840306 wait complete. 2024-09-16T00:31:33.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.212+0000 7f9894dc8640 1 -- 172.21.15.38:0/2046662510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9890100a80 msgr2=0x7f9890100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:33.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.212+0000 7f9894dc8640 1 --2- 172.21.15.38:0/2046662510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9890100a80 0x7f9890100ee0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9884009cd0 tx=0x7f988402a8e0 comp rx=0 tx=0).stop 2024-09-16T00:31:33.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.212+0000 7f9894dc8640 1 -- 172.21.15.38:0/2046662510 shutdown_connections 2024-09-16T00:31:33.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.212+0000 7f9894dc8640 1 --2- 172.21.15.38:0/2046662510 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9890101420 0x7f989010d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:33.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.212+0000 7f9894dc8640 1 --2- 172.21.15.38:0/2046662510 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9890100a80 0x7f9890100ee0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:33.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.212+0000 7f9894dc8640 1 --2- 172.21.15.38:0/2046662510 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9890106a80 0x7f9890106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:33.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.212+0000 7f9894dc8640 1 -- 172.21.15.38:0/2046662510 >> 172.21.15.38:0/2046662510 conn(0x7f98900fc7a0 msgr2=0x7f98900febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:33.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.212+0000 7f9894dc8640 1 -- 172.21.15.38:0/2046662510 shutdown_connections 2024-09-16T00:31:33.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.212+0000 7f9894dc8640 1 -- 172.21.15.38:0/2046662510 wait complete. 2024-09-16T00:31:33.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.213+0000 7f9894dc8640 1 Processor -- start 2024-09-16T00:31:33.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.213+0000 7f9894dc8640 1 -- start start 2024-09-16T00:31:33.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.213+0000 7f9894dc8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9890100a80 0x7f98900fce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:33.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.213+0000 7f9894dc8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9890101420 0x7f98900fd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:33.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.213+0000 7f9894dc8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9890106a80 0x7f98900fd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:33.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.213+0000 7f9894dc8640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9890110ea0 con 0x7f9890100a80 2024-09-16T00:31:33.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.213+0000 7f9894dc8640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9890110d20 con 0x7f9890106a80 2024-09-16T00:31:33.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.213+0000 7f9894dc8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9890111020 con 0x7f9890101420 2024-09-16T00:31:33.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.213+0000 7f988e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9890100a80 0x7f98900fce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:33.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.213+0000 7f988e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9890100a80 0x7f98900fce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38804/0 (socket says 172.21.15.38:38804) 2024-09-16T00:31:33.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.213+0000 7f988e59c640 1 -- 172.21.15.38:0/1582678696 learned_addr learned my addr 172.21.15.38:0/1582678696 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:31:33.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.213+0000 7f988ed9d640 1 --2- 172.21.15.38:0/1582678696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9890106a80 0x7f98900fd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:33.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.214+0000 7f988dd9b640 1 --2- 172.21.15.38:0/1582678696 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9890101420 0x7f98900fd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:33.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.214+0000 7f988e59c640 1 -- 172.21.15.38:0/1582678696 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9890101420 msgr2=0x7f98900fd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:33.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.214+0000 7f988e59c640 1 --2- 172.21.15.38:0/1582678696 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9890101420 0x7f98900fd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:33.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.214+0000 7f988e59c640 1 -- 172.21.15.38:0/1582678696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9890106a80 msgr2=0x7f98900fd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:33.229 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.214+0000 7f988e59c640 1 --2- 172.21.15.38:0/1582678696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9890106a80 0x7f98900fd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:33.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.214+0000 7f988e59c640 1 -- 172.21.15.38:0/1582678696 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98901a69f0 con 0x7f9890100a80 2024-09-16T00:31:33.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.214+0000 7f988e59c640 1 --2- 172.21.15.38:0/1582678696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9890100a80 0x7f98900fce60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f988000b4a0 tx=0x7f988000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:33.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.214+0000 7f987f7fe640 1 -- 172.21.15.38:0/1582678696 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9880014070 con 0x7f9890100a80 2024-09-16T00:31:33.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.214+0000 7f9894dc8640 1 -- 172.21.15.38:0/1582678696 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98901a6ce0 con 0x7f9890100a80 2024-09-16T00:31:33.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.214+0000 7f987f7fe640 1 -- 172.21.15.38:0/1582678696 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9880010af0 con 0x7f9890100a80 2024-09-16T00:31:33.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.214+0000 7f987f7fe640 1 -- 172.21.15.38:0/1582678696 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9880017d10 con 0x7f9890100a80 2024-09-16T00:31:33.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.214+0000 7f9894dc8640 1 -- 172.21.15.38:0/1582678696 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98901a7210 con 0x7f9890100a80 2024-09-16T00:31:33.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.215+0000 7f9894dc8640 1 -- 172.21.15.38:0/1582678696 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f98900628c0 con 0x7f9890100a80 2024-09-16T00:31:33.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.216+0000 7f987f7fe640 1 -- 172.21.15.38:0/1582678696 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f9880017070 con 0x7f9890100a80 2024-09-16T00:31:33.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.217+0000 7f987f7fe640 1 --2- 172.21.15.38:0/1582678696 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f98780a32d0 0x7f98780a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:33.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.217+0000 7f987f7fe640 1 -- 172.21.15.38:0/1582678696 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f98800cf4e0 con 0x7f9890100a80 2024-09-16T00:31:33.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.217+0000 7f988dd9b640 1 --2- 172.21.15.38:0/1582678696 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f98780a32d0 0x7f98780a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:33.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.217+0000 7f987f7fe640 1 --2- 172.21.15.38:0/1582678696 >> [v2:172.21.15.122:6824/1886446061,v1:172.21.15.122:6825/1886446061] conn(0x7f98780ae420 0x7f98780b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:33.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.217+0000 7f987f7fe640 1 -- 172.21.15.38:0/1582678696 --> [v2:172.21.15.122:6824/1886446061,v1:172.21.15.122:6825/1886446061] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f988000dec0 con 0x7f98780ae420 2024-09-16T00:31:33.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.217+0000 7f987f7fe640 1 -- 172.21.15.38:0/1582678696 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f98800cf930 con 0x7f9890100a80 2024-09-16T00:31:33.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.217+0000 7f988dd9b640 1 --2- 172.21.15.38:0/1582678696 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f98780a32d0 0x7f98780a5790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9884006010 tx=0x7f9884000f40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:33.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.218+0000 7f988ed9d640 1 --2- 172.21.15.38:0/1582678696 >> [v2:172.21.15.122:6824/1886446061,v1:172.21.15.122:6825/1886446061] conn(0x7f98780ae420 0x7f98780b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:33.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.218+0000 7f988ed9d640 1 --2- 172.21.15.38:0/1582678696 >> [v2:172.21.15.122:6824/1886446061,v1:172.21.15.122:6825/1886446061] conn(0x7f98780ae420 0x7f98780b08c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:33.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.219+0000 7f987f7fe640 1 -- 172.21.15.38:0/1582678696 <== osd.11 v2:172.21.15.122:6824/1886446061 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f988000dec0 con 0x7f98780ae420 2024-09-16T00:31:33.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.232+0000 7f9894dc8640 1 -- 172.21.15.38:0/1582678696 --> [v2:172.21.15.122:6824/1886446061,v1:172.21.15.122:6825/1886446061] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9890062cc0 con 0x7f98780ae420 2024-09-16T00:31:33.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.233+0000 7f987f7fe640 1 -- 172.21.15.38:0/1582678696 <== osd.11 v2:172.21.15.122:6824/1886446061 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9890062cc0 con 0x7f98780ae420 2024-09-16T00:31:33.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.234+0000 7f9894dc8640 1 -- 172.21.15.38:0/1582678696 >> [v2:172.21.15.122:6824/1886446061,v1:172.21.15.122:6825/1886446061] conn(0x7f98780ae420 msgr2=0x7f98780b08c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:33.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.234+0000 7f9894dc8640 1 --2- 172.21.15.38:0/1582678696 >> [v2:172.21.15.122:6824/1886446061,v1:172.21.15.122:6825/1886446061] conn(0x7f98780ae420 0x7f98780b08c0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:33.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.234+0000 7f9894dc8640 1 -- 172.21.15.38:0/1582678696 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f98780a32d0 msgr2=0x7f98780a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:33.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.234+0000 7f9894dc8640 1 --2- 172.21.15.38:0/1582678696 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f98780a32d0 0x7f98780a5790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9884006010 tx=0x7f9884000f40 comp rx=0 tx=0).stop 2024-09-16T00:31:33.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.234+0000 7f9894dc8640 1 -- 172.21.15.38:0/1582678696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9890100a80 msgr2=0x7f98900fce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:33.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.234+0000 7f9894dc8640 1 --2- 172.21.15.38:0/1582678696 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9890100a80 0x7f98900fce60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f988000b4a0 tx=0x7f988000b960 comp rx=0 tx=0).stop 2024-09-16T00:31:33.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.234+0000 7f988ed9d640 1 -- 172.21.15.38:0/1582678696 reap_dead start 2024-09-16T00:31:33.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.234+0000 7f9894dc8640 1 -- 172.21.15.38:0/1582678696 shutdown_connections 2024-09-16T00:31:33.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.234+0000 7f9894dc8640 1 -- 172.21.15.38:0/1582678696 >> 172.21.15.38:0/1582678696 conn(0x7f98900fc7a0 msgr2=0x7f989006aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:33.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.234+0000 7f9894dc8640 1 -- 172.21.15.38:0/1582678696 shutdown_connections 2024-09-16T00:31:33.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.234+0000 7f9894dc8640 1 -- 172.21.15.38:0/1582678696 wait complete. 2024-09-16T00:31:33.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:33 smithi038 ceph-mon[30945]: pgmap v325: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 901 B/s wr, 7 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-09-16T00:31:33.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:33 smithi122 ceph-mon[35412]: pgmap v325: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 901 B/s wr, 7 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-09-16T00:31:33.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:33 smithi073 ceph-mon[35254]: pgmap v325: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 901 B/s wr, 7 op/s; 6 B/s, 1 keys/s, 0 objects/s recovering 2024-09-16T00:31:33.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.679+0000 7fdd73884640 1 -- 172.21.15.38:0/1743831141 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd6c102d30 msgr2=0x7fdd6c103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:33.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.679+0000 7fdd73884640 1 --2- 172.21.15.38:0/1743831141 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd6c102d30 0x7fdd6c103110 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdd68009930 tx=0x7fdd6802a420 comp rx=0 tx=0).stop 2024-09-16T00:31:33.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.679+0000 7fdd73884640 1 -- 172.21.15.38:0/1743831141 shutdown_connections 2024-09-16T00:31:33.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.679+0000 7fdd73884640 1 --2- 172.21.15.38:0/1743831141 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd6c1082a0 0x7fdd6c10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:33.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.679+0000 7fdd73884640 1 --2- 172.21.15.38:0/1743831141 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd6c1036e0 0x7fdd6c107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:33.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.679+0000 7fdd73884640 1 --2- 172.21.15.38:0/1743831141 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd6c102d30 0x7fdd6c103110 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:33.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.679+0000 7fdd73884640 1 -- 172.21.15.38:0/1743831141 >> 172.21.15.38:0/1743831141 conn(0x7fdd6c0fc7d0 msgr2=0x7fdd6c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:33.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.679+0000 7fdd73884640 1 -- 172.21.15.38:0/1743831141 shutdown_connections 2024-09-16T00:31:33.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.679+0000 7fdd73884640 1 -- 172.21.15.38:0/1743831141 wait complete. 2024-09-16T00:31:33.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.679+0000 7fdd73884640 1 Processor -- start 2024-09-16T00:31:33.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.680+0000 7fdd73884640 1 -- start start 2024-09-16T00:31:33.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.680+0000 7fdd73884640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd6c102d30 0x7fdd6c100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:33.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.680+0000 7fdd73884640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd6c1036e0 0x7fdd6c100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:33.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.680+0000 7fdd73884640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd6c1082a0 0x7fdd6c101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:33.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.680+0000 7fdd73884640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6c1120a0 con 0x7fdd6c1082a0 2024-09-16T00:31:33.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.680+0000 7fdd73884640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6c111f20 con 0x7fdd6c102d30 2024-09-16T00:31:33.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.680+0000 7fdd73884640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdd6c112220 con 0x7fdd6c1036e0 2024-09-16T00:31:33.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.680+0000 7fdd71e21640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd6c1082a0 0x7fdd6c101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:33.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.680+0000 7fdd71e21640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd6c1082a0 0x7fdd6c101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38816/0 (socket says 172.21.15.38:38816) 2024-09-16T00:31:33.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.680+0000 7fdd71e21640 1 -- 172.21.15.38:0/2896073950 learned_addr learned my addr 172.21.15.38:0/2896073950 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:31:33.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.680+0000 7fdd70e1f640 1 --2- 172.21.15.38:0/2896073950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd6c1036e0 0x7fdd6c100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:33.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.680+0000 7fdd71620640 1 --2- 172.21.15.38:0/2896073950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd6c102d30 0x7fdd6c100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:33.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.680+0000 7fdd71e21640 1 -- 172.21.15.38:0/2896073950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd6c1036e0 msgr2=0x7fdd6c100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:33.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.680+0000 7fdd71e21640 1 --2- 172.21.15.38:0/2896073950 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd6c1036e0 0x7fdd6c100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:33.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.680+0000 7fdd71e21640 1 -- 172.21.15.38:0/2896073950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd6c102d30 msgr2=0x7fdd6c100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:33.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.680+0000 7fdd71e21640 1 --2- 172.21.15.38:0/2896073950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd6c102d30 0x7fdd6c100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:33.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.680+0000 7fdd71e21640 1 -- 172.21.15.38:0/2896073950 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd6c1a6ab0 con 0x7fdd6c1082a0 2024-09-16T00:31:33.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.680+0000 7fdd71620640 1 --2- 172.21.15.38:0/2896073950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd6c102d30 0x7fdd6c100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:31:33.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.680+0000 7fdd71e21640 1 --2- 172.21.15.38:0/2896073950 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd6c1082a0 0x7fdd6c101410 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400eb10 tx=0x7fdd6400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:33.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.680+0000 7fdd627fc640 1 -- 172.21.15.38:0/2896073950 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd64004b20 con 0x7fdd6c1082a0 2024-09-16T00:31:33.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.681+0000 7fdd627fc640 1 -- 172.21.15.38:0/2896073950 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd64004cc0 con 0x7fdd6c1082a0 2024-09-16T00:31:33.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.681+0000 7fdd73884640 1 -- 172.21.15.38:0/2896073950 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd6c1a6da0 con 0x7fdd6c1082a0 2024-09-16T00:31:33.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.681+0000 7fdd627fc640 1 -- 172.21.15.38:0/2896073950 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd64010760 con 0x7fdd6c1082a0 2024-09-16T00:31:33.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.681+0000 7fdd73884640 1 -- 172.21.15.38:0/2896073950 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd6c1a72b0 con 0x7fdd6c1082a0 2024-09-16T00:31:33.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.681+0000 7fdd73884640 1 -- 172.21.15.38:0/2896073950 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdd6c062900 con 0x7fdd6c1082a0 2024-09-16T00:31:33.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.683+0000 7fdd627fc640 1 -- 172.21.15.38:0/2896073950 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fdd64005020 con 0x7fdd6c1082a0 2024-09-16T00:31:33.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.683+0000 7fdd627fc640 1 --2- 172.21.15.38:0/2896073950 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fdd580a32d0 0x7fdd580a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:33.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.683+0000 7fdd71620640 1 --2- 172.21.15.38:0/2896073950 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fdd580a32d0 0x7fdd580a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:33.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.683+0000 7fdd627fc640 1 -- 172.21.15.38:0/2896073950 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7fdd640d0a90 con 0x7fdd6c1082a0 2024-09-16T00:31:33.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.684+0000 7fdd627fc640 1 --2- 172.21.15.38:0/2896073950 >> [v2:172.21.15.122:6808/2954413058,v1:172.21.15.122:6809/2954413058] conn(0x7fdd580ae420 0x7fdd580b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:33.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.684+0000 7fdd627fc640 1 -- 172.21.15.38:0/2896073950 --> [v2:172.21.15.122:6808/2954413058,v1:172.21.15.122:6809/2954413058] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdd6401ee50 con 0x7fdd580ae420 2024-09-16T00:31:33.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.684+0000 7fdd627fc640 1 -- 172.21.15.38:0/2896073950 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fdd640d0ee0 con 0x7fdd6c1082a0 2024-09-16T00:31:33.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.684+0000 7fdd71620640 1 --2- 172.21.15.38:0/2896073950 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fdd580a32d0 0x7fdd580a5790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdd68000c00 tx=0x7fdd68005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:33.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.684+0000 7fdd70e1f640 1 --2- 172.21.15.38:0/2896073950 >> [v2:172.21.15.122:6808/2954413058,v1:172.21.15.122:6809/2954413058] conn(0x7fdd580ae420 0x7fdd580b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:33.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.684+0000 7fdd70e1f640 1 --2- 172.21.15.38:0/2896073950 >> [v2:172.21.15.122:6808/2954413058,v1:172.21.15.122:6809/2954413058] conn(0x7fdd580ae420 0x7fdd580b08c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:33.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.685+0000 7fdd627fc640 1 -- 172.21.15.38:0/2896073950 <== osd.5 v2:172.21.15.122:6808/2954413058 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdd6401ee50 con 0x7fdd580ae420 2024-09-16T00:31:33.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.700+0000 7fdd73884640 1 -- 172.21.15.38:0/2896073950 --> [v2:172.21.15.122:6808/2954413058,v1:172.21.15.122:6809/2954413058] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdd6c062d80 con 0x7fdd580ae420 2024-09-16T00:31:33.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.700+0000 7fdd627fc640 1 -- 172.21.15.38:0/2896073950 <== osd.5 v2:172.21.15.122:6808/2954413058 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdd6c062d80 con 0x7fdd580ae420 2024-09-16T00:31:33.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.701+0000 7fdd73884640 1 -- 172.21.15.38:0/2896073950 >> [v2:172.21.15.122:6808/2954413058,v1:172.21.15.122:6809/2954413058] conn(0x7fdd580ae420 msgr2=0x7fdd580b08c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:33.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.701+0000 7fdd73884640 1 --2- 172.21.15.38:0/2896073950 >> [v2:172.21.15.122:6808/2954413058,v1:172.21.15.122:6809/2954413058] conn(0x7fdd580ae420 0x7fdd580b08c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:33.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.701+0000 7fdd73884640 1 -- 172.21.15.38:0/2896073950 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fdd580a32d0 msgr2=0x7fdd580a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:33.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.701+0000 7fdd73884640 1 --2- 172.21.15.38:0/2896073950 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fdd580a32d0 0x7fdd580a5790 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdd68000c00 tx=0x7fdd68005a20 comp rx=0 tx=0).stop 2024-09-16T00:31:33.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.701+0000 7fdd73884640 1 -- 172.21.15.38:0/2896073950 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd6c1082a0 msgr2=0x7fdd6c101410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:33.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.701+0000 7fdd73884640 1 --2- 172.21.15.38:0/2896073950 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd6c1082a0 0x7fdd6c101410 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdd6400eb10 tx=0x7fdd6400c2d0 comp rx=0 tx=0).stop 2024-09-16T00:31:33.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.701+0000 7fdd71e21640 1 -- 172.21.15.38:0/2896073950 reap_dead start 2024-09-16T00:31:33.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.701+0000 7fdd73884640 1 -- 172.21.15.38:0/2896073950 shutdown_connections 2024-09-16T00:31:33.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.701+0000 7fdd73884640 1 -- 172.21.15.38:0/2896073950 >> 172.21.15.38:0/2896073950 conn(0x7fdd6c0fc7d0 msgr2=0x7fdd6c105de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:33.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.701+0000 7fdd73884640 1 -- 172.21.15.38:0/2896073950 shutdown_connections 2024-09-16T00:31:33.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:33.701+0000 7fdd73884640 1 -- 172.21.15.38:0/2896073950 wait complete. 2024-09-16T00:31:34.188 INFO:teuthology.orchestra.run.smithi038.stdout:244813135914 2024-09-16T00:31:34.189 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-09-16T00:31:34.587 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.520+0000 7f458fb0e640 1 -- 172.21.15.38:0/1503915025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4588102d30 msgr2=0x7f4588103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:34.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.520+0000 7f458fb0e640 1 --2- 172.21.15.38:0/1503915025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4588102d30 0x7f4588103110 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4584009930 tx=0x7f458402a420 comp rx=0 tx=0).stop 2024-09-16T00:31:34.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.520+0000 7f458fb0e640 1 -- 172.21.15.38:0/1503915025 shutdown_connections 2024-09-16T00:31:34.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.520+0000 7f458fb0e640 1 --2- 172.21.15.38:0/1503915025 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f45881082a0 0x7f458810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:34.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.520+0000 7f458fb0e640 1 --2- 172.21.15.38:0/1503915025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45881036e0 0x7f4588107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:34.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.520+0000 7f458fb0e640 1 --2- 172.21.15.38:0/1503915025 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4588102d30 0x7f4588103110 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:34.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.520+0000 7f458fb0e640 1 -- 172.21.15.38:0/1503915025 >> 172.21.15.38:0/1503915025 conn(0x7f45880fc7d0 msgr2=0x7f45880febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:34.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.521+0000 7f458fb0e640 1 -- 172.21.15.38:0/1503915025 shutdown_connections 2024-09-16T00:31:34.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.521+0000 7f458fb0e640 1 -- 172.21.15.38:0/1503915025 wait complete. 2024-09-16T00:31:34.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.521+0000 7f458fb0e640 1 Processor -- start 2024-09-16T00:31:34.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.521+0000 7f458fb0e640 1 -- start start 2024-09-16T00:31:34.588 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.521+0000 7f458fb0e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4588102d30 0x7f4588198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:34.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.521+0000 7f458fb0e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45881036e0 0x7f4588198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:34.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.521+0000 7f458fb0e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45881082a0 0x7f458819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:34.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.521+0000 7f458fb0e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f45881120a0 con 0x7f45881082a0 2024-09-16T00:31:34.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.521+0000 7f458fb0e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4588111f20 con 0x7f45881036e0 2024-09-16T00:31:34.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.521+0000 7f458fb0e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4588112220 con 0x7f4588102d30 2024-09-16T00:31:34.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.521+0000 7f458e0ab640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45881082a0 0x7f458819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:34.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.521+0000 7f458e0ab640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45881082a0 0x7f458819ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38782/0 (socket says 172.21.15.38:38782) 2024-09-16T00:31:34.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.521+0000 7f458e0ab640 1 -- 172.21.15.38:0/2568835549 learned_addr learned my addr 172.21.15.38:0/2568835549 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:31:34.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.522+0000 7f458d0a9640 1 --2- 172.21.15.38:0/2568835549 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45881036e0 0x7f4588198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:34.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.522+0000 7f458d8aa640 1 --2- 172.21.15.38:0/2568835549 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4588102d30 0x7f4588198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:34.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.522+0000 7f458e0ab640 1 -- 172.21.15.38:0/2568835549 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4588102d30 msgr2=0x7f4588198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:34.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.522+0000 7f458e0ab640 1 --2- 172.21.15.38:0/2568835549 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4588102d30 0x7f4588198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:34.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.522+0000 7f458e0ab640 1 -- 172.21.15.38:0/2568835549 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45881036e0 msgr2=0x7f4588198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:34.589 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.522+0000 7f458e0ab640 1 --2- 172.21.15.38:0/2568835549 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45881036e0 0x7f4588198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:34.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.522+0000 7f458e0ab640 1 -- 172.21.15.38:0/2568835549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f458819d510 con 0x7f45881082a0 2024-09-16T00:31:34.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.522+0000 7f458e0ab640 1 --2- 172.21.15.38:0/2568835549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45881082a0 0x7f458819ce30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f458000cce0 tx=0x7f458000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:34.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.522+0000 7f457effd640 1 -- 172.21.15.38:0/2568835549 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f458000f450 con 0x7f45881082a0 2024-09-16T00:31:34.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.522+0000 7f457effd640 1 -- 172.21.15.38:0/2568835549 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f458000fad0 con 0x7f45881082a0 2024-09-16T00:31:34.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.522+0000 7f458fb0e640 1 -- 172.21.15.38:0/2568835549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f458819d800 con 0x7f45881082a0 2024-09-16T00:31:34.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.522+0000 7f457effd640 1 -- 172.21.15.38:0/2568835549 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4580018c10 con 0x7f45881082a0 2024-09-16T00:31:34.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.522+0000 7f458fb0e640 1 -- 172.21.15.38:0/2568835549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45881a50e0 con 0x7f45881082a0 2024-09-16T00:31:34.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.523+0000 7f458fb0e640 1 -- 172.21.15.38:0/2568835549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4588062980 con 0x7f45881082a0 2024-09-16T00:31:34.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.524+0000 7f457effd640 1 -- 172.21.15.38:0/2568835549 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f458000f5f0 con 0x7f45881082a0 2024-09-16T00:31:34.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.525+0000 7f457effd640 1 --2- 172.21.15.38:0/2568835549 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f45740a32d0 0x7f45740a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:34.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.525+0000 7f458d8aa640 1 --2- 172.21.15.38:0/2568835549 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f45740a32d0 0x7f45740a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:34.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.525+0000 7f457effd640 1 -- 172.21.15.38:0/2568835549 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f45800d05f0 con 0x7f45881082a0 2024-09-16T00:31:34.590 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.525+0000 7f457effd640 1 --2- 172.21.15.38:0/2568835549 >> [v2:172.21.15.73:6800/3811690336,v1:172.21.15.73:6801/3811690336] conn(0x7f45740ae420 0x7f45740b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:34.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.525+0000 7f457effd640 1 -- 172.21.15.38:0/2568835549 --> [v2:172.21.15.73:6800/3811690336,v1:172.21.15.73:6801/3811690336] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4580015e70 con 0x7f45740ae420 2024-09-16T00:31:34.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.525+0000 7f457effd640 1 -- 172.21.15.38:0/2568835549 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f45800d0a40 con 0x7f45881082a0 2024-09-16T00:31:34.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.526+0000 7f458d8aa640 1 --2- 172.21.15.38:0/2568835549 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f45740a32d0 0x7f45740a5790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4584009900 tx=0x7f45840058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:34.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.526+0000 7f458d0a9640 1 --2- 172.21.15.38:0/2568835549 >> [v2:172.21.15.73:6800/3811690336,v1:172.21.15.73:6801/3811690336] conn(0x7f45740ae420 0x7f45740b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:34.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.526+0000 7f458d0a9640 1 --2- 172.21.15.38:0/2568835549 >> [v2:172.21.15.73:6800/3811690336,v1:172.21.15.73:6801/3811690336] conn(0x7f45740ae420 0x7f45740b08c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:34.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.527+0000 7f457effd640 1 -- 172.21.15.38:0/2568835549 <== osd.1 v2:172.21.15.73:6800/3811690336 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4580015e70 con 0x7f45740ae420 2024-09-16T00:31:34.591 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.540+0000 7f458fb0e640 1 -- 172.21.15.38:0/2568835549 --> [v2:172.21.15.73:6800/3811690336,v1:172.21.15.73:6801/3811690336] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4588062e00 con 0x7f45740ae420 2024-09-16T00:31:34.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.541+0000 7f457effd640 1 -- 172.21.15.38:0/2568835549 <== osd.1 v2:172.21.15.73:6800/3811690336 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f4588062e00 con 0x7f45740ae420 2024-09-16T00:31:34.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.541+0000 7f458fb0e640 1 -- 172.21.15.38:0/2568835549 >> [v2:172.21.15.73:6800/3811690336,v1:172.21.15.73:6801/3811690336] conn(0x7f45740ae420 msgr2=0x7f45740b08c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:34.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.541+0000 7f458fb0e640 1 --2- 172.21.15.38:0/2568835549 >> [v2:172.21.15.73:6800/3811690336,v1:172.21.15.73:6801/3811690336] conn(0x7f45740ae420 0x7f45740b08c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:34.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.541+0000 7f458fb0e640 1 -- 172.21.15.38:0/2568835549 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f45740a32d0 msgr2=0x7f45740a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:34.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.541+0000 7f458fb0e640 1 --2- 172.21.15.38:0/2568835549 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f45740a32d0 0x7f45740a5790 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4584009900 tx=0x7f45840058d0 comp rx=0 tx=0).stop 2024-09-16T00:31:34.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.541+0000 7f458fb0e640 1 -- 172.21.15.38:0/2568835549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45881082a0 msgr2=0x7f458819ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:34.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.541+0000 7f458fb0e640 1 --2- 172.21.15.38:0/2568835549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f45881082a0 0x7f458819ce30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f458000cce0 tx=0x7f458000a620 comp rx=0 tx=0).stop 2024-09-16T00:31:34.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.541+0000 7f458e0ab640 1 -- 172.21.15.38:0/2568835549 reap_dead start 2024-09-16T00:31:34.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.541+0000 7f458fb0e640 1 -- 172.21.15.38:0/2568835549 shutdown_connections 2024-09-16T00:31:34.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.541+0000 7f458fb0e640 1 -- 172.21.15.38:0/2568835549 >> 172.21.15.38:0/2568835549 conn(0x7f45880fc7d0 msgr2=0x7f4588105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:34.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.542+0000 7f458fb0e640 1 -- 172.21.15.38:0/2568835549 shutdown_connections 2024-09-16T00:31:34.592 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:32.542+0000 7f458fb0e640 1 -- 172.21.15.38:0/2568835549 wait complete. 2024-09-16T00:31:35.214 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:35 smithi038 ceph-mon[30945]: pgmap v326: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 812 B/s wr, 6 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-09-16T00:31:35.304 INFO:teuthology.orchestra.run.smithi038.stdout:55834574927 2024-09-16T00:31:35.304 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-16T00:31:35.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:35 smithi122 ceph-mon[35412]: pgmap v326: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 812 B/s wr, 6 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-09-16T00:31:35.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.463+0000 7f777da4c640 1 -- 172.21.15.38:0/2484148189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77781082a0 msgr2=0x7f777810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:35.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.463+0000 7f777da4c640 1 --2- 172.21.15.38:0/2484148189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77781082a0 0x7f777810ec40 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7770009cd0 tx=0x7f777002a8e0 comp rx=0 tx=0).stop 2024-09-16T00:31:35.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.463+0000 7f777da4c640 1 -- 172.21.15.38:0/2484148189 shutdown_connections 2024-09-16T00:31:35.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.463+0000 7f777da4c640 1 --2- 172.21.15.38:0/2484148189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77781082a0 0x7f777810ec40 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:35.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.463+0000 7f777da4c640 1 --2- 172.21.15.38:0/2484148189 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f77781036e0 0x7f7778107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:35.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.463+0000 7f777da4c640 1 --2- 172.21.15.38:0/2484148189 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7778102d30 0x7f7778103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:35.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.463+0000 7f777da4c640 1 -- 172.21.15.38:0/2484148189 >> 172.21.15.38:0/2484148189 conn(0x7f77780fc7d0 msgr2=0x7f77780febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:35.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.463+0000 7f777da4c640 1 -- 172.21.15.38:0/2484148189 shutdown_connections 2024-09-16T00:31:35.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.463+0000 7f777da4c640 1 -- 172.21.15.38:0/2484148189 wait complete. 2024-09-16T00:31:35.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.464+0000 7f777da4c640 1 Processor -- start 2024-09-16T00:31:35.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.464+0000 7f777da4c640 1 -- start start 2024-09-16T00:31:35.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.464+0000 7f777da4c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7778102d30 0x7f7778198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:35.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.464+0000 7f777da4c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f77781036e0 0x7f7778198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:35.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.464+0000 7f777da4c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77781082a0 0x7f777819ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:35.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.464+0000 7f777da4c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f77781120a0 con 0x7f7778102d30 2024-09-16T00:31:35.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.464+0000 7f777da4c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7778111f20 con 0x7f77781036e0 2024-09-16T00:31:35.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.464+0000 7f777da4c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7778112220 con 0x7f77781082a0 2024-09-16T00:31:35.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.465+0000 7f7776ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7778102d30 0x7f7778198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:35.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.465+0000 7f7776ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7778102d30 0x7f7778198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38846/0 (socket says 172.21.15.38:38846) 2024-09-16T00:31:35.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.465+0000 7f7776ffd640 1 -- 172.21.15.38:0/372588585 learned_addr learned my addr 172.21.15.38:0/372588585 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:31:35.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.465+0000 7f77767fc640 1 --2- 172.21.15.38:0/372588585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f77781036e0 0x7f7778198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:35.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.465+0000 7f77777fe640 1 --2- 172.21.15.38:0/372588585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77781082a0 0x7f777819ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:35.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.465+0000 7f77767fc640 1 -- 172.21.15.38:0/372588585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77781082a0 msgr2=0x7f777819ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:35.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.465+0000 7f77767fc640 1 --2- 172.21.15.38:0/372588585 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f77781082a0 0x7f777819ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:35.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.465+0000 7f77767fc640 1 -- 172.21.15.38:0/372588585 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7778102d30 msgr2=0x7f7778198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:35.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.465+0000 7f77767fc640 1 --2- 172.21.15.38:0/372588585 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7778102d30 0x7f7778198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:35.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.465+0000 7f77767fc640 1 -- 172.21.15.38:0/372588585 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f777819d4e0 con 0x7f77781036e0 2024-09-16T00:31:35.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.465+0000 7f7776ffd640 1 --2- 172.21.15.38:0/372588585 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7778102d30 0x7f7778198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:31:35.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.465+0000 7f77767fc640 1 --2- 172.21.15.38:0/372588585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f77781036e0 0x7f7778198a70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f77780697d0 tx=0x7f776c00ee50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:35.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.466+0000 7f777ca4a640 1 -- 172.21.15.38:0/372588585 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c00cc40 con 0x7f77781036e0 2024-09-16T00:31:35.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.466+0000 7f777ca4a640 1 -- 172.21.15.38:0/372588585 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f776c00cde0 con 0x7f77781036e0 2024-09-16T00:31:35.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.466+0000 7f777da4c640 1 -- 172.21.15.38:0/372588585 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f777819d7d0 con 0x7f77781036e0 2024-09-16T00:31:35.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.466+0000 7f777ca4a640 1 -- 172.21.15.38:0/372588585 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c0188e0 con 0x7f77781036e0 2024-09-16T00:31:35.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.466+0000 7f777da4c640 1 -- 172.21.15.38:0/372588585 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77781a50b0 con 0x7f77781036e0 2024-09-16T00:31:35.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.467+0000 7f777da4c640 1 -- 172.21.15.38:0/372588585 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7778062980 con 0x7f77781036e0 2024-09-16T00:31:35.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.468+0000 7f777ca4a640 1 -- 172.21.15.38:0/372588585 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f776c018a80 con 0x7f77781036e0 2024-09-16T00:31:35.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.469+0000 7f777ca4a640 1 --2- 172.21.15.38:0/372588585 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f77600a3350 0x7f77600a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:35.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.469+0000 7f7776ffd640 1 --2- 172.21.15.38:0/372588585 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f77600a3350 0x7f77600a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:35.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.469+0000 7f777ca4a640 1 -- 172.21.15.38:0/372588585 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f776c0d0490 con 0x7f77781036e0 2024-09-16T00:31:35.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.470+0000 7f777ca4a640 1 --2- 172.21.15.38:0/372588585 >> [v2:172.21.15.38:6802/1622028921,v1:172.21.15.38:6803/1622028921] conn(0x7f77600ae4a0 0x7f77600b0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:35.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.470+0000 7f777ca4a640 1 -- 172.21.15.38:0/372588585 --> [v2:172.21.15.38:6802/1622028921,v1:172.21.15.38:6803/1622028921] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f77600b1010 con 0x7f77600ae4a0 2024-09-16T00:31:35.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.470+0000 7f777ca4a640 1 -- 172.21.15.38:0/372588585 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f776c0d0840 con 0x7f77781036e0 2024-09-16T00:31:35.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.470+0000 7f77777fe640 1 --2- 172.21.15.38:0/372588585 >> [v2:172.21.15.38:6802/1622028921,v1:172.21.15.38:6803/1622028921] conn(0x7f77600ae4a0 0x7f77600b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:35.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.470+0000 7f7776ffd640 1 --2- 172.21.15.38:0/372588585 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f77600a3350 0x7f77600a5810 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7768005fd0 tx=0x7f776800b480 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:35.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.470+0000 7f77777fe640 1 --2- 172.21.15.38:0/372588585 >> [v2:172.21.15.38:6802/1622028921,v1:172.21.15.38:6803/1622028921] conn(0x7f77600ae4a0 0x7f77600b0940 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:35.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.471+0000 7f777ca4a640 1 -- 172.21.15.38:0/372588585 <== osd.0 v2:172.21.15.38:6802/1622028921 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f77600b1010 con 0x7f77600ae4a0 2024-09-16T00:31:35.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.485+0000 7f777da4c640 1 -- 172.21.15.38:0/372588585 --> [v2:172.21.15.38:6802/1622028921,v1:172.21.15.38:6803/1622028921] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7778062d80 con 0x7f77600ae4a0 2024-09-16T00:31:35.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.485+0000 7f777ca4a640 1 -- 172.21.15.38:0/372588585 <== osd.0 v2:172.21.15.38:6802/1622028921 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7778062d80 con 0x7f77600ae4a0 2024-09-16T00:31:35.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.486+0000 7f777da4c640 1 -- 172.21.15.38:0/372588585 >> [v2:172.21.15.38:6802/1622028921,v1:172.21.15.38:6803/1622028921] conn(0x7f77600ae4a0 msgr2=0x7f77600b0940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:35.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.486+0000 7f777da4c640 1 --2- 172.21.15.38:0/372588585 >> [v2:172.21.15.38:6802/1622028921,v1:172.21.15.38:6803/1622028921] conn(0x7f77600ae4a0 0x7f77600b0940 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:35.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.486+0000 7f777da4c640 1 -- 172.21.15.38:0/372588585 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f77600a3350 msgr2=0x7f77600a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:35.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.486+0000 7f777da4c640 1 --2- 172.21.15.38:0/372588585 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f77600a3350 0x7f77600a5810 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7768005fd0 tx=0x7f776800b480 comp rx=0 tx=0).stop 2024-09-16T00:31:35.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.486+0000 7f777da4c640 1 -- 172.21.15.38:0/372588585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f77781036e0 msgr2=0x7f7778198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:35.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.486+0000 7f777da4c640 1 --2- 172.21.15.38:0/372588585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f77781036e0 0x7f7778198a70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f77780697d0 tx=0x7f776c00ee50 comp rx=0 tx=0).stop 2024-09-16T00:31:35.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.486+0000 7f77777fe640 1 -- 172.21.15.38:0/372588585 reap_dead start 2024-09-16T00:31:35.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.486+0000 7f777da4c640 1 -- 172.21.15.38:0/372588585 shutdown_connections 2024-09-16T00:31:35.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.486+0000 7f777da4c640 1 -- 172.21.15.38:0/372588585 >> 172.21.15.38:0/372588585 conn(0x7f77780fc7d0 msgr2=0x7f7778105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:35.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.486+0000 7f777da4c640 1 -- 172.21.15.38:0/372588585 shutdown_connections 2024-09-16T00:31:35.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.486+0000 7f777da4c640 1 -- 172.21.15.38:0/372588585 wait complete. 2024-09-16T00:31:35.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:35 smithi073 ceph-mon[35254]: pgmap v326: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 812 B/s wr, 6 op/s; 5 B/s, 0 keys/s, 0 objects/s recovering 2024-09-16T00:31:35.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.674+0000 7f6196cf1640 1 -- 172.21.15.38:0/2274436817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6190102dd0 msgr2=0x7f61901031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:35.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.674+0000 7f6196cf1640 1 --2- 172.21.15.38:0/2274436817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6190102dd0 0x7f61901031b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f6184009930 tx=0x7f618402a420 comp rx=0 tx=0).stop 2024-09-16T00:31:35.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.674+0000 7f6196cf1640 1 -- 172.21.15.38:0/2274436817 shutdown_connections 2024-09-16T00:31:35.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.674+0000 7f6196cf1640 1 --2- 172.21.15.38:0/2274436817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6190108300 0x7f619010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:35.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.674+0000 7f6196cf1640 1 --2- 172.21.15.38:0/2274436817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6190103780 0x7f6190107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:35.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.674+0000 7f6196cf1640 1 --2- 172.21.15.38:0/2274436817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6190102dd0 0x7f61901031b0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:35.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.674+0000 7f6196cf1640 1 -- 172.21.15.38:0/2274436817 >> 172.21.15.38:0/2274436817 conn(0x7f61900fc7d0 msgr2=0x7f61900febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:35.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.674+0000 7f6196cf1640 1 -- 172.21.15.38:0/2274436817 shutdown_connections 2024-09-16T00:31:35.688 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.674+0000 7f6196cf1640 1 -- 172.21.15.38:0/2274436817 wait complete. 2024-09-16T00:31:35.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.674+0000 7f6196cf1640 1 Processor -- start 2024-09-16T00:31:35.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.674+0000 7f6196cf1640 1 -- start start 2024-09-16T00:31:35.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.675+0000 7f6196cf1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6190102dd0 0x7f6190100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:35.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.675+0000 7f6196cf1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6190103780 0x7f6190100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:35.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.675+0000 7f6196cf1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6190108300 0x7f61901014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:35.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.675+0000 7f6196cf1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6190112090 con 0x7f6190103780 2024-09-16T00:31:35.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.675+0000 7f6196cf1640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6190111f10 con 0x7f6190102dd0 2024-09-16T00:31:35.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.675+0000 7f6196cf1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6190112210 con 0x7f6190108300 2024-09-16T00:31:35.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.675+0000 7f618ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6190103780 0x7f6190100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:35.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.675+0000 7f618ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6190103780 0x7f6190100f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:38852/0 (socket says 172.21.15.38:38852) 2024-09-16T00:31:35.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.675+0000 7f618ffff640 1 -- 172.21.15.38:0/3409025064 learned_addr learned my addr 172.21.15.38:0/3409025064 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:31:35.689 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.675+0000 7f6194a8d640 1 --2- 172.21.15.38:0/3409025064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6190102dd0 0x7f6190100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:35.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.675+0000 7f619528e640 1 --2- 172.21.15.38:0/3409025064 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6190108300 0x7f61901014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:35.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.675+0000 7f618ffff640 1 -- 172.21.15.38:0/3409025064 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6190108300 msgr2=0x7f61901014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:35.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.675+0000 7f618ffff640 1 --2- 172.21.15.38:0/3409025064 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6190108300 0x7f61901014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:35.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.675+0000 7f618ffff640 1 -- 172.21.15.38:0/3409025064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6190102dd0 msgr2=0x7f6190100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:35.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.675+0000 7f618ffff640 1 --2- 172.21.15.38:0/3409025064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6190102dd0 0x7f6190100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:35.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.675+0000 7f618ffff640 1 -- 172.21.15.38:0/3409025064 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61901a6a80 con 0x7f6190103780 2024-09-16T00:31:35.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.675+0000 7f618ffff640 1 --2- 172.21.15.38:0/3409025064 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6190103780 0x7f6190100f70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f617c00c8c0 tx=0x7f617c00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:35.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.676+0000 7f618dffb640 1 -- 172.21.15.38:0/3409025064 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c007bb0 con 0x7f6190103780 2024-09-16T00:31:35.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.676+0000 7f6196cf1640 1 -- 172.21.15.38:0/3409025064 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61901a6d70 con 0x7f6190103780 2024-09-16T00:31:35.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.676+0000 7f618dffb640 1 -- 172.21.15.38:0/3409025064 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f617c007d50 con 0x7f6190103780 2024-09-16T00:31:35.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.676+0000 7f618dffb640 1 -- 172.21.15.38:0/3409025064 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f617c018830 con 0x7f6190103780 2024-09-16T00:31:35.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.676+0000 7f6196cf1640 1 -- 172.21.15.38:0/3409025064 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61901a7220 con 0x7f6190103780 2024-09-16T00:31:35.690 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.677+0000 7f6196cf1640 1 -- 172.21.15.38:0/3409025064 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f61900628c0 con 0x7f6190103780 2024-09-16T00:31:35.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.678+0000 7f618dffb640 1 -- 172.21.15.38:0/3409025064 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f617c018ad0 con 0x7f6190103780 2024-09-16T00:31:35.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.679+0000 7f618dffb640 1 --2- 172.21.15.38:0/3409025064 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f61800a32d0 0x7f61800a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:35.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.679+0000 7f618dffb640 1 -- 172.21.15.38:0/3409025064 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f617c0cff20 con 0x7f6190103780 2024-09-16T00:31:35.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.679+0000 7f6194a8d640 1 --2- 172.21.15.38:0/3409025064 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f61800a32d0 0x7f61800a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:35.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.679+0000 7f618dffb640 1 --2- 172.21.15.38:0/3409025064 >> [v2:172.21.15.73:6824/43297139,v1:172.21.15.73:6825/43297139] conn(0x7f61800ae420 0x7f61800b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:35.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.679+0000 7f618dffb640 1 -- 172.21.15.38:0/3409025064 --> [v2:172.21.15.73:6824/43297139,v1:172.21.15.73:6825/43297139] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f617c012ec0 con 0x7f61800ae420 2024-09-16T00:31:35.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.679+0000 7f618dffb640 1 -- 172.21.15.38:0/3409025064 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f617c0d2540 con 0x7f6190103780 2024-09-16T00:31:35.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.679+0000 7f6194a8d640 1 --2- 172.21.15.38:0/3409025064 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f61800a32d0 0x7f61800a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6184009900 tx=0x7f61840058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:35.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.679+0000 7f619528e640 1 --2- 172.21.15.38:0/3409025064 >> [v2:172.21.15.73:6824/43297139,v1:172.21.15.73:6825/43297139] conn(0x7f61800ae420 0x7f61800b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:35.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.680+0000 7f619528e640 1 --2- 172.21.15.38:0/3409025064 >> [v2:172.21.15.73:6824/43297139,v1:172.21.15.73:6825/43297139] conn(0x7f61800ae420 0x7f61800b08c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:35.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.681+0000 7f618dffb640 1 -- 172.21.15.38:0/3409025064 <== osd.10 v2:172.21.15.73:6824/43297139 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f617c012ec0 con 0x7f61800ae420 2024-09-16T00:31:35.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.695+0000 7f6196cf1640 1 -- 172.21.15.38:0/3409025064 --> [v2:172.21.15.73:6824/43297139,v1:172.21.15.73:6825/43297139] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6190062cf0 con 0x7f61800ae420 2024-09-16T00:31:35.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.696+0000 7f618dffb640 1 -- 172.21.15.38:0/3409025064 <== osd.10 v2:172.21.15.73:6824/43297139 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6190062cf0 con 0x7f61800ae420 2024-09-16T00:31:35.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.696+0000 7f6196cf1640 1 -- 172.21.15.38:0/3409025064 >> [v2:172.21.15.73:6824/43297139,v1:172.21.15.73:6825/43297139] conn(0x7f61800ae420 msgr2=0x7f61800b08c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:35.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.696+0000 7f6196cf1640 1 --2- 172.21.15.38:0/3409025064 >> [v2:172.21.15.73:6824/43297139,v1:172.21.15.73:6825/43297139] conn(0x7f61800ae420 0x7f61800b08c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:35.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.696+0000 7f6196cf1640 1 -- 172.21.15.38:0/3409025064 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f61800a32d0 msgr2=0x7f61800a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:35.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.697+0000 7f6196cf1640 1 --2- 172.21.15.38:0/3409025064 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f61800a32d0 0x7f61800a5790 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6184009900 tx=0x7f61840058d0 comp rx=0 tx=0).stop 2024-09-16T00:31:35.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.697+0000 7f6196cf1640 1 -- 172.21.15.38:0/3409025064 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6190103780 msgr2=0x7f6190100f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:35.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.697+0000 7f6196cf1640 1 --2- 172.21.15.38:0/3409025064 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6190103780 0x7f6190100f70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f617c00c8c0 tx=0x7f617c00cd80 comp rx=0 tx=0).stop 2024-09-16T00:31:35.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.697+0000 7f619528e640 1 -- 172.21.15.38:0/3409025064 reap_dead start 2024-09-16T00:31:35.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.697+0000 7f6196cf1640 1 -- 172.21.15.38:0/3409025064 shutdown_connections 2024-09-16T00:31:35.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.697+0000 7f6196cf1640 1 -- 172.21.15.38:0/3409025064 >> 172.21.15.38:0/3409025064 conn(0x7f61900fc7d0 msgr2=0x7f6190105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:35.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.697+0000 7f6196cf1640 1 -- 172.21.15.38:0/3409025064 shutdown_connections 2024-09-16T00:31:35.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:35.697+0000 7f6196cf1640 1 -- 172.21.15.38:0/3409025064 wait complete. 2024-09-16T00:31:35.860 INFO:teuthology.orchestra.run.smithi038.stdout:210453397551 2024-09-16T00:31:35.860 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-16T00:31:36.799 INFO:teuthology.orchestra.run.smithi038.stdout:326417514523 2024-09-16T00:31:36.799 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-09-16T00:31:37.122 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:37.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:37 smithi038 ceph-mon[30945]: pgmap v327: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 7 op/s; 4 B/s, 0 keys/s, 0 objects/s recovering 2024-09-16T00:31:37.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:37 smithi122 ceph-mon[35412]: pgmap v327: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 7 op/s; 4 B/s, 0 keys/s, 0 objects/s recovering 2024-09-16T00:31:37.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:37 smithi073 ceph-mon[35254]: pgmap v327: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 682 B/s wr, 7 op/s; 4 B/s, 0 keys/s, 0 objects/s recovering 2024-09-16T00:31:37.603 INFO:teuthology.orchestra.run.smithi038.stdout:107374182466 2024-09-16T00:31:37.604 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-16T00:31:37.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.774+0000 7fe6def23640 1 -- 172.21.15.38:0/3265537826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6d8100e20 msgr2=0x7fe6d8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:37.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.774+0000 7fe6def23640 1 --2- 172.21.15.38:0/3265537826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6d8100e20 0x7fe6d8101200 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc009900 tx=0x7fe6cc02a480 comp rx=0 tx=0).stop 2024-09-16T00:31:37.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.774+0000 7fe6def23640 1 -- 172.21.15.38:0/3265537826 shutdown_connections 2024-09-16T00:31:37.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.774+0000 7fe6def23640 1 --2- 172.21.15.38:0/3265537826 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6d8102ef0 0x7fe6d810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:37.789 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.774+0000 7fe6def23640 1 --2- 172.21.15.38:0/3265537826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6d8102570 0x7fe6d81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:37.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.774+0000 7fe6def23640 1 --2- 172.21.15.38:0/3265537826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6d8100e20 0x7fe6d8101200 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:37.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.774+0000 7fe6def23640 1 -- 172.21.15.38:0/3265537826 >> 172.21.15.38:0/3265537826 conn(0x7fe6d80fc7d0 msgr2=0x7fe6d80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:37.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.774+0000 7fe6def23640 1 -- 172.21.15.38:0/3265537826 shutdown_connections 2024-09-16T00:31:37.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.774+0000 7fe6def23640 1 -- 172.21.15.38:0/3265537826 wait complete. 2024-09-16T00:31:37.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.774+0000 7fe6def23640 1 Processor -- start 2024-09-16T00:31:37.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.774+0000 7fe6def23640 1 -- start start 2024-09-16T00:31:37.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.775+0000 7fe6def23640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6d8100e20 0x7fe6d8199360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:37.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.775+0000 7fe6def23640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6d8102570 0x7fe6d81998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:37.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.775+0000 7fe6def23640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6d8102ef0 0x7fe6d81a43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:37.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.775+0000 7fe6def23640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d8110f50 con 0x7fe6d8100e20 2024-09-16T00:31:37.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.775+0000 7fe6def23640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d8110dd0 con 0x7fe6d8102570 2024-09-16T00:31:37.790 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.775+0000 7fe6def23640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d81110d0 con 0x7fe6d8102ef0 2024-09-16T00:31:37.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.775+0000 7fe6dccbf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6d8100e20 0x7fe6d8199360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:37.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.775+0000 7fe6dccbf640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6d8100e20 0x7fe6d8199360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40684/0 (socket says 172.21.15.38:40684) 2024-09-16T00:31:37.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.775+0000 7fe6dccbf640 1 -- 172.21.15.38:0/1906927656 learned_addr learned my addr 172.21.15.38:0/1906927656 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:31:37.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.775+0000 7fe6dd4c0640 1 --2- 172.21.15.38:0/1906927656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6d8102ef0 0x7fe6d81a43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:37.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.775+0000 7fe6d7fff640 1 --2- 172.21.15.38:0/1906927656 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6d8102570 0x7fe6d81998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:37.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.775+0000 7fe6dccbf640 1 -- 172.21.15.38:0/1906927656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6d8102ef0 msgr2=0x7fe6d81a43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:37.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.775+0000 7fe6dccbf640 1 --2- 172.21.15.38:0/1906927656 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fe6d8102ef0 0x7fe6d81a43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:37.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.775+0000 7fe6dccbf640 1 -- 172.21.15.38:0/1906927656 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6d8102570 msgr2=0x7fe6d81998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:37.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.775+0000 7fe6dccbf640 1 --2- 172.21.15.38:0/1906927656 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6d8102570 0x7fe6d81998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:37.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.775+0000 7fe6dccbf640 1 -- 172.21.15.38:0/1906927656 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d81a4920 con 0x7fe6d8100e20 2024-09-16T00:31:37.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.775+0000 7fe6d7fff640 1 --2- 172.21.15.38:0/1906927656 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6d8102570 0x7fe6d81998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:31:37.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.775+0000 7fe6dccbf640 1 --2- 172.21.15.38:0/1906927656 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6d8100e20 0x7fe6d8199360 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc006010 tx=0x7fe6cc003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:37.791 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.775+0000 7fe6d5ffb640 1 -- 172.21.15.38:0/1906927656 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc03f070 con 0x7fe6d8100e20 2024-09-16T00:31:37.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.776+0000 7fe6d5ffb640 1 -- 172.21.15.38:0/1906927656 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fe6cc031c30 con 0x7fe6d8100e20 2024-09-16T00:31:37.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.776+0000 7fe6d5ffb640 1 -- 172.21.15.38:0/1906927656 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6cc035030 con 0x7fe6d8100e20 2024-09-16T00:31:37.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.776+0000 7fe6def23640 1 -- 172.21.15.38:0/1906927656 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6d81a4af0 con 0x7fe6d8100e20 2024-09-16T00:31:37.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.776+0000 7fe6def23640 1 -- 172.21.15.38:0/1906927656 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6d81a4ef0 con 0x7fe6d8100e20 2024-09-16T00:31:37.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.777+0000 7fe6def23640 1 -- 172.21.15.38:0/1906927656 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe6d8062980 con 0x7fe6d8100e20 2024-09-16T00:31:37.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.778+0000 7fe6d5ffb640 1 -- 172.21.15.38:0/1906927656 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fe6cc042070 con 0x7fe6d8100e20 2024-09-16T00:31:37.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.779+0000 7fe6d5ffb640 1 --2- 172.21.15.38:0/1906927656 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fe6c80a32d0 0x7fe6c80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:37.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.779+0000 7fe6d5ffb640 1 -- 172.21.15.38:0/1906927656 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7fe6cc0f0ef0 con 0x7fe6d8100e20 2024-09-16T00:31:37.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.779+0000 7fe6d7fff640 1 --2- 172.21.15.38:0/1906927656 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fe6c80a32d0 0x7fe6c80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:37.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.779+0000 7fe6d5ffb640 1 --2- 172.21.15.38:0/1906927656 >> [v2:172.21.15.38:6826/445727833,v1:172.21.15.38:6827/445727833] conn(0x7fe6c80ae420 0x7fe6c80b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:37.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.779+0000 7fe6d5ffb640 1 -- 172.21.15.38:0/1906927656 --> [v2:172.21.15.38:6826/445727833,v1:172.21.15.38:6827/445727833] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe6c80b0f90 con 0x7fe6c80ae420 2024-09-16T00:31:37.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.779+0000 7fe6d5ffb640 1 -- 172.21.15.38:0/1906927656 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fe6cc0f12a0 con 0x7fe6d8100e20 2024-09-16T00:31:37.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.779+0000 7fe6dd4c0640 1 --2- 172.21.15.38:0/1906927656 >> [v2:172.21.15.38:6826/445727833,v1:172.21.15.38:6827/445727833] conn(0x7fe6c80ae420 0x7fe6c80b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:37.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.779+0000 7fe6d7fff640 1 --2- 172.21.15.38:0/1906927656 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fe6c80a32d0 0x7fe6c80a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe6d806aac0 tx=0x7fe6c400b5f0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:37.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.780+0000 7fe6dd4c0640 1 --2- 172.21.15.38:0/1906927656 >> [v2:172.21.15.38:6826/445727833,v1:172.21.15.38:6827/445727833] conn(0x7fe6c80ae420 0x7fe6c80b08c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:37.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.780+0000 7fe6d5ffb640 1 -- 172.21.15.38:0/1906927656 <== osd.9 v2:172.21.15.38:6826/445727833 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe6c80b0f90 con 0x7fe6c80ae420 2024-09-16T00:31:37.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.799+0000 7fe6def23640 1 -- 172.21.15.38:0/1906927656 --> [v2:172.21.15.38:6826/445727833,v1:172.21.15.38:6827/445727833] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe6d8062e00 con 0x7fe6c80ae420 2024-09-16T00:31:37.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.800+0000 7fe6d5ffb640 1 -- 172.21.15.38:0/1906927656 <== osd.9 v2:172.21.15.38:6826/445727833 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe6d8062e00 con 0x7fe6c80ae420 2024-09-16T00:31:37.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.800+0000 7fe6def23640 1 -- 172.21.15.38:0/1906927656 >> [v2:172.21.15.38:6826/445727833,v1:172.21.15.38:6827/445727833] conn(0x7fe6c80ae420 msgr2=0x7fe6c80b08c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:37.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.800+0000 7fe6def23640 1 --2- 172.21.15.38:0/1906927656 >> [v2:172.21.15.38:6826/445727833,v1:172.21.15.38:6827/445727833] conn(0x7fe6c80ae420 0x7fe6c80b08c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:37.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.800+0000 7fe6def23640 1 -- 172.21.15.38:0/1906927656 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fe6c80a32d0 msgr2=0x7fe6c80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:37.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.800+0000 7fe6def23640 1 --2- 172.21.15.38:0/1906927656 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fe6c80a32d0 0x7fe6c80a5790 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe6d806aac0 tx=0x7fe6c400b5f0 comp rx=0 tx=0).stop 2024-09-16T00:31:37.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.800+0000 7fe6def23640 1 -- 172.21.15.38:0/1906927656 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6d8100e20 msgr2=0x7fe6d8199360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:37.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.800+0000 7fe6def23640 1 --2- 172.21.15.38:0/1906927656 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fe6d8100e20 0x7fe6d8199360 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc006010 tx=0x7fe6cc003c10 comp rx=0 tx=0).stop 2024-09-16T00:31:37.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.800+0000 7fe6dd4c0640 1 -- 172.21.15.38:0/1906927656 reap_dead start 2024-09-16T00:31:37.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.801+0000 7fe6def23640 1 -- 172.21.15.38:0/1906927656 shutdown_connections 2024-09-16T00:31:37.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.801+0000 7fe6def23640 1 -- 172.21.15.38:0/1906927656 >> 172.21.15.38:0/1906927656 conn(0x7fe6d80fc7d0 msgr2=0x7fe6d80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:37.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.801+0000 7fe6def23640 1 -- 172.21.15.38:0/1906927656 shutdown_connections 2024-09-16T00:31:37.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.801+0000 7fe6def23640 1 -- 172.21.15.38:0/1906927656 wait complete. 2024-09-16T00:31:38.528 INFO:teuthology.orchestra.run.smithi038.stdout:274877906982 2024-09-16T00:31:38.529 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-09-16T00:31:38.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.992+0000 7fc5e2775640 1 -- 172.21.15.38:0/1132273588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5dc102d90 msgr2=0x7fc5dc103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:38.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.992+0000 7fc5e2775640 1 --2- 172.21.15.38:0/1132273588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5dc102d90 0x7fc5dc103170 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009930 tx=0x7fc5cc02a420 comp rx=0 tx=0).stop 2024-09-16T00:31:38.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.993+0000 7fc5e2775640 1 -- 172.21.15.38:0/1132273588 shutdown_connections 2024-09-16T00:31:38.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.993+0000 7fc5e2775640 1 --2- 172.21.15.38:0/1132273588 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5dc108290 0x7fc5dc10ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:38.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.993+0000 7fc5e2775640 1 --2- 172.21.15.38:0/1132273588 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5dc103740 0x7fc5dc107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:38.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.993+0000 7fc5e2775640 1 --2- 172.21.15.38:0/1132273588 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5dc102d90 0x7fc5dc103170 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:38.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.993+0000 7fc5e2775640 1 -- 172.21.15.38:0/1132273588 >> 172.21.15.38:0/1132273588 conn(0x7fc5dc0fc7d0 msgr2=0x7fc5dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:38.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.993+0000 7fc5e2775640 1 -- 172.21.15.38:0/1132273588 shutdown_connections 2024-09-16T00:31:38.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.993+0000 7fc5e2775640 1 -- 172.21.15.38:0/1132273588 wait complete. 2024-09-16T00:31:38.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.993+0000 7fc5e2775640 1 Processor -- start 2024-09-16T00:31:38.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.993+0000 7fc5e2775640 1 -- start start 2024-09-16T00:31:38.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.994+0000 7fc5e2775640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5dc102d90 0x7fc5dc1a4890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:38.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.994+0000 7fc5e2775640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5dc103740 0x7fc5dc1a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:38.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.994+0000 7fc5e2775640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5dc108290 0x7fc5dc10ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:38.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.994+0000 7fc5e2775640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc112090 con 0x7fc5dc108290 2024-09-16T00:31:38.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.994+0000 7fc5e2775640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc111f10 con 0x7fc5dc103740 2024-09-16T00:31:38.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.994+0000 7fc5e2775640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc5dc112210 con 0x7fc5dc102d90 2024-09-16T00:31:38.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.994+0000 7fc5e0d12640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5dc108290 0x7fc5dc10ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:38.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.994+0000 7fc5e0d12640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5dc108290 0x7fc5dc10ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40656/0 (socket says 172.21.15.38:40656) 2024-09-16T00:31:38.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.994+0000 7fc5e0d12640 1 -- 172.21.15.38:0/4261528955 learned_addr learned my addr 172.21.15.38:0/4261528955 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:31:38.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.994+0000 7fc5dbfff640 1 --2- 172.21.15.38:0/4261528955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5dc102d90 0x7fc5dc1a4890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:38.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.994+0000 7fc5db7fe640 1 --2- 172.21.15.38:0/4261528955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5dc103740 0x7fc5dc1a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:38.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.994+0000 7fc5e0d12640 1 -- 172.21.15.38:0/4261528955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5dc102d90 msgr2=0x7fc5dc1a4890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:38.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.994+0000 7fc5e0d12640 1 --2- 172.21.15.38:0/4261528955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc5dc102d90 0x7fc5dc1a4890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:38.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.994+0000 7fc5e0d12640 1 -- 172.21.15.38:0/4261528955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5dc103740 msgr2=0x7fc5dc1a4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:38.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.994+0000 7fc5e0d12640 1 --2- 172.21.15.38:0/4261528955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5dc103740 0x7fc5dc1a4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:38.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.995+0000 7fc5e0d12640 1 -- 172.21.15.38:0/4261528955 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5dc10e6b0 con 0x7fc5dc108290 2024-09-16T00:31:38.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.995+0000 7fc5db7fe640 1 --2- 172.21.15.38:0/4261528955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5dc103740 0x7fc5dc1a4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:31:38.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.995+0000 7fc5e0d12640 1 --2- 172.21.15.38:0/4261528955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5dc108290 0x7fc5dc10ddf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc5d400cce0 tx=0x7fc5d400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:38.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.995+0000 7fc5d97fa640 1 -- 172.21.15.38:0/4261528955 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d400f450 con 0x7fc5dc108290 2024-09-16T00:31:38.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.995+0000 7fc5d97fa640 1 -- 172.21.15.38:0/4261528955 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc5d400fad0 con 0x7fc5dc108290 2024-09-16T00:31:38.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.995+0000 7fc5e2775640 1 -- 172.21.15.38:0/4261528955 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5dc10e9a0 con 0x7fc5dc108290 2024-09-16T00:31:38.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.995+0000 7fc5d97fa640 1 -- 172.21.15.38:0/4261528955 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5d4018c10 con 0x7fc5dc108290 2024-09-16T00:31:38.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.995+0000 7fc5e2775640 1 -- 172.21.15.38:0/4261528955 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5dc1ab830 con 0x7fc5dc108290 2024-09-16T00:31:38.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.996+0000 7fc5e2775640 1 -- 172.21.15.38:0/4261528955 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc5dc062980 con 0x7fc5dc108290 2024-09-16T00:31:38.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.997+0000 7fc5d97fa640 1 -- 172.21.15.38:0/4261528955 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fc5d400f5f0 con 0x7fc5dc108290 2024-09-16T00:31:38.743 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.998+0000 7fc5d97fa640 1 --2- 172.21.15.38:0/4261528955 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fc5c40a32d0 0x7fc5c40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:38.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.998+0000 7fc5d97fa640 1 -- 172.21.15.38:0/4261528955 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7fc5d40d0710 con 0x7fc5dc108290 2024-09-16T00:31:38.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.998+0000 7fc5dbfff640 1 --2- 172.21.15.38:0/4261528955 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fc5c40a32d0 0x7fc5c40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:38.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.998+0000 7fc5d97fa640 1 --2- 172.21.15.38:0/4261528955 >> [v2:172.21.15.38:6818/1415719550,v1:172.21.15.38:6819/1415719550] conn(0x7fc5c40ae420 0x7fc5c40b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:38.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.998+0000 7fc5d97fa640 1 -- 172.21.15.38:0/4261528955 --> [v2:172.21.15.38:6818/1415719550,v1:172.21.15.38:6819/1415719550] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc5c40b0f90 con 0x7fc5c40ae420 2024-09-16T00:31:38.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.998+0000 7fc5d97fa640 1 -- 172.21.15.38:0/4261528955 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fc5d40d0ac0 con 0x7fc5dc108290 2024-09-16T00:31:38.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.999+0000 7fc5db7fe640 1 --2- 172.21.15.38:0/4261528955 >> [v2:172.21.15.38:6818/1415719550,v1:172.21.15.38:6819/1415719550] conn(0x7fc5c40ae420 0x7fc5c40b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:38.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.999+0000 7fc5dbfff640 1 --2- 172.21.15.38:0/4261528955 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fc5c40a32d0 0x7fc5c40a5790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009900 tx=0x7fc5cc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:38.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:36.999+0000 7fc5db7fe640 1 --2- 172.21.15.38:0/4261528955 >> [v2:172.21.15.38:6818/1415719550,v1:172.21.15.38:6819/1415719550] conn(0x7fc5c40ae420 0x7fc5c40b08c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:38.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.000+0000 7fc5d97fa640 1 -- 172.21.15.38:0/4261528955 <== osd.6 v2:172.21.15.38:6818/1415719550 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc5c40b0f90 con 0x7fc5c40ae420 2024-09-16T00:31:38.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.013+0000 7fc5e2775640 1 -- 172.21.15.38:0/4261528955 --> [v2:172.21.15.38:6818/1415719550,v1:172.21.15.38:6819/1415719550] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc5dc062e00 con 0x7fc5c40ae420 2024-09-16T00:31:38.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.014+0000 7fc5d97fa640 1 -- 172.21.15.38:0/4261528955 <== osd.6 v2:172.21.15.38:6818/1415719550 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc5dc062e00 con 0x7fc5c40ae420 2024-09-16T00:31:38.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.014+0000 7fc5e2775640 1 -- 172.21.15.38:0/4261528955 >> [v2:172.21.15.38:6818/1415719550,v1:172.21.15.38:6819/1415719550] conn(0x7fc5c40ae420 msgr2=0x7fc5c40b08c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:38.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.014+0000 7fc5e2775640 1 --2- 172.21.15.38:0/4261528955 >> [v2:172.21.15.38:6818/1415719550,v1:172.21.15.38:6819/1415719550] conn(0x7fc5c40ae420 0x7fc5c40b08c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:38.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.014+0000 7fc5e2775640 1 -- 172.21.15.38:0/4261528955 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fc5c40a32d0 msgr2=0x7fc5c40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:38.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.014+0000 7fc5e2775640 1 --2- 172.21.15.38:0/4261528955 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fc5c40a32d0 0x7fc5c40a5790 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc009900 tx=0x7fc5cc0058d0 comp rx=0 tx=0).stop 2024-09-16T00:31:38.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.014+0000 7fc5e2775640 1 -- 172.21.15.38:0/4261528955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5dc108290 msgr2=0x7fc5dc10ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:38.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.014+0000 7fc5e2775640 1 --2- 172.21.15.38:0/4261528955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc5dc108290 0x7fc5dc10ddf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc5d400cce0 tx=0x7fc5d400a620 comp rx=0 tx=0).stop 2024-09-16T00:31:38.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.014+0000 7fc5e0d12640 1 -- 172.21.15.38:0/4261528955 reap_dead start 2024-09-16T00:31:38.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.014+0000 7fc5e2775640 1 -- 172.21.15.38:0/4261528955 shutdown_connections 2024-09-16T00:31:38.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.014+0000 7fc5e2775640 1 -- 172.21.15.38:0/4261528955 >> 172.21.15.38:0/4261528955 conn(0x7fc5dc0fc7d0 msgr2=0x7fc5dc1016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:38.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.014+0000 7fc5e2775640 1 -- 172.21.15.38:0/4261528955 shutdown_connections 2024-09-16T00:31:38.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:37.014+0000 7fc5e2775640 1 -- 172.21.15.38:0/4261528955 wait complete. 2024-09-16T00:31:39.397 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:39 smithi038 ceph-mon[30945]: pgmap v328: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-16T00:31:39.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:39 smithi122 ceph-mon[35412]: pgmap v328: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-16T00:31:39.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:39 smithi073 ceph-mon[35254]: pgmap v328: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-16T00:31:39.745 INFO:teuthology.orchestra.run.smithi038.stdout:188978561078 2024-09-16T00:31:39.745 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-16T00:31:39.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.919+0000 7fdb60c97640 1 -- 172.21.15.38:0/718510551 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb5c1027c0 msgr2=0x7fdb5c106810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:39.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.919+0000 7fdb60c97640 1 --2- 172.21.15.38:0/718510551 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb5c1027c0 0x7fdb5c106810 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fdb50009900 tx=0x7fdb5002a410 comp rx=0 tx=0).stop 2024-09-16T00:31:39.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.919+0000 7fdb60c97640 1 -- 172.21.15.38:0/718510551 shutdown_connections 2024-09-16T00:31:39.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.919+0000 7fdb60c97640 1 --2- 172.21.15.38:0/718510551 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb5c106f40 0x7fdb5c10eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:39.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.919+0000 7fdb60c97640 1 --2- 172.21.15.38:0/718510551 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb5c1027c0 0x7fdb5c106810 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:39.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.919+0000 7fdb60c97640 1 --2- 172.21.15.38:0/718510551 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb5c101e10 0x7fdb5c1021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:39.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.919+0000 7fdb60c97640 1 -- 172.21.15.38:0/718510551 >> 172.21.15.38:0/718510551 conn(0x7fdb5c0fd8d0 msgr2=0x7fdb5c0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:39.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.919+0000 7fdb60c97640 1 -- 172.21.15.38:0/718510551 shutdown_connections 2024-09-16T00:31:39.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.919+0000 7fdb60c97640 1 -- 172.21.15.38:0/718510551 wait complete. 2024-09-16T00:31:39.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.920+0000 7fdb60c97640 1 Processor -- start 2024-09-16T00:31:39.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.920+0000 7fdb60c97640 1 -- start start 2024-09-16T00:31:39.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.920+0000 7fdb60c97640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb5c101e10 0x7fdb5c198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:39.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.920+0000 7fdb60c97640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb5c1027c0 0x7fdb5c1989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:39.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.920+0000 7fdb60c97640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb5c106f40 0x7fdb5c19cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:39.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.920+0000 7fdb60c97640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c111fa0 con 0x7fdb5c101e10 2024-09-16T00:31:39.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.920+0000 7fdb60c97640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c111e20 con 0x7fdb5c106f40 2024-09-16T00:31:39.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.920+0000 7fdb60c97640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdb5c112120 con 0x7fdb5c1027c0 2024-09-16T00:31:39.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.920+0000 7fdb5a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb5c101e10 0x7fdb5c198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:39.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.920+0000 7fdb5a59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb5c101e10 0x7fdb5c198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40702/0 (socket says 172.21.15.38:40702) 2024-09-16T00:31:39.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.920+0000 7fdb5a59c640 1 -- 172.21.15.38:0/1311810240 learned_addr learned my addr 172.21.15.38:0/1311810240 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:31:39.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.920+0000 7fdb5ad9d640 1 --2- 172.21.15.38:0/1311810240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb5c106f40 0x7fdb5c19cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:39.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.920+0000 7fdb59d9b640 1 --2- 172.21.15.38:0/1311810240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb5c1027c0 0x7fdb5c1989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:39.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.920+0000 7fdb5a59c640 1 -- 172.21.15.38:0/1311810240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb5c1027c0 msgr2=0x7fdb5c1989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:39.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.920+0000 7fdb5a59c640 1 --2- 172.21.15.38:0/1311810240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb5c1027c0 0x7fdb5c1989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:39.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.920+0000 7fdb5a59c640 1 -- 172.21.15.38:0/1311810240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb5c106f40 msgr2=0x7fdb5c19cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:39.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.920+0000 7fdb5a59c640 1 --2- 172.21.15.38:0/1311810240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb5c106f40 0x7fdb5c19cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:39.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.921+0000 7fdb5a59c640 1 -- 172.21.15.38:0/1311810240 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb5c19d430 con 0x7fdb5c101e10 2024-09-16T00:31:39.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.921+0000 7fdb59d9b640 1 --2- 172.21.15.38:0/1311810240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb5c1027c0 0x7fdb5c1989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:31:39.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.921+0000 7fdb5a59c640 1 --2- 172.21.15.38:0/1311810240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb5c101e10 0x7fdb5c198480 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c00d770 tx=0x7fdb4c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:39.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.921+0000 7fdb4b7fe640 1 -- 172.21.15.38:0/1311810240 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c017070 con 0x7fdb5c101e10 2024-09-16T00:31:39.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.921+0000 7fdb60c97640 1 -- 172.21.15.38:0/1311810240 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb5c19d720 con 0x7fdb5c101e10 2024-09-16T00:31:39.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.921+0000 7fdb4b7fe640 1 -- 172.21.15.38:0/1311810240 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb4c005500 con 0x7fdb5c101e10 2024-09-16T00:31:39.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.921+0000 7fdb4b7fe640 1 -- 172.21.15.38:0/1311810240 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c004e00 con 0x7fdb5c101e10 2024-09-16T00:31:39.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.921+0000 7fdb60c97640 1 -- 172.21.15.38:0/1311810240 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb5c1a5000 con 0x7fdb5c101e10 2024-09-16T00:31:39.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.922+0000 7fdb60c97640 1 -- 172.21.15.38:0/1311810240 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdb5c062980 con 0x7fdb5c101e10 2024-09-16T00:31:39.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.923+0000 7fdb4b7fe640 1 -- 172.21.15.38:0/1311810240 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fdb4c004fa0 con 0x7fdb5c101e10 2024-09-16T00:31:39.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.924+0000 7fdb4b7fe640 1 --2- 172.21.15.38:0/1311810240 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fdb440a32d0 0x7fdb440a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:39.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.924+0000 7fdb4b7fe640 1 -- 172.21.15.38:0/1311810240 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7fdb4c0cfdd0 con 0x7fdb5c101e10 2024-09-16T00:31:39.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.924+0000 7fdb59d9b640 1 --2- 172.21.15.38:0/1311810240 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fdb440a32d0 0x7fdb440a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:39.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.924+0000 7fdb4b7fe640 1 --2- 172.21.15.38:0/1311810240 >> [v2:172.21.15.73:6808/1469927628,v1:172.21.15.73:6809/1469927628] conn(0x7fdb440ae420 0x7fdb440b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:39.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.924+0000 7fdb4b7fe640 1 -- 172.21.15.38:0/1311810240 --> [v2:172.21.15.73:6808/1469927628,v1:172.21.15.73:6809/1469927628] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdb4c015e80 con 0x7fdb440ae420 2024-09-16T00:31:39.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.924+0000 7fdb4b7fe640 1 -- 172.21.15.38:0/1311810240 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fdb4c0d0220 con 0x7fdb5c101e10 2024-09-16T00:31:39.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.925+0000 7fdb59d9b640 1 --2- 172.21.15.38:0/1311810240 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fdb440a32d0 0x7fdb440a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c0fdcf0 tx=0x7fdb50005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:39.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.925+0000 7fdb5ad9d640 1 --2- 172.21.15.38:0/1311810240 >> [v2:172.21.15.73:6808/1469927628,v1:172.21.15.73:6809/1469927628] conn(0x7fdb440ae420 0x7fdb440b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:39.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.925+0000 7fdb5ad9d640 1 --2- 172.21.15.38:0/1311810240 >> [v2:172.21.15.73:6808/1469927628,v1:172.21.15.73:6809/1469927628] conn(0x7fdb440ae420 0x7fdb440b08c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:39.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.926+0000 7fdb4b7fe640 1 -- 172.21.15.38:0/1311810240 <== osd.4 v2:172.21.15.73:6808/1469927628 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdb4c015e80 con 0x7fdb440ae420 2024-09-16T00:31:39.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.940+0000 7fdb60c97640 1 -- 172.21.15.38:0/1311810240 --> [v2:172.21.15.73:6808/1469927628,v1:172.21.15.73:6809/1469927628] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdb5c062e00 con 0x7fdb440ae420 2024-09-16T00:31:39.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.941+0000 7fdb4b7fe640 1 -- 172.21.15.38:0/1311810240 <== osd.4 v2:172.21.15.73:6808/1469927628 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdb5c062e00 con 0x7fdb440ae420 2024-09-16T00:31:39.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.941+0000 7fdb60c97640 1 -- 172.21.15.38:0/1311810240 >> [v2:172.21.15.73:6808/1469927628,v1:172.21.15.73:6809/1469927628] conn(0x7fdb440ae420 msgr2=0x7fdb440b08c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:39.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.941+0000 7fdb60c97640 1 --2- 172.21.15.38:0/1311810240 >> [v2:172.21.15.73:6808/1469927628,v1:172.21.15.73:6809/1469927628] conn(0x7fdb440ae420 0x7fdb440b08c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:39.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.941+0000 7fdb60c97640 1 -- 172.21.15.38:0/1311810240 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fdb440a32d0 msgr2=0x7fdb440a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:39.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.941+0000 7fdb60c97640 1 --2- 172.21.15.38:0/1311810240 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fdb440a32d0 0x7fdb440a5790 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fdb5c0fdcf0 tx=0x7fdb50005a20 comp rx=0 tx=0).stop 2024-09-16T00:31:39.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.941+0000 7fdb60c97640 1 -- 172.21.15.38:0/1311810240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb5c101e10 msgr2=0x7fdb5c198480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:39.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.941+0000 7fdb60c97640 1 --2- 172.21.15.38:0/1311810240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb5c101e10 0x7fdb5c198480 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c00d770 tx=0x7fdb4c00dc30 comp rx=0 tx=0).stop 2024-09-16T00:31:39.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.941+0000 7fdb5ad9d640 1 -- 172.21.15.38:0/1311810240 reap_dead start 2024-09-16T00:31:39.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.942+0000 7fdb60c97640 1 -- 172.21.15.38:0/1311810240 shutdown_connections 2024-09-16T00:31:39.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.942+0000 7fdb60c97640 1 -- 172.21.15.38:0/1311810240 >> 172.21.15.38:0/1311810240 conn(0x7fdb5c0fd8d0 msgr2=0x7fdb5c0fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:39.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.942+0000 7fdb60c97640 1 -- 172.21.15.38:0/1311810240 shutdown_connections 2024-09-16T00:31:39.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:39.942+0000 7fdb60c97640 1 -- 172.21.15.38:0/1311810240 wait complete. 2024-09-16T00:31:41.060 INFO:teuthology.orchestra.run.smithi038.stdout:137438953537 2024-09-16T00:31:41.061 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-16T00:31:41.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.245+0000 7f3fad2b3640 1 -- 172.21.15.38:0/1711476273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3fa8074f40 msgr2=0x7f3fa810de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:41.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.245+0000 7f3fad2b3640 1 --2- 172.21.15.38:0/1711476273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3fa8074f40 0x7f3fa810de80 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c009900 tx=0x7f3f9c02a480 comp rx=0 tx=0).stop 2024-09-16T00:31:41.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.245+0000 7f3fad2b3640 1 -- 172.21.15.38:0/1711476273 shutdown_connections 2024-09-16T00:31:41.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.245+0000 7f3fad2b3640 1 --2- 172.21.15.38:0/1711476273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3fa810e3c0 0x7f3fa8110810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:41.269 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.245+0000 7f3fad2b3640 1 --2- 172.21.15.38:0/1711476273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3fa8074f40 0x7f3fa810de80 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:41.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.245+0000 7f3fad2b3640 1 --2- 172.21.15.38:0/1711476273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fa8074620 0x7f3fa8074a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:41.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.245+0000 7f3fad2b3640 1 -- 172.21.15.38:0/1711476273 >> 172.21.15.38:0/1711476273 conn(0x7f3fa80fd8c0 msgr2=0x7f3fa80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:41.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.245+0000 7f3fad2b3640 1 -- 172.21.15.38:0/1711476273 shutdown_connections 2024-09-16T00:31:41.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.246+0000 7f3fad2b3640 1 -- 172.21.15.38:0/1711476273 wait complete. 2024-09-16T00:31:41.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.246+0000 7f3fad2b3640 1 Processor -- start 2024-09-16T00:31:41.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.246+0000 7f3fad2b3640 1 -- start start 2024-09-16T00:31:41.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.247+0000 7f3fad2b3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fa8074620 0x7f3fa819a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:41.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.247+0000 7f3fad2b3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3fa8074f40 0x7f3fa819abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:41.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.247+0000 7f3fad2b3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3fa810e3c0 0x7f3fa819ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:41.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.247+0000 7f3fad2b3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa8114160 con 0x7f3fa810e3c0 2024-09-16T00:31:41.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.247+0000 7f3fad2b3640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa8113fe0 con 0x7f3fa8074f40 2024-09-16T00:31:41.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.247+0000 7f3fad2b3640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3fa81142e0 con 0x7f3fa8074620 2024-09-16T00:31:41.270 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.247+0000 7f3fa77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3fa810e3c0 0x7f3fa819ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:41.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.247+0000 7f3fa77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3fa810e3c0 0x7f3fa819ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40726/0 (socket says 172.21.15.38:40726) 2024-09-16T00:31:41.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.247+0000 7f3fa77fe640 1 -- 172.21.15.38:0/2958504701 learned_addr learned my addr 172.21.15.38:0/2958504701 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:31:41.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.247+0000 7f3fa6ffd640 1 --2- 172.21.15.38:0/2958504701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fa8074620 0x7f3fa819a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:41.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.247+0000 7f3fa67fc640 1 --2- 172.21.15.38:0/2958504701 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3fa8074f40 0x7f3fa819abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:41.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.247+0000 7f3fa77fe640 1 -- 172.21.15.38:0/2958504701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fa8074620 msgr2=0x7f3fa819a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:41.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.247+0000 7f3fa77fe640 1 --2- 172.21.15.38:0/2958504701 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3fa8074620 0x7f3fa819a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:41.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.247+0000 7f3fa77fe640 1 -- 172.21.15.38:0/2958504701 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3fa8074f40 msgr2=0x7f3fa819abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:41.271 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.247+0000 7f3fa77fe640 1 --2- 172.21.15.38:0/2958504701 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3fa8074f40 0x7f3fa819abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:41.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.247+0000 7f3fa77fe640 1 -- 172.21.15.38:0/2958504701 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fa819f630 con 0x7f3fa810e3c0 2024-09-16T00:31:41.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.247+0000 7f3fa77fe640 1 --2- 172.21.15.38:0/2958504701 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3fa810e3c0 0x7f3fa819ef50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3fa000eaa0 tx=0x7f3fa000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:41.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.248+0000 7f3f8ffff640 1 -- 172.21.15.38:0/2958504701 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa000cdb0 con 0x7f3fa810e3c0 2024-09-16T00:31:41.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.248+0000 7f3f8ffff640 1 -- 172.21.15.38:0/2958504701 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3fa0005020 con 0x7f3fa810e3c0 2024-09-16T00:31:41.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.248+0000 7f3fad2b3640 1 -- 172.21.15.38:0/2958504701 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fa819f920 con 0x7f3fa810e3c0 2024-09-16T00:31:41.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.248+0000 7f3f8ffff640 1 -- 172.21.15.38:0/2958504701 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fa00116b0 con 0x7f3fa810e3c0 2024-09-16T00:31:41.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.248+0000 7f3fad2b3640 1 -- 172.21.15.38:0/2958504701 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fa81a7200 con 0x7f3fa810e3c0 2024-09-16T00:31:41.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.249+0000 7f3fad2b3640 1 -- 172.21.15.38:0/2958504701 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3fa8062980 con 0x7f3fa810e3c0 2024-09-16T00:31:41.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.250+0000 7f3f8ffff640 1 -- 172.21.15.38:0/2958504701 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f3fa0005560 con 0x7f3fa810e3c0 2024-09-16T00:31:41.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.251+0000 7f3f8ffff640 1 --2- 172.21.15.38:0/2958504701 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f3f900a32d0 0x7f3f900a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:41.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.251+0000 7f3f8ffff640 1 -- 172.21.15.38:0/2958504701 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f3fa00d0340 con 0x7f3fa810e3c0 2024-09-16T00:31:41.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.251+0000 7f3fa6ffd640 1 --2- 172.21.15.38:0/2958504701 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f3f900a32d0 0x7f3f900a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:41.272 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.251+0000 7f3f8ffff640 1 --2- 172.21.15.38:0/2958504701 >> [v2:172.21.15.122:6800/1754741655,v1:172.21.15.122:6801/1754741655] conn(0x7f3f900ae420 0x7f3f900b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:41.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.251+0000 7f3f8ffff640 1 -- 172.21.15.38:0/2958504701 --> [v2:172.21.15.122:6800/1754741655,v1:172.21.15.122:6801/1754741655] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3fa0014ec0 con 0x7f3f900ae420 2024-09-16T00:31:41.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.251+0000 7f3f8ffff640 1 -- 172.21.15.38:0/2958504701 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f3fa00d0790 con 0x7f3fa810e3c0 2024-09-16T00:31:41.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.251+0000 7f3fa6ffd640 1 --2- 172.21.15.38:0/2958504701 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f3f900a32d0 0x7f3f900a5790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3f98005fd0 tx=0x7f3f9800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:41.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.251+0000 7f3fa67fc640 1 --2- 172.21.15.38:0/2958504701 >> [v2:172.21.15.122:6800/1754741655,v1:172.21.15.122:6801/1754741655] conn(0x7f3f900ae420 0x7f3f900b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:41.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.252+0000 7f3fa67fc640 1 --2- 172.21.15.38:0/2958504701 >> [v2:172.21.15.122:6800/1754741655,v1:172.21.15.122:6801/1754741655] conn(0x7f3f900ae420 0x7f3f900b08c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:41.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.253+0000 7f3f8ffff640 1 -- 172.21.15.38:0/2958504701 <== osd.2 v2:172.21.15.122:6800/1754741655 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3fa0014ec0 con 0x7f3f900ae420 2024-09-16T00:31:41.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.267+0000 7f3fad2b3640 1 -- 172.21.15.38:0/2958504701 --> [v2:172.21.15.122:6800/1754741655,v1:172.21.15.122:6801/1754741655] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3fa8062e00 con 0x7f3f900ae420 2024-09-16T00:31:41.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.268+0000 7f3f8ffff640 1 -- 172.21.15.38:0/2958504701 <== osd.2 v2:172.21.15.122:6800/1754741655 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3fa8062e00 con 0x7f3f900ae420 2024-09-16T00:31:41.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.268+0000 7f3fad2b3640 1 -- 172.21.15.38:0/2958504701 >> [v2:172.21.15.122:6800/1754741655,v1:172.21.15.122:6801/1754741655] conn(0x7f3f900ae420 msgr2=0x7f3f900b08c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:41.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.268+0000 7f3fad2b3640 1 --2- 172.21.15.38:0/2958504701 >> [v2:172.21.15.122:6800/1754741655,v1:172.21.15.122:6801/1754741655] conn(0x7f3f900ae420 0x7f3f900b08c0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:41.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.268+0000 7f3fad2b3640 1 -- 172.21.15.38:0/2958504701 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f3f900a32d0 msgr2=0x7f3f900a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:41.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.268+0000 7f3fad2b3640 1 --2- 172.21.15.38:0/2958504701 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f3f900a32d0 0x7f3f900a5790 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3f98005fd0 tx=0x7f3f9800b390 comp rx=0 tx=0).stop 2024-09-16T00:31:41.273 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.268+0000 7f3fad2b3640 1 -- 172.21.15.38:0/2958504701 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3fa810e3c0 msgr2=0x7f3fa819ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:41.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.268+0000 7f3fad2b3640 1 --2- 172.21.15.38:0/2958504701 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3fa810e3c0 0x7f3fa819ef50 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3fa000eaa0 tx=0x7f3fa000ef60 comp rx=0 tx=0).stop 2024-09-16T00:31:41.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.268+0000 7f3fa77fe640 1 -- 172.21.15.38:0/2958504701 reap_dead start 2024-09-16T00:31:41.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.268+0000 7f3fad2b3640 1 -- 172.21.15.38:0/2958504701 shutdown_connections 2024-09-16T00:31:41.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.268+0000 7f3fad2b3640 1 -- 172.21.15.38:0/2958504701 >> 172.21.15.38:0/2958504701 conn(0x7f3fa80fd8c0 msgr2=0x7f3fa80fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:41.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.269+0000 7f3fad2b3640 1 -- 172.21.15.38:0/2958504701 shutdown_connections 2024-09-16T00:31:41.274 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:41.269+0000 7f3fad2b3640 1 -- 172.21.15.38:0/2958504701 wait complete. 2024-09-16T00:31:41.280 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:41 smithi038 ceph-mon[30945]: pgmap v329: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-16T00:31:41.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:41 smithi122 ceph-mon[35412]: pgmap v329: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-16T00:31:41.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:41 smithi073 ceph-mon[35254]: pgmap v329: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-16T00:31:41.733 INFO:teuthology.orchestra.run.smithi038.stdout:34359738453 2024-09-16T00:31:41.733 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-16T00:31:42.390 INFO:teuthology.orchestra.run.smithi038.stdout:296352743457 2024-09-16T00:31:42.391 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-09-16T00:31:42.857 INFO:teuthology.orchestra.run.smithi038.stdout:158913790010 2024-09-16T00:31:42.857 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-16T00:31:43.378 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:43 smithi038 ceph-mon[30945]: pgmap v330: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T00:31:43.379 INFO:teuthology.orchestra.run.smithi038.stdout:77309411404 2024-09-16T00:31:43.379 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-16T00:31:43.417 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:43.454 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:43.457 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:43.457 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:43.462 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:43.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:43 smithi122 ceph-mon[35412]: pgmap v330: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T00:31:43.467 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:43.471 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:43.486 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:43.487 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:43.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:43 smithi073 ceph-mon[35254]: pgmap v330: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T00:31:44.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.404+0000 7f295657d640 1 -- 172.21.15.38:0/321680158 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2950102790 msgr2=0x7f29501067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:44.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.404+0000 7f295657d640 1 --2- 172.21.15.38:0/321680158 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2950102790 0x7f29501067e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2944009900 tx=0x7f294402a410 comp rx=0 tx=0).stop 2024-09-16T00:31:44.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.404+0000 7f295657d640 1 -- 172.21.15.38:0/321680158 shutdown_connections 2024-09-16T00:31:44.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.404+0000 7f295657d640 1 --2- 172.21.15.38:0/321680158 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2950106f10 0x7f295010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.431 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.404+0000 7f295657d640 1 --2- 172.21.15.38:0/321680158 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2950102790 0x7f29501067e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.404+0000 7f295657d640 1 --2- 172.21.15.38:0/321680158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2950101de0 0x7f29501021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.404+0000 7f295657d640 1 -- 172.21.15.38:0/321680158 >> 172.21.15.38:0/321680158 conn(0x7f29500fd8c0 msgr2=0x7f29500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:44.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.404+0000 7f295657d640 1 -- 172.21.15.38:0/321680158 shutdown_connections 2024-09-16T00:31:44.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.404+0000 7f295657d640 1 -- 172.21.15.38:0/321680158 wait complete. 2024-09-16T00:31:44.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.405+0000 7f295657d640 1 Processor -- start 2024-09-16T00:31:44.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.405+0000 7f295657d640 1 -- start start 2024-09-16T00:31:44.432 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.405+0000 7f295657d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2950101de0 0x7f2950198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:44.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.405+0000 7f295657d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2950102790 0x7f2950198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:44.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.405+0000 7f295657d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2950106f10 0x7f295019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:44.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.405+0000 7f295657d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2950111f70 con 0x7f2950106f10 2024-09-16T00:31:44.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.405+0000 7f295657d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2950111df0 con 0x7f2950102790 2024-09-16T00:31:44.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.405+0000 7f295657d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f29501120f0 con 0x7f2950101de0 2024-09-16T00:31:44.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.405+0000 7f2954b1a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2950106f10 0x7f295019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:44.433 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.406+0000 7f2954b1a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2950106f10 0x7f295019cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40742/0 (socket says 172.21.15.38:40742) 2024-09-16T00:31:44.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.406+0000 7f2954b1a640 1 -- 172.21.15.38:0/4119149707 learned_addr learned my addr 172.21.15.38:0/4119149707 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:31:44.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.406+0000 7f294ffff640 1 --2- 172.21.15.38:0/4119149707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2950101de0 0x7f2950198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:44.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.406+0000 7f294f7fe640 1 --2- 172.21.15.38:0/4119149707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2950102790 0x7f2950198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:44.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.406+0000 7f2954b1a640 1 -- 172.21.15.38:0/4119149707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2950101de0 msgr2=0x7f2950198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:44.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.406+0000 7f2954b1a640 1 --2- 172.21.15.38:0/4119149707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2950101de0 0x7f2950198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.406+0000 7f2954b1a640 1 -- 172.21.15.38:0/4119149707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2950102790 msgr2=0x7f2950198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:44.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.406+0000 7f2954b1a640 1 --2- 172.21.15.38:0/4119149707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2950102790 0x7f2950198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.434 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.406+0000 7f2954b1a640 1 -- 172.21.15.38:0/4119149707 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f295019d3c0 con 0x7f2950106f10 2024-09-16T00:31:44.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.406+0000 7f294f7fe640 1 --2- 172.21.15.38:0/4119149707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2950102790 0x7f2950198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:31:44.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.406+0000 7f2954b1a640 1 --2- 172.21.15.38:0/4119149707 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2950106f10 0x7f295019cce0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f294800cce0 tx=0x7f294800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:44.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.406+0000 7f294d7fa640 1 -- 172.21.15.38:0/4119149707 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f294800f450 con 0x7f2950106f10 2024-09-16T00:31:44.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.406+0000 7f294d7fa640 1 -- 172.21.15.38:0/4119149707 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f294800fad0 con 0x7f2950106f10 2024-09-16T00:31:44.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.406+0000 7f295657d640 1 -- 172.21.15.38:0/4119149707 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f295019d6b0 con 0x7f2950106f10 2024-09-16T00:31:44.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.406+0000 7f294d7fa640 1 -- 172.21.15.38:0/4119149707 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2948018c10 con 0x7f2950106f10 2024-09-16T00:31:44.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.406+0000 7f295657d640 1 -- 172.21.15.38:0/4119149707 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29501a4f90 con 0x7f2950106f10 2024-09-16T00:31:44.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.407+0000 7f295657d640 1 -- 172.21.15.38:0/4119149707 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29501021c0 con 0x7f2950106f10 2024-09-16T00:31:44.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.409+0000 7f294d7fa640 1 -- 172.21.15.38:0/4119149707 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f294800f5f0 con 0x7f2950106f10 2024-09-16T00:31:44.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.409+0000 7f294d7fa640 1 --2- 172.21.15.38:0/4119149707 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f29380a32d0 0x7f29380a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:44.435 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.409+0000 7f294ffff640 1 --2- 172.21.15.38:0/4119149707 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f29380a32d0 0x7f29380a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:44.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.409+0000 7f294d7fa640 1 -- 172.21.15.38:0/4119149707 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f29480d04e0 con 0x7f2950106f10 2024-09-16T00:31:44.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.410+0000 7f294ffff640 1 --2- 172.21.15.38:0/4119149707 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f29380a32d0 0x7f29380a5790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f29400058c0 tx=0x7f294000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:44.436 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.413+0000 7f294d7fa640 1 -- 172.21.15.38:0/4119149707 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2948014030 con 0x7f2950106f10 2024-09-16T00:31:44.603 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.603+0000 7f295657d640 1 -- 172.21.15.38:0/4119149707 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f2950199770 con 0x7f2950106f10 2024-09-16T00:31:44.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.603+0000 7f294d7fa640 1 -- 172.21.15.38:0/4119149707 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f294809cdf0 con 0x7f2950106f10 2024-09-16T00:31:44.604 INFO:teuthology.orchestra.run.smithi038.stdout:210453397553 2024-09-16T00:31:44.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.605+0000 7f295657d640 1 -- 172.21.15.38:0/4119149707 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f29380a32d0 msgr2=0x7f29380a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:44.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.605+0000 7f295657d640 1 --2- 172.21.15.38:0/4119149707 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f29380a32d0 0x7f29380a5790 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f29400058c0 tx=0x7f294000a2d0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.605+0000 7f295657d640 1 -- 172.21.15.38:0/4119149707 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2950106f10 msgr2=0x7f295019cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:44.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.605+0000 7f295657d640 1 --2- 172.21.15.38:0/4119149707 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2950106f10 0x7f295019cce0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f294800cce0 tx=0x7f294800a620 comp rx=0 tx=0).stop 2024-09-16T00:31:44.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.605+0000 7f295657d640 1 -- 172.21.15.38:0/4119149707 shutdown_connections 2024-09-16T00:31:44.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.605+0000 7f295657d640 1 --2- 172.21.15.38:0/4119149707 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f29380a32d0 0x7f29380a5790 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.605+0000 7f295657d640 1 --2- 172.21.15.38:0/4119149707 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2950106f10 0x7f295019cce0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.605+0000 7f295657d640 1 --2- 172.21.15.38:0/4119149707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2950102790 0x7f2950198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.605+0000 7f295657d640 1 --2- 172.21.15.38:0/4119149707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2950101de0 0x7f2950198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:44.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.605+0000 7f295657d640 1 -- 172.21.15.38:0/4119149707 >> 172.21.15.38:0/4119149707 conn(0x7f29500fd8c0 msgr2=0x7f29500febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:44.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.605+0000 7f295657d640 1 -- 172.21.15.38:0/4119149707 shutdown_connections 2024-09-16T00:31:44.607 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.605+0000 7f295657d640 1 -- 172.21.15.38:0/4119149707 wait complete. 2024-09-16T00:31:45.037 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.021+0000 7f0ff70fe640 1 -- 172.21.15.38:0/2287238857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ff0102ef0 msgr2=0x7f0ff010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:45.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.021+0000 7f0ff70fe640 1 --2- 172.21.15.38:0/2287238857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ff0102ef0 0x7f0ff010da40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8009ca0 tx=0x7f0fe802a950 comp rx=0 tx=0).stop 2024-09-16T00:31:45.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.021+0000 7f0ff70fe640 1 -- 172.21.15.38:0/2287238857 shutdown_connections 2024-09-16T00:31:45.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.021+0000 7f0ff70fe640 1 --2- 172.21.15.38:0/2287238857 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ff0102ef0 0x7f0ff010da40 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:45.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.021+0000 7f0ff70fe640 1 --2- 172.21.15.38:0/2287238857 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff0102570 0x7f0ff01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:45.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.021+0000 7f0ff70fe640 1 --2- 172.21.15.38:0/2287238857 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ff0100e20 0x7f0ff0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:45.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.021+0000 7f0ff70fe640 1 -- 172.21.15.38:0/2287238857 >> 172.21.15.38:0/2287238857 conn(0x7f0ff00fc7d0 msgr2=0x7f0ff00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:45.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.021+0000 7f0ff70fe640 1 -- 172.21.15.38:0/2287238857 shutdown_connections 2024-09-16T00:31:45.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.021+0000 7f0ff70fe640 1 -- 172.21.15.38:0/2287238857 wait complete. 2024-09-16T00:31:45.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.022+0000 7f0ff70fe640 1 Processor -- start 2024-09-16T00:31:45.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.022+0000 7f0ff70fe640 1 -- start start 2024-09-16T00:31:45.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.022+0000 7f0ff70fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ff0100e20 0x7f0ff00fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:45.038 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.022+0000 7f0ff70fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ff0102570 0x7f0ff00fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:45.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.022+0000 7f0ff70fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff0102ef0 0x7f0ff00fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:45.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.022+0000 7f0ff70fe640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff0110f80 con 0x7f0ff0102570 2024-09-16T00:31:45.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.022+0000 7f0ff70fe640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff0110e00 con 0x7f0ff0102ef0 2024-09-16T00:31:45.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.022+0000 7f0ff70fe640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0ff0111100 con 0x7f0ff0100e20 2024-09-16T00:31:45.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.022+0000 7f0feffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ff0102570 0x7f0ff00fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:45.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.022+0000 7f0feffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ff0102570 0x7f0ff00fd1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40764/0 (socket says 172.21.15.38:40764) 2024-09-16T00:31:45.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.022+0000 7f0feffff640 1 -- 172.21.15.38:0/3083017630 learned_addr learned my addr 172.21.15.38:0/3083017630 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:31:45.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.022+0000 7f0ff4e9a640 1 --2- 172.21.15.38:0/3083017630 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ff0100e20 0x7f0ff00fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:45.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.022+0000 7f0ff569b640 1 --2- 172.21.15.38:0/3083017630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff0102ef0 0x7f0ff00fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:45.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.023+0000 7f0feffff640 1 -- 172.21.15.38:0/3083017630 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ff0100e20 msgr2=0x7f0ff00fcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:45.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.023+0000 7f0feffff640 1 --2- 172.21.15.38:0/3083017630 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ff0100e20 0x7f0ff00fcc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:45.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.023+0000 7f0feffff640 1 -- 172.21.15.38:0/3083017630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff0102ef0 msgr2=0x7f0ff00fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:45.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.023+0000 7f0feffff640 1 --2- 172.21.15.38:0/3083017630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff0102ef0 0x7f0ff00fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:45.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.023+0000 7f0feffff640 1 -- 172.21.15.38:0/3083017630 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ff01a68b0 con 0x7f0ff0102570 2024-09-16T00:31:45.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.023+0000 7f0feffff640 1 --2- 172.21.15.38:0/3083017630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ff0102570 0x7f0ff00fd1c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc00c920 tx=0x7f0fdc00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:45.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.023+0000 7f0fedffb640 1 -- 172.21.15.38:0/3083017630 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc007c70 con 0x7f0ff0102570 2024-09-16T00:31:45.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.023+0000 7f0ff70fe640 1 -- 172.21.15.38:0/3083017630 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ff01a6ba0 con 0x7f0ff0102570 2024-09-16T00:31:45.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.023+0000 7f0fedffb640 1 -- 172.21.15.38:0/3083017630 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0fdc007e10 con 0x7f0ff0102570 2024-09-16T00:31:45.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.023+0000 7f0fedffb640 1 -- 172.21.15.38:0/3083017630 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fdc018890 con 0x7f0ff0102570 2024-09-16T00:31:45.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.023+0000 7f0ff70fe640 1 -- 172.21.15.38:0/3083017630 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ff01a70e0 con 0x7f0ff0102570 2024-09-16T00:31:45.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.024+0000 7f0ff70fe640 1 -- 172.21.15.38:0/3083017630 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ff0101200 con 0x7f0ff0102570 2024-09-16T00:31:45.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.025+0000 7f0fedffb640 1 -- 172.21.15.38:0/3083017630 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f0fdc018a70 con 0x7f0ff0102570 2024-09-16T00:31:45.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.026+0000 7f0fedffb640 1 --2- 172.21.15.38:0/3083017630 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f0fe00a32d0 0x7f0fe00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:45.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.026+0000 7f0fedffb640 1 -- 172.21.15.38:0/3083017630 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f0fdc0cff60 con 0x7f0ff0102570 2024-09-16T00:31:45.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.026+0000 7f0ff4e9a640 1 --2- 172.21.15.38:0/3083017630 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f0fe00a32d0 0x7f0fe00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:45.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.027+0000 7f0ff4e9a640 1 --2- 172.21.15.38:0/3083017630 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f0fe00a32d0 0x7f0fe00a5790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4009b60 tx=0x7f0fe4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:45.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.030+0000 7f0fedffb640 1 -- 172.21.15.38:0/3083017630 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0fdc014030 con 0x7f0ff0102570 2024-09-16T00:31:45.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.901+0000 7f9ed195e640 1 -- 172.21.15.38:0/2748993017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ecc100e20 msgr2=0x7f9ecc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:45.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.901+0000 7f9ed195e640 1 --2- 172.21.15.38:0/2748993017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ecc100e20 0x7f9ecc101200 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4009900 tx=0x7f9eb402a480 comp rx=0 tx=0).stop 2024-09-16T00:31:45.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.901+0000 7f9ed195e640 1 -- 172.21.15.38:0/2748993017 shutdown_connections 2024-09-16T00:31:45.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.901+0000 7f9ed195e640 1 --2- 172.21.15.38:0/2748993017 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ecc102ef0 0x7f9ecc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:45.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.901+0000 7f9ed195e640 1 --2- 172.21.15.38:0/2748993017 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ecc102570 0x7f9ecc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:45.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.901+0000 7f9ed195e640 1 --2- 172.21.15.38:0/2748993017 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ecc100e20 0x7f9ecc101200 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:45.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.901+0000 7f9ed195e640 1 -- 172.21.15.38:0/2748993017 >> 172.21.15.38:0/2748993017 conn(0x7f9ecc0fc7d0 msgr2=0x7f9ecc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:45.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.902+0000 7f9ed195e640 1 -- 172.21.15.38:0/2748993017 shutdown_connections 2024-09-16T00:31:45.155 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.902+0000 7f9ed195e640 1 -- 172.21.15.38:0/2748993017 wait complete. 2024-09-16T00:31:45.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.902+0000 7f9ed195e640 1 Processor -- start 2024-09-16T00:31:45.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.902+0000 7f9ed195e640 1 -- start start 2024-09-16T00:31:45.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.903+0000 7f9ed195e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ecc100e20 0x7f9ecc103750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:45.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.903+0000 7f9ed195e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ecc102570 0x7f9ecc103c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:45.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.903+0000 7f9ed195e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ecc102ef0 0x7f9ecc1041d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:45.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.903+0000 7f9ed195e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc110fd0 con 0x7f9ecc100e20 2024-09-16T00:31:45.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.903+0000 7f9ed195e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc110e50 con 0x7f9ecc102ef0 2024-09-16T00:31:45.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.903+0000 7f9ed195e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9ecc111150 con 0x7f9ecc102570 2024-09-16T00:31:45.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.903+0000 7f9ecaffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ecc100e20 0x7f9ecc103750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:45.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.903+0000 7f9ecaffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ecc100e20 0x7f9ecc103750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40754/0 (socket says 172.21.15.38:40754) 2024-09-16T00:31:45.156 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.903+0000 7f9ecaffd640 1 -- 172.21.15.38:0/70747136 learned_addr learned my addr 172.21.15.38:0/70747136 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:31:45.157 INFO:teuthology.orchestra.run.smithi038.stdout:107374182469 2024-09-16T00:31:45.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.903+0000 7f9ecb7fe640 1 --2- 172.21.15.38:0/70747136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ecc102ef0 0x7f9ecc1041d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:45.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.903+0000 7f9eca7fc640 1 --2- 172.21.15.38:0/70747136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ecc102570 0x7f9ecc103c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:45.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.903+0000 7f9ecaffd640 1 -- 172.21.15.38:0/70747136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ecc102570 msgr2=0x7f9ecc103c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:45.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.903+0000 7f9ecaffd640 1 --2- 172.21.15.38:0/70747136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ecc102570 0x7f9ecc103c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:45.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.903+0000 7f9ecaffd640 1 -- 172.21.15.38:0/70747136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ecc102ef0 msgr2=0x7f9ecc1041d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:45.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.903+0000 7f9ecaffd640 1 --2- 172.21.15.38:0/70747136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ecc102ef0 0x7f9ecc1041d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:45.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.903+0000 7f9ecaffd640 1 -- 172.21.15.38:0/70747136 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ecc1a6920 con 0x7f9ecc100e20 2024-09-16T00:31:45.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.903+0000 7f9ecb7fe640 1 --2- 172.21.15.38:0/70747136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ecc102ef0 0x7f9ecc1041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:31:45.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.903+0000 7f9ecaffd640 1 --2- 172.21.15.38:0/70747136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ecc100e20 0x7f9ecc103750 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9eb402aa70 tx=0x7f9eb4003b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:45.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.903+0000 7f9ed095c640 1 -- 172.21.15.38:0/70747136 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb403f070 con 0x7f9ecc100e20 2024-09-16T00:31:45.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.904+0000 7f9ed095c640 1 -- 172.21.15.38:0/70747136 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9eb4031c20 con 0x7f9ecc100e20 2024-09-16T00:31:45.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.904+0000 7f9ed195e640 1 -- 172.21.15.38:0/70747136 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ecc1a6bb0 con 0x7f9ecc100e20 2024-09-16T00:31:45.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.904+0000 7f9ed095c640 1 -- 172.21.15.38:0/70747136 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4035030 con 0x7f9ecc100e20 2024-09-16T00:31:45.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.904+0000 7f9ed195e640 1 -- 172.21.15.38:0/70747136 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ecc1a7090 con 0x7f9ecc100e20 2024-09-16T00:31:45.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.905+0000 7f9ed195e640 1 -- 172.21.15.38:0/70747136 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ecc101200 con 0x7f9ecc100e20 2024-09-16T00:31:45.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.906+0000 7f9ed095c640 1 -- 172.21.15.38:0/70747136 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f9eb4042070 con 0x7f9ecc100e20 2024-09-16T00:31:45.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.907+0000 7f9ed095c640 1 --2- 172.21.15.38:0/70747136 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f9eb80a32d0 0x7f9eb80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:45.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.907+0000 7f9eca7fc640 1 --2- 172.21.15.38:0/70747136 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f9eb80a32d0 0x7f9eb80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:45.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.907+0000 7f9ed095c640 1 -- 172.21.15.38:0/70747136 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f9eb40f06c0 con 0x7f9ecc100e20 2024-09-16T00:31:45.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.908+0000 7f9eca7fc640 1 --2- 172.21.15.38:0/70747136 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f9eb80a32d0 0x7f9eb80a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc005fd0 tx=0x7f9ebc005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:45.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:44.911+0000 7f9ed095c640 1 -- 172.21.15.38:0/70747136 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9eb4031dd0 con 0x7f9ecc100e20 2024-09-16T00:31:45.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.093+0000 7f9ed195e640 1 -- 172.21.15.38:0/70747136 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f9ecc1a7290 con 0x7f9ecc100e20 2024-09-16T00:31:45.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.093+0000 7f9ed095c640 1 -- 172.21.15.38:0/70747136 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f9eb40bcee0 con 0x7f9ecc100e20 2024-09-16T00:31:45.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.095+0000 7f9ed195e640 1 -- 172.21.15.38:0/70747136 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f9eb80a32d0 msgr2=0x7f9eb80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:45.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.095+0000 7f9ed195e640 1 --2- 172.21.15.38:0/70747136 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f9eb80a32d0 0x7f9eb80a5790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f9ebc005fd0 tx=0x7f9ebc005850 comp rx=0 tx=0).stop 2024-09-16T00:31:45.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.095+0000 7f9ed195e640 1 -- 172.21.15.38:0/70747136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ecc100e20 msgr2=0x7f9ecc103750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:45.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.095+0000 7f9ed195e640 1 --2- 172.21.15.38:0/70747136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ecc100e20 0x7f9ecc103750 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f9eb402aa70 tx=0x7f9eb4003b60 comp rx=0 tx=0).stop 2024-09-16T00:31:45.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.095+0000 7f9ed195e640 1 -- 172.21.15.38:0/70747136 shutdown_connections 2024-09-16T00:31:45.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.095+0000 7f9ed195e640 1 --2- 172.21.15.38:0/70747136 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f9eb80a32d0 0x7f9eb80a5790 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:45.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.095+0000 7f9ed195e640 1 --2- 172.21.15.38:0/70747136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ecc102ef0 0x7f9ecc1041d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:45.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.095+0000 7f9ed195e640 1 --2- 172.21.15.38:0/70747136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9ecc102570 0x7f9ecc103c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:45.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.095+0000 7f9ed195e640 1 --2- 172.21.15.38:0/70747136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9ecc100e20 0x7f9ecc103750 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:45.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.095+0000 7f9ed195e640 1 -- 172.21.15.38:0/70747136 >> 172.21.15.38:0/70747136 conn(0x7f9ecc0fc7d0 msgr2=0x7f9ecc06a0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:45.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.095+0000 7f9ed195e640 1 -- 172.21.15.38:0/70747136 shutdown_connections 2024-09-16T00:31:45.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.095+0000 7f9ed195e640 1 -- 172.21.15.38:0/70747136 wait complete. 2024-09-16T00:31:45.160 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:45 smithi038 ceph-mon[30945]: pgmap v331: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-16T00:31:45.160 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:45 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/4119149707' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-16T00:31:45.160 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:45 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/70747136' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-16T00:31:45.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.213+0000 7f0ff70fe640 1 -- 172.21.15.38:0/3083017630 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f0ff010a0a0 con 0x7f0ff0102570 2024-09-16T00:31:45.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.214+0000 7f0fedffb640 1 -- 172.21.15.38:0/3083017630 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0fdc09c7e0 con 0x7f0ff0102570 2024-09-16T00:31:45.214 INFO:teuthology.orchestra.run.smithi038.stdout:34359738454 2024-09-16T00:31:45.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.216+0000 7f0ff70fe640 1 -- 172.21.15.38:0/3083017630 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f0fe00a32d0 msgr2=0x7f0fe00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:45.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.216+0000 7f0ff70fe640 1 --2- 172.21.15.38:0/3083017630 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f0fe00a32d0 0x7f0fe00a5790 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f0fe4009b60 tx=0x7f0fe4005f90 comp rx=0 tx=0).stop 2024-09-16T00:31:45.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.216+0000 7f0ff70fe640 1 -- 172.21.15.38:0/3083017630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ff0102570 msgr2=0x7f0ff00fd1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:45.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.216+0000 7f0ff70fe640 1 --2- 172.21.15.38:0/3083017630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ff0102570 0x7f0ff00fd1c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc00c920 tx=0x7f0fdc00cde0 comp rx=0 tx=0).stop 2024-09-16T00:31:45.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.216+0000 7f0ff70fe640 1 -- 172.21.15.38:0/3083017630 shutdown_connections 2024-09-16T00:31:45.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.216+0000 7f0ff70fe640 1 --2- 172.21.15.38:0/3083017630 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f0fe00a32d0 0x7f0fe00a5790 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:45.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.216+0000 7f0ff70fe640 1 --2- 172.21.15.38:0/3083017630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff0102ef0 0x7f0ff00fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:45.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.216+0000 7f0ff70fe640 1 --2- 172.21.15.38:0/3083017630 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0ff0102570 0x7f0ff00fd1c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:45.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.216+0000 7f0ff70fe640 1 --2- 172.21.15.38:0/3083017630 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0ff0100e20 0x7f0ff00fcc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:45.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.216+0000 7f0ff70fe640 1 -- 172.21.15.38:0/3083017630 >> 172.21.15.38:0/3083017630 conn(0x7f0ff00fc7d0 msgr2=0x7f0ff006a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:45.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.216+0000 7f0ff70fe640 1 -- 172.21.15.38:0/3083017630 shutdown_connections 2024-09-16T00:31:45.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:45.216+0000 7f0ff70fe640 1 -- 172.21.15.38:0/3083017630 wait complete. 2024-09-16T00:31:45.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:45 smithi122 ceph-mon[35412]: pgmap v331: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-16T00:31:45.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:45 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/4119149707' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-16T00:31:45.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:45 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/70747136' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-16T00:31:45.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:45 smithi073 ceph-mon[35254]: pgmap v331: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-16T00:31:45.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:45 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/4119149707' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-16T00:31:45.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:45 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/70747136' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-16T00:31:46.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.171+0000 7faaa4fe5640 1 -- 172.21.15.38:0/2343697318 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faaa0102d90 msgr2=0x7faaa0103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:46.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.171+0000 7faaa4fe5640 1 --2- 172.21.15.38:0/2343697318 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faaa0102d90 0x7faaa0103170 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7faa90009930 tx=0x7faa9002a420 comp rx=0 tx=0).stop 2024-09-16T00:31:46.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.171+0000 7faaa4fe5640 1 -- 172.21.15.38:0/2343697318 shutdown_connections 2024-09-16T00:31:46.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.171+0000 7faaa4fe5640 1 --2- 172.21.15.38:0/2343697318 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faaa0108290 0x7faaa010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:46.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.171+0000 7faaa4fe5640 1 --2- 172.21.15.38:0/2343697318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faaa0103740 0x7faaa0107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:46.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.171+0000 7faaa4fe5640 1 --2- 172.21.15.38:0/2343697318 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faaa0102d90 0x7faaa0103170 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:46.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.171+0000 7faaa4fe5640 1 -- 172.21.15.38:0/2343697318 >> 172.21.15.38:0/2343697318 conn(0x7faaa00fc7d0 msgr2=0x7faaa00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:46.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.171+0000 7faaa4fe5640 1 -- 172.21.15.38:0/2343697318 shutdown_connections 2024-09-16T00:31:46.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.171+0000 7faaa4fe5640 1 -- 172.21.15.38:0/2343697318 wait complete. 2024-09-16T00:31:46.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.171+0000 7faaa4fe5640 1 Processor -- start 2024-09-16T00:31:46.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.172+0000 7faaa4fe5640 1 -- start start 2024-09-16T00:31:46.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.172+0000 7faaa4fe5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faaa0102d90 0x7faaa01009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:46.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.172+0000 7faaa4fe5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faaa0103740 0x7faaa0100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:46.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.172+0000 7faaa4fe5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faaa0108290 0x7faaa0101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:46.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.172+0000 7faaa4fe5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7faaa0112090 con 0x7faaa0108290 2024-09-16T00:31:46.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.172+0000 7faaa4fe5640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7faaa0111f10 con 0x7faaa0103740 2024-09-16T00:31:46.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.172+0000 7faaa4fe5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faaa0112210 con 0x7faaa0102d90 2024-09-16T00:31:46.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.172+0000 7faa9ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faaa0108290 0x7faaa0101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:46.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.172+0000 7faa9ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faaa0108290 0x7faaa0101470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36972/0 (socket says 172.21.15.38:36972) 2024-09-16T00:31:46.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.172+0000 7faa9ed9d640 1 -- 172.21.15.38:0/1778565889 learned_addr learned my addr 172.21.15.38:0/1778565889 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:31:46.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.172+0000 7faa9dd9b640 1 --2- 172.21.15.38:0/1778565889 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faaa0103740 0x7faaa0100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:46.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.172+0000 7faa9e59c640 1 --2- 172.21.15.38:0/1778565889 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faaa0102d90 0x7faaa01009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:46.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.172+0000 7faa9ed9d640 1 -- 172.21.15.38:0/1778565889 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faaa0102d90 msgr2=0x7faaa01009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:46.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.172+0000 7faa9ed9d640 1 --2- 172.21.15.38:0/1778565889 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faaa0102d90 0x7faaa01009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:46.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.172+0000 7faa9ed9d640 1 -- 172.21.15.38:0/1778565889 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faaa0103740 msgr2=0x7faaa0100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:46.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.172+0000 7faa9ed9d640 1 --2- 172.21.15.38:0/1778565889 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faaa0103740 0x7faaa0100f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:46.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.172+0000 7faa9ed9d640 1 -- 172.21.15.38:0/1778565889 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaa01a6a90 con 0x7faaa0108290 2024-09-16T00:31:46.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.172+0000 7faa9ed9d640 1 --2- 172.21.15.38:0/1778565889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faaa0108290 0x7faaa0101470 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7faa9800eaa0 tx=0x7faa9800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:46.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.173+0000 7faa8f7fe640 1 -- 172.21.15.38:0/1778565889 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9800cdb0 con 0x7faaa0108290 2024-09-16T00:31:46.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.173+0000 7faa8f7fe640 1 -- 172.21.15.38:0/1778565889 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faa98005020 con 0x7faaa0108290 2024-09-16T00:31:46.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.173+0000 7faaa4fe5640 1 -- 172.21.15.38:0/1778565889 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaa01a6d80 con 0x7faaa0108290 2024-09-16T00:31:46.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.173+0000 7faa8f7fe640 1 -- 172.21.15.38:0/1778565889 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa980116b0 con 0x7faaa0108290 2024-09-16T00:31:46.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.173+0000 7faaa4fe5640 1 -- 172.21.15.38:0/1778565889 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaa01a7270 con 0x7faaa0108290 2024-09-16T00:31:46.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.174+0000 7faaa4fe5640 1 -- 172.21.15.38:0/1778565889 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faaa006a7f0 con 0x7faaa0108290 2024-09-16T00:31:46.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.175+0000 7faa8f7fe640 1 -- 172.21.15.38:0/1778565889 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7faa98005560 con 0x7faaa0108290 2024-09-16T00:31:46.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.176+0000 7faa8f7fe640 1 --2- 172.21.15.38:0/1778565889 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7faa880a32d0 0x7faa880a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:46.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.176+0000 7faa9e59c640 1 --2- 172.21.15.38:0/1778565889 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7faa880a32d0 0x7faa880a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:46.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.176+0000 7faa8f7fe640 1 -- 172.21.15.38:0/1778565889 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7faa980cff90 con 0x7faaa0108290 2024-09-16T00:31:46.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.176+0000 7faa9e59c640 1 --2- 172.21.15.38:0/1778565889 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7faa880a32d0 0x7faa880a5790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7faa90009900 tx=0x7faa900058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:46.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.180+0000 7faa8f7fe640 1 -- 172.21.15.38:0/1778565889 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7faa98016030 con 0x7faaa0108290 2024-09-16T00:31:46.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.367+0000 7faaa4fe5640 1 -- 172.21.15.38:0/1778565889 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7faaa010a2a0 con 0x7faaa0108290 2024-09-16T00:31:46.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.368+0000 7faa8f7fe640 1 -- 172.21.15.38:0/1778565889 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7faa9809c160 con 0x7faaa0108290 2024-09-16T00:31:46.369 INFO:teuthology.orchestra.run.smithi038.stdout:244813135917 2024-09-16T00:31:46.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.370+0000 7faaa4fe5640 1 -- 172.21.15.38:0/1778565889 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7faa880a32d0 msgr2=0x7faa880a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:46.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.370+0000 7faaa4fe5640 1 --2- 172.21.15.38:0/1778565889 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7faa880a32d0 0x7faa880a5790 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7faa90009900 tx=0x7faa900058d0 comp rx=0 tx=0).stop 2024-09-16T00:31:46.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.370+0000 7faaa4fe5640 1 -- 172.21.15.38:0/1778565889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faaa0108290 msgr2=0x7faaa0101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:46.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.370+0000 7faaa4fe5640 1 --2- 172.21.15.38:0/1778565889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faaa0108290 0x7faaa0101470 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7faa9800eaa0 tx=0x7faa9800ef60 comp rx=0 tx=0).stop 2024-09-16T00:31:46.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.370+0000 7faaa4fe5640 1 -- 172.21.15.38:0/1778565889 shutdown_connections 2024-09-16T00:31:46.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.370+0000 7faaa4fe5640 1 --2- 172.21.15.38:0/1778565889 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7faa880a32d0 0x7faa880a5790 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:46.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.370+0000 7faaa4fe5640 1 --2- 172.21.15.38:0/1778565889 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faaa0108290 0x7faaa0101470 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:46.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.370+0000 7faaa4fe5640 1 --2- 172.21.15.38:0/1778565889 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faaa0103740 0x7faaa0100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:46.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.370+0000 7faaa4fe5640 1 --2- 172.21.15.38:0/1778565889 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faaa0102d90 0x7faaa01009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:46.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.370+0000 7faaa4fe5640 1 -- 172.21.15.38:0/1778565889 >> 172.21.15.38:0/1778565889 conn(0x7faaa00fc7d0 msgr2=0x7faaa0106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:46.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.370+0000 7faaa4fe5640 1 -- 172.21.15.38:0/1778565889 shutdown_connections 2024-09-16T00:31:46.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.370+0000 7faaa4fe5640 1 -- 172.21.15.38:0/1778565889 wait complete. 2024-09-16T00:31:46.380 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:46 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/3083017630' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-16T00:31:46.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:46 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/3083017630' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-16T00:31:46.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:46 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/3083017630' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-16T00:31:47.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:47 smithi038 ceph-mon[30945]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:31:47.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:47 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/4268830076' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-16T00:31:47.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:47 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/1778565889' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-16T00:31:47.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:47 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/737951492' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-16T00:31:47.358 INFO:tasks.cephadm.ceph_manager.ceph:need seq 244813135914 got 244813135917 for osd.8 2024-09-16T00:31:47.358 DEBUG:teuthology.parallel:result is None 2024-09-16T00:31:47.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.596+0000 7fa2caa37640 1 -- 172.21.15.38:0/890145826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2c4103ff0 msgr2=0x7fa2c41043d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:47.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.596+0000 7fa2caa37640 1 --2- 172.21.15.38:0/890145826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2c4103ff0 0x7fa2c41043d0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4009900 tx=0x7fa2b402a410 comp rx=0 tx=0).stop 2024-09-16T00:31:47.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.596+0000 7fa2caa37640 1 -- 172.21.15.38:0/890145826 shutdown_connections 2024-09-16T00:31:47.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.596+0000 7fa2caa37640 1 --2- 172.21.15.38:0/890145826 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2c4109120 0x7fa2c4110ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:47.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.596+0000 7fa2caa37640 1 --2- 172.21.15.38:0/890145826 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2c41049a0 0x7fa2c41089f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:47.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.596+0000 7fa2caa37640 1 --2- 172.21.15.38:0/890145826 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2c4103ff0 0x7fa2c41043d0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:47.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.596+0000 7fa2caa37640 1 -- 172.21.15.38:0/890145826 >> 172.21.15.38:0/890145826 conn(0x7fa2c40ffab0 msgr2=0x7fa2c4101ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:47.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.596+0000 7fa2caa37640 1 -- 172.21.15.38:0/890145826 shutdown_connections 2024-09-16T00:31:47.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.596+0000 7fa2caa37640 1 -- 172.21.15.38:0/890145826 wait complete. 2024-09-16T00:31:47.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.597+0000 7fa2caa37640 1 Processor -- start 2024-09-16T00:31:47.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.597+0000 7fa2caa37640 1 -- start start 2024-09-16T00:31:47.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.597+0000 7fa2caa37640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2c4103ff0 0x7fa2c41a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:47.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.597+0000 7fa2caa37640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2c41049a0 0x7fa2c41a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:47.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.597+0000 7fa2caa37640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2c4109120 0x7fa2c419a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:47.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.597+0000 7fa2caa37640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c4114220 con 0x7fa2c4103ff0 2024-09-16T00:31:47.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.597+0000 7fa2caa37640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c41140a0 con 0x7fa2c41049a0 2024-09-16T00:31:47.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.597+0000 7fa2caa37640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa2c41143a0 con 0x7fa2c4109120 2024-09-16T00:31:47.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.597+0000 7fa2c3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2c4103ff0 0x7fa2c41a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:47.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.597+0000 7fa2c3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2c4103ff0 0x7fa2c41a0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:36988/0 (socket says 172.21.15.38:36988) 2024-09-16T00:31:47.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.597+0000 7fa2c3fff640 1 -- 172.21.15.38:0/737951492 learned_addr learned my addr 172.21.15.38:0/737951492 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:31:47.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.598+0000 7fa2c8fd4640 1 --2- 172.21.15.38:0/737951492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2c4109120 0x7fa2c419a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:47.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.598+0000 7fa2c37fe640 1 --2- 172.21.15.38:0/737951492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2c41049a0 0x7fa2c41a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:47.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.598+0000 7fa2c3fff640 1 -- 172.21.15.38:0/737951492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2c4109120 msgr2=0x7fa2c419a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:47.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.598+0000 7fa2c3fff640 1 --2- 172.21.15.38:0/737951492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2c4109120 0x7fa2c419a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:47.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.598+0000 7fa2c3fff640 1 -- 172.21.15.38:0/737951492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2c41049a0 msgr2=0x7fa2c41a0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:47.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.598+0000 7fa2c3fff640 1 --2- 172.21.15.38:0/737951492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2c41049a0 0x7fa2c41a0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:47.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.598+0000 7fa2c3fff640 1 -- 172.21.15.38:0/737951492 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2c419ad00 con 0x7fa2c4103ff0 2024-09-16T00:31:47.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.598+0000 7fa2c37fe640 1 --2- 172.21.15.38:0/737951492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2c41049a0 0x7fa2c41a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:31:47.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.598+0000 7fa2c3fff640 1 --2- 172.21.15.38:0/737951492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2c4103ff0 0x7fa2c41a0410 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4005ab0 tx=0x7fa2b40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:47.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.598+0000 7fa2c17fa640 1 -- 172.21.15.38:0/737951492 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b403f030 con 0x7fa2c4103ff0 2024-09-16T00:31:47.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.598+0000 7fa2caa37640 1 -- 172.21.15.38:0/737951492 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2c419af90 con 0x7fa2c4103ff0 2024-09-16T00:31:47.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.598+0000 7fa2c17fa640 1 -- 172.21.15.38:0/737951492 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa2b4003ce0 con 0x7fa2c4103ff0 2024-09-16T00:31:47.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.598+0000 7fa2c17fa640 1 -- 172.21.15.38:0/737951492 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2b4035070 con 0x7fa2c4103ff0 2024-09-16T00:31:47.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.598+0000 7fa2caa37640 1 -- 172.21.15.38:0/737951492 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2c41a72e0 con 0x7fa2c4103ff0 2024-09-16T00:31:47.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.599+0000 7fa2caa37640 1 -- 172.21.15.38:0/737951492 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2c4104450 con 0x7fa2c4103ff0 2024-09-16T00:31:47.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.600+0000 7fa2c17fa640 1 -- 172.21.15.38:0/737951492 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fa2b4042030 con 0x7fa2c4103ff0 2024-09-16T00:31:47.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.601+0000 7fa2c17fa640 1 --2- 172.21.15.38:0/737951492 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fa2ac0a32d0 0x7fa2ac0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:47.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.601+0000 7fa2c17fa640 1 -- 172.21.15.38:0/737951492 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7fa2b40f0780 con 0x7fa2c4103ff0 2024-09-16T00:31:47.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.601+0000 7fa2c37fe640 1 --2- 172.21.15.38:0/737951492 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fa2ac0a32d0 0x7fa2ac0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:47.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.602+0000 7fa2c37fe640 1 --2- 172.21.15.38:0/737951492 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fa2ac0a32d0 0x7fa2ac0a5790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8006fd0 tx=0x7fa2b8008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:47.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.605+0000 7fa2c17fa640 1 -- 172.21.15.38:0/737951492 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa2b40bd000 con 0x7fa2c4103ff0 2024-09-16T00:31:47.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.786+0000 7fa2caa37640 1 -- 172.21.15.38:0/737951492 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fa2c410c300 con 0x7fa2c4103ff0 2024-09-16T00:31:47.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.786+0000 7fa2c17fa640 1 -- 172.21.15.38:0/737951492 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa2b40bc900 con 0x7fa2c4103ff0 2024-09-16T00:31:47.453 INFO:teuthology.orchestra.run.smithi038.stdout:55834574930 2024-09-16T00:31:47.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.788+0000 7fa2caa37640 1 -- 172.21.15.38:0/737951492 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fa2ac0a32d0 msgr2=0x7fa2ac0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:47.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.788+0000 7fa2caa37640 1 --2- 172.21.15.38:0/737951492 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fa2ac0a32d0 0x7fa2ac0a5790 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa2b8006fd0 tx=0x7fa2b8008040 comp rx=0 tx=0).stop 2024-09-16T00:31:47.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.788+0000 7fa2caa37640 1 -- 172.21.15.38:0/737951492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2c4103ff0 msgr2=0x7fa2c41a0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:47.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.788+0000 7fa2caa37640 1 --2- 172.21.15.38:0/737951492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2c4103ff0 0x7fa2c41a0410 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fa2b4005ab0 tx=0x7fa2b40042f0 comp rx=0 tx=0).stop 2024-09-16T00:31:47.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.788+0000 7fa2caa37640 1 -- 172.21.15.38:0/737951492 shutdown_connections 2024-09-16T00:31:47.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.788+0000 7fa2caa37640 1 --2- 172.21.15.38:0/737951492 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fa2ac0a32d0 0x7fa2ac0a5790 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:47.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.788+0000 7fa2caa37640 1 --2- 172.21.15.38:0/737951492 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa2c4109120 0x7fa2c419a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:47.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.788+0000 7fa2caa37640 1 --2- 172.21.15.38:0/737951492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa2c41049a0 0x7fa2c41a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:47.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.788+0000 7fa2caa37640 1 --2- 172.21.15.38:0/737951492 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa2c4103ff0 0x7fa2c41a0410 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:47.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.788+0000 7fa2caa37640 1 -- 172.21.15.38:0/737951492 >> 172.21.15.38:0/737951492 conn(0x7fa2c40ffab0 msgr2=0x7fa2c4100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:47.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.788+0000 7fa2caa37640 1 -- 172.21.15.38:0/737951492 shutdown_connections 2024-09-16T00:31:47.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.788+0000 7fa2caa37640 1 -- 172.21.15.38:0/737951492 wait complete. 2024-09-16T00:31:47.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:47 smithi122 ceph-mon[35412]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:31:47.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:47 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/4268830076' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-16T00:31:47.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:47 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/1778565889' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-16T00:31:47.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:47 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/737951492' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-16T00:31:47.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:47 smithi073 ceph-mon[35254]: pgmap v332: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:31:47.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:47 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/4268830076' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-09-16T00:31:47.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:47 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/1778565889' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-09-16T00:31:47.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:47 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/737951492' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-16T00:31:48.520 INFO:tasks.cephadm.ceph_manager.ceph:need seq 210453397551 got 210453397553 for osd.7 2024-09-16T00:31:48.520 DEBUG:teuthology.parallel:result is None 2024-09-16T00:31:48.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.678+0000 7f2605be3640 1 -- 172.21.15.38:0/1030191376 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2600101bc0 msgr2=0x7f260010d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:48.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.678+0000 7f2605be3640 1 --2- 172.21.15.38:0/1030191376 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2600101bc0 0x7f260010d910 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f25f8009ca0 tx=0x7f25f802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:31:48.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.678+0000 7f2605be3640 1 -- 172.21.15.38:0/1030191376 shutdown_connections 2024-09-16T00:31:48.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.678+0000 7f2605be3640 1 --2- 172.21.15.38:0/1030191376 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2600101bc0 0x7f260010d910 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:48.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.678+0000 7f2605be3640 1 --2- 172.21.15.38:0/1030191376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2600101240 0x7f2600101680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:48.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.678+0000 7f2605be3640 1 --2- 172.21.15.38:0/1030191376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2600105a90 0x7f2600105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:48.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.678+0000 7f2605be3640 1 -- 172.21.15.38:0/1030191376 >> 172.21.15.38:0/1030191376 conn(0x7f2600077210 msgr2=0x7f26000fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:48.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.678+0000 7f2605be3640 1 -- 172.21.15.38:0/1030191376 shutdown_connections 2024-09-16T00:31:48.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.678+0000 7f2605be3640 1 -- 172.21.15.38:0/1030191376 wait complete. 2024-09-16T00:31:48.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.678+0000 7f2605be3640 1 Processor -- start 2024-09-16T00:31:48.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.679+0000 7f2605be3640 1 -- start start 2024-09-16T00:31:48.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.679+0000 7f2605be3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2600101240 0x7f2600199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:48.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.679+0000 7f2605be3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2600101bc0 0x7f2600199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:48.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.679+0000 7f2605be3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2600105a90 0x7f26001a4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:48.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.679+0000 7f2605be3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2600110fc0 con 0x7f2600101240 2024-09-16T00:31:48.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.679+0000 7f2605be3640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2600110e40 con 0x7f2600101bc0 2024-09-16T00:31:48.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.679+0000 7f2605be3640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2600111140 con 0x7f2600105a90 2024-09-16T00:31:48.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.679+0000 7f25ff7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2600101240 0x7f2600199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:48.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.679+0000 7f25ff7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2600101240 0x7f2600199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37004/0 (socket says 172.21.15.38:37004) 2024-09-16T00:31:48.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.679+0000 7f25ff7fe640 1 -- 172.21.15.38:0/3928648042 learned_addr learned my addr 172.21.15.38:0/3928648042 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:31:48.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.679+0000 7f25feffd640 1 --2- 172.21.15.38:0/3928648042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2600101bc0 0x7f2600199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:48.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.679+0000 7f25fffff640 1 --2- 172.21.15.38:0/3928648042 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2600105a90 0x7f26001a4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:48.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.679+0000 7f25ff7fe640 1 -- 172.21.15.38:0/3928648042 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2600105a90 msgr2=0x7f26001a4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:48.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.679+0000 7f25ff7fe640 1 --2- 172.21.15.38:0/3928648042 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2600105a90 0x7f26001a4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:48.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.679+0000 7f25ff7fe640 1 -- 172.21.15.38:0/3928648042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2600101bc0 msgr2=0x7f2600199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:48.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.679+0000 7f25ff7fe640 1 --2- 172.21.15.38:0/3928648042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2600101bc0 0x7f2600199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:48.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.679+0000 7f25ff7fe640 1 -- 172.21.15.38:0/3928648042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26001a47d0 con 0x7f2600101240 2024-09-16T00:31:48.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.680+0000 7f25ff7fe640 1 --2- 172.21.15.38:0/3928648042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2600101240 0x7f2600199210 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f25f000c920 tx=0x7f25f000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:48.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.680+0000 7f25fcff9640 1 -- 172.21.15.38:0/3928648042 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f0007c70 con 0x7f2600101240 2024-09-16T00:31:48.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.680+0000 7f2605be3640 1 -- 172.21.15.38:0/3928648042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26001a49a0 con 0x7f2600101240 2024-09-16T00:31:48.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.680+0000 7f25fcff9640 1 -- 172.21.15.38:0/3928648042 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f25f0007e10 con 0x7f2600101240 2024-09-16T00:31:48.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.680+0000 7f2605be3640 1 -- 172.21.15.38:0/3928648042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26001a4f30 con 0x7f2600101240 2024-09-16T00:31:48.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.680+0000 7f25fcff9640 1 -- 172.21.15.38:0/3928648042 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f000f450 con 0x7f2600101240 2024-09-16T00:31:48.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.681+0000 7f2605be3640 1 -- 172.21.15.38:0/3928648042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2600101680 con 0x7f2600101240 2024-09-16T00:31:48.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.682+0000 7f25fcff9640 1 -- 172.21.15.38:0/3928648042 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f25f0004f80 con 0x7f2600101240 2024-09-16T00:31:48.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.683+0000 7f25fcff9640 1 --2- 172.21.15.38:0/3928648042 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f25e80a32d0 0x7f25e80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:48.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.683+0000 7f25fcff9640 1 -- 172.21.15.38:0/3928648042 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f25f00d00e0 con 0x7f2600101240 2024-09-16T00:31:48.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.683+0000 7f25feffd640 1 --2- 172.21.15.38:0/3928648042 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f25e80a32d0 0x7f25e80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:48.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.683+0000 7f25feffd640 1 --2- 172.21.15.38:0/3928648042 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f25e80a32d0 0x7f25e80a5790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f25f4005fd0 tx=0x7f25f4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:48.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.687+0000 7f25fcff9640 1 -- 172.21.15.38:0/3928648042 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f25f0014030 con 0x7f2600101240 2024-09-16T00:31:48.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.873+0000 7f2605be3640 1 -- 172.21.15.38:0/3928648042 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f2600100760 con 0x7f2600101240 2024-09-16T00:31:48.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.873+0000 7f25fcff9640 1 -- 172.21.15.38:0/3928648042 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f25f009c960 con 0x7f2600101240 2024-09-16T00:31:48.877 INFO:teuthology.orchestra.run.smithi038.stdout:137438953538 2024-09-16T00:31:48.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.875+0000 7f2605be3640 1 -- 172.21.15.38:0/3928648042 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f25e80a32d0 msgr2=0x7f25e80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:48.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.875+0000 7f2605be3640 1 --2- 172.21.15.38:0/3928648042 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f25e80a32d0 0x7f25e80a5790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f25f4005fd0 tx=0x7f25f4005850 comp rx=0 tx=0).stop 2024-09-16T00:31:48.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.875+0000 7f2605be3640 1 -- 172.21.15.38:0/3928648042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2600101240 msgr2=0x7f2600199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:48.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.875+0000 7f2605be3640 1 --2- 172.21.15.38:0/3928648042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2600101240 0x7f2600199210 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f25f000c920 tx=0x7f25f000cde0 comp rx=0 tx=0).stop 2024-09-16T00:31:48.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.875+0000 7f2605be3640 1 -- 172.21.15.38:0/3928648042 shutdown_connections 2024-09-16T00:31:48.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.875+0000 7f2605be3640 1 --2- 172.21.15.38:0/3928648042 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f25e80a32d0 0x7f25e80a5790 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:48.878 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.875+0000 7f2605be3640 1 --2- 172.21.15.38:0/3928648042 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2600105a90 0x7f26001a4290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:48.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.875+0000 7f2605be3640 1 --2- 172.21.15.38:0/3928648042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2600101bc0 0x7f2600199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:48.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.875+0000 7f2605be3640 1 --2- 172.21.15.38:0/3928648042 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2600101240 0x7f2600199210 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:48.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.875+0000 7f2605be3640 1 -- 172.21.15.38:0/3928648042 >> 172.21.15.38:0/3928648042 conn(0x7f2600077210 msgr2=0x7f2600103790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:48.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.876+0000 7f2605be3640 1 -- 172.21.15.38:0/3928648042 shutdown_connections 2024-09-16T00:31:48.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:48.876+0000 7f2605be3640 1 -- 172.21.15.38:0/3928648042 wait complete. 2024-09-16T00:31:49.219 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574927 got 55834574930 for osd.1 2024-09-16T00:31:49.219 DEBUG:teuthology.parallel:result is None 2024-09-16T00:31:49.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.039+0000 7f3e7f711640 1 -- 172.21.15.38:0/703980955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e78102790 msgr2=0x7f3e781067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:49.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.039+0000 7f3e7f711640 1 --2- 172.21.15.38:0/703980955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e78102790 0x7f3e781067e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f3e6c009900 tx=0x7f3e6c02a410 comp rx=0 tx=0).stop 2024-09-16T00:31:49.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.039+0000 7f3e7f711640 1 -- 172.21.15.38:0/703980955 shutdown_connections 2024-09-16T00:31:49.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.039+0000 7f3e7f711640 1 --2- 172.21.15.38:0/703980955 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e78106f10 0x7f3e7810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:49.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.039+0000 7f3e7f711640 1 --2- 172.21.15.38:0/703980955 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e78102790 0x7f3e781067e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:49.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.039+0000 7f3e7f711640 1 --2- 172.21.15.38:0/703980955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e78101de0 0x7f3e781021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:49.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.039+0000 7f3e7f711640 1 -- 172.21.15.38:0/703980955 >> 172.21.15.38:0/703980955 conn(0x7f3e780fd8c0 msgr2=0x7f3e780ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:49.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.039+0000 7f3e7f711640 1 -- 172.21.15.38:0/703980955 shutdown_connections 2024-09-16T00:31:49.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.039+0000 7f3e7f711640 1 -- 172.21.15.38:0/703980955 wait complete. 2024-09-16T00:31:49.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.040+0000 7f3e7f711640 1 Processor -- start 2024-09-16T00:31:49.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.040+0000 7f3e7f711640 1 -- start start 2024-09-16T00:31:49.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.040+0000 7f3e7f711640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e78101de0 0x7f3e78198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:49.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.040+0000 7f3e7f711640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e78102790 0x7f3e78198990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:49.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.040+0000 7f3e7f711640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e78106f10 0x7f3e7819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:49.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.040+0000 7f3e7f711640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3e78111f70 con 0x7f3e78101de0 2024-09-16T00:31:49.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.040+0000 7f3e7f711640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3e78111df0 con 0x7f3e78102790 2024-09-16T00:31:49.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.040+0000 7f3e7f711640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3e781120f0 con 0x7f3e78106f10 2024-09-16T00:31:49.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.040+0000 7f3e7d4ad640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e78101de0 0x7f3e78198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:49.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.040+0000 7f3e7d4ad640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e78101de0 0x7f3e78198450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40782/0 (socket says 172.21.15.38:40782) 2024-09-16T00:31:49.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.040+0000 7f3e7d4ad640 1 -- 172.21.15.38:0/4268830076 learned_addr learned my addr 172.21.15.38:0/4268830076 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:31:49.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.040+0000 7f3e7dcae640 1 --2- 172.21.15.38:0/4268830076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e78106f10 0x7f3e7819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:49.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.040+0000 7f3e7ccac640 1 --2- 172.21.15.38:0/4268830076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e78102790 0x7f3e78198990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:49.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.041+0000 7f3e7d4ad640 1 -- 172.21.15.38:0/4268830076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e78106f10 msgr2=0x7f3e7819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:49.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.041+0000 7f3e7d4ad640 1 --2- 172.21.15.38:0/4268830076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e78106f10 0x7f3e7819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:49.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.041+0000 7f3e7d4ad640 1 -- 172.21.15.38:0/4268830076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e78102790 msgr2=0x7f3e78198990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:49.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.041+0000 7f3e7d4ad640 1 --2- 172.21.15.38:0/4268830076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e78102790 0x7f3e78198990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:49.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.041+0000 7f3e7d4ad640 1 -- 172.21.15.38:0/4268830076 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e7819d400 con 0x7f3e78101de0 2024-09-16T00:31:49.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.041+0000 7f3e7d4ad640 1 --2- 172.21.15.38:0/4268830076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e78101de0 0x7f3e78198450 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3e6800c890 tx=0x7f3e6800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:49.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.041+0000 7f3e767fc640 1 -- 172.21.15.38:0/4268830076 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e68004220 con 0x7f3e78101de0 2024-09-16T00:31:49.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.041+0000 7f3e767fc640 1 -- 172.21.15.38:0/4268830076 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3e680043c0 con 0x7f3e78101de0 2024-09-16T00:31:49.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.041+0000 7f3e7f711640 1 -- 172.21.15.38:0/4268830076 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e7819d6f0 con 0x7f3e78101de0 2024-09-16T00:31:49.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.041+0000 7f3e767fc640 1 -- 172.21.15.38:0/4268830076 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e680049e0 con 0x7f3e78101de0 2024-09-16T00:31:49.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.041+0000 7f3e7f711640 1 -- 172.21.15.38:0/4268830076 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e781a4fd0 con 0x7f3e78101de0 2024-09-16T00:31:49.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.042+0000 7f3e7f711640 1 -- 172.21.15.38:0/4268830076 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e781021c0 con 0x7f3e78101de0 2024-09-16T00:31:49.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.043+0000 7f3e767fc640 1 -- 172.21.15.38:0/4268830076 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f3e680079b0 con 0x7f3e78101de0 2024-09-16T00:31:49.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.044+0000 7f3e767fc640 1 --2- 172.21.15.38:0/4268830076 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f3e600a3350 0x7f3e600a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:49.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.044+0000 7f3e7ccac640 1 --2- 172.21.15.38:0/4268830076 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f3e600a3350 0x7f3e600a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:49.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.044+0000 7f3e767fc640 1 -- 172.21.15.38:0/4268830076 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f3e680cfdf0 con 0x7f3e78101de0 2024-09-16T00:31:49.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.045+0000 7f3e7ccac640 1 --2- 172.21.15.38:0/4268830076 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f3e600a3350 0x7f3e600a5810 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3e780fe170 tx=0x7f3e6c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:49.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.047+0000 7f3e767fc640 1 -- 172.21.15.38:0/4268830076 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3e68014030 con 0x7f3e78101de0 2024-09-16T00:31:49.296 INFO:teuthology.orchestra.run.smithi038.stdout:274877906984 2024-09-16T00:31:49.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.235+0000 7f3e7f711640 1 -- 172.21.15.38:0/4268830076 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f3e7810a150 con 0x7f3e78101de0 2024-09-16T00:31:49.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.235+0000 7f3e767fc640 1 -- 172.21.15.38:0/4268830076 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f3e6809c6f0 con 0x7f3e78101de0 2024-09-16T00:31:49.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.238+0000 7f3e7f711640 1 -- 172.21.15.38:0/4268830076 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f3e600a3350 msgr2=0x7f3e600a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:49.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.238+0000 7f3e7f711640 1 --2- 172.21.15.38:0/4268830076 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f3e600a3350 0x7f3e600a5810 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f3e780fe170 tx=0x7f3e6c005a20 comp rx=0 tx=0).stop 2024-09-16T00:31:49.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.238+0000 7f3e7f711640 1 -- 172.21.15.38:0/4268830076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e78101de0 msgr2=0x7f3e78198450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:49.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.238+0000 7f3e7f711640 1 --2- 172.21.15.38:0/4268830076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e78101de0 0x7f3e78198450 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f3e6800c890 tx=0x7f3e6800cd50 comp rx=0 tx=0).stop 2024-09-16T00:31:49.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.238+0000 7f3e7f711640 1 -- 172.21.15.38:0/4268830076 shutdown_connections 2024-09-16T00:31:49.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.238+0000 7f3e7f711640 1 --2- 172.21.15.38:0/4268830076 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f3e600a3350 0x7f3e600a5810 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:49.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.238+0000 7f3e7f711640 1 --2- 172.21.15.38:0/4268830076 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3e78106f10 0x7f3e7819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:49.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.238+0000 7f3e7f711640 1 --2- 172.21.15.38:0/4268830076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e78102790 0x7f3e78198990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:49.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.238+0000 7f3e7f711640 1 --2- 172.21.15.38:0/4268830076 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3e78101de0 0x7f3e78198450 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:49.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.238+0000 7f3e7f711640 1 -- 172.21.15.38:0/4268830076 >> 172.21.15.38:0/4268830076 conn(0x7f3e780fd8c0 msgr2=0x7f3e780fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:49.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.238+0000 7f3e7f711640 1 -- 172.21.15.38:0/4268830076 shutdown_connections 2024-09-16T00:31:49.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:46.238+0000 7f3e7f711640 1 -- 172.21.15.38:0/4268830076 wait complete. 2024-09-16T00:31:49.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:49 smithi038 ceph-mon[30945]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:31:49.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:49 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/3928648042' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-16T00:31:49.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:49 smithi122 ceph-mon[35412]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:31:49.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:49 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/3928648042' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-16T00:31:49.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:49 smithi073 ceph-mon[35254]: pgmap v333: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:31:49.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:49 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/3928648042' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-16T00:31:50.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.050+0000 7f6e9281b640 1 -- 172.21.15.38:0/1245087738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e8c103660 msgr2=0x7f6e8c10bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:50.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.050+0000 7f6e9281b640 1 --2- 172.21.15.38:0/1245087738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e8c103660 0x7f6e8c10bb50 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6e80009900 tx=0x7f6e8002a480 comp rx=0 tx=0).stop 2024-09-16T00:31:50.074 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.050+0000 7f6e9281b640 1 -- 172.21.15.38:0/1245087738 shutdown_connections 2024-09-16T00:31:50.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.050+0000 7f6e9281b640 1 --2- 172.21.15.38:0/1245087738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e8c10c170 0x7f6e8c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:50.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.050+0000 7f6e9281b640 1 --2- 172.21.15.38:0/1245087738 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e8c103660 0x7f6e8c10bb50 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:50.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.050+0000 7f6e9281b640 1 --2- 172.21.15.38:0/1245087738 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e8c102d40 0x7f6e8c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:50.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.050+0000 7f6e9281b640 1 -- 172.21.15.38:0/1245087738 >> 172.21.15.38:0/1245087738 conn(0x7f6e8c0fc7a0 msgr2=0x7f6e8c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:50.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.050+0000 7f6e9281b640 1 -- 172.21.15.38:0/1245087738 shutdown_connections 2024-09-16T00:31:50.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.050+0000 7f6e9281b640 1 -- 172.21.15.38:0/1245087738 wait complete. 2024-09-16T00:31:50.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.051+0000 7f6e9281b640 1 Processor -- start 2024-09-16T00:31:50.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.051+0000 7f6e9281b640 1 -- start start 2024-09-16T00:31:50.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.051+0000 7f6e9281b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e8c102d40 0x7f6e8c198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:50.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.051+0000 7f6e9281b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e8c103660 0x7f6e8c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:50.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.051+0000 7f6e9281b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e8c10c170 0x7f6e8c19cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:50.075 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.051+0000 7f6e9281b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6e8c111fb0 con 0x7f6e8c10c170 2024-09-16T00:31:50.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.051+0000 7f6e9281b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6e8c111e30 con 0x7f6e8c102d40 2024-09-16T00:31:50.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.051+0000 7f6e9281b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6e8c112130 con 0x7f6e8c103660 2024-09-16T00:31:50.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.051+0000 7f6e90db8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e8c10c170 0x7f6e8c19cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:50.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.051+0000 7f6e90db8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e8c10c170 0x7f6e8c19cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37034/0 (socket says 172.21.15.38:37034) 2024-09-16T00:31:50.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.051+0000 7f6e90db8640 1 -- 172.21.15.38:0/3981650745 learned_addr learned my addr 172.21.15.38:0/3981650745 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:31:50.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.052+0000 7f6e8bfff640 1 --2- 172.21.15.38:0/3981650745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e8c102d40 0x7f6e8c198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:50.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.052+0000 7f6e8b7fe640 1 --2- 172.21.15.38:0/3981650745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e8c103660 0x7f6e8c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:50.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.052+0000 7f6e90db8640 1 -- 172.21.15.38:0/3981650745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e8c103660 msgr2=0x7f6e8c198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:50.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.052+0000 7f6e90db8640 1 --2- 172.21.15.38:0/3981650745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e8c103660 0x7f6e8c198970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:50.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.052+0000 7f6e90db8640 1 -- 172.21.15.38:0/3981650745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e8c102d40 msgr2=0x7f6e8c198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:50.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.052+0000 7f6e90db8640 1 --2- 172.21.15.38:0/3981650745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e8c102d40 0x7f6e8c198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:50.076 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.052+0000 7f6e90db8640 1 -- 172.21.15.38:0/3981650745 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e8c19d3e0 con 0x7f6e8c10c170 2024-09-16T00:31:50.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.052+0000 7f6e8bfff640 1 --2- 172.21.15.38:0/3981650745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e8c102d40 0x7f6e8c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:31:50.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.052+0000 7f6e90db8640 1 --2- 172.21.15.38:0/3981650745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e8c10c170 0x7f6e8c19cd00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6e8400b970 tx=0x7f6e8400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:50.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.052+0000 7f6e897fa640 1 -- 172.21.15.38:0/3981650745 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e84019070 con 0x7f6e8c10c170 2024-09-16T00:31:50.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.052+0000 7f6e897fa640 1 -- 172.21.15.38:0/3981650745 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e8400cb40 con 0x7f6e8c10c170 2024-09-16T00:31:50.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.052+0000 7f6e9281b640 1 -- 172.21.15.38:0/3981650745 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e8c19d6d0 con 0x7f6e8c10c170 2024-09-16T00:31:50.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.052+0000 7f6e897fa640 1 -- 172.21.15.38:0/3981650745 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e84007650 con 0x7f6e8c10c170 2024-09-16T00:31:50.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.052+0000 7f6e9281b640 1 -- 172.21.15.38:0/3981650745 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e8c1a4fb0 con 0x7f6e8c10c170 2024-09-16T00:31:50.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.053+0000 7f6e9281b640 1 -- 172.21.15.38:0/3981650745 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e8c103120 con 0x7f6e8c10c170 2024-09-16T00:31:50.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.054+0000 7f6e897fa640 1 -- 172.21.15.38:0/3981650745 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f6e8401c070 con 0x7f6e8c10c170 2024-09-16T00:31:50.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.055+0000 7f6e897fa640 1 --2- 172.21.15.38:0/3981650745 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f6e740a32d0 0x7f6e740a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:50.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.055+0000 7f6e8bfff640 1 --2- 172.21.15.38:0/3981650745 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f6e740a32d0 0x7f6e740a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:50.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.055+0000 7f6e897fa640 1 -- 172.21.15.38:0/3981650745 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f6e840cf370 con 0x7f6e8c10c170 2024-09-16T00:31:50.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.056+0000 7f6e8bfff640 1 --2- 172.21.15.38:0/3981650745 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f6e740a32d0 0x7f6e740a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c00abe0 tx=0x7f6e7c00a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:50.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.060+0000 7f6e897fa640 1 -- 172.21.15.38:0/3981650745 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6e8409bd00 con 0x7f6e8c10c170 2024-09-16T00:31:50.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.260+0000 7f6e9281b640 1 -- 172.21.15.38:0/3981650745 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f6e8c199790 con 0x7f6e8c10c170 2024-09-16T00:31:50.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.260+0000 7f6e897fa640 1 -- 172.21.15.38:0/3981650745 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6e8409b600 con 0x7f6e8c10c170 2024-09-16T00:31:50.261 INFO:teuthology.orchestra.run.smithi038.stdout:188978561080 2024-09-16T00:31:50.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.262+0000 7f6e9281b640 1 -- 172.21.15.38:0/3981650745 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f6e740a32d0 msgr2=0x7f6e740a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:50.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.262+0000 7f6e9281b640 1 --2- 172.21.15.38:0/3981650745 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f6e740a32d0 0x7f6e740a5790 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6e7c00abe0 tx=0x7f6e7c00a5c0 comp rx=0 tx=0).stop 2024-09-16T00:31:50.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.262+0000 7f6e9281b640 1 -- 172.21.15.38:0/3981650745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e8c10c170 msgr2=0x7f6e8c19cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:50.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.262+0000 7f6e9281b640 1 --2- 172.21.15.38:0/3981650745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e8c10c170 0x7f6e8c19cd00 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6e8400b970 tx=0x7f6e8400be30 comp rx=0 tx=0).stop 2024-09-16T00:31:50.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.262+0000 7f6e9281b640 1 -- 172.21.15.38:0/3981650745 shutdown_connections 2024-09-16T00:31:50.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.262+0000 7f6e9281b640 1 --2- 172.21.15.38:0/3981650745 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f6e740a32d0 0x7f6e740a5790 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:50.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.262+0000 7f6e9281b640 1 --2- 172.21.15.38:0/3981650745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e8c10c170 0x7f6e8c19cd00 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:50.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.262+0000 7f6e9281b640 1 --2- 172.21.15.38:0/3981650745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e8c103660 0x7f6e8c198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:50.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.262+0000 7f6e9281b640 1 --2- 172.21.15.38:0/3981650745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e8c102d40 0x7f6e8c198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:50.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.262+0000 7f6e9281b640 1 -- 172.21.15.38:0/3981650745 >> 172.21.15.38:0/3981650745 conn(0x7f6e8c0fc7a0 msgr2=0x7f6e8c0fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:50.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.262+0000 7f6e9281b640 1 -- 172.21.15.38:0/3981650745 shutdown_connections 2024-09-16T00:31:50.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.262+0000 7f6e9281b640 1 -- 172.21.15.38:0/3981650745 wait complete. 2024-09-16T00:31:50.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.714+0000 7fdb4f09e640 1 -- 172.21.15.38:0/3228332269 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb48103660 msgr2=0x7fdb4810bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:50.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.714+0000 7fdb4f09e640 1 --2- 172.21.15.38:0/3228332269 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb48103660 0x7fdb4810bb50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fdb34009900 tx=0x7fdb3402a480 comp rx=0 tx=0).stop 2024-09-16T00:31:50.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.714+0000 7fdb4f09e640 1 -- 172.21.15.38:0/3228332269 shutdown_connections 2024-09-16T00:31:50.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.714+0000 7fdb4f09e640 1 --2- 172.21.15.38:0/3228332269 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb4810c170 0x7fdb4810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:50.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.714+0000 7fdb4f09e640 1 --2- 172.21.15.38:0/3228332269 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb48103660 0x7fdb4810bb50 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:50.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.714+0000 7fdb4f09e640 1 --2- 172.21.15.38:0/3228332269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb48102d40 0x7fdb48103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:50.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.714+0000 7fdb4f09e640 1 -- 172.21.15.38:0/3228332269 >> 172.21.15.38:0/3228332269 conn(0x7fdb480fc7a0 msgr2=0x7fdb480febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:50.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.714+0000 7fdb4f09e640 1 -- 172.21.15.38:0/3228332269 shutdown_connections 2024-09-16T00:31:50.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.714+0000 7fdb4f09e640 1 -- 172.21.15.38:0/3228332269 wait complete. 2024-09-16T00:31:50.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.715+0000 7fdb4f09e640 1 Processor -- start 2024-09-16T00:31:50.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.715+0000 7fdb4f09e640 1 -- start start 2024-09-16T00:31:50.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.715+0000 7fdb4f09e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb48102d40 0x7fdb48198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:50.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.715+0000 7fdb4f09e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb48103660 0x7fdb48198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:50.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.715+0000 7fdb4f09e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb4810c170 0x7fdb4819ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:50.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.715+0000 7fdb4f09e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdb48112030 con 0x7fdb4810c170 2024-09-16T00:31:50.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.715+0000 7fdb4f09e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdb48111eb0 con 0x7fdb48102d40 2024-09-16T00:31:50.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.715+0000 7fdb4f09e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdb481121b0 con 0x7fdb48103660 2024-09-16T00:31:50.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.716+0000 7fdb4d63b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb4810c170 0x7fdb4819ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:50.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.716+0000 7fdb4d63b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb4810c170 0x7fdb4819ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37050/0 (socket says 172.21.15.38:37050) 2024-09-16T00:31:50.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.716+0000 7fdb4d63b640 1 -- 172.21.15.38:0/2679944476 learned_addr learned my addr 172.21.15.38:0/2679944476 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:31:50.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.716+0000 7fdb47fff640 1 --2- 172.21.15.38:0/2679944476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb48103660 0x7fdb48198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:50.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.716+0000 7fdb4ce3a640 1 --2- 172.21.15.38:0/2679944476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb48102d40 0x7fdb48198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:50.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.716+0000 7fdb4d63b640 1 -- 172.21.15.38:0/2679944476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb48103660 msgr2=0x7fdb48198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:50.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.716+0000 7fdb4d63b640 1 --2- 172.21.15.38:0/2679944476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb48103660 0x7fdb48198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:50.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.716+0000 7fdb4d63b640 1 -- 172.21.15.38:0/2679944476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb48102d40 msgr2=0x7fdb48198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:50.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.716+0000 7fdb4d63b640 1 --2- 172.21.15.38:0/2679944476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb48102d40 0x7fdb48198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:50.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.716+0000 7fdb4d63b640 1 -- 172.21.15.38:0/2679944476 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb4819d540 con 0x7fdb4810c170 2024-09-16T00:31:50.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.716+0000 7fdb4d63b640 1 --2- 172.21.15.38:0/2679944476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb4810c170 0x7fdb4819ce60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fdb4000b970 tx=0x7fdb4000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:50.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.716+0000 7fdb45ffb640 1 -- 172.21.15.38:0/2679944476 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb40019070 con 0x7fdb4810c170 2024-09-16T00:31:50.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.716+0000 7fdb45ffb640 1 -- 172.21.15.38:0/2679944476 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdb4000cb40 con 0x7fdb4810c170 2024-09-16T00:31:50.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.716+0000 7fdb4f09e640 1 -- 172.21.15.38:0/2679944476 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb4819d830 con 0x7fdb4810c170 2024-09-16T00:31:50.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.716+0000 7fdb45ffb640 1 -- 172.21.15.38:0/2679944476 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb40007650 con 0x7fdb4810c170 2024-09-16T00:31:50.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.717+0000 7fdb4f09e640 1 -- 172.21.15.38:0/2679944476 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb481a5110 con 0x7fdb4810c170 2024-09-16T00:31:50.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.717+0000 7fdb4f09e640 1 -- 172.21.15.38:0/2679944476 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb481031a0 con 0x7fdb4810c170 2024-09-16T00:31:50.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.719+0000 7fdb45ffb640 1 -- 172.21.15.38:0/2679944476 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fdb4001c070 con 0x7fdb4810c170 2024-09-16T00:31:50.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.719+0000 7fdb45ffb640 1 --2- 172.21.15.38:0/2679944476 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fdb380a32d0 0x7fdb380a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:50.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.719+0000 7fdb4ce3a640 1 --2- 172.21.15.38:0/2679944476 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fdb380a32d0 0x7fdb380a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:50.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.719+0000 7fdb45ffb640 1 -- 172.21.15.38:0/2679944476 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7fdb400cf2e0 con 0x7fdb4810c170 2024-09-16T00:31:50.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.720+0000 7fdb4ce3a640 1 --2- 172.21.15.38:0/2679944476 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fdb380a32d0 0x7fdb380a5790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c009890 tx=0x7fdb3c008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:50.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.724+0000 7fdb45ffb640 1 -- 172.21.15.38:0/2679944476 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdb4009bc70 con 0x7fdb4810c170 2024-09-16T00:31:50.901 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.901+0000 7fdb4f09e640 1 -- 172.21.15.38:0/2679944476 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7fdb48106c50 con 0x7fdb4810c170 2024-09-16T00:31:50.902 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.901+0000 7fdb45ffb640 1 -- 172.21.15.38:0/2679944476 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fdb400bb070 con 0x7fdb4810c170 2024-09-16T00:31:50.902 INFO:teuthology.orchestra.run.smithi038.stdout:326417514526 2024-09-16T00:31:50.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.903+0000 7fdb4f09e640 1 -- 172.21.15.38:0/2679944476 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fdb380a32d0 msgr2=0x7fdb380a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:50.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.903+0000 7fdb4f09e640 1 --2- 172.21.15.38:0/2679944476 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fdb380a32d0 0x7fdb380a5790 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c009890 tx=0x7fdb3c008040 comp rx=0 tx=0).stop 2024-09-16T00:31:50.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.903+0000 7fdb4f09e640 1 -- 172.21.15.38:0/2679944476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb4810c170 msgr2=0x7fdb4819ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:50.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.903+0000 7fdb4f09e640 1 --2- 172.21.15.38:0/2679944476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb4810c170 0x7fdb4819ce60 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fdb4000b970 tx=0x7fdb4000be30 comp rx=0 tx=0).stop 2024-09-16T00:31:50.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.903+0000 7fdb4f09e640 1 -- 172.21.15.38:0/2679944476 shutdown_connections 2024-09-16T00:31:50.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.903+0000 7fdb4f09e640 1 --2- 172.21.15.38:0/2679944476 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fdb380a32d0 0x7fdb380a5790 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:50.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.903+0000 7fdb4f09e640 1 --2- 172.21.15.38:0/2679944476 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdb4810c170 0x7fdb4819ce60 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:50.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.903+0000 7fdb4f09e640 1 --2- 172.21.15.38:0/2679944476 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdb48103660 0x7fdb48198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:50.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.903+0000 7fdb4f09e640 1 --2- 172.21.15.38:0/2679944476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb48102d40 0x7fdb48198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:50.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.903+0000 7fdb4f09e640 1 -- 172.21.15.38:0/2679944476 >> 172.21.15.38:0/2679944476 conn(0x7fdb480fc7a0 msgr2=0x7fdb480fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:50.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.904+0000 7fdb4f09e640 1 -- 172.21.15.38:0/2679944476 shutdown_connections 2024-09-16T00:31:50.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:50.904+0000 7fdb4f09e640 1 -- 172.21.15.38:0/2679944476 wait complete. 2024-09-16T00:31:51.214 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:51 smithi038 ceph-mon[30945]: pgmap v334: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-09-16T00:31:51.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:51 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/3981650745' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-16T00:31:51.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:51 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/2679944476' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-16T00:31:51.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:51 smithi122 ceph-mon[35412]: pgmap v334: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-09-16T00:31:51.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:51 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/3981650745' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-16T00:31:51.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:51 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/2679944476' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-16T00:31:51.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:51 smithi073 ceph-mon[35254]: pgmap v334: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 6 op/s 2024-09-16T00:31:51.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:51 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/3981650745' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-16T00:31:51.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:51 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/2679944476' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-09-16T00:31:51.617 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738453 got 34359738454 for osd.0 2024-09-16T00:31:51.617 DEBUG:teuthology.parallel:result is None 2024-09-16T00:31:51.741 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182466 got 107374182469 for osd.3 2024-09-16T00:31:51.742 DEBUG:teuthology.parallel:result is None 2024-09-16T00:31:52.279 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:52.317 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906982 got 274877906984 for osd.9 2024-09-16T00:31:52.317 DEBUG:teuthology.parallel:result is None 2024-09-16T00:31:52.973 INFO:tasks.cephadm.ceph_manager.ceph:need seq 188978561078 got 188978561080 for osd.6 2024-09-16T00:31:52.974 DEBUG:teuthology.parallel:result is None 2024-09-16T00:31:53.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:53 smithi038 ceph-mon[30945]: pgmap v335: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-16T00:31:53.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:53 smithi122 ceph-mon[35412]: pgmap v335: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-16T00:31:53.492 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953537 got 137438953538 for osd.4 2024-09-16T00:31:53.492 DEBUG:teuthology.parallel:result is None 2024-09-16T00:31:53.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:53 smithi073 ceph-mon[35254]: pgmap v335: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2024-09-16T00:31:53.737 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:54.181 INFO:tasks.cephadm.ceph_manager.ceph:need seq 326417514523 got 326417514526 for osd.11 2024-09-16T00:31:54.182 DEBUG:teuthology.parallel:result is None 2024-09-16T00:31:54.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.449+0000 7f5531103640 1 -- 172.21.15.38:0/3620657934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f552c100e20 msgr2=0x7f552c101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:54.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.449+0000 7f5531103640 1 --2- 172.21.15.38:0/3620657934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f552c100e20 0x7f552c101200 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f551c009900 tx=0x7f551c02a480 comp rx=0 tx=0).stop 2024-09-16T00:31:54.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.449+0000 7f5531103640 1 -- 172.21.15.38:0/3620657934 shutdown_connections 2024-09-16T00:31:54.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.449+0000 7f5531103640 1 --2- 172.21.15.38:0/3620657934 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f552c102ef0 0x7f552c10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:54.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.449+0000 7f5531103640 1 --2- 172.21.15.38:0/3620657934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f552c102570 0x7f552c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:54.563 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.449+0000 7f5531103640 1 --2- 172.21.15.38:0/3620657934 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f552c100e20 0x7f552c101200 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:54.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.449+0000 7f5531103640 1 -- 172.21.15.38:0/3620657934 >> 172.21.15.38:0/3620657934 conn(0x7f552c0fc7d0 msgr2=0x7f552c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:54.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.449+0000 7f5531103640 1 -- 172.21.15.38:0/3620657934 shutdown_connections 2024-09-16T00:31:54.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.449+0000 7f5531103640 1 -- 172.21.15.38:0/3620657934 wait complete. 2024-09-16T00:31:54.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.450+0000 7f5531103640 1 Processor -- start 2024-09-16T00:31:54.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.450+0000 7f5531103640 1 -- start start 2024-09-16T00:31:54.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.450+0000 7f5531103640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f552c100e20 0x7f552c19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:54.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.450+0000 7f5531103640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f552c102570 0x7f552c19ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:54.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.450+0000 7f5531103640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f552c102ef0 0x7f552c19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:54.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.450+0000 7f5531103640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f552c112060 con 0x7f552c102ef0 2024-09-16T00:31:54.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.450+0000 7f5531103640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f552c111ee0 con 0x7f552c102570 2024-09-16T00:31:54.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.450+0000 7f5531103640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f552c1121e0 con 0x7f552c100e20 2024-09-16T00:31:54.564 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.451+0000 7f552b59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f552c102ef0 0x7f552c19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:54.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.451+0000 7f552a59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f552c102570 0x7f552c19ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:54.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.451+0000 7f552b59e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f552c102ef0 0x7f552c19efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37070/0 (socket says 172.21.15.38:37070) 2024-09-16T00:31:54.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.451+0000 7f552a59c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f552c102570 0x7f552c19ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.38:56182/0 (socket says 172.21.15.38:56182) 2024-09-16T00:31:54.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.451+0000 7f552b59e640 1 -- 172.21.15.38:0/1097916870 learned_addr learned my addr 172.21.15.38:0/1097916870 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:31:54.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.451+0000 7f552ad9d640 1 --2- 172.21.15.38:0/1097916870 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f552c100e20 0x7f552c19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:54.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.451+0000 7f552a59c640 1 -- 172.21.15.38:0/1097916870 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f552c100e20 msgr2=0x7f552c19a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:54.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.451+0000 7f552a59c640 1 --2- 172.21.15.38:0/1097916870 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f552c100e20 0x7f552c19a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:54.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.451+0000 7f552a59c640 1 -- 172.21.15.38:0/1097916870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f552c102ef0 msgr2=0x7f552c19efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:54.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.451+0000 7f552a59c640 1 --2- 172.21.15.38:0/1097916870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f552c102ef0 0x7f552c19efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:54.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.451+0000 7f552a59c640 1 -- 172.21.15.38:0/1097916870 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f552c19f6a0 con 0x7f552c102570 2024-09-16T00:31:54.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.451+0000 7f552a59c640 1 --2- 172.21.15.38:0/1097916870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f552c102570 0x7f552c19ac30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f552000e9f0 tx=0x7f552000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:54.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.452+0000 7f5513fff640 1 -- 172.21.15.38:0/1097916870 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552000ccc0 con 0x7f552c102570 2024-09-16T00:31:54.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.452+0000 7f5531103640 1 -- 172.21.15.38:0/1097916870 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f552c19f990 con 0x7f552c102570 2024-09-16T00:31:54.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.452+0000 7f5513fff640 1 -- 172.21.15.38:0/1097916870 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5520010910 con 0x7f552c102570 2024-09-16T00:31:54.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.452+0000 7f5513fff640 1 -- 172.21.15.38:0/1097916870 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5520018960 con 0x7f552c102570 2024-09-16T00:31:54.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.452+0000 7f5531103640 1 -- 172.21.15.38:0/1097916870 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f552c1a7270 con 0x7f552c102570 2024-09-16T00:31:54.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.453+0000 7f5531103640 1 -- 172.21.15.38:0/1097916870 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f552c101200 con 0x7f552c102570 2024-09-16T00:31:54.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.454+0000 7f5513fff640 1 -- 172.21.15.38:0/1097916870 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f5520010ac0 con 0x7f552c102570 2024-09-16T00:31:54.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.455+0000 7f5513fff640 1 --2- 172.21.15.38:0/1097916870 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f55140a32d0 0x7f55140a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:54.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.455+0000 7f5513fff640 1 -- 172.21.15.38:0/1097916870 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f55200d02b0 con 0x7f552c102570 2024-09-16T00:31:54.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.455+0000 7f552ad9d640 1 --2- 172.21.15.38:0/1097916870 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f55140a32d0 0x7f55140a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:54.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.456+0000 7f552ad9d640 1 --2- 172.21.15.38:0/1097916870 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f55140a32d0 0x7f55140a5790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f551c009230 tx=0x7f551c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:54.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.458+0000 7f5513fff640 1 -- 172.21.15.38:0/1097916870 <== mon.2 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f552009c4e0 con 0x7f552c102570 2024-09-16T00:31:54.608 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:54.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.640+0000 7f5531103640 1 -- 172.21.15.38:0/1097916870 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f552c10fb10 con 0x7f552c102570 2024-09-16T00:31:54.642 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.641+0000 7f5513fff640 1 -- 172.21.15.38:0/1097916870 <== mon.2 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f55200a07e0 con 0x7f552c102570 2024-09-16T00:31:54.642 INFO:teuthology.orchestra.run.smithi038.stdout:296352743460 2024-09-16T00:31:54.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.643+0000 7f5531103640 1 -- 172.21.15.38:0/1097916870 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f55140a32d0 msgr2=0x7f55140a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:54.644 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.643+0000 7f5531103640 1 --2- 172.21.15.38:0/1097916870 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f55140a32d0 0x7f55140a5790 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f551c009230 tx=0x7f551c0058d0 comp rx=0 tx=0).stop 2024-09-16T00:31:54.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.643+0000 7f5531103640 1 -- 172.21.15.38:0/1097916870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f552c102570 msgr2=0x7f552c19ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:54.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.643+0000 7f5531103640 1 --2- 172.21.15.38:0/1097916870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f552c102570 0x7f552c19ac30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f552000e9f0 tx=0x7f552000eeb0 comp rx=0 tx=0).stop 2024-09-16T00:31:54.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.643+0000 7f5531103640 1 -- 172.21.15.38:0/1097916870 shutdown_connections 2024-09-16T00:31:54.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.643+0000 7f5531103640 1 --2- 172.21.15.38:0/1097916870 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f55140a32d0 0x7f55140a5790 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:54.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.643+0000 7f5531103640 1 --2- 172.21.15.38:0/1097916870 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f552c102ef0 0x7f552c19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:54.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.643+0000 7f5531103640 1 --2- 172.21.15.38:0/1097916870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f552c102570 0x7f552c19ac30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:54.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.643+0000 7f5531103640 1 --2- 172.21.15.38:0/1097916870 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f552c100e20 0x7f552c19a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:54.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.643+0000 7f5531103640 1 -- 172.21.15.38:0/1097916870 >> 172.21.15.38:0/1097916870 conn(0x7f552c0fc7d0 msgr2=0x7f552c105f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:54.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.643+0000 7f5531103640 1 -- 172.21.15.38:0/1097916870 shutdown_connections 2024-09-16T00:31:54.645 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:54.643+0000 7f5531103640 1 -- 172.21.15.38:0/1097916870 wait complete. 2024-09-16T00:31:55.449 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:55 smithi038 ceph-mon[30945]: pgmap v336: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-16T00:31:55.449 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:55 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/1097916870' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-16T00:31:55.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:55 smithi122 ceph-mon[35412]: pgmap v336: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-16T00:31:55.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:55 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/1097916870' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-16T00:31:55.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:55 smithi073 ceph-mon[35254]: pgmap v336: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-16T00:31:55.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:55 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/1097916870' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-09-16T00:31:55.903 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743457 got 296352743460 for osd.10 2024-09-16T00:31:55.903 DEBUG:teuthology.parallel:result is None 2024-09-16T00:31:56.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.110+0000 7f51093ba640 1 -- 172.21.15.38:0/3331079470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5104106f10 msgr2=0x7f510410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:56.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.110+0000 7f51093ba640 1 --2- 172.21.15.38:0/3331079470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5104106f10 0x7f510410ea90 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f50fc009ca0 tx=0x7f50fc02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.110+0000 7f51093ba640 1 -- 172.21.15.38:0/3331079470 shutdown_connections 2024-09-16T00:31:56.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.110+0000 7f51093ba640 1 --2- 172.21.15.38:0/3331079470 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5104106f10 0x7f510410ea90 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.110+0000 7f51093ba640 1 --2- 172.21.15.38:0/3331079470 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5104102790 0x7f51041067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.110+0000 7f51093ba640 1 --2- 172.21.15.38:0/3331079470 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5104101de0 0x7f51041021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.110+0000 7f51093ba640 1 -- 172.21.15.38:0/3331079470 >> 172.21.15.38:0/3331079470 conn(0x7f51040fd8c0 msgr2=0x7f51040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:56.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.110+0000 7f51093ba640 1 -- 172.21.15.38:0/3331079470 shutdown_connections 2024-09-16T00:31:56.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.110+0000 7f51093ba640 1 -- 172.21.15.38:0/3331079470 wait complete. 2024-09-16T00:31:56.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.111+0000 7f51093ba640 1 Processor -- start 2024-09-16T00:31:56.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.111+0000 7f51093ba640 1 -- start start 2024-09-16T00:31:56.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.112+0000 7f51093ba640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5104101de0 0x7f5104198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:56.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.112+0000 7f51093ba640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5104102790 0x7f5104198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:56.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.112+0000 7f51093ba640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5104106f10 0x7f510419cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:56.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.112+0000 7f51093ba640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5104111f70 con 0x7f5104106f10 2024-09-16T00:31:56.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.112+0000 7f51093ba640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5104111df0 con 0x7f5104102790 2024-09-16T00:31:56.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.112+0000 7f51093ba640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f51041120f0 con 0x7f5104101de0 2024-09-16T00:31:56.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.112+0000 7f51037fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5104106f10 0x7f510419cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:56.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.112+0000 7f51037fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5104106f10 0x7f510419cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:37090/0 (socket says 172.21.15.38:37090) 2024-09-16T00:31:56.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.112+0000 7f51037fe640 1 -- 172.21.15.38:0/661461525 learned_addr learned my addr 172.21.15.38:0/661461525 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:31:56.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.112+0000 7f51027fc640 1 --2- 172.21.15.38:0/661461525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5104102790 0x7f5104198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:56.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.112+0000 7f5102ffd640 1 --2- 172.21.15.38:0/661461525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5104101de0 0x7f5104198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:56.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.112+0000 7f51037fe640 1 -- 172.21.15.38:0/661461525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5104101de0 msgr2=0x7f5104198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:56.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.112+0000 7f51037fe640 1 --2- 172.21.15.38:0/661461525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5104101de0 0x7f5104198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.112+0000 7f51037fe640 1 -- 172.21.15.38:0/661461525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5104102790 msgr2=0x7f5104198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:56.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.112+0000 7f51037fe640 1 --2- 172.21.15.38:0/661461525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5104102790 0x7f5104198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.112+0000 7f51037fe640 1 -- 172.21.15.38:0/661461525 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f510419d2e0 con 0x7f5104106f10 2024-09-16T00:31:56.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.112+0000 7f51037fe640 1 --2- 172.21.15.38:0/661461525 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5104106f10 0x7f510419cc00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f50fc031a20 tx=0x7f50fc031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:56.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.113+0000 7f50ebfff640 1 -- 172.21.15.38:0/661461525 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50fc003b70 con 0x7f5104106f10 2024-09-16T00:31:56.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.113+0000 7f51093ba640 1 -- 172.21.15.38:0/661461525 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f510419d570 con 0x7f5104106f10 2024-09-16T00:31:56.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.113+0000 7f50ebfff640 1 -- 172.21.15.38:0/661461525 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f50fc003d10 con 0x7f5104106f10 2024-09-16T00:31:56.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.113+0000 7f50ebfff640 1 -- 172.21.15.38:0/661461525 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50fc02fc80 con 0x7f5104106f10 2024-09-16T00:31:56.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.113+0000 7f51093ba640 1 -- 172.21.15.38:0/661461525 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51041a4eb0 con 0x7f5104106f10 2024-09-16T00:31:56.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.114+0000 7f51093ba640 1 -- 172.21.15.38:0/661461525 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51041021c0 con 0x7f5104106f10 2024-09-16T00:31:56.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.115+0000 7f50ebfff640 1 -- 172.21.15.38:0/661461525 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f50fc0046a0 con 0x7f5104106f10 2024-09-16T00:31:56.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.116+0000 7f50ebfff640 1 --2- 172.21.15.38:0/661461525 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f50ec0a32d0 0x7f50ec0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:56.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.116+0000 7f5102ffd640 1 --2- 172.21.15.38:0/661461525 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f50ec0a32d0 0x7f50ec0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:56.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.116+0000 7f50ebfff640 1 -- 172.21.15.38:0/661461525 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f50fc0f0ff0 con 0x7f5104106f10 2024-09-16T00:31:56.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.117+0000 7f5102ffd640 1 --2- 172.21.15.38:0/661461525 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f50ec0a32d0 0x7f50ec0a5790 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f50f400bbe0 tx=0x7f50f400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:56.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.119+0000 7f50ebfff640 1 -- 172.21.15.38:0/661461525 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f50fc0bda10 con 0x7f5104106f10 2024-09-16T00:31:56.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.298+0000 7f51093ba640 1 -- 172.21.15.38:0/661461525 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f51041a50e0 con 0x7f5104106f10 2024-09-16T00:31:56.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.298+0000 7f50ebfff640 1 -- 172.21.15.38:0/661461525 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f50fc0bd310 con 0x7f5104106f10 2024-09-16T00:31:56.300 INFO:teuthology.orchestra.run.smithi038.stdout:158913790014 2024-09-16T00:31:56.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.300+0000 7f51093ba640 1 -- 172.21.15.38:0/661461525 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f50ec0a32d0 msgr2=0x7f50ec0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:56.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.300+0000 7f51093ba640 1 --2- 172.21.15.38:0/661461525 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f50ec0a32d0 0x7f50ec0a5790 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f50f400bbe0 tx=0x7f50f400b390 comp rx=0 tx=0).stop 2024-09-16T00:31:56.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.300+0000 7f51093ba640 1 -- 172.21.15.38:0/661461525 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5104106f10 msgr2=0x7f510419cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:56.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.300+0000 7f51093ba640 1 --2- 172.21.15.38:0/661461525 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5104106f10 0x7f510419cc00 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f50fc031a20 tx=0x7f50fc031a50 comp rx=0 tx=0).stop 2024-09-16T00:31:56.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.301+0000 7f51093ba640 1 -- 172.21.15.38:0/661461525 shutdown_connections 2024-09-16T00:31:56.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.301+0000 7f51093ba640 1 --2- 172.21.15.38:0/661461525 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f50ec0a32d0 0x7f50ec0a5790 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.301+0000 7f51093ba640 1 --2- 172.21.15.38:0/661461525 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5104106f10 0x7f510419cc00 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.301+0000 7f51093ba640 1 --2- 172.21.15.38:0/661461525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5104102790 0x7f5104198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.301+0000 7f51093ba640 1 --2- 172.21.15.38:0/661461525 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5104101de0 0x7f5104198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.301+0000 7f51093ba640 1 -- 172.21.15.38:0/661461525 >> 172.21.15.38:0/661461525 conn(0x7f51040fd8c0 msgr2=0x7f51040ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:56.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.301+0000 7f51093ba640 1 -- 172.21.15.38:0/661461525 shutdown_connections 2024-09-16T00:31:56.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.301+0000 7f51093ba640 1 -- 172.21.15.38:0/661461525 wait complete. 2024-09-16T00:31:56.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.342+0000 7f81cb256640 1 -- 172.21.15.38:0/2954726395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81c4102db0 msgr2=0x7f81c4103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:56.384 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.342+0000 7f81cb256640 1 --2- 172.21.15.38:0/2954726395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81c4102db0 0x7f81c4103190 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f81b4009930 tx=0x7f81b402a420 comp rx=0 tx=0).stop 2024-09-16T00:31:56.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.342+0000 7f81cb256640 1 -- 172.21.15.38:0/2954726395 shutdown_connections 2024-09-16T00:31:56.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.342+0000 7f81cb256640 1 --2- 172.21.15.38:0/2954726395 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81c41082e0 0x7f81c410ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.342+0000 7f81cb256640 1 --2- 172.21.15.38:0/2954726395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81c4103760 0x7f81c4107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.342+0000 7f81cb256640 1 --2- 172.21.15.38:0/2954726395 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81c4102db0 0x7f81c4103190 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.342+0000 7f81cb256640 1 -- 172.21.15.38:0/2954726395 >> 172.21.15.38:0/2954726395 conn(0x7f81c40fc7d0 msgr2=0x7f81c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:56.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.342+0000 7f81cb256640 1 -- 172.21.15.38:0/2954726395 shutdown_connections 2024-09-16T00:31:56.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.342+0000 7f81cb256640 1 -- 172.21.15.38:0/2954726395 wait complete. 2024-09-16T00:31:56.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.343+0000 7f81cb256640 1 Processor -- start 2024-09-16T00:31:56.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.343+0000 7f81cb256640 1 -- start start 2024-09-16T00:31:56.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.343+0000 7f81cb256640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81c4102db0 0x7f81c4100a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:56.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.343+0000 7f81cb256640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81c4103760 0x7f81c4100f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:56.385 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.343+0000 7f81cb256640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81c41082e0 0x7f81c4101490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:56.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.343+0000 7f81cb256640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f81c41120e0 con 0x7f81c4102db0 2024-09-16T00:31:56.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.343+0000 7f81cb256640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f81c4111f60 con 0x7f81c4103760 2024-09-16T00:31:56.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.343+0000 7f81cb256640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f81c4112260 con 0x7f81c41082e0 2024-09-16T00:31:56.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.343+0000 7f81c8ff2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81c4102db0 0x7f81c4100a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:56.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.343+0000 7f81c8ff2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81c4102db0 0x7f81c4100a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34970/0 (socket says 172.21.15.38:34970) 2024-09-16T00:31:56.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.343+0000 7f81c8ff2640 1 -- 172.21.15.38:0/1316326983 learned_addr learned my addr 172.21.15.38:0/1316326983 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:31:56.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.343+0000 7f81c3fff640 1 --2- 172.21.15.38:0/1316326983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81c4103760 0x7f81c4100f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:56.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.343+0000 7f81c97f3640 1 --2- 172.21.15.38:0/1316326983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81c41082e0 0x7f81c4101490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:56.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.343+0000 7f81c8ff2640 1 -- 172.21.15.38:0/1316326983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81c41082e0 msgr2=0x7f81c4101490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:56.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.343+0000 7f81c8ff2640 1 --2- 172.21.15.38:0/1316326983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81c41082e0 0x7f81c4101490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.343+0000 7f81c8ff2640 1 -- 172.21.15.38:0/1316326983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81c4103760 msgr2=0x7f81c4100f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:56.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.343+0000 7f81c8ff2640 1 --2- 172.21.15.38:0/1316326983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81c4103760 0x7f81c4100f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.386 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.343+0000 7f81c8ff2640 1 -- 172.21.15.38:0/1316326983 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c41a6a70 con 0x7f81c4102db0 2024-09-16T00:31:56.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.343+0000 7f81c3fff640 1 --2- 172.21.15.38:0/1316326983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81c4103760 0x7f81c4100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:31:56.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.344+0000 7f81c8ff2640 1 --2- 172.21.15.38:0/1316326983 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81c4102db0 0x7f81c4100a10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f81b402a940 tx=0x7f81b4003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:56.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.344+0000 7f81c1ffb640 1 -- 172.21.15.38:0/1316326983 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b403f070 con 0x7f81c4102db0 2024-09-16T00:31:56.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.344+0000 7f81cb256640 1 -- 172.21.15.38:0/1316326983 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81c41a6d00 con 0x7f81c4102db0 2024-09-16T00:31:56.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.344+0000 7f81c1ffb640 1 -- 172.21.15.38:0/1316326983 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f81b4031c20 con 0x7f81c4102db0 2024-09-16T00:31:56.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.344+0000 7f81c1ffb640 1 -- 172.21.15.38:0/1316326983 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b4035030 con 0x7f81c4102db0 2024-09-16T00:31:56.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.344+0000 7f81cb256640 1 -- 172.21.15.38:0/1316326983 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81c41a71d0 con 0x7f81c4102db0 2024-09-16T00:31:56.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.345+0000 7f81cb256640 1 -- 172.21.15.38:0/1316326983 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81c4103190 con 0x7f81c4102db0 2024-09-16T00:31:56.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.346+0000 7f81c1ffb640 1 -- 172.21.15.38:0/1316326983 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f81b4042070 con 0x7f81c4102db0 2024-09-16T00:31:56.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.347+0000 7f81c1ffb640 1 --2- 172.21.15.38:0/1316326983 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f81ac0a32d0 0x7f81ac0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:56.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.347+0000 7f81c1ffb640 1 -- 172.21.15.38:0/1316326983 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f81b40f0cc0 con 0x7f81c4102db0 2024-09-16T00:31:56.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.347+0000 7f81c3fff640 1 --2- 172.21.15.38:0/1316326983 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f81ac0a32d0 0x7f81ac0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:56.387 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.347+0000 7f81c3fff640 1 --2- 172.21.15.38:0/1316326983 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f81ac0a32d0 0x7f81ac0a5790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f81c41024a0 tx=0x7f81b8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:56.388 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.350+0000 7f81c1ffb640 1 -- 172.21.15.38:0/1316326983 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f81b4031dd0 con 0x7f81c4102db0 2024-09-16T00:31:56.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.527+0000 7f81cb256640 1 -- 172.21.15.38:0/1316326983 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f81c410a2c0 con 0x7f81c4102db0 2024-09-16T00:31:56.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.527+0000 7f81c1ffb640 1 -- 172.21.15.38:0/1316326983 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f81b40bd540 con 0x7f81c4102db0 2024-09-16T00:31:56.530 INFO:teuthology.orchestra.run.smithi038.stdout:77309411406 2024-09-16T00:31:56.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.530+0000 7f81cb256640 1 -- 172.21.15.38:0/1316326983 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f81ac0a32d0 msgr2=0x7f81ac0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:56.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.530+0000 7f81cb256640 1 --2- 172.21.15.38:0/1316326983 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f81ac0a32d0 0x7f81ac0a5790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f81c41024a0 tx=0x7f81b8005f90 comp rx=0 tx=0).stop 2024-09-16T00:31:56.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.530+0000 7f81cb256640 1 -- 172.21.15.38:0/1316326983 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81c4102db0 msgr2=0x7f81c4100a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:56.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.530+0000 7f81cb256640 1 --2- 172.21.15.38:0/1316326983 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81c4102db0 0x7f81c4100a10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f81b402a940 tx=0x7f81b4003b90 comp rx=0 tx=0).stop 2024-09-16T00:31:56.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.530+0000 7f81cb256640 1 -- 172.21.15.38:0/1316326983 shutdown_connections 2024-09-16T00:31:56.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.530+0000 7f81cb256640 1 --2- 172.21.15.38:0/1316326983 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f81ac0a32d0 0x7f81ac0a5790 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.530+0000 7f81cb256640 1 --2- 172.21.15.38:0/1316326983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f81c41082e0 0x7f81c4101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.530+0000 7f81cb256640 1 --2- 172.21.15.38:0/1316326983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81c4103760 0x7f81c4100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.530+0000 7f81cb256640 1 --2- 172.21.15.38:0/1316326983 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f81c4102db0 0x7f81c4100a10 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:56.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.530+0000 7f81cb256640 1 -- 172.21.15.38:0/1316326983 >> 172.21.15.38:0/1316326983 conn(0x7f81c40fc7d0 msgr2=0x7f81c4105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:56.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.530+0000 7f81cb256640 1 -- 172.21.15.38:0/1316326983 shutdown_connections 2024-09-16T00:31:56.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:56.530+0000 7f81cb256640 1 -- 172.21.15.38:0/1316326983 wait complete. 2024-09-16T00:31:57.171 INFO:tasks.cephadm.ceph_manager.ceph:need seq 158913790010 got 158913790014 for osd.5 2024-09-16T00:31:57.171 DEBUG:teuthology.parallel:result is None 2024-09-16T00:31:57.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:57 smithi038 ceph-mon[30945]: pgmap v337: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:31:57.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:57 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/661461525' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-16T00:31:57.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:57 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/1316326983' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-16T00:31:57.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:57 smithi122 ceph-mon[35412]: pgmap v337: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:31:57.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:57 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/661461525' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-16T00:31:57.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:57 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/1316326983' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-16T00:31:57.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:57 smithi073 ceph-mon[35254]: pgmap v337: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:31:57.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:57 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/661461525' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-16T00:31:57.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:57 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/1316326983' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-16T00:31:57.950 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411404 got 77309411406 for osd.2 2024-09-16T00:31:57.950 DEBUG:teuthology.parallel:result is None 2024-09-16T00:31:57.950 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-16T00:31:57.950 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T00:31:58.213 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:31:58.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.692+0000 7fdd68935640 1 -- 172.21.15.38:0/1340761306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd64102570 msgr2=0x7fdd641029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:58.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.692+0000 7fdd68935640 1 --2- 172.21.15.38:0/1340761306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd64102570 0x7fdd641029b0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fdd58009900 tx=0x7fdd5802a480 comp rx=0 tx=0).stop 2024-09-16T00:31:58.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.692+0000 7fdd68935640 1 -- 172.21.15.38:0/1340761306 shutdown_connections 2024-09-16T00:31:58.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.692+0000 7fdd68935640 1 --2- 172.21.15.38:0/1340761306 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd64102ef0 0x7fdd6410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.692+0000 7fdd68935640 1 --2- 172.21.15.38:0/1340761306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd64102570 0x7fdd641029b0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.720 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.692+0000 7fdd68935640 1 --2- 172.21.15.38:0/1340761306 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd64100e20 0x7fdd64101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.692+0000 7fdd68935640 1 -- 172.21.15.38:0/1340761306 >> 172.21.15.38:0/1340761306 conn(0x7fdd640fc7d0 msgr2=0x7fdd640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:58.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.692+0000 7fdd68935640 1 -- 172.21.15.38:0/1340761306 shutdown_connections 2024-09-16T00:31:58.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.692+0000 7fdd68935640 1 -- 172.21.15.38:0/1340761306 wait complete. 2024-09-16T00:31:58.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.693+0000 7fdd68935640 1 Processor -- start 2024-09-16T00:31:58.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.693+0000 7fdd68935640 1 -- start start 2024-09-16T00:31:58.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.693+0000 7fdd68935640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd64100e20 0x7fdd641973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:58.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.693+0000 7fdd68935640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd64102570 0x7fdd64197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:58.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.693+0000 7fdd68935640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd64102ef0 0x7fdd6419bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:58.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.693+0000 7fdd68935640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdd64110f80 con 0x7fdd64102ef0 2024-09-16T00:31:58.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.693+0000 7fdd68935640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdd64110e00 con 0x7fdd64102570 2024-09-16T00:31:58.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.693+0000 7fdd68935640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdd64111100 con 0x7fdd64100e20 2024-09-16T00:31:58.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.693+0000 7fdd62d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd64102ef0 0x7fdd6419bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:58.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.693+0000 7fdd62d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd64102ef0 0x7fdd6419bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35002/0 (socket says 172.21.15.38:35002) 2024-09-16T00:31:58.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.693+0000 7fdd62d9d640 1 -- 172.21.15.38:0/2352681935 learned_addr learned my addr 172.21.15.38:0/2352681935 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:31:58.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.693+0000 7fdd6259c640 1 --2- 172.21.15.38:0/2352681935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd64100e20 0x7fdd641973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:58.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.693+0000 7fdd61d9b640 1 --2- 172.21.15.38:0/2352681935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd64102570 0x7fdd64197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:58.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.693+0000 7fdd62d9d640 1 -- 172.21.15.38:0/2352681935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd64100e20 msgr2=0x7fdd641973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:58.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.693+0000 7fdd62d9d640 1 --2- 172.21.15.38:0/2352681935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd64100e20 0x7fdd641973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.693+0000 7fdd62d9d640 1 -- 172.21.15.38:0/2352681935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd64102570 msgr2=0x7fdd64197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:58.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.693+0000 7fdd62d9d640 1 --2- 172.21.15.38:0/2352681935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd64102570 0x7fdd64197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.693+0000 7fdd62d9d640 1 -- 172.21.15.38:0/2352681935 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd6419c390 con 0x7fdd64102ef0 2024-09-16T00:31:58.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.693+0000 7fdd61d9b640 1 --2- 172.21.15.38:0/2352681935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd64102570 0x7fdd64197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:31:58.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.694+0000 7fdd62d9d640 1 --2- 172.21.15.38:0/2352681935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd64102ef0 0x7fdd6419bcb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c00eaa0 tx=0x7fdd5c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:58.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.694+0000 7fdd537fe640 1 -- 172.21.15.38:0/2352681935 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c00cdb0 con 0x7fdd64102ef0 2024-09-16T00:31:58.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.694+0000 7fdd537fe640 1 -- 172.21.15.38:0/2352681935 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdd5c005020 con 0x7fdd64102ef0 2024-09-16T00:31:58.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.694+0000 7fdd68935640 1 -- 172.21.15.38:0/2352681935 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd6419c680 con 0x7fdd64102ef0 2024-09-16T00:31:58.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.694+0000 7fdd537fe640 1 -- 172.21.15.38:0/2352681935 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5c0116b0 con 0x7fdd64102ef0 2024-09-16T00:31:58.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.694+0000 7fdd68935640 1 -- 172.21.15.38:0/2352681935 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd641a3f60 con 0x7fdd64102ef0 2024-09-16T00:31:58.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.695+0000 7fdd68935640 1 -- 172.21.15.38:0/2352681935 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd64101200 con 0x7fdd64102ef0 2024-09-16T00:31:58.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.696+0000 7fdd537fe640 1 -- 172.21.15.38:0/2352681935 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fdd5c005560 con 0x7fdd64102ef0 2024-09-16T00:31:58.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.697+0000 7fdd537fe640 1 --2- 172.21.15.38:0/2352681935 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fdd4c0a32d0 0x7fdd4c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:31:58.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.697+0000 7fdd537fe640 1 -- 172.21.15.38:0/2352681935 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7fdd5c0d0100 con 0x7fdd64102ef0 2024-09-16T00:31:58.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.697+0000 7fdd6259c640 1 --2- 172.21.15.38:0/2352681935 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fdd4c0a32d0 0x7fdd4c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:31:58.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.697+0000 7fdd6259c640 1 --2- 172.21.15.38:0/2352681935 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fdd4c0a32d0 0x7fdd4c0a5790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdd54005fd0 tx=0x7fdd54005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:31:58.723 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.701+0000 7fdd537fe640 1 -- 172.21.15.38:0/2352681935 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdd5c016030 con 0x7fdd64102ef0 2024-09-16T00:31:58.874 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.873+0000 7fdd68935640 1 -- 172.21.15.38:0/2352681935 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fdd64062630 con 0x7fdd4c0a32d0 2024-09-16T00:31:58.883 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.881+0000 7fdd537fe640 1 -- 172.21.15.38:0/2352681935 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340370 (secure 0 0 0) 0x7fdd64062630 con 0x7fdd4c0a32d0 2024-09-16T00:31:58.886 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:31:58.888 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-16T00:31:58.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.885+0000 7fdd68935640 1 -- 172.21.15.38:0/2352681935 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fdd4c0a32d0 msgr2=0x7fdd4c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:58.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.885+0000 7fdd68935640 1 --2- 172.21.15.38:0/2352681935 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fdd4c0a32d0 0x7fdd4c0a5790 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fdd54005fd0 tx=0x7fdd54005850 comp rx=0 tx=0).stop 2024-09-16T00:31:58.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.885+0000 7fdd68935640 1 -- 172.21.15.38:0/2352681935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd64102ef0 msgr2=0x7fdd6419bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:31:58.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.885+0000 7fdd68935640 1 --2- 172.21.15.38:0/2352681935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd64102ef0 0x7fdd6419bcb0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fdd5c00eaa0 tx=0x7fdd5c00ef60 comp rx=0 tx=0).stop 2024-09-16T00:31:58.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.885+0000 7fdd68935640 1 -- 172.21.15.38:0/2352681935 shutdown_connections 2024-09-16T00:31:58.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.885+0000 7fdd68935640 1 --2- 172.21.15.38:0/2352681935 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fdd4c0a32d0 0x7fdd4c0a5790 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.885+0000 7fdd68935640 1 --2- 172.21.15.38:0/2352681935 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdd64102ef0 0x7fdd6419bcb0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.885+0000 7fdd68935640 1 --2- 172.21.15.38:0/2352681935 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd64102570 0x7fdd64197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.885+0000 7fdd68935640 1 --2- 172.21.15.38:0/2352681935 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdd64100e20 0x7fdd641973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:31:58.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.885+0000 7fdd68935640 1 -- 172.21.15.38:0/2352681935 >> 172.21.15.38:0/2352681935 conn(0x7fdd640fc7d0 msgr2=0x7fdd641068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:31:58.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.885+0000 7fdd68935640 1 -- 172.21.15.38:0/2352681935 shutdown_connections 2024-09-16T00:31:58.892 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:31:58.885+0000 7fdd68935640 1 -- 172.21.15.38:0/2352681935 wait complete. 2024-09-16T00:31:59.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:31:59 smithi038 ceph-mon[30945]: pgmap v338: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:31:59.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:31:59 smithi122 ceph-mon[35412]: pgmap v338: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:31:59.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:31:59 smithi073 ceph-mon[35254]: pgmap v338: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:31:59.689 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":338,"stamp":"2024-09-16T00:31:57.529714+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656681,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":458,"num_read_kb":348,"num_write":349,"num_write_kb":3055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1180792,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":281,"ondisk_log_size":281,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379984,"kb_used_data":14036,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561840,"statfs":{"total":1151940427776,"available":1151551324160,"internally_reserved":0,"allocated":14372864,"data_stored":7999037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003285"},"pg_stats":[{"pgid":"2.1f","version":"88'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442635+0000","last_change":"2024-09-16T00:30:23.933355+0000","last_active":"2024-09-16T00:31:23.442635+0000","last_peered":"2024-09-16T00:31:23.442635+0000","last_clean":"2024-09-16T00:31:23.442635+0000","last_became_active":"2024-09-16T00:30:23.933057+0000","last_became_peered":"2024-09-16T00:30:23.933057+0000","last_unstale":"2024-09-16T00:31:23.442635+0000","last_undegraded":"2024-09-16T00:31:23.442635+0000","last_fullsized":"2024-09-16T00:31:23.442635+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:23:01.713440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.445196+0000","last_change":"2024-09-16T00:30:26.084537+0000","last_active":"2024-09-16T00:31:23.445196+0000","last_peered":"2024-09-16T00:31:23.445196+0000","last_clean":"2024-09-16T00:31:23.445196+0000","last_became_active":"2024-09-16T00:30:26.084226+0000","last_became_peered":"2024-09-16T00:30:26.084226+0000","last_unstale":"2024-09-16T00:31:23.445196+0000","last_undegraded":"2024-09-16T00:31:23.445196+0000","last_fullsized":"2024-09-16T00:31:23.445196+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:43:41.628877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442440+0000","last_change":"2024-09-16T00:30:33.154873+0000","last_active":"2024-09-16T00:31:23.442440+0000","last_peered":"2024-09-16T00:31:23.442440+0000","last_clean":"2024-09-16T00:31:23.442440+0000","last_became_active":"2024-09-16T00:30:33.154433+0000","last_became_peered":"2024-09-16T00:30:33.154433+0000","last_unstale":"2024-09-16T00:31:23.442440+0000","last_undegraded":"2024-09-16T00:31:23.442440+0000","last_fullsized":"2024-09-16T00:31:23.442440+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:50.805806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.445394+0000","last_change":"2024-09-16T00:30:24.422274+0000","last_active":"2024-09-16T00:31:23.445394+0000","last_peered":"2024-09-16T00:31:23.445394+0000","last_clean":"2024-09-16T00:31:23.445394+0000","last_became_active":"2024-09-16T00:30:24.421902+0000","last_became_peered":"2024-09-16T00:30:24.421902+0000","last_unstale":"2024-09-16T00:31:23.445394+0000","last_undegraded":"2024-09-16T00:31:23.445394+0000","last_fullsized":"2024-09-16T00:31:23.445394+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:01.910187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.577309+0000","last_change":"2024-09-16T00:30:26.088165+0000","last_active":"2024-09-16T00:31:23.577309+0000","last_peered":"2024-09-16T00:31:23.577309+0000","last_clean":"2024-09-16T00:31:23.577309+0000","last_became_active":"2024-09-16T00:30:26.087593+0000","last_became_peered":"2024-09-16T00:30:26.087593+0000","last_unstale":"2024-09-16T00:31:23.577309+0000","last_undegraded":"2024-09-16T00:31:23.577309+0000","last_fullsized":"2024-09-16T00:31:23.577309+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:54.149930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.445078+0000","last_change":"2024-09-16T00:30:33.154586+0000","last_active":"2024-09-16T00:31:23.445078+0000","last_peered":"2024-09-16T00:31:23.445078+0000","last_clean":"2024-09-16T00:31:23.445078+0000","last_became_active":"2024-09-16T00:30:33.154209+0000","last_became_peered":"2024-09-16T00:30:33.154209+0000","last_unstale":"2024-09-16T00:31:23.445078+0000","last_undegraded":"2024-09-16T00:31:23.445078+0000","last_fullsized":"2024-09-16T00:31:23.445078+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:45.776895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.444802+0000","last_change":"2024-09-16T00:30:23.932442+0000","last_active":"2024-09-16T00:31:23.444802+0000","last_peered":"2024-09-16T00:31:23.444802+0000","last_clean":"2024-09-16T00:31:23.444802+0000","last_became_active":"2024-09-16T00:30:23.932009+0000","last_became_peered":"2024-09-16T00:30:23.932009+0000","last_unstale":"2024-09-16T00:31:23.444802+0000","last_undegraded":"2024-09-16T00:31:23.444802+0000","last_fullsized":"2024-09-16T00:31:23.444802+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:09:27.078187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.440646+0000","last_change":"2024-09-16T00:30:26.087715+0000","last_active":"2024-09-16T00:31:23.440646+0000","last_peered":"2024-09-16T00:31:23.440646+0000","last_clean":"2024-09-16T00:31:23.440646+0000","last_became_active":"2024-09-16T00:30:26.087415+0000","last_became_peered":"2024-09-16T00:30:26.087415+0000","last_unstale":"2024-09-16T00:31:23.440646+0000","last_undegraded":"2024-09-16T00:31:23.440646+0000","last_fullsized":"2024-09-16T00:31:23.440646+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:52.373992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.445658+0000","last_change":"2024-09-16T00:30:33.152328+0000","last_active":"2024-09-16T00:31:23.445658+0000","last_peered":"2024-09-16T00:31:23.445658+0000","last_clean":"2024-09-16T00:31:23.445658+0000","last_became_active":"2024-09-16T00:30:33.151954+0000","last_became_peered":"2024-09-16T00:30:33.151954+0000","last_unstale":"2024-09-16T00:31:23.445658+0000","last_undegraded":"2024-09-16T00:31:23.445658+0000","last_fullsized":"2024-09-16T00:31:23.445658+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:36.546729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.443216+0000","last_change":"2024-09-16T00:30:23.928988+0000","last_active":"2024-09-16T00:31:23.443216+0000","last_peered":"2024-09-16T00:31:23.443216+0000","last_clean":"2024-09-16T00:31:23.443216+0000","last_became_active":"2024-09-16T00:30:23.928638+0000","last_became_peered":"2024-09-16T00:30:23.928638+0000","last_unstale":"2024-09-16T00:31:23.443216+0000","last_undegraded":"2024-09-16T00:31:23.443216+0000","last_fullsized":"2024-09-16T00:31:23.443216+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:45:58.203455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.446550+0000","last_change":"2024-09-16T00:31:23.446550+0000","last_active":"2024-09-16T00:31:23.446550+0000","last_peered":"2024-09-16T00:31:23.446550+0000","last_clean":"2024-09-16T00:31:23.446550+0000","last_became_active":"2024-09-16T00:31:23.446179+0000","last_became_peered":"2024-09-16T00:31:23.446179+0000","last_unstale":"2024-09-16T00:31:23.446550+0000","last_undegraded":"2024-09-16T00:31:23.446550+0000","last_fullsized":"2024-09-16T00:31:23.446550+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:04:21.280174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.445269+0000","last_change":"2024-09-16T00:30:33.147564+0000","last_active":"2024-09-16T00:31:23.445269+0000","last_peered":"2024-09-16T00:31:23.445269+0000","last_clean":"2024-09-16T00:31:23.445269+0000","last_became_active":"2024-09-16T00:30:33.147119+0000","last_became_peered":"2024-09-16T00:30:33.147119+0000","last_unstale":"2024-09-16T00:31:23.445269+0000","last_undegraded":"2024-09-16T00:31:23.445269+0000","last_fullsized":"2024-09-16T00:31:23.445269+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:00.509122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442254+0000","last_change":"2024-09-16T00:30:26.080954+0000","last_active":"2024-09-16T00:31:23.442254+0000","last_peered":"2024-09-16T00:31:23.442254+0000","last_clean":"2024-09-16T00:31:23.442254+0000","last_became_active":"2024-09-16T00:30:26.080624+0000","last_became_peered":"2024-09-16T00:30:26.080624+0000","last_unstale":"2024-09-16T00:31:23.442254+0000","last_undegraded":"2024-09-16T00:31:23.442254+0000","last_fullsized":"2024-09-16T00:31:23.442254+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:05:10.440980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442620+0000","last_change":"2024-09-16T00:30:23.933715+0000","last_active":"2024-09-16T00:31:23.442620+0000","last_peered":"2024-09-16T00:31:23.442620+0000","last_clean":"2024-09-16T00:31:23.442620+0000","last_became_active":"2024-09-16T00:30:23.933275+0000","last_became_peered":"2024-09-16T00:30:23.933275+0000","last_unstale":"2024-09-16T00:31:23.442620+0000","last_undegraded":"2024-09-16T00:31:23.442620+0000","last_fullsized":"2024-09-16T00:31:23.442620+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:40:44.846053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.445456+0000","last_change":"2024-09-16T00:30:33.604602+0000","last_active":"2024-09-16T00:31:23.445456+0000","last_peered":"2024-09-16T00:31:23.445456+0000","last_clean":"2024-09-16T00:31:23.445456+0000","last_became_active":"2024-09-16T00:30:33.604107+0000","last_became_peered":"2024-09-16T00:30:33.604107+0000","last_unstale":"2024-09-16T00:31:23.445456+0000","last_undegraded":"2024-09-16T00:31:23.445456+0000","last_fullsized":"2024-09-16T00:31:23.445456+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:41.254445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.444747+0000","last_change":"2024-09-16T00:30:26.080612+0000","last_active":"2024-09-16T00:31:23.444747+0000","last_peered":"2024-09-16T00:31:23.444747+0000","last_clean":"2024-09-16T00:31:23.444747+0000","last_became_active":"2024-09-16T00:30:26.080317+0000","last_became_peered":"2024-09-16T00:30:26.080317+0000","last_unstale":"2024-09-16T00:31:23.444747+0000","last_undegraded":"2024-09-16T00:31:23.444747+0000","last_fullsized":"2024-09-16T00:31:23.444747+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:23.814241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442788+0000","last_change":"2024-09-16T00:30:23.936778+0000","last_active":"2024-09-16T00:31:23.442788+0000","last_peered":"2024-09-16T00:31:23.442788+0000","last_clean":"2024-09-16T00:31:23.442788+0000","last_became_active":"2024-09-16T00:30:23.936265+0000","last_became_peered":"2024-09-16T00:30:23.936265+0000","last_unstale":"2024-09-16T00:31:23.442788+0000","last_undegraded":"2024-09-16T00:31:23.442788+0000","last_fullsized":"2024-09-16T00:31:23.442788+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:35:28.288476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.445546+0000","last_change":"2024-09-16T00:30:33.148340+0000","last_active":"2024-09-16T00:31:23.445546+0000","last_peered":"2024-09-16T00:31:23.445546+0000","last_clean":"2024-09-16T00:31:23.445546+0000","last_became_active":"2024-09-16T00:30:33.147289+0000","last_became_peered":"2024-09-16T00:30:33.147289+0000","last_unstale":"2024-09-16T00:31:23.445546+0000","last_undegraded":"2024-09-16T00:31:23.445546+0000","last_fullsized":"2024-09-16T00:31:23.445546+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:21.259254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442247+0000","last_change":"2024-09-16T00:30:26.077325+0000","last_active":"2024-09-16T00:31:23.442247+0000","last_peered":"2024-09-16T00:31:23.442247+0000","last_clean":"2024-09-16T00:31:23.442247+0000","last_became_active":"2024-09-16T00:30:26.077002+0000","last_became_peered":"2024-09-16T00:30:26.077002+0000","last_unstale":"2024-09-16T00:31:23.442247+0000","last_undegraded":"2024-09-16T00:31:23.442247+0000","last_fullsized":"2024-09-16T00:31:23.442247+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:12:38.569247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.445059+0000","last_change":"2024-09-16T00:30:23.933025+0000","last_active":"2024-09-16T00:31:23.445059+0000","last_peered":"2024-09-16T00:31:23.445059+0000","last_clean":"2024-09-16T00:31:23.445059+0000","last_became_active":"2024-09-16T00:30:23.932669+0000","last_became_peered":"2024-09-16T00:30:23.932669+0000","last_unstale":"2024-09-16T00:31:23.445059+0000","last_undegraded":"2024-09-16T00:31:23.445059+0000","last_fullsized":"2024-09-16T00:31:23.445059+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:20.593665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442296+0000","last_change":"2024-09-16T00:30:33.151844+0000","last_active":"2024-09-16T00:31:23.442296+0000","last_peered":"2024-09-16T00:31:23.442296+0000","last_clean":"2024-09-16T00:31:23.442296+0000","last_became_active":"2024-09-16T00:30:33.151311+0000","last_became_peered":"2024-09-16T00:30:33.151311+0000","last_unstale":"2024-09-16T00:31:23.442296+0000","last_undegraded":"2024-09-16T00:31:23.442296+0000","last_fullsized":"2024-09-16T00:31:23.442296+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:51.785233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442430+0000","last_change":"2024-09-16T00:30:26.076104+0000","last_active":"2024-09-16T00:31:23.442430+0000","last_peered":"2024-09-16T00:31:23.442430+0000","last_clean":"2024-09-16T00:31:23.442430+0000","last_became_active":"2024-09-16T00:30:26.075764+0000","last_became_peered":"2024-09-16T00:30:26.075764+0000","last_unstale":"2024-09-16T00:31:23.442430+0000","last_undegraded":"2024-09-16T00:31:23.442430+0000","last_fullsized":"2024-09-16T00:31:23.442430+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:31.228533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.440275+0000","last_change":"2024-09-16T00:30:24.422411+0000","last_active":"2024-09-16T00:31:23.440275+0000","last_peered":"2024-09-16T00:31:23.440275+0000","last_clean":"2024-09-16T00:31:23.440275+0000","last_became_active":"2024-09-16T00:30:24.422102+0000","last_became_peered":"2024-09-16T00:30:24.422102+0000","last_unstale":"2024-09-16T00:31:23.440275+0000","last_undegraded":"2024-09-16T00:31:23.440275+0000","last_fullsized":"2024-09-16T00:31:23.440275+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:06:00.572070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.445691+0000","last_change":"2024-09-16T00:30:33.605397+0000","last_active":"2024-09-16T00:31:23.445691+0000","last_peered":"2024-09-16T00:31:23.445691+0000","last_clean":"2024-09-16T00:31:23.445691+0000","last_became_active":"2024-09-16T00:30:33.604978+0000","last_became_peered":"2024-09-16T00:30:33.604978+0000","last_unstale":"2024-09-16T00:31:23.445691+0000","last_undegraded":"2024-09-16T00:31:23.445691+0000","last_fullsized":"2024-09-16T00:31:23.445691+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:35:16.766056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.577206+0000","last_change":"2024-09-16T00:30:26.082684+0000","last_active":"2024-09-16T00:31:23.577206+0000","last_peered":"2024-09-16T00:31:23.577206+0000","last_clean":"2024-09-16T00:31:23.577206+0000","last_became_active":"2024-09-16T00:30:26.082335+0000","last_became_peered":"2024-09-16T00:30:26.082335+0000","last_unstale":"2024-09-16T00:31:23.577206+0000","last_undegraded":"2024-09-16T00:31:23.577206+0000","last_fullsized":"2024-09-16T00:31:23.577206+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:51:21.389627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.445962+0000","last_change":"2024-09-16T00:30:24.587840+0000","last_active":"2024-09-16T00:31:23.445962+0000","last_peered":"2024-09-16T00:31:23.445962+0000","last_clean":"2024-09-16T00:31:23.445962+0000","last_became_active":"2024-09-16T00:30:24.587436+0000","last_became_peered":"2024-09-16T00:30:24.587436+0000","last_unstale":"2024-09-16T00:31:23.445962+0000","last_undegraded":"2024-09-16T00:31:23.445962+0000","last_fullsized":"2024-09-16T00:31:23.445962+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:06.758585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.445527+0000","last_change":"2024-09-16T00:30:33.145051+0000","last_active":"2024-09-16T00:31:23.445527+0000","last_peered":"2024-09-16T00:31:23.445527+0000","last_clean":"2024-09-16T00:31:23.445527+0000","last_became_active":"2024-09-16T00:30:33.144585+0000","last_became_peered":"2024-09-16T00:30:33.144585+0000","last_unstale":"2024-09-16T00:31:23.445527+0000","last_undegraded":"2024-09-16T00:31:23.445527+0000","last_fullsized":"2024-09-16T00:31:23.445527+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:48:23.588495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.444802+0000","last_change":"2024-09-16T00:31:23.444802+0000","last_active":"2024-09-16T00:31:23.444802+0000","last_peered":"2024-09-16T00:31:23.444802+0000","last_clean":"2024-09-16T00:31:23.444802+0000","last_became_active":"2024-09-16T00:31:23.444447+0000","last_became_peered":"2024-09-16T00:31:23.444447+0000","last_unstale":"2024-09-16T00:31:23.444802+0000","last_undegraded":"2024-09-16T00:31:23.444802+0000","last_fullsized":"2024-09-16T00:31:23.444802+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:46:37.285250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.6","version":"88'11","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442253+0000","last_change":"2024-09-16T00:30:23.933284+0000","last_active":"2024-09-16T00:31:23.442253+0000","last_peered":"2024-09-16T00:31:23.442253+0000","last_clean":"2024-09-16T00:31:23.442253+0000","last_became_active":"2024-09-16T00:30:23.932920+0000","last_became_peered":"2024-09-16T00:30:23.932920+0000","last_unstale":"2024-09-16T00:31:23.442253+0000","last_undegraded":"2024-09-16T00:31:23.442253+0000","last_fullsized":"2024-09-16T00:31:23.442253+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:38.199391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.444402+0000","last_change":"2024-09-16T00:30:33.604308+0000","last_active":"2024-09-16T00:31:23.444402+0000","last_peered":"2024-09-16T00:31:23.444402+0000","last_clean":"2024-09-16T00:31:23.444402+0000","last_became_active":"2024-09-16T00:30:33.603889+0000","last_became_peered":"2024-09-16T00:30:33.603889+0000","last_unstale":"2024-09-16T00:31:23.444402+0000","last_undegraded":"2024-09-16T00:31:23.444402+0000","last_fullsized":"2024-09-16T00:31:23.444402+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:32:55.173063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.439777+0000","last_change":"2024-09-16T00:30:26.078515+0000","last_active":"2024-09-16T00:31:23.439777+0000","last_peered":"2024-09-16T00:31:23.439777+0000","last_clean":"2024-09-16T00:31:23.439777+0000","last_became_active":"2024-09-16T00:30:26.078173+0000","last_became_peered":"2024-09-16T00:30:26.078173+0000","last_unstale":"2024-09-16T00:31:23.439777+0000","last_undegraded":"2024-09-16T00:31:23.439777+0000","last_fullsized":"2024-09-16T00:31:23.439777+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:40:31.471154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.444907+0000","last_change":"2024-09-16T00:30:24.423209+0000","last_active":"2024-09-16T00:31:23.444907+0000","last_peered":"2024-09-16T00:31:23.444907+0000","last_clean":"2024-09-16T00:31:23.444907+0000","last_became_active":"2024-09-16T00:30:24.422157+0000","last_became_peered":"2024-09-16T00:30:24.422157+0000","last_unstale":"2024-09-16T00:31:23.444907+0000","last_undegraded":"2024-09-16T00:31:23.444907+0000","last_fullsized":"2024-09-16T00:31:23.444907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:28.262392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442754+0000","last_change":"2024-09-16T00:30:33.146204+0000","last_active":"2024-09-16T00:31:23.442754+0000","last_peered":"2024-09-16T00:31:23.442754+0000","last_clean":"2024-09-16T00:31:23.442754+0000","last_became_active":"2024-09-16T00:30:33.145553+0000","last_became_peered":"2024-09-16T00:30:33.145553+0000","last_unstale":"2024-09-16T00:31:23.442754+0000","last_undegraded":"2024-09-16T00:31:23.442754+0000","last_fullsized":"2024-09-16T00:31:23.442754+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:22.096050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.441856+0000","last_change":"2024-09-16T00:30:26.070153+0000","last_active":"2024-09-16T00:31:23.441856+0000","last_peered":"2024-09-16T00:31:23.441856+0000","last_clean":"2024-09-16T00:31:23.441856+0000","last_became_active":"2024-09-16T00:30:26.069792+0000","last_became_peered":"2024-09-16T00:30:26.069792+0000","last_unstale":"2024-09-16T00:31:23.441856+0000","last_undegraded":"2024-09-16T00:31:23.441856+0000","last_fullsized":"2024-09-16T00:31:23.441856+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:28.164242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442132+0000","last_change":"2024-09-16T00:30:24.423261+0000","last_active":"2024-09-16T00:31:23.442132+0000","last_peered":"2024-09-16T00:31:23.442132+0000","last_clean":"2024-09-16T00:31:23.442132+0000","last_became_active":"2024-09-16T00:30:24.422650+0000","last_became_peered":"2024-09-16T00:30:24.422650+0000","last_unstale":"2024-09-16T00:31:23.442132+0000","last_undegraded":"2024-09-16T00:31:23.442132+0000","last_fullsized":"2024-09-16T00:31:23.442132+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:00.211744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.443327+0000","last_change":"2024-09-16T00:30:33.604492+0000","last_active":"2024-09-16T00:31:23.443327+0000","last_peered":"2024-09-16T00:31:23.443327+0000","last_clean":"2024-09-16T00:31:23.443327+0000","last_became_active":"2024-09-16T00:30:33.604015+0000","last_became_peered":"2024-09-16T00:30:33.604015+0000","last_unstale":"2024-09-16T00:31:23.443327+0000","last_undegraded":"2024-09-16T00:31:23.443327+0000","last_fullsized":"2024-09-16T00:31:23.443327+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:40:22.516583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442253+0000","last_change":"2024-09-16T00:30:26.075517+0000","last_active":"2024-09-16T00:31:23.442253+0000","last_peered":"2024-09-16T00:31:23.442253+0000","last_clean":"2024-09-16T00:31:23.442253+0000","last_became_active":"2024-09-16T00:30:26.075113+0000","last_became_peered":"2024-09-16T00:30:26.075113+0000","last_unstale":"2024-09-16T00:31:23.442253+0000","last_undegraded":"2024-09-16T00:31:23.442253+0000","last_fullsized":"2024-09-16T00:31:23.442253+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:11.427010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442390+0000","last_change":"2024-09-16T00:30:23.933639+0000","last_active":"2024-09-16T00:31:23.442390+0000","last_peered":"2024-09-16T00:31:23.442390+0000","last_clean":"2024-09-16T00:31:23.442390+0000","last_became_active":"2024-09-16T00:30:23.933322+0000","last_became_peered":"2024-09-16T00:30:23.933322+0000","last_unstale":"2024-09-16T00:31:23.442390+0000","last_undegraded":"2024-09-16T00:31:23.442390+0000","last_fullsized":"2024-09-16T00:31:23.442390+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:59:59.082702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.439665+0000","last_change":"2024-09-16T00:30:33.148508+0000","last_active":"2024-09-16T00:31:23.439665+0000","last_peered":"2024-09-16T00:31:23.439665+0000","last_clean":"2024-09-16T00:31:23.439665+0000","last_became_active":"2024-09-16T00:30:33.148091+0000","last_became_peered":"2024-09-16T00:30:33.148091+0000","last_unstale":"2024-09-16T00:31:23.439665+0000","last_undegraded":"2024-09-16T00:31:23.439665+0000","last_fullsized":"2024-09-16T00:31:23.439665+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:50.052526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.440040+0000","last_change":"2024-09-16T00:30:26.080374+0000","last_active":"2024-09-16T00:31:23.440040+0000","last_peered":"2024-09-16T00:31:23.440040+0000","last_clean":"2024-09-16T00:31:23.440040+0000","last_became_active":"2024-09-16T00:30:26.080044+0000","last_became_peered":"2024-09-16T00:30:26.080044+0000","last_unstale":"2024-09-16T00:31:23.440040+0000","last_undegraded":"2024-09-16T00:31:23.440040+0000","last_fullsized":"2024-09-16T00:31:23.440040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:38:36.034903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":90,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:53.404778+0000","last_change":"2024-09-16T00:30:24.422137+0000","last_active":"2024-09-16T00:31:53.404778+0000","last_peered":"2024-09-16T00:31:53.404778+0000","last_clean":"2024-09-16T00:31:53.404778+0000","last_became_active":"2024-09-16T00:30:24.421778+0000","last_became_peered":"2024-09-16T00:30:24.421778+0000","last_unstale":"2024-09-16T00:31:53.404778+0000","last_undegraded":"2024-09-16T00:31:53.404778+0000","last_fullsized":"2024-09-16T00:31:53.404778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:54:21.964099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.439834+0000","last_change":"2024-09-16T00:30:33.604263+0000","last_active":"2024-09-16T00:31:23.439834+0000","last_peered":"2024-09-16T00:31:23.439834+0000","last_clean":"2024-09-16T00:31:23.439834+0000","last_became_active":"2024-09-16T00:30:33.603804+0000","last_became_peered":"2024-09-16T00:30:33.603804+0000","last_unstale":"2024-09-16T00:31:23.439834+0000","last_undegraded":"2024-09-16T00:31:23.439834+0000","last_fullsized":"2024-09-16T00:31:23.439834+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:08:37.590827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.439285+0000","last_change":"2024-09-16T00:30:26.082916+0000","last_active":"2024-09-16T00:31:23.439285+0000","last_peered":"2024-09-16T00:31:23.439285+0000","last_clean":"2024-09-16T00:31:23.439285+0000","last_became_active":"2024-09-16T00:30:26.082606+0000","last_became_peered":"2024-09-16T00:30:26.082606+0000","last_unstale":"2024-09-16T00:31:23.439285+0000","last_undegraded":"2024-09-16T00:31:23.439285+0000","last_fullsized":"2024-09-16T00:31:23.439285+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:56.777640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.441880+0000","last_change":"2024-09-16T00:30:23.937602+0000","last_active":"2024-09-16T00:31:23.441880+0000","last_peered":"2024-09-16T00:31:23.441880+0000","last_clean":"2024-09-16T00:31:23.441880+0000","last_became_active":"2024-09-16T00:30:23.937261+0000","last_became_peered":"2024-09-16T00:30:23.937261+0000","last_unstale":"2024-09-16T00:31:23.441880+0000","last_undegraded":"2024-09-16T00:31:23.441880+0000","last_fullsized":"2024-09-16T00:31:23.441880+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:24.698758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.440724+0000","last_change":"2024-09-16T00:30:33.605536+0000","last_active":"2024-09-16T00:31:23.440724+0000","last_peered":"2024-09-16T00:31:23.440724+0000","last_clean":"2024-09-16T00:31:23.440724+0000","last_became_active":"2024-09-16T00:30:33.605073+0000","last_became_peered":"2024-09-16T00:30:33.605073+0000","last_unstale":"2024-09-16T00:31:23.440724+0000","last_undegraded":"2024-09-16T00:31:23.440724+0000","last_fullsized":"2024-09-16T00:31:23.440724+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:11.143052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.444986+0000","last_change":"2024-09-16T00:30:26.084507+0000","last_active":"2024-09-16T00:31:23.444986+0000","last_peered":"2024-09-16T00:31:23.444986+0000","last_clean":"2024-09-16T00:31:23.444986+0000","last_became_active":"2024-09-16T00:30:26.084152+0000","last_became_peered":"2024-09-16T00:30:26.084152+0000","last_unstale":"2024-09-16T00:31:23.444986+0000","last_undegraded":"2024-09-16T00:31:23.444986+0000","last_fullsized":"2024-09-16T00:31:23.444986+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:27:21.557409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.445869+0000","last_change":"2024-09-16T00:30:23.932686+0000","last_active":"2024-09-16T00:31:23.445869+0000","last_peered":"2024-09-16T00:31:23.445869+0000","last_clean":"2024-09-16T00:31:23.445869+0000","last_became_active":"2024-09-16T00:30:23.932343+0000","last_became_peered":"2024-09-16T00:30:23.932343+0000","last_unstale":"2024-09-16T00:31:23.445869+0000","last_undegraded":"2024-09-16T00:31:23.445869+0000","last_fullsized":"2024-09-16T00:31:23.445869+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:13:44.185712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":347,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442083+0000","last_change":"2024-09-16T00:29:11.143642+0000","last_active":"2024-09-16T00:31:23.442083+0000","last_peered":"2024-09-16T00:31:23.442083+0000","last_clean":"2024-09-16T00:31:23.442083+0000","last_became_active":"2024-09-16T00:29:11.143227+0000","last_became_peered":"2024-09-16T00:29:11.143227+0000","last_unstale":"2024-09-16T00:31:23.442083+0000","last_undegraded":"2024-09-16T00:31:23.442083+0000","last_fullsized":"2024-09-16T00:31:23.442083+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:25:42.361316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:25:42.361316+0000","last_clean_scrub_stamp":"2024-09-16T00:25:42.361316+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:29:12.740474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442899+0000","last_change":"2024-09-16T00:30:33.155555+0000","last_active":"2024-09-16T00:31:23.442899+0000","last_peered":"2024-09-16T00:31:23.442899+0000","last_clean":"2024-09-16T00:31:23.442899+0000","last_became_active":"2024-09-16T00:30:33.155170+0000","last_became_peered":"2024-09-16T00:30:33.155170+0000","last_unstale":"2024-09-16T00:31:23.442899+0000","last_undegraded":"2024-09-16T00:31:23.442899+0000","last_fullsized":"2024-09-16T00:31:23.442899+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:34:46.214422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.443059+0000","last_change":"2024-09-16T00:30:26.076003+0000","last_active":"2024-09-16T00:31:23.443059+0000","last_peered":"2024-09-16T00:31:23.443059+0000","last_clean":"2024-09-16T00:31:23.443059+0000","last_became_active":"2024-09-16T00:30:26.075666+0000","last_became_peered":"2024-09-16T00:30:26.075666+0000","last_unstale":"2024-09-16T00:31:23.443059+0000","last_undegraded":"2024-09-16T00:31:23.443059+0000","last_fullsized":"2024-09-16T00:31:23.443059+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:30.113380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.444547+0000","last_change":"2024-09-16T00:30:24.587469+0000","last_active":"2024-09-16T00:31:23.444547+0000","last_peered":"2024-09-16T00:31:23.444547+0000","last_clean":"2024-09-16T00:31:23.444547+0000","last_became_active":"2024-09-16T00:30:24.587049+0000","last_became_peered":"2024-09-16T00:30:24.587049+0000","last_unstale":"2024-09-16T00:31:23.444547+0000","last_undegraded":"2024-09-16T00:31:23.444547+0000","last_fullsized":"2024-09-16T00:31:23.444547+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:17:45.934505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.440438+0000","last_change":"2024-09-16T00:30:33.145921+0000","last_active":"2024-09-16T00:31:23.440438+0000","last_peered":"2024-09-16T00:31:23.440438+0000","last_clean":"2024-09-16T00:31:23.440438+0000","last_became_active":"2024-09-16T00:30:33.145450+0000","last_became_peered":"2024-09-16T00:30:33.145450+0000","last_unstale":"2024-09-16T00:31:23.440438+0000","last_undegraded":"2024-09-16T00:31:23.440438+0000","last_fullsized":"2024-09-16T00:31:23.440438+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:53:51.698686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442375+0000","last_change":"2024-09-16T00:30:26.087940+0000","last_active":"2024-09-16T00:31:23.442375+0000","last_peered":"2024-09-16T00:31:23.442375+0000","last_clean":"2024-09-16T00:31:23.442375+0000","last_became_active":"2024-09-16T00:30:26.087599+0000","last_became_peered":"2024-09-16T00:30:26.087599+0000","last_unstale":"2024-09-16T00:31:23.442375+0000","last_undegraded":"2024-09-16T00:31:23.442375+0000","last_fullsized":"2024-09-16T00:31:23.442375+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:43.934839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.440967+0000","last_change":"2024-09-16T00:30:23.930986+0000","last_active":"2024-09-16T00:31:23.440967+0000","last_peered":"2024-09-16T00:31:23.440967+0000","last_clean":"2024-09-16T00:31:23.440967+0000","last_became_active":"2024-09-16T00:30:23.930588+0000","last_became_peered":"2024-09-16T00:30:23.930588+0000","last_unstale":"2024-09-16T00:31:23.440967+0000","last_undegraded":"2024-09-16T00:31:23.440967+0000","last_fullsized":"2024-09-16T00:31:23.440967+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:04:41.902358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.576963+0000","last_change":"2024-09-16T00:30:33.154894+0000","last_active":"2024-09-16T00:31:23.576963+0000","last_peered":"2024-09-16T00:31:23.576963+0000","last_clean":"2024-09-16T00:31:23.576963+0000","last_became_active":"2024-09-16T00:30:33.154490+0000","last_became_peered":"2024-09-16T00:30:33.154490+0000","last_unstale":"2024-09-16T00:31:23.576963+0000","last_undegraded":"2024-09-16T00:31:23.576963+0000","last_fullsized":"2024-09-16T00:31:23.576963+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:59:53.854212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442565+0000","last_change":"2024-09-16T00:30:26.077069+0000","last_active":"2024-09-16T00:31:23.442565+0000","last_peered":"2024-09-16T00:31:23.442565+0000","last_clean":"2024-09-16T00:31:23.442565+0000","last_became_active":"2024-09-16T00:30:26.076718+0000","last_became_peered":"2024-09-16T00:30:26.076718+0000","last_unstale":"2024-09-16T00:31:23.442565+0000","last_undegraded":"2024-09-16T00:31:23.442565+0000","last_fullsized":"2024-09-16T00:31:23.442565+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:25:37.541704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.439926+0000","last_change":"2024-09-16T00:30:23.932081+0000","last_active":"2024-09-16T00:31:23.439926+0000","last_peered":"2024-09-16T00:31:23.439926+0000","last_clean":"2024-09-16T00:31:23.439926+0000","last_became_active":"2024-09-16T00:30:23.931729+0000","last_became_peered":"2024-09-16T00:30:23.931729+0000","last_unstale":"2024-09-16T00:31:23.439926+0000","last_undegraded":"2024-09-16T00:31:23.439926+0000","last_fullsized":"2024-09-16T00:31:23.439926+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:20:05.178580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.439866+0000","last_change":"2024-09-16T00:30:33.147738+0000","last_active":"2024-09-16T00:31:23.439866+0000","last_peered":"2024-09-16T00:31:23.439866+0000","last_clean":"2024-09-16T00:31:23.439866+0000","last_became_active":"2024-09-16T00:30:33.147326+0000","last_became_peered":"2024-09-16T00:30:33.147326+0000","last_unstale":"2024-09-16T00:31:23.439866+0000","last_undegraded":"2024-09-16T00:31:23.439866+0000","last_fullsized":"2024-09-16T00:31:23.439866+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:44:55.123980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.443143+0000","last_change":"2024-09-16T00:30:26.079312+0000","last_active":"2024-09-16T00:31:23.443143+0000","last_peered":"2024-09-16T00:31:23.443143+0000","last_clean":"2024-09-16T00:31:23.443143+0000","last_became_active":"2024-09-16T00:30:26.078986+0000","last_became_peered":"2024-09-16T00:30:26.078986+0000","last_unstale":"2024-09-16T00:31:23.443143+0000","last_undegraded":"2024-09-16T00:31:23.443143+0000","last_fullsized":"2024-09-16T00:31:23.443143+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:02:06.602850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.452513+0000","last_change":"2024-09-16T00:31:23.452102+0000","last_active":"2024-09-16T00:31:23.452513+0000","last_peered":"2024-09-16T00:31:23.452513+0000","last_clean":"2024-09-16T00:31:23.452513+0000","last_became_active":"2024-09-16T00:31:23.445974+0000","last_became_peered":"2024-09-16T00:31:23.445974+0000","last_unstale":"2024-09-16T00:31:23.452513+0000","last_undegraded":"2024-09-16T00:31:23.452513+0000","last_fullsized":"2024-09-16T00:31:23.452513+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:52.314494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.439965+0000","last_change":"2024-09-16T00:30:33.149154+0000","last_active":"2024-09-16T00:31:23.439965+0000","last_peered":"2024-09-16T00:31:23.439965+0000","last_clean":"2024-09-16T00:31:23.439965+0000","last_became_active":"2024-09-16T00:30:33.148763+0000","last_became_peered":"2024-09-16T00:30:33.148763+0000","last_unstale":"2024-09-16T00:31:23.439965+0000","last_undegraded":"2024-09-16T00:31:23.439965+0000","last_fullsized":"2024-09-16T00:31:23.439965+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:58:05.884919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.440188+0000","last_change":"2024-09-16T00:30:26.086905+0000","last_active":"2024-09-16T00:31:23.440188+0000","last_peered":"2024-09-16T00:31:23.440188+0000","last_clean":"2024-09-16T00:31:23.440188+0000","last_became_active":"2024-09-16T00:30:26.086519+0000","last_became_peered":"2024-09-16T00:30:26.086519+0000","last_unstale":"2024-09-16T00:31:23.440188+0000","last_undegraded":"2024-09-16T00:31:23.440188+0000","last_fullsized":"2024-09-16T00:31:23.440188+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:51:08.787679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442314+0000","last_change":"2024-09-16T00:30:24.422893+0000","last_active":"2024-09-16T00:31:23.442314+0000","last_peered":"2024-09-16T00:31:23.442314+0000","last_clean":"2024-09-16T00:31:23.442314+0000","last_became_active":"2024-09-16T00:30:24.422073+0000","last_became_peered":"2024-09-16T00:30:24.422073+0000","last_unstale":"2024-09-16T00:31:23.442314+0000","last_undegraded":"2024-09-16T00:31:23.442314+0000","last_fullsized":"2024-09-16T00:31:23.442314+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:15:11.375957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.439835+0000","last_change":"2024-09-16T00:30:33.152582+0000","last_active":"2024-09-16T00:31:23.439835+0000","last_peered":"2024-09-16T00:31:23.439835+0000","last_clean":"2024-09-16T00:31:23.439835+0000","last_became_active":"2024-09-16T00:30:33.152180+0000","last_became_peered":"2024-09-16T00:30:33.152180+0000","last_unstale":"2024-09-16T00:31:23.439835+0000","last_undegraded":"2024-09-16T00:31:23.439835+0000","last_fullsized":"2024-09-16T00:31:23.439835+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:41:10.499391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.441715+0000","last_change":"2024-09-16T00:30:26.086052+0000","last_active":"2024-09-16T00:31:23.441715+0000","last_peered":"2024-09-16T00:31:23.441715+0000","last_clean":"2024-09-16T00:31:23.441715+0000","last_became_active":"2024-09-16T00:30:26.085756+0000","last_became_peered":"2024-09-16T00:30:26.085756+0000","last_unstale":"2024-09-16T00:31:23.441715+0000","last_undegraded":"2024-09-16T00:31:23.441715+0000","last_fullsized":"2024-09-16T00:31:23.441715+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:52.582166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":89,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:51.402232+0000","last_change":"2024-09-16T00:30:23.933528+0000","last_active":"2024-09-16T00:31:51.402232+0000","last_peered":"2024-09-16T00:31:51.402232+0000","last_clean":"2024-09-16T00:31:51.402232+0000","last_became_active":"2024-09-16T00:30:23.933186+0000","last_became_peered":"2024-09-16T00:30:23.933186+0000","last_unstale":"2024-09-16T00:31:51.402232+0000","last_undegraded":"2024-09-16T00:31:51.402232+0000","last_fullsized":"2024-09-16T00:31:51.402232+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:11:00.489676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.578399+0000","last_change":"2024-09-16T00:31:23.578056+0000","last_active":"2024-09-16T00:31:23.578399+0000","last_peered":"2024-09-16T00:31:23.578399+0000","last_clean":"2024-09-16T00:31:23.578399+0000","last_became_active":"2024-09-16T00:31:23.577721+0000","last_became_peered":"2024-09-16T00:31:23.577721+0000","last_unstale":"2024-09-16T00:31:23.578399+0000","last_undegraded":"2024-09-16T00:31:23.578399+0000","last_fullsized":"2024-09-16T00:31:23.578399+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:53.632440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442597+0000","last_change":"2024-09-16T00:30:26.084509+0000","last_active":"2024-09-16T00:31:23.442597+0000","last_peered":"2024-09-16T00:31:23.442597+0000","last_clean":"2024-09-16T00:31:23.442597+0000","last_became_active":"2024-09-16T00:30:26.084188+0000","last_became_peered":"2024-09-16T00:30:26.084188+0000","last_unstale":"2024-09-16T00:31:23.442597+0000","last_undegraded":"2024-09-16T00:31:23.442597+0000","last_fullsized":"2024-09-16T00:31:23.442597+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:21:48.981277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.440467+0000","last_change":"2024-09-16T00:30:23.933499+0000","last_active":"2024-09-16T00:31:23.440467+0000","last_peered":"2024-09-16T00:31:23.440467+0000","last_clean":"2024-09-16T00:31:23.440467+0000","last_became_active":"2024-09-16T00:30:23.933082+0000","last_became_peered":"2024-09-16T00:30:23.933082+0000","last_unstale":"2024-09-16T00:31:23.440467+0000","last_undegraded":"2024-09-16T00:31:23.440467+0000","last_fullsized":"2024-09-16T00:31:23.440467+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:42:18.998433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442466+0000","last_change":"2024-09-16T00:30:33.154748+0000","last_active":"2024-09-16T00:31:23.442466+0000","last_peered":"2024-09-16T00:31:23.442466+0000","last_clean":"2024-09-16T00:31:23.442466+0000","last_became_active":"2024-09-16T00:30:33.154305+0000","last_became_peered":"2024-09-16T00:30:33.154305+0000","last_unstale":"2024-09-16T00:31:23.442466+0000","last_undegraded":"2024-09-16T00:31:23.442466+0000","last_fullsized":"2024-09-16T00:31:23.442466+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:57:34.651844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.439471+0000","last_change":"2024-09-16T00:30:26.080517+0000","last_active":"2024-09-16T00:31:23.439471+0000","last_peered":"2024-09-16T00:31:23.439471+0000","last_clean":"2024-09-16T00:31:23.439471+0000","last_became_active":"2024-09-16T00:30:26.080197+0000","last_became_peered":"2024-09-16T00:30:26.080197+0000","last_unstale":"2024-09-16T00:31:23.439471+0000","last_undegraded":"2024-09-16T00:31:23.439471+0000","last_fullsized":"2024-09-16T00:31:23.439471+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:15.508562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.457335+0000","last_change":"2024-09-16T00:31:23.457047+0000","last_active":"2024-09-16T00:31:23.457335+0000","last_peered":"2024-09-16T00:31:23.457335+0000","last_clean":"2024-09-16T00:31:23.457335+0000","last_became_active":"2024-09-16T00:31:23.447014+0000","last_became_peered":"2024-09-16T00:31:23.447014+0000","last_unstale":"2024-09-16T00:31:23.457335+0000","last_undegraded":"2024-09-16T00:31:23.457335+0000","last_fullsized":"2024-09-16T00:31:23.457335+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:34:42.040228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.440632+0000","last_change":"2024-09-16T00:30:33.145803+0000","last_active":"2024-09-16T00:31:23.440632+0000","last_peered":"2024-09-16T00:31:23.440632+0000","last_clean":"2024-09-16T00:31:23.440632+0000","last_became_active":"2024-09-16T00:30:33.145366+0000","last_became_peered":"2024-09-16T00:30:33.145366+0000","last_unstale":"2024-09-16T00:31:23.440632+0000","last_undegraded":"2024-09-16T00:31:23.440632+0000","last_fullsized":"2024-09-16T00:31:23.440632+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:47:28.689305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.444640+0000","last_change":"2024-09-16T00:30:26.079415+0000","last_active":"2024-09-16T00:31:23.444640+0000","last_peered":"2024-09-16T00:31:23.444640+0000","last_clean":"2024-09-16T00:31:23.444640+0000","last_became_active":"2024-09-16T00:30:26.079075+0000","last_became_peered":"2024-09-16T00:30:26.079075+0000","last_unstale":"2024-09-16T00:31:23.444640+0000","last_undegraded":"2024-09-16T00:31:23.444640+0000","last_fullsized":"2024-09-16T00:31:23.444640+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:18:18.236342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'32","reported_seq":176,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:53.405164+0000","last_change":"2024-09-16T00:30:23.933172+0000","last_active":"2024-09-16T00:31:53.405164+0000","last_peered":"2024-09-16T00:31:53.405164+0000","last_clean":"2024-09-16T00:31:53.405164+0000","last_became_active":"2024-09-16T00:30:23.932492+0000","last_became_peered":"2024-09-16T00:30:23.932492+0000","last_unstale":"2024-09-16T00:31:53.405164+0000","last_undegraded":"2024-09-16T00:31:53.405164+0000","last_fullsized":"2024-09-16T00:31:53.405164+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:29.169448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62076,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":110,"num_read_kb":117,"num_write":33,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.446248+0000","last_change":"2024-09-16T00:31:23.445673+0000","last_active":"2024-09-16T00:31:23.446248+0000","last_peered":"2024-09-16T00:31:23.446248+0000","last_clean":"2024-09-16T00:31:23.446248+0000","last_became_active":"2024-09-16T00:31:23.445299+0000","last_became_peered":"2024-09-16T00:31:23.445299+0000","last_unstale":"2024-09-16T00:31:23.446248+0000","last_undegraded":"2024-09-16T00:31:23.446248+0000","last_fullsized":"2024-09-16T00:31:23.446248+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:07:44.235340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.440745+0000","last_change":"2024-09-16T00:30:26.085419+0000","last_active":"2024-09-16T00:31:23.440745+0000","last_peered":"2024-09-16T00:31:23.440745+0000","last_clean":"2024-09-16T00:31:23.440745+0000","last_became_active":"2024-09-16T00:30:26.085103+0000","last_became_peered":"2024-09-16T00:30:26.085103+0000","last_unstale":"2024-09-16T00:31:23.440745+0000","last_undegraded":"2024-09-16T00:31:23.440745+0000","last_fullsized":"2024-09-16T00:31:23.440745+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:56:26.352779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.440913+0000","last_change":"2024-09-16T00:30:24.422646+0000","last_active":"2024-09-16T00:31:23.440913+0000","last_peered":"2024-09-16T00:31:23.440913+0000","last_clean":"2024-09-16T00:31:23.440913+0000","last_became_active":"2024-09-16T00:30:24.422065+0000","last_became_peered":"2024-09-16T00:30:24.422065+0000","last_unstale":"2024-09-16T00:31:23.440913+0000","last_undegraded":"2024-09-16T00:31:23.440913+0000","last_fullsized":"2024-09-16T00:31:23.440913+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:53:12.401613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.445491+0000","last_change":"2024-09-16T00:30:33.147692+0000","last_active":"2024-09-16T00:31:23.445491+0000","last_peered":"2024-09-16T00:31:23.445491+0000","last_clean":"2024-09-16T00:31:23.445491+0000","last_became_active":"2024-09-16T00:30:33.147115+0000","last_became_peered":"2024-09-16T00:30:33.147115+0000","last_unstale":"2024-09-16T00:31:23.445491+0000","last_undegraded":"2024-09-16T00:31:23.445491+0000","last_fullsized":"2024-09-16T00:31:23.445491+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:44:04.536894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.577333+0000","last_change":"2024-09-16T00:30:26.088427+0000","last_active":"2024-09-16T00:31:23.577333+0000","last_peered":"2024-09-16T00:31:23.577333+0000","last_clean":"2024-09-16T00:31:23.577333+0000","last_became_active":"2024-09-16T00:30:26.086454+0000","last_became_peered":"2024-09-16T00:30:26.086454+0000","last_unstale":"2024-09-16T00:31:23.577333+0000","last_undegraded":"2024-09-16T00:31:23.577333+0000","last_fullsized":"2024-09-16T00:31:23.577333+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:54:29.679959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.453484+0000","last_change":"2024-09-16T00:31:23.453484+0000","last_active":"2024-09-16T00:31:23.453484+0000","last_peered":"2024-09-16T00:31:23.453484+0000","last_clean":"2024-09-16T00:31:23.453484+0000","last_became_active":"2024-09-16T00:31:23.446700+0000","last_became_peered":"2024-09-16T00:31:23.446700+0000","last_unstale":"2024-09-16T00:31:23.453484+0000","last_undegraded":"2024-09-16T00:31:23.453484+0000","last_fullsized":"2024-09-16T00:31:23.453484+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:43:26.465657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.441598+0000","last_change":"2024-09-16T00:30:33.156510+0000","last_active":"2024-09-16T00:31:23.441598+0000","last_peered":"2024-09-16T00:31:23.441598+0000","last_clean":"2024-09-16T00:31:23.441598+0000","last_became_active":"2024-09-16T00:30:33.156088+0000","last_became_peered":"2024-09-16T00:30:33.156088+0000","last_unstale":"2024-09-16T00:31:23.441598+0000","last_undegraded":"2024-09-16T00:31:23.441598+0000","last_fullsized":"2024-09-16T00:31:23.441598+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:34.230901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.440674+0000","last_change":"2024-09-16T00:30:26.082598+0000","last_active":"2024-09-16T00:31:23.440674+0000","last_peered":"2024-09-16T00:31:23.440674+0000","last_clean":"2024-09-16T00:31:23.440674+0000","last_became_active":"2024-09-16T00:30:26.082286+0000","last_became_peered":"2024-09-16T00:30:26.082286+0000","last_unstale":"2024-09-16T00:31:23.440674+0000","last_undegraded":"2024-09-16T00:31:23.440674+0000","last_fullsized":"2024-09-16T00:31:23.440674+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:22:18.579149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":88,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:52.403582+0000","last_change":"2024-09-16T00:30:23.931097+0000","last_active":"2024-09-16T00:31:52.403582+0000","last_peered":"2024-09-16T00:31:52.403582+0000","last_clean":"2024-09-16T00:31:52.403582+0000","last_became_active":"2024-09-16T00:30:23.930719+0000","last_became_peered":"2024-09-16T00:30:23.930719+0000","last_unstale":"2024-09-16T00:31:52.403582+0000","last_undegraded":"2024-09-16T00:31:52.403582+0000","last_fullsized":"2024-09-16T00:31:52.403582+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:13:47.887754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":51,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.444716+0000","last_change":"2024-09-16T00:30:33.142581+0000","last_active":"2024-09-16T00:31:23.444716+0000","last_peered":"2024-09-16T00:31:23.444716+0000","last_clean":"2024-09-16T00:31:23.444716+0000","last_became_active":"2024-09-16T00:30:33.142263+0000","last_became_peered":"2024-09-16T00:30:33.142263+0000","last_unstale":"2024-09-16T00:31:23.444716+0000","last_undegraded":"2024-09-16T00:31:23.444716+0000","last_fullsized":"2024-09-16T00:31:23.444716+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:10:06.456011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.577032+0000","last_change":"2024-09-16T00:30:33.152084+0000","last_active":"2024-09-16T00:31:23.577032+0000","last_peered":"2024-09-16T00:31:23.577032+0000","last_clean":"2024-09-16T00:31:23.577032+0000","last_became_active":"2024-09-16T00:30:33.151681+0000","last_became_peered":"2024-09-16T00:30:33.151681+0000","last_unstale":"2024-09-16T00:31:23.577032+0000","last_undegraded":"2024-09-16T00:31:23.577032+0000","last_fullsized":"2024-09-16T00:31:23.577032+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:36.966498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.441760+0000","last_change":"2024-09-16T00:30:26.084200+0000","last_active":"2024-09-16T00:31:23.441760+0000","last_peered":"2024-09-16T00:31:23.441760+0000","last_clean":"2024-09-16T00:31:23.441760+0000","last_became_active":"2024-09-16T00:30:26.083858+0000","last_became_peered":"2024-09-16T00:30:26.083858+0000","last_unstale":"2024-09-16T00:31:23.441760+0000","last_undegraded":"2024-09-16T00:31:23.441760+0000","last_fullsized":"2024-09-16T00:31:23.441760+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:42.712624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.440233+0000","last_change":"2024-09-16T00:30:23.932399+0000","last_active":"2024-09-16T00:31:23.440233+0000","last_peered":"2024-09-16T00:31:23.440233+0000","last_clean":"2024-09-16T00:31:23.440233+0000","last_became_active":"2024-09-16T00:30:23.931996+0000","last_became_peered":"2024-09-16T00:30:23.931996+0000","last_unstale":"2024-09-16T00:31:23.440233+0000","last_undegraded":"2024-09-16T00:31:23.440233+0000","last_fullsized":"2024-09-16T00:31:23.440233+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:47:09.634507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442880+0000","last_change":"2024-09-16T00:30:33.154563+0000","last_active":"2024-09-16T00:31:23.442880+0000","last_peered":"2024-09-16T00:31:23.442880+0000","last_clean":"2024-09-16T00:31:23.442880+0000","last_became_active":"2024-09-16T00:30:33.154105+0000","last_became_peered":"2024-09-16T00:30:33.154105+0000","last_unstale":"2024-09-16T00:31:23.442880+0000","last_undegraded":"2024-09-16T00:31:23.442880+0000","last_fullsized":"2024-09-16T00:31:23.442880+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:59.966255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.444616+0000","last_change":"2024-09-16T00:30:26.075126+0000","last_active":"2024-09-16T00:31:23.444616+0000","last_peered":"2024-09-16T00:31:23.444616+0000","last_clean":"2024-09-16T00:31:23.444616+0000","last_became_active":"2024-09-16T00:30:26.074819+0000","last_became_peered":"2024-09-16T00:30:26.074819+0000","last_unstale":"2024-09-16T00:31:23.444616+0000","last_undegraded":"2024-09-16T00:31:23.444616+0000","last_fullsized":"2024-09-16T00:31:23.444616+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:36.996346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.439321+0000","last_change":"2024-09-16T00:30:23.929943+0000","last_active":"2024-09-16T00:31:23.439321+0000","last_peered":"2024-09-16T00:31:23.439321+0000","last_clean":"2024-09-16T00:31:23.439321+0000","last_became_active":"2024-09-16T00:30:23.929578+0000","last_became_peered":"2024-09-16T00:30:23.929578+0000","last_unstale":"2024-09-16T00:31:23.439321+0000","last_undegraded":"2024-09-16T00:31:23.439321+0000","last_fullsized":"2024-09-16T00:31:23.439321+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:05:12.505543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442749+0000","last_change":"2024-09-16T00:30:23.935758+0000","last_active":"2024-09-16T00:31:23.442749+0000","last_peered":"2024-09-16T00:31:23.442749+0000","last_clean":"2024-09-16T00:31:23.442749+0000","last_became_active":"2024-09-16T00:30:23.935321+0000","last_became_peered":"2024-09-16T00:30:23.935321+0000","last_unstale":"2024-09-16T00:31:23.442749+0000","last_undegraded":"2024-09-16T00:31:23.442749+0000","last_fullsized":"2024-09-16T00:31:23.442749+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:35:40.513435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.440500+0000","last_change":"2024-09-16T00:30:26.086316+0000","last_active":"2024-09-16T00:31:23.440500+0000","last_peered":"2024-09-16T00:31:23.440500+0000","last_clean":"2024-09-16T00:31:23.440500+0000","last_became_active":"2024-09-16T00:30:26.085800+0000","last_became_peered":"2024-09-16T00:30:26.085800+0000","last_unstale":"2024-09-16T00:31:23.440500+0000","last_undegraded":"2024-09-16T00:31:23.440500+0000","last_fullsized":"2024-09-16T00:31:23.440500+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:05:36.277127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.444019+0000","last_change":"2024-09-16T00:30:33.155665+0000","last_active":"2024-09-16T00:31:23.444019+0000","last_peered":"2024-09-16T00:31:23.444019+0000","last_clean":"2024-09-16T00:31:23.444019+0000","last_became_active":"2024-09-16T00:30:33.155253+0000","last_became_peered":"2024-09-16T00:30:33.155253+0000","last_unstale":"2024-09-16T00:31:23.444019+0000","last_undegraded":"2024-09-16T00:31:23.444019+0000","last_fullsized":"2024-09-16T00:31:23.444019+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:37.545232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":93,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:51.402659+0000","last_change":"2024-09-16T00:30:23.934487+0000","last_active":"2024-09-16T00:31:51.402659+0000","last_peered":"2024-09-16T00:31:51.402659+0000","last_clean":"2024-09-16T00:31:51.402659+0000","last_became_active":"2024-09-16T00:30:23.934137+0000","last_became_peered":"2024-09-16T00:30:23.934137+0000","last_unstale":"2024-09-16T00:31:51.402659+0000","last_undegraded":"2024-09-16T00:31:51.402659+0000","last_fullsized":"2024-09-16T00:31:51.402659+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:14:33.088776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":53,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.443467+0000","last_change":"2024-09-16T00:30:26.077533+0000","last_active":"2024-09-16T00:31:23.443467+0000","last_peered":"2024-09-16T00:31:23.443467+0000","last_clean":"2024-09-16T00:31:23.443467+0000","last_became_active":"2024-09-16T00:30:26.076443+0000","last_became_peered":"2024-09-16T00:30:26.076443+0000","last_unstale":"2024-09-16T00:31:23.443467+0000","last_undegraded":"2024-09-16T00:31:23.443467+0000","last_fullsized":"2024-09-16T00:31:23.443467+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:39:17.495521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.443290+0000","last_change":"2024-09-16T00:30:33.146123+0000","last_active":"2024-09-16T00:31:23.443290+0000","last_peered":"2024-09-16T00:31:23.443290+0000","last_clean":"2024-09-16T00:31:23.443290+0000","last_became_active":"2024-09-16T00:30:33.145716+0000","last_became_peered":"2024-09-16T00:30:33.145716+0000","last_unstale":"2024-09-16T00:31:23.443290+0000","last_undegraded":"2024-09-16T00:31:23.443290+0000","last_fullsized":"2024-09-16T00:31:23.443290+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:14:27.281367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442159+0000","last_change":"2024-09-16T00:30:26.083155+0000","last_active":"2024-09-16T00:31:23.442159+0000","last_peered":"2024-09-16T00:31:23.442159+0000","last_clean":"2024-09-16T00:31:23.442159+0000","last_became_active":"2024-09-16T00:30:26.082782+0000","last_became_peered":"2024-09-16T00:30:26.082782+0000","last_unstale":"2024-09-16T00:31:23.442159+0000","last_undegraded":"2024-09-16T00:31:23.442159+0000","last_fullsized":"2024-09-16T00:31:23.442159+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:15.929343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.447453+0000","last_change":"2024-09-16T00:31:23.447453+0000","last_active":"2024-09-16T00:31:23.447453+0000","last_peered":"2024-09-16T00:31:23.447453+0000","last_clean":"2024-09-16T00:31:23.447453+0000","last_became_active":"2024-09-16T00:31:23.445507+0000","last_became_peered":"2024-09-16T00:31:23.445507+0000","last_unstale":"2024-09-16T00:31:23.447453+0000","last_undegraded":"2024-09-16T00:31:23.447453+0000","last_fullsized":"2024-09-16T00:31:23.447453+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:18:50.885809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442695+0000","last_change":"2024-09-16T00:30:26.076252+0000","last_active":"2024-09-16T00:31:23.442695+0000","last_peered":"2024-09-16T00:31:23.442695+0000","last_clean":"2024-09-16T00:31:23.442695+0000","last_became_active":"2024-09-16T00:30:26.075922+0000","last_became_peered":"2024-09-16T00:30:26.075922+0000","last_unstale":"2024-09-16T00:31:23.442695+0000","last_undegraded":"2024-09-16T00:31:23.442695+0000","last_fullsized":"2024-09-16T00:31:23.442695+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:28:04.822817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.577020+0000","last_change":"2024-09-16T00:30:26.085123+0000","last_active":"2024-09-16T00:31:23.577020+0000","last_peered":"2024-09-16T00:31:23.577020+0000","last_clean":"2024-09-16T00:31:23.577020+0000","last_became_active":"2024-09-16T00:30:26.084782+0000","last_became_peered":"2024-09-16T00:30:26.084782+0000","last_unstale":"2024-09-16T00:31:23.577020+0000","last_undegraded":"2024-09-16T00:31:23.577020+0000","last_fullsized":"2024-09-16T00:31:23.577020+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:09:33.766340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.439865+0000","last_change":"2024-09-16T00:30:26.077048+0000","last_active":"2024-09-16T00:31:23.439865+0000","last_peered":"2024-09-16T00:31:23.439865+0000","last_clean":"2024-09-16T00:31:23.439865+0000","last_became_active":"2024-09-16T00:30:26.076719+0000","last_became_peered":"2024-09-16T00:30:26.076719+0000","last_unstale":"2024-09-16T00:31:23.439865+0000","last_undegraded":"2024-09-16T00:31:23.439865+0000","last_fullsized":"2024-09-16T00:31:23.439865+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:58.584246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442997+0000","last_change":"2024-09-16T00:30:26.078446+0000","last_active":"2024-09-16T00:31:23.442997+0000","last_peered":"2024-09-16T00:31:23.442997+0000","last_clean":"2024-09-16T00:31:23.442997+0000","last_became_active":"2024-09-16T00:30:26.078145+0000","last_became_peered":"2024-09-16T00:30:26.078145+0000","last_unstale":"2024-09-16T00:31:23.442997+0000","last_undegraded":"2024-09-16T00:31:23.442997+0000","last_fullsized":"2024-09-16T00:31:23.442997+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:52:28.739834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.444338+0000","last_change":"2024-09-16T00:30:26.075875+0000","last_active":"2024-09-16T00:31:23.444338+0000","last_peered":"2024-09-16T00:31:23.444338+0000","last_clean":"2024-09-16T00:31:23.444338+0000","last_became_active":"2024-09-16T00:30:26.075424+0000","last_became_peered":"2024-09-16T00:30:26.075424+0000","last_unstale":"2024-09-16T00:31:23.444338+0000","last_undegraded":"2024-09-16T00:31:23.444338+0000","last_fullsized":"2024-09-16T00:31:23.444338+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:21:07.958507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.445311+0000","last_change":"2024-09-16T00:30:26.077694+0000","last_active":"2024-09-16T00:31:23.445311+0000","last_peered":"2024-09-16T00:31:23.445311+0000","last_clean":"2024-09-16T00:31:23.445311+0000","last_became_active":"2024-09-16T00:30:26.077374+0000","last_became_peered":"2024-09-16T00:30:26.077374+0000","last_unstale":"2024-09-16T00:31:23.445311+0000","last_undegraded":"2024-09-16T00:31:23.445311+0000","last_fullsized":"2024-09-16T00:31:23.445311+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:16:30.822050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.439962+0000","last_change":"2024-09-16T00:30:26.085931+0000","last_active":"2024-09-16T00:31:23.439962+0000","last_peered":"2024-09-16T00:31:23.439962+0000","last_clean":"2024-09-16T00:31:23.439962+0000","last_became_active":"2024-09-16T00:30:26.085420+0000","last_became_peered":"2024-09-16T00:30:26.085420+0000","last_unstale":"2024-09-16T00:31:23.439962+0000","last_undegraded":"2024-09-16T00:31:23.439962+0000","last_fullsized":"2024-09-16T00:31:23.439962+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:20:36.019203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.444885+0000","last_change":"2024-09-16T00:30:26.074217+0000","last_active":"2024-09-16T00:31:23.444885+0000","last_peered":"2024-09-16T00:31:23.444885+0000","last_clean":"2024-09-16T00:31:23.444885+0000","last_became_active":"2024-09-16T00:30:26.073856+0000","last_became_peered":"2024-09-16T00:30:26.073856+0000","last_unstale":"2024-09-16T00:31:23.444885+0000","last_undegraded":"2024-09-16T00:31:23.444885+0000","last_fullsized":"2024-09-16T00:31:23.444885+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:53:57.078465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.440598+0000","last_change":"2024-09-16T00:30:26.084248+0000","last_active":"2024-09-16T00:31:23.440598+0000","last_peered":"2024-09-16T00:31:23.440598+0000","last_clean":"2024-09-16T00:31:23.440598+0000","last_became_active":"2024-09-16T00:30:26.083530+0000","last_became_peered":"2024-09-16T00:30:26.083530+0000","last_unstale":"2024-09-16T00:31:23.440598+0000","last_undegraded":"2024-09-16T00:31:23.440598+0000","last_fullsized":"2024-09-16T00:31:23.440598+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:16:49.483881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.439703+0000","last_change":"2024-09-16T00:30:26.080803+0000","last_active":"2024-09-16T00:31:23.439703+0000","last_peered":"2024-09-16T00:31:23.439703+0000","last_clean":"2024-09-16T00:31:23.439703+0000","last_became_active":"2024-09-16T00:30:26.080482+0000","last_became_peered":"2024-09-16T00:30:26.080482+0000","last_unstale":"2024-09-16T00:31:23.439703+0000","last_undegraded":"2024-09-16T00:31:23.439703+0000","last_fullsized":"2024-09-16T00:31:23.439703+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:35:41.741025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442585+0000","last_change":"2024-09-16T00:30:26.077680+0000","last_active":"2024-09-16T00:31:23.442585+0000","last_peered":"2024-09-16T00:31:23.442585+0000","last_clean":"2024-09-16T00:31:23.442585+0000","last_became_active":"2024-09-16T00:30:26.077371+0000","last_became_peered":"2024-09-16T00:30:26.077371+0000","last_unstale":"2024-09-16T00:31:23.442585+0000","last_undegraded":"2024-09-16T00:31:23.442585+0000","last_fullsized":"2024-09-16T00:31:23.442585+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:31:34.338617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.443453+0000","last_change":"2024-09-16T00:30:26.071531+0000","last_active":"2024-09-16T00:31:23.443453+0000","last_peered":"2024-09-16T00:31:23.443453+0000","last_clean":"2024-09-16T00:31:23.443453+0000","last_became_active":"2024-09-16T00:30:26.071189+0000","last_became_peered":"2024-09-16T00:30:26.071189+0000","last_unstale":"2024-09-16T00:31:23.443453+0000","last_undegraded":"2024-09-16T00:31:23.443453+0000","last_fullsized":"2024-09-16T00:31:23.443453+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:29:18.406272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442215+0000","last_change":"2024-09-16T00:30:26.085398+0000","last_active":"2024-09-16T00:31:23.442215+0000","last_peered":"2024-09-16T00:31:23.442215+0000","last_clean":"2024-09-16T00:31:23.442215+0000","last_became_active":"2024-09-16T00:30:26.085076+0000","last_became_peered":"2024-09-16T00:30:26.085076+0000","last_unstale":"2024-09-16T00:31:23.442215+0000","last_undegraded":"2024-09-16T00:31:23.442215+0000","last_fullsized":"2024-09-16T00:31:23.442215+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:20:20.864660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.443283+0000","last_change":"2024-09-16T00:30:26.080953+0000","last_active":"2024-09-16T00:31:23.443283+0000","last_peered":"2024-09-16T00:31:23.443283+0000","last_clean":"2024-09-16T00:31:23.443283+0000","last_became_active":"2024-09-16T00:30:26.080185+0000","last_became_peered":"2024-09-16T00:30:26.080185+0000","last_unstale":"2024-09-16T00:31:23.443283+0000","last_undegraded":"2024-09-16T00:31:23.443283+0000","last_fullsized":"2024-09-16T00:31:23.443283+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:56.769182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.440107+0000","last_change":"2024-09-16T00:30:26.079320+0000","last_active":"2024-09-16T00:31:23.440107+0000","last_peered":"2024-09-16T00:31:23.440107+0000","last_clean":"2024-09-16T00:31:23.440107+0000","last_became_active":"2024-09-16T00:30:26.078980+0000","last_became_peered":"2024-09-16T00:30:26.078980+0000","last_unstale":"2024-09-16T00:31:23.440107+0000","last_undegraded":"2024-09-16T00:31:23.440107+0000","last_fullsized":"2024-09-16T00:31:23.440107+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:03:15.386657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.576817+0000","last_change":"2024-09-16T00:30:26.084948+0000","last_active":"2024-09-16T00:31:23.576817+0000","last_peered":"2024-09-16T00:31:23.576817+0000","last_clean":"2024-09-16T00:31:23.576817+0000","last_became_active":"2024-09-16T00:30:26.084602+0000","last_became_peered":"2024-09-16T00:30:26.084602+0000","last_unstale":"2024-09-16T00:31:23.576817+0000","last_undegraded":"2024-09-16T00:31:23.576817+0000","last_fullsized":"2024-09-16T00:31:23.576817+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:01.709332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.576775+0000","last_change":"2024-09-16T00:30:26.080491+0000","last_active":"2024-09-16T00:31:23.576775+0000","last_peered":"2024-09-16T00:31:23.576775+0000","last_clean":"2024-09-16T00:31:23.576775+0000","last_became_active":"2024-09-16T00:30:26.080189+0000","last_became_peered":"2024-09-16T00:30:26.080189+0000","last_unstale":"2024-09-16T00:31:23.576775+0000","last_undegraded":"2024-09-16T00:31:23.576775+0000","last_fullsized":"2024-09-16T00:31:23.576775+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:38.358919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.439700+0000","last_change":"2024-09-16T00:30:26.077410+0000","last_active":"2024-09-16T00:31:23.439700+0000","last_peered":"2024-09-16T00:31:23.439700+0000","last_clean":"2024-09-16T00:31:23.439700+0000","last_became_active":"2024-09-16T00:30:26.076498+0000","last_became_peered":"2024-09-16T00:30:26.076498+0000","last_unstale":"2024-09-16T00:31:23.439700+0000","last_undegraded":"2024-09-16T00:31:23.439700+0000","last_fullsized":"2024-09-16T00:31:23.439700+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:54:17.712774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.445826+0000","last_change":"2024-09-16T00:30:26.074500+0000","last_active":"2024-09-16T00:31:23.445826+0000","last_peered":"2024-09-16T00:31:23.445826+0000","last_clean":"2024-09-16T00:31:23.445826+0000","last_became_active":"2024-09-16T00:30:26.074177+0000","last_became_peered":"2024-09-16T00:30:26.074177+0000","last_unstale":"2024-09-16T00:31:23.445826+0000","last_undegraded":"2024-09-16T00:31:23.445826+0000","last_fullsized":"2024-09-16T00:31:23.445826+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:43:13.792231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.443052+0000","last_change":"2024-09-16T00:30:26.078358+0000","last_active":"2024-09-16T00:31:23.443052+0000","last_peered":"2024-09-16T00:31:23.443052+0000","last_clean":"2024-09-16T00:31:23.443052+0000","last_became_active":"2024-09-16T00:30:26.077993+0000","last_became_peered":"2024-09-16T00:30:26.077993+0000","last_unstale":"2024-09-16T00:31:23.443052+0000","last_undegraded":"2024-09-16T00:31:23.443052+0000","last_fullsized":"2024-09-16T00:31:23.443052+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:25:10.711394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.446836+0000","last_change":"2024-09-16T00:31:23.446836+0000","last_active":"2024-09-16T00:31:23.446836+0000","last_peered":"2024-09-16T00:31:23.446836+0000","last_clean":"2024-09-16T00:31:23.446836+0000","last_became_active":"2024-09-16T00:31:23.446471+0000","last_became_peered":"2024-09-16T00:31:23.446471+0000","last_unstale":"2024-09-16T00:31:23.446836+0000","last_undegraded":"2024-09-16T00:31:23.446836+0000","last_fullsized":"2024-09-16T00:31:23.446836+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:08:34.887618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.441321+0000","last_change":"2024-09-16T00:30:26.086313+0000","last_active":"2024-09-16T00:31:23.441321+0000","last_peered":"2024-09-16T00:31:23.441321+0000","last_clean":"2024-09-16T00:31:23.441321+0000","last_became_active":"2024-09-16T00:30:26.085760+0000","last_became_peered":"2024-09-16T00:30:26.085760+0000","last_unstale":"2024-09-16T00:31:23.441321+0000","last_undegraded":"2024-09-16T00:31:23.441321+0000","last_fullsized":"2024-09-16T00:31:23.441321+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:40:38.236789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.444527+0000","last_change":"2024-09-16T00:30:26.085289+0000","last_active":"2024-09-16T00:31:23.444527+0000","last_peered":"2024-09-16T00:31:23.444527+0000","last_clean":"2024-09-16T00:31:23.444527+0000","last_became_active":"2024-09-16T00:30:26.084995+0000","last_became_peered":"2024-09-16T00:30:26.084995+0000","last_unstale":"2024-09-16T00:31:23.444527+0000","last_undegraded":"2024-09-16T00:31:23.444527+0000","last_fullsized":"2024-09-16T00:31:23.444527+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:45.667892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.443030+0000","last_change":"2024-09-16T00:30:26.088690+0000","last_active":"2024-09-16T00:31:23.443030+0000","last_peered":"2024-09-16T00:31:23.443030+0000","last_clean":"2024-09-16T00:31:23.443030+0000","last_became_active":"2024-09-16T00:30:26.087700+0000","last_became_peered":"2024-09-16T00:30:26.087700+0000","last_unstale":"2024-09-16T00:31:23.443030+0000","last_undegraded":"2024-09-16T00:31:23.443030+0000","last_fullsized":"2024-09-16T00:31:23.443030+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:39:51.445324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442664+0000","last_change":"2024-09-16T00:30:26.080215+0000","last_active":"2024-09-16T00:31:23.442664+0000","last_peered":"2024-09-16T00:31:23.442664+0000","last_clean":"2024-09-16T00:31:23.442664+0000","last_became_active":"2024-09-16T00:30:26.079038+0000","last_became_peered":"2024-09-16T00:30:26.079038+0000","last_unstale":"2024-09-16T00:31:23.442664+0000","last_undegraded":"2024-09-16T00:31:23.442664+0000","last_fullsized":"2024-09-16T00:31:23.442664+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:01.573960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442533+0000","last_change":"2024-09-16T00:30:26.080966+0000","last_active":"2024-09-16T00:31:23.442533+0000","last_peered":"2024-09-16T00:31:23.442533+0000","last_clean":"2024-09-16T00:31:23.442533+0000","last_became_active":"2024-09-16T00:30:26.080641+0000","last_became_peered":"2024-09-16T00:30:26.080641+0000","last_unstale":"2024-09-16T00:31:23.442533+0000","last_undegraded":"2024-09-16T00:31:23.442533+0000","last_fullsized":"2024-09-16T00:31:23.442533+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:33:01.538169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.445705+0000","last_change":"2024-09-16T00:30:26.072644+0000","last_active":"2024-09-16T00:31:23.445705+0000","last_peered":"2024-09-16T00:31:23.445705+0000","last_clean":"2024-09-16T00:31:23.445705+0000","last_became_active":"2024-09-16T00:30:26.072336+0000","last_became_peered":"2024-09-16T00:30:26.072336+0000","last_unstale":"2024-09-16T00:31:23.445705+0000","last_undegraded":"2024-09-16T00:31:23.445705+0000","last_fullsized":"2024-09-16T00:31:23.445705+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:44:45.760057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.577011+0000","last_change":"2024-09-16T00:30:26.079155+0000","last_active":"2024-09-16T00:31:23.577011+0000","last_peered":"2024-09-16T00:31:23.577011+0000","last_clean":"2024-09-16T00:31:23.577011+0000","last_became_active":"2024-09-16T00:30:26.078820+0000","last_became_peered":"2024-09-16T00:30:26.078820+0000","last_unstale":"2024-09-16T00:31:23.577011+0000","last_undegraded":"2024-09-16T00:31:23.577011+0000","last_fullsized":"2024-09-16T00:31:23.577011+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:35.071143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.577087+0000","last_change":"2024-09-16T00:30:26.081537+0000","last_active":"2024-09-16T00:31:23.577087+0000","last_peered":"2024-09-16T00:31:23.577087+0000","last_clean":"2024-09-16T00:31:23.577087+0000","last_became_active":"2024-09-16T00:30:26.080744+0000","last_became_peered":"2024-09-16T00:30:26.080744+0000","last_unstale":"2024-09-16T00:31:23.577087+0000","last_undegraded":"2024-09-16T00:31:23.577087+0000","last_fullsized":"2024-09-16T00:31:23.577087+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:46:38.179888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.443231+0000","last_change":"2024-09-16T00:30:26.081066+0000","last_active":"2024-09-16T00:31:23.443231+0000","last_peered":"2024-09-16T00:31:23.443231+0000","last_clean":"2024-09-16T00:31:23.443231+0000","last_became_active":"2024-09-16T00:30:26.080630+0000","last_became_peered":"2024-09-16T00:30:26.080630+0000","last_unstale":"2024-09-16T00:31:23.443231+0000","last_undegraded":"2024-09-16T00:31:23.443231+0000","last_fullsized":"2024-09-16T00:31:23.443231+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:44:42.644792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66079,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":332,"num_read_kb":239,"num_write":166,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":198237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514528,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1028096,"data_stored":504478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743461,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32852,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712300,"statfs":{"total":95995035648,"available":95961395200,"internally_reserved":0,"allocated":1708032,"data_stored":1153766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906986,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1040384,"data_stored":508754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135920,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1032192,"data_stored":504556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397556,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28624,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716528,"statfs":{"total":95995035648,"available":95965724672,"internally_reserved":0,"allocated":1630208,"data_stored":1100489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561082,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1015808,"data_stored":499834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790014,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1638400,"data_stored":1096077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953540,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1028096,"data_stored":504484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182472,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1028096,"data_stored":500028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":563529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1081344,"data_stored":561820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1040384,"data_stored":501222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:31:59.696 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T00:31:59.959 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:00.373 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:00 smithi038 ceph-mon[30945]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:32:00.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:00 smithi122 ceph-mon[35412]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:32:00.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.454+0000 7ff84bd16640 1 -- 172.21.15.38:0/3120104080 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff844106a80 msgr2=0x7ff844106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:00.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.454+0000 7ff84bd16640 1 --2- 172.21.15.38:0/3120104080 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff844106a80 0x7ff844106e60 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7ff840009930 tx=0x7ff84002a420 comp rx=0 tx=0).stop 2024-09-16T00:32:00.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.454+0000 7ff84bd16640 1 -- 172.21.15.38:0/3120104080 shutdown_connections 2024-09-16T00:32:00.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.454+0000 7ff84bd16640 1 --2- 172.21.15.38:0/3120104080 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff844101420 0x7ff84410d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.454+0000 7ff84bd16640 1 --2- 172.21.15.38:0/3120104080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff844100a80 0x7ff844100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.454+0000 7ff84bd16640 1 --2- 172.21.15.38:0/3120104080 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff844106a80 0x7ff844106e60 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.454+0000 7ff84bd16640 1 -- 172.21.15.38:0/3120104080 >> 172.21.15.38:0/3120104080 conn(0x7ff8440fc7a0 msgr2=0x7ff8440febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:00.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.454+0000 7ff84bd16640 1 -- 172.21.15.38:0/3120104080 shutdown_connections 2024-09-16T00:32:00.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.454+0000 7ff84bd16640 1 -- 172.21.15.38:0/3120104080 wait complete. 2024-09-16T00:32:00.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.454+0000 7ff84bd16640 1 Processor -- start 2024-09-16T00:32:00.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.455+0000 7ff84bd16640 1 -- start start 2024-09-16T00:32:00.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.455+0000 7ff84bd16640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff844100a80 0x7ff844199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:00.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.455+0000 7ff84bd16640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff844101420 0x7ff844199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:00.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.455+0000 7ff84bd16640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff844106a80 0x7ff8441a4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:00.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.455+0000 7ff84bd16640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff844110ea0 con 0x7ff844106a80 2024-09-16T00:32:00.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.455+0000 7ff84bd16640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff844110d20 con 0x7ff844100a80 2024-09-16T00:32:00.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.455+0000 7ff84bd16640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff844111020 con 0x7ff844101420 2024-09-16T00:32:00.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.455+0000 7ff84a2b3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff844106a80 0x7ff8441a4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:00.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.455+0000 7ff84a2b3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff844106a80 0x7ff8441a4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35012/0 (socket says 172.21.15.38:35012) 2024-09-16T00:32:00.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.455+0000 7ff84a2b3640 1 -- 172.21.15.38:0/3029384675 learned_addr learned my addr 172.21.15.38:0/3029384675 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:00.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.455+0000 7ff849ab2640 1 --2- 172.21.15.38:0/3029384675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff844100a80 0x7ff844199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:00.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.455+0000 7ff8492b1640 1 --2- 172.21.15.38:0/3029384675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff844101420 0x7ff844199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:00.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.455+0000 7ff84a2b3640 1 -- 172.21.15.38:0/3029384675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff844101420 msgr2=0x7ff844199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:00.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.455+0000 7ff84a2b3640 1 --2- 172.21.15.38:0/3029384675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff844101420 0x7ff844199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.455+0000 7ff84a2b3640 1 -- 172.21.15.38:0/3029384675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff844100a80 msgr2=0x7ff844199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:00.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.455+0000 7ff84a2b3640 1 --2- 172.21.15.38:0/3029384675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff844100a80 0x7ff844199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.455+0000 7ff84a2b3640 1 -- 172.21.15.38:0/3029384675 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8441a48d0 con 0x7ff844106a80 2024-09-16T00:32:00.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.456+0000 7ff84a2b3640 1 --2- 172.21.15.38:0/3029384675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff844106a80 0x7ff8441a4390 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff83c00b970 tx=0x7ff83c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:00.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.456+0000 7ff83affd640 1 -- 172.21.15.38:0/3029384675 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83c019070 con 0x7ff844106a80 2024-09-16T00:32:00.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.456+0000 7ff84bd16640 1 -- 172.21.15.38:0/3029384675 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8441a4aa0 con 0x7ff844106a80 2024-09-16T00:32:00.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.456+0000 7ff83affd640 1 -- 172.21.15.38:0/3029384675 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff83c00cb40 con 0x7ff844106a80 2024-09-16T00:32:00.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.456+0000 7ff83affd640 1 -- 172.21.15.38:0/3029384675 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff83c007650 con 0x7ff844106a80 2024-09-16T00:32:00.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.456+0000 7ff84bd16640 1 -- 172.21.15.38:0/3029384675 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8441a4fb0 con 0x7ff844106a80 2024-09-16T00:32:00.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.458+0000 7ff84bd16640 1 -- 172.21.15.38:0/3029384675 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff844100ee0 con 0x7ff844106a80 2024-09-16T00:32:00.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.459+0000 7ff83affd640 1 -- 172.21.15.38:0/3029384675 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7ff83c01c070 con 0x7ff844106a80 2024-09-16T00:32:00.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.460+0000 7ff83affd640 1 --2- 172.21.15.38:0/3029384675 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ff8300a32d0 0x7ff8300a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:00.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.460+0000 7ff849ab2640 1 --2- 172.21.15.38:0/3029384675 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ff8300a32d0 0x7ff8300a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:00.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.460+0000 7ff83affd640 1 -- 172.21.15.38:0/3029384675 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7ff83c0cf790 con 0x7ff844106a80 2024-09-16T00:32:00.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.461+0000 7ff849ab2640 1 --2- 172.21.15.38:0/3029384675 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ff8300a32d0 0x7ff8300a5790 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff840009900 tx=0x7ff8400058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:00.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.464+0000 7ff83affd640 1 -- 172.21.15.38:0/3029384675 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff83c09ba70 con 0x7ff844106a80 2024-09-16T00:32:00.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:00 smithi073 ceph-mon[35254]: from='client.15015 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:32:00.639 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.639+0000 7ff84bd16640 1 -- 172.21.15.38:0/3029384675 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff844109060 con 0x7ff8300a32d0 2024-09-16T00:32:00.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.649+0000 7ff83affd640 1 -- 172.21.15.38:0/3029384675 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340370 (secure 0 0 0) 0x7ff840031860 con 0x7ff8300a32d0 2024-09-16T00:32:00.653 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:32:00.655 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-16T00:32:00.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.653+0000 7ff84bd16640 1 -- 172.21.15.38:0/3029384675 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ff8300a32d0 msgr2=0x7ff8300a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:00.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.653+0000 7ff84bd16640 1 --2- 172.21.15.38:0/3029384675 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ff8300a32d0 0x7ff8300a5790 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff840009900 tx=0x7ff8400058d0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.653+0000 7ff84bd16640 1 -- 172.21.15.38:0/3029384675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff844106a80 msgr2=0x7ff8441a4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:00.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.653+0000 7ff84bd16640 1 --2- 172.21.15.38:0/3029384675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff844106a80 0x7ff8441a4390 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff83c00b970 tx=0x7ff83c00be30 comp rx=0 tx=0).stop 2024-09-16T00:32:00.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.653+0000 7ff84bd16640 1 -- 172.21.15.38:0/3029384675 shutdown_connections 2024-09-16T00:32:00.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.653+0000 7ff84bd16640 1 --2- 172.21.15.38:0/3029384675 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ff8300a32d0 0x7ff8300a5790 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.657 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.653+0000 7ff84bd16640 1 --2- 172.21.15.38:0/3029384675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff844106a80 0x7ff8441a4390 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.653+0000 7ff84bd16640 1 --2- 172.21.15.38:0/3029384675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff844101420 0x7ff844199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.653+0000 7ff84bd16640 1 --2- 172.21.15.38:0/3029384675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff844100a80 0x7ff844199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:00.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.653+0000 7ff84bd16640 1 -- 172.21.15.38:0/3029384675 >> 172.21.15.38:0/3029384675 conn(0x7ff8440fc7a0 msgr2=0x7ff8440fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:00.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.653+0000 7ff84bd16640 1 -- 172.21.15.38:0/3029384675 shutdown_connections 2024-09-16T00:32:00.658 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:00.653+0000 7ff84bd16640 1 -- 172.21.15.38:0/3029384675 wait complete. 2024-09-16T00:32:01.388 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:01 smithi038 ceph-mon[30945]: pgmap v339: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T00:32:01.418 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2024-09-16T00:31:59.530447+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":656681,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":483,"num_read_kb":358,"num_write":349,"num_write_kb":3055,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1180792,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":281,"ondisk_log_size":281,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":379984,"kb_used_data":14036,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561840,"statfs":{"total":1151940427776,"available":1151551324160,"internally_reserved":0,"allocated":14372864,"data_stored":7999037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003515"},"pg_stats":[{"pgid":"2.1f","version":"88'9","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442635+0000","last_change":"2024-09-16T00:30:23.933355+0000","last_active":"2024-09-16T00:31:23.442635+0000","last_peered":"2024-09-16T00:31:23.442635+0000","last_clean":"2024-09-16T00:31:23.442635+0000","last_became_active":"2024-09-16T00:30:23.933057+0000","last_became_peered":"2024-09-16T00:30:23.933057+0000","last_unstale":"2024-09-16T00:31:23.442635+0000","last_undegraded":"2024-09-16T00:31:23.442635+0000","last_fullsized":"2024-09-16T00:31:23.442635+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:23:01.713440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.445196+0000","last_change":"2024-09-16T00:30:26.084537+0000","last_active":"2024-09-16T00:31:23.445196+0000","last_peered":"2024-09-16T00:31:23.445196+0000","last_clean":"2024-09-16T00:31:23.445196+0000","last_became_active":"2024-09-16T00:30:26.084226+0000","last_became_peered":"2024-09-16T00:30:26.084226+0000","last_unstale":"2024-09-16T00:31:23.445196+0000","last_undegraded":"2024-09-16T00:31:23.445196+0000","last_fullsized":"2024-09-16T00:31:23.445196+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:43:41.628877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442440+0000","last_change":"2024-09-16T00:30:33.154873+0000","last_active":"2024-09-16T00:31:23.442440+0000","last_peered":"2024-09-16T00:31:23.442440+0000","last_clean":"2024-09-16T00:31:23.442440+0000","last_became_active":"2024-09-16T00:30:33.154433+0000","last_became_peered":"2024-09-16T00:30:33.154433+0000","last_unstale":"2024-09-16T00:31:23.442440+0000","last_undegraded":"2024-09-16T00:31:23.442440+0000","last_fullsized":"2024-09-16T00:31:23.442440+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:50.805806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.445394+0000","last_change":"2024-09-16T00:30:24.422274+0000","last_active":"2024-09-16T00:31:23.445394+0000","last_peered":"2024-09-16T00:31:23.445394+0000","last_clean":"2024-09-16T00:31:23.445394+0000","last_became_active":"2024-09-16T00:30:24.421902+0000","last_became_peered":"2024-09-16T00:30:24.421902+0000","last_unstale":"2024-09-16T00:31:23.445394+0000","last_undegraded":"2024-09-16T00:31:23.445394+0000","last_fullsized":"2024-09-16T00:31:23.445394+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:01.910187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.577309+0000","last_change":"2024-09-16T00:30:26.088165+0000","last_active":"2024-09-16T00:31:23.577309+0000","last_peered":"2024-09-16T00:31:23.577309+0000","last_clean":"2024-09-16T00:31:23.577309+0000","last_became_active":"2024-09-16T00:30:26.087593+0000","last_became_peered":"2024-09-16T00:30:26.087593+0000","last_unstale":"2024-09-16T00:31:23.577309+0000","last_undegraded":"2024-09-16T00:31:23.577309+0000","last_fullsized":"2024-09-16T00:31:23.577309+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:54.149930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.445078+0000","last_change":"2024-09-16T00:30:33.154586+0000","last_active":"2024-09-16T00:31:23.445078+0000","last_peered":"2024-09-16T00:31:23.445078+0000","last_clean":"2024-09-16T00:31:23.445078+0000","last_became_active":"2024-09-16T00:30:33.154209+0000","last_became_peered":"2024-09-16T00:30:33.154209+0000","last_unstale":"2024-09-16T00:31:23.445078+0000","last_undegraded":"2024-09-16T00:31:23.445078+0000","last_fullsized":"2024-09-16T00:31:23.445078+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:45.776895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.444802+0000","last_change":"2024-09-16T00:30:23.932442+0000","last_active":"2024-09-16T00:31:23.444802+0000","last_peered":"2024-09-16T00:31:23.444802+0000","last_clean":"2024-09-16T00:31:23.444802+0000","last_became_active":"2024-09-16T00:30:23.932009+0000","last_became_peered":"2024-09-16T00:30:23.932009+0000","last_unstale":"2024-09-16T00:31:23.444802+0000","last_undegraded":"2024-09-16T00:31:23.444802+0000","last_fullsized":"2024-09-16T00:31:23.444802+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:09:27.078187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.440646+0000","last_change":"2024-09-16T00:30:26.087715+0000","last_active":"2024-09-16T00:31:23.440646+0000","last_peered":"2024-09-16T00:31:23.440646+0000","last_clean":"2024-09-16T00:31:23.440646+0000","last_became_active":"2024-09-16T00:30:26.087415+0000","last_became_peered":"2024-09-16T00:30:26.087415+0000","last_unstale":"2024-09-16T00:31:23.440646+0000","last_undegraded":"2024-09-16T00:31:23.440646+0000","last_fullsized":"2024-09-16T00:31:23.440646+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:52.373992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.445658+0000","last_change":"2024-09-16T00:30:33.152328+0000","last_active":"2024-09-16T00:31:23.445658+0000","last_peered":"2024-09-16T00:31:23.445658+0000","last_clean":"2024-09-16T00:31:23.445658+0000","last_became_active":"2024-09-16T00:30:33.151954+0000","last_became_peered":"2024-09-16T00:30:33.151954+0000","last_unstale":"2024-09-16T00:31:23.445658+0000","last_undegraded":"2024-09-16T00:31:23.445658+0000","last_fullsized":"2024-09-16T00:31:23.445658+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:36.546729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.443216+0000","last_change":"2024-09-16T00:30:23.928988+0000","last_active":"2024-09-16T00:31:23.443216+0000","last_peered":"2024-09-16T00:31:23.443216+0000","last_clean":"2024-09-16T00:31:23.443216+0000","last_became_active":"2024-09-16T00:30:23.928638+0000","last_became_peered":"2024-09-16T00:30:23.928638+0000","last_unstale":"2024-09-16T00:31:23.443216+0000","last_undegraded":"2024-09-16T00:31:23.443216+0000","last_fullsized":"2024-09-16T00:31:23.443216+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:45:58.203455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.446550+0000","last_change":"2024-09-16T00:31:23.446550+0000","last_active":"2024-09-16T00:31:23.446550+0000","last_peered":"2024-09-16T00:31:23.446550+0000","last_clean":"2024-09-16T00:31:23.446550+0000","last_became_active":"2024-09-16T00:31:23.446179+0000","last_became_peered":"2024-09-16T00:31:23.446179+0000","last_unstale":"2024-09-16T00:31:23.446550+0000","last_undegraded":"2024-09-16T00:31:23.446550+0000","last_fullsized":"2024-09-16T00:31:23.446550+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:04:21.280174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.445269+0000","last_change":"2024-09-16T00:30:33.147564+0000","last_active":"2024-09-16T00:31:23.445269+0000","last_peered":"2024-09-16T00:31:23.445269+0000","last_clean":"2024-09-16T00:31:23.445269+0000","last_became_active":"2024-09-16T00:30:33.147119+0000","last_became_peered":"2024-09-16T00:30:33.147119+0000","last_unstale":"2024-09-16T00:31:23.445269+0000","last_undegraded":"2024-09-16T00:31:23.445269+0000","last_fullsized":"2024-09-16T00:31:23.445269+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:00.509122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442254+0000","last_change":"2024-09-16T00:30:26.080954+0000","last_active":"2024-09-16T00:31:23.442254+0000","last_peered":"2024-09-16T00:31:23.442254+0000","last_clean":"2024-09-16T00:31:23.442254+0000","last_became_active":"2024-09-16T00:30:26.080624+0000","last_became_peered":"2024-09-16T00:30:26.080624+0000","last_unstale":"2024-09-16T00:31:23.442254+0000","last_undegraded":"2024-09-16T00:31:23.442254+0000","last_fullsized":"2024-09-16T00:31:23.442254+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:05:10.440980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442620+0000","last_change":"2024-09-16T00:30:23.933715+0000","last_active":"2024-09-16T00:31:23.442620+0000","last_peered":"2024-09-16T00:31:23.442620+0000","last_clean":"2024-09-16T00:31:23.442620+0000","last_became_active":"2024-09-16T00:30:23.933275+0000","last_became_peered":"2024-09-16T00:30:23.933275+0000","last_unstale":"2024-09-16T00:31:23.442620+0000","last_undegraded":"2024-09-16T00:31:23.442620+0000","last_fullsized":"2024-09-16T00:31:23.442620+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:40:44.846053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.445456+0000","last_change":"2024-09-16T00:30:33.604602+0000","last_active":"2024-09-16T00:31:23.445456+0000","last_peered":"2024-09-16T00:31:23.445456+0000","last_clean":"2024-09-16T00:31:23.445456+0000","last_became_active":"2024-09-16T00:30:33.604107+0000","last_became_peered":"2024-09-16T00:30:33.604107+0000","last_unstale":"2024-09-16T00:31:23.445456+0000","last_undegraded":"2024-09-16T00:31:23.445456+0000","last_fullsized":"2024-09-16T00:31:23.445456+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:41.254445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.444747+0000","last_change":"2024-09-16T00:30:26.080612+0000","last_active":"2024-09-16T00:31:23.444747+0000","last_peered":"2024-09-16T00:31:23.444747+0000","last_clean":"2024-09-16T00:31:23.444747+0000","last_became_active":"2024-09-16T00:30:26.080317+0000","last_became_peered":"2024-09-16T00:30:26.080317+0000","last_unstale":"2024-09-16T00:31:23.444747+0000","last_undegraded":"2024-09-16T00:31:23.444747+0000","last_fullsized":"2024-09-16T00:31:23.444747+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:23.814241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442788+0000","last_change":"2024-09-16T00:30:23.936778+0000","last_active":"2024-09-16T00:31:23.442788+0000","last_peered":"2024-09-16T00:31:23.442788+0000","last_clean":"2024-09-16T00:31:23.442788+0000","last_became_active":"2024-09-16T00:30:23.936265+0000","last_became_peered":"2024-09-16T00:30:23.936265+0000","last_unstale":"2024-09-16T00:31:23.442788+0000","last_undegraded":"2024-09-16T00:31:23.442788+0000","last_fullsized":"2024-09-16T00:31:23.442788+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:35:28.288476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.445546+0000","last_change":"2024-09-16T00:30:33.148340+0000","last_active":"2024-09-16T00:31:23.445546+0000","last_peered":"2024-09-16T00:31:23.445546+0000","last_clean":"2024-09-16T00:31:23.445546+0000","last_became_active":"2024-09-16T00:30:33.147289+0000","last_became_peered":"2024-09-16T00:30:33.147289+0000","last_unstale":"2024-09-16T00:31:23.445546+0000","last_undegraded":"2024-09-16T00:31:23.445546+0000","last_fullsized":"2024-09-16T00:31:23.445546+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:21.259254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442247+0000","last_change":"2024-09-16T00:30:26.077325+0000","last_active":"2024-09-16T00:31:23.442247+0000","last_peered":"2024-09-16T00:31:23.442247+0000","last_clean":"2024-09-16T00:31:23.442247+0000","last_became_active":"2024-09-16T00:30:26.077002+0000","last_became_peered":"2024-09-16T00:30:26.077002+0000","last_unstale":"2024-09-16T00:31:23.442247+0000","last_undegraded":"2024-09-16T00:31:23.442247+0000","last_fullsized":"2024-09-16T00:31:23.442247+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:12:38.569247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.445059+0000","last_change":"2024-09-16T00:30:23.933025+0000","last_active":"2024-09-16T00:31:23.445059+0000","last_peered":"2024-09-16T00:31:23.445059+0000","last_clean":"2024-09-16T00:31:23.445059+0000","last_became_active":"2024-09-16T00:30:23.932669+0000","last_became_peered":"2024-09-16T00:30:23.932669+0000","last_unstale":"2024-09-16T00:31:23.445059+0000","last_undegraded":"2024-09-16T00:31:23.445059+0000","last_fullsized":"2024-09-16T00:31:23.445059+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:20.593665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442296+0000","last_change":"2024-09-16T00:30:33.151844+0000","last_active":"2024-09-16T00:31:23.442296+0000","last_peered":"2024-09-16T00:31:23.442296+0000","last_clean":"2024-09-16T00:31:23.442296+0000","last_became_active":"2024-09-16T00:30:33.151311+0000","last_became_peered":"2024-09-16T00:30:33.151311+0000","last_unstale":"2024-09-16T00:31:23.442296+0000","last_undegraded":"2024-09-16T00:31:23.442296+0000","last_fullsized":"2024-09-16T00:31:23.442296+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:18:51.785233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442430+0000","last_change":"2024-09-16T00:30:26.076104+0000","last_active":"2024-09-16T00:31:23.442430+0000","last_peered":"2024-09-16T00:31:23.442430+0000","last_clean":"2024-09-16T00:31:23.442430+0000","last_became_active":"2024-09-16T00:30:26.075764+0000","last_became_peered":"2024-09-16T00:30:26.075764+0000","last_unstale":"2024-09-16T00:31:23.442430+0000","last_undegraded":"2024-09-16T00:31:23.442430+0000","last_fullsized":"2024-09-16T00:31:23.442430+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:31.228533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.440275+0000","last_change":"2024-09-16T00:30:24.422411+0000","last_active":"2024-09-16T00:31:23.440275+0000","last_peered":"2024-09-16T00:31:23.440275+0000","last_clean":"2024-09-16T00:31:23.440275+0000","last_became_active":"2024-09-16T00:30:24.422102+0000","last_became_peered":"2024-09-16T00:30:24.422102+0000","last_unstale":"2024-09-16T00:31:23.440275+0000","last_undegraded":"2024-09-16T00:31:23.440275+0000","last_fullsized":"2024-09-16T00:31:23.440275+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:06:00.572070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.445691+0000","last_change":"2024-09-16T00:30:33.605397+0000","last_active":"2024-09-16T00:31:23.445691+0000","last_peered":"2024-09-16T00:31:23.445691+0000","last_clean":"2024-09-16T00:31:23.445691+0000","last_became_active":"2024-09-16T00:30:33.604978+0000","last_became_peered":"2024-09-16T00:30:33.604978+0000","last_unstale":"2024-09-16T00:31:23.445691+0000","last_undegraded":"2024-09-16T00:31:23.445691+0000","last_fullsized":"2024-09-16T00:31:23.445691+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:35:16.766056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.577206+0000","last_change":"2024-09-16T00:30:26.082684+0000","last_active":"2024-09-16T00:31:23.577206+0000","last_peered":"2024-09-16T00:31:23.577206+0000","last_clean":"2024-09-16T00:31:23.577206+0000","last_became_active":"2024-09-16T00:30:26.082335+0000","last_became_peered":"2024-09-16T00:30:26.082335+0000","last_unstale":"2024-09-16T00:31:23.577206+0000","last_undegraded":"2024-09-16T00:31:23.577206+0000","last_fullsized":"2024-09-16T00:31:23.577206+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:51:21.389627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.445962+0000","last_change":"2024-09-16T00:30:24.587840+0000","last_active":"2024-09-16T00:31:23.445962+0000","last_peered":"2024-09-16T00:31:23.445962+0000","last_clean":"2024-09-16T00:31:23.445962+0000","last_became_active":"2024-09-16T00:30:24.587436+0000","last_became_peered":"2024-09-16T00:30:24.587436+0000","last_unstale":"2024-09-16T00:31:23.445962+0000","last_undegraded":"2024-09-16T00:31:23.445962+0000","last_fullsized":"2024-09-16T00:31:23.445962+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:06.758585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.445527+0000","last_change":"2024-09-16T00:30:33.145051+0000","last_active":"2024-09-16T00:31:23.445527+0000","last_peered":"2024-09-16T00:31:23.445527+0000","last_clean":"2024-09-16T00:31:23.445527+0000","last_became_active":"2024-09-16T00:30:33.144585+0000","last_became_peered":"2024-09-16T00:30:33.144585+0000","last_unstale":"2024-09-16T00:31:23.445527+0000","last_undegraded":"2024-09-16T00:31:23.445527+0000","last_fullsized":"2024-09-16T00:31:23.445527+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:48:23.588495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.444802+0000","last_change":"2024-09-16T00:31:23.444802+0000","last_active":"2024-09-16T00:31:23.444802+0000","last_peered":"2024-09-16T00:31:23.444802+0000","last_clean":"2024-09-16T00:31:23.444802+0000","last_became_active":"2024-09-16T00:31:23.444447+0000","last_became_peered":"2024-09-16T00:31:23.444447+0000","last_unstale":"2024-09-16T00:31:23.444802+0000","last_undegraded":"2024-09-16T00:31:23.444802+0000","last_fullsized":"2024-09-16T00:31:23.444802+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:46:37.285250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.6","version":"88'11","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442253+0000","last_change":"2024-09-16T00:30:23.933284+0000","last_active":"2024-09-16T00:31:23.442253+0000","last_peered":"2024-09-16T00:31:23.442253+0000","last_clean":"2024-09-16T00:31:23.442253+0000","last_became_active":"2024-09-16T00:30:23.932920+0000","last_became_peered":"2024-09-16T00:30:23.932920+0000","last_unstale":"2024-09-16T00:31:23.442253+0000","last_undegraded":"2024-09-16T00:31:23.442253+0000","last_fullsized":"2024-09-16T00:31:23.442253+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:38.199391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.444402+0000","last_change":"2024-09-16T00:30:33.604308+0000","last_active":"2024-09-16T00:31:23.444402+0000","last_peered":"2024-09-16T00:31:23.444402+0000","last_clean":"2024-09-16T00:31:23.444402+0000","last_became_active":"2024-09-16T00:30:33.603889+0000","last_became_peered":"2024-09-16T00:30:33.603889+0000","last_unstale":"2024-09-16T00:31:23.444402+0000","last_undegraded":"2024-09-16T00:31:23.444402+0000","last_fullsized":"2024-09-16T00:31:23.444402+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:32:55.173063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.439777+0000","last_change":"2024-09-16T00:30:26.078515+0000","last_active":"2024-09-16T00:31:23.439777+0000","last_peered":"2024-09-16T00:31:23.439777+0000","last_clean":"2024-09-16T00:31:23.439777+0000","last_became_active":"2024-09-16T00:30:26.078173+0000","last_became_peered":"2024-09-16T00:30:26.078173+0000","last_unstale":"2024-09-16T00:31:23.439777+0000","last_undegraded":"2024-09-16T00:31:23.439777+0000","last_fullsized":"2024-09-16T00:31:23.439777+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:40:31.471154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.444907+0000","last_change":"2024-09-16T00:30:24.423209+0000","last_active":"2024-09-16T00:31:23.444907+0000","last_peered":"2024-09-16T00:31:23.444907+0000","last_clean":"2024-09-16T00:31:23.444907+0000","last_became_active":"2024-09-16T00:30:24.422157+0000","last_became_peered":"2024-09-16T00:30:24.422157+0000","last_unstale":"2024-09-16T00:31:23.444907+0000","last_undegraded":"2024-09-16T00:31:23.444907+0000","last_fullsized":"2024-09-16T00:31:23.444907+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:28.262392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442754+0000","last_change":"2024-09-16T00:30:33.146204+0000","last_active":"2024-09-16T00:31:23.442754+0000","last_peered":"2024-09-16T00:31:23.442754+0000","last_clean":"2024-09-16T00:31:23.442754+0000","last_became_active":"2024-09-16T00:30:33.145553+0000","last_became_peered":"2024-09-16T00:30:33.145553+0000","last_unstale":"2024-09-16T00:31:23.442754+0000","last_undegraded":"2024-09-16T00:31:23.442754+0000","last_fullsized":"2024-09-16T00:31:23.442754+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:22.096050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.441856+0000","last_change":"2024-09-16T00:30:26.070153+0000","last_active":"2024-09-16T00:31:23.441856+0000","last_peered":"2024-09-16T00:31:23.441856+0000","last_clean":"2024-09-16T00:31:23.441856+0000","last_became_active":"2024-09-16T00:30:26.069792+0000","last_became_peered":"2024-09-16T00:30:26.069792+0000","last_unstale":"2024-09-16T00:31:23.441856+0000","last_undegraded":"2024-09-16T00:31:23.441856+0000","last_fullsized":"2024-09-16T00:31:23.441856+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:28.164242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442132+0000","last_change":"2024-09-16T00:30:24.423261+0000","last_active":"2024-09-16T00:31:23.442132+0000","last_peered":"2024-09-16T00:31:23.442132+0000","last_clean":"2024-09-16T00:31:23.442132+0000","last_became_active":"2024-09-16T00:30:24.422650+0000","last_became_peered":"2024-09-16T00:30:24.422650+0000","last_unstale":"2024-09-16T00:31:23.442132+0000","last_undegraded":"2024-09-16T00:31:23.442132+0000","last_fullsized":"2024-09-16T00:31:23.442132+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:00.211744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.443327+0000","last_change":"2024-09-16T00:30:33.604492+0000","last_active":"2024-09-16T00:31:23.443327+0000","last_peered":"2024-09-16T00:31:23.443327+0000","last_clean":"2024-09-16T00:31:23.443327+0000","last_became_active":"2024-09-16T00:30:33.604015+0000","last_became_peered":"2024-09-16T00:30:33.604015+0000","last_unstale":"2024-09-16T00:31:23.443327+0000","last_undegraded":"2024-09-16T00:31:23.443327+0000","last_fullsized":"2024-09-16T00:31:23.443327+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:40:22.516583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442253+0000","last_change":"2024-09-16T00:30:26.075517+0000","last_active":"2024-09-16T00:31:23.442253+0000","last_peered":"2024-09-16T00:31:23.442253+0000","last_clean":"2024-09-16T00:31:23.442253+0000","last_became_active":"2024-09-16T00:30:26.075113+0000","last_became_peered":"2024-09-16T00:30:26.075113+0000","last_unstale":"2024-09-16T00:31:23.442253+0000","last_undegraded":"2024-09-16T00:31:23.442253+0000","last_fullsized":"2024-09-16T00:31:23.442253+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:11.427010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442390+0000","last_change":"2024-09-16T00:30:23.933639+0000","last_active":"2024-09-16T00:31:23.442390+0000","last_peered":"2024-09-16T00:31:23.442390+0000","last_clean":"2024-09-16T00:31:23.442390+0000","last_became_active":"2024-09-16T00:30:23.933322+0000","last_became_peered":"2024-09-16T00:30:23.933322+0000","last_unstale":"2024-09-16T00:31:23.442390+0000","last_undegraded":"2024-09-16T00:31:23.442390+0000","last_fullsized":"2024-09-16T00:31:23.442390+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:59:59.082702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.439665+0000","last_change":"2024-09-16T00:30:33.148508+0000","last_active":"2024-09-16T00:31:23.439665+0000","last_peered":"2024-09-16T00:31:23.439665+0000","last_clean":"2024-09-16T00:31:23.439665+0000","last_became_active":"2024-09-16T00:30:33.148091+0000","last_became_peered":"2024-09-16T00:30:33.148091+0000","last_unstale":"2024-09-16T00:31:23.439665+0000","last_undegraded":"2024-09-16T00:31:23.439665+0000","last_fullsized":"2024-09-16T00:31:23.439665+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:50.052526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.440040+0000","last_change":"2024-09-16T00:30:26.080374+0000","last_active":"2024-09-16T00:31:23.440040+0000","last_peered":"2024-09-16T00:31:23.440040+0000","last_clean":"2024-09-16T00:31:23.440040+0000","last_became_active":"2024-09-16T00:30:26.080044+0000","last_became_peered":"2024-09-16T00:30:26.080044+0000","last_unstale":"2024-09-16T00:31:23.440040+0000","last_undegraded":"2024-09-16T00:31:23.440040+0000","last_fullsized":"2024-09-16T00:31:23.440040+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:38:36.034903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":90,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:53.404778+0000","last_change":"2024-09-16T00:30:24.422137+0000","last_active":"2024-09-16T00:31:53.404778+0000","last_peered":"2024-09-16T00:31:53.404778+0000","last_clean":"2024-09-16T00:31:53.404778+0000","last_became_active":"2024-09-16T00:30:24.421778+0000","last_became_peered":"2024-09-16T00:30:24.421778+0000","last_unstale":"2024-09-16T00:31:53.404778+0000","last_undegraded":"2024-09-16T00:31:53.404778+0000","last_fullsized":"2024-09-16T00:31:53.404778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:54:21.964099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.439834+0000","last_change":"2024-09-16T00:30:33.604263+0000","last_active":"2024-09-16T00:31:23.439834+0000","last_peered":"2024-09-16T00:31:23.439834+0000","last_clean":"2024-09-16T00:31:23.439834+0000","last_became_active":"2024-09-16T00:30:33.603804+0000","last_became_peered":"2024-09-16T00:30:33.603804+0000","last_unstale":"2024-09-16T00:31:23.439834+0000","last_undegraded":"2024-09-16T00:31:23.439834+0000","last_fullsized":"2024-09-16T00:31:23.439834+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:08:37.590827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.439285+0000","last_change":"2024-09-16T00:30:26.082916+0000","last_active":"2024-09-16T00:31:23.439285+0000","last_peered":"2024-09-16T00:31:23.439285+0000","last_clean":"2024-09-16T00:31:23.439285+0000","last_became_active":"2024-09-16T00:30:26.082606+0000","last_became_peered":"2024-09-16T00:30:26.082606+0000","last_unstale":"2024-09-16T00:31:23.439285+0000","last_undegraded":"2024-09-16T00:31:23.439285+0000","last_fullsized":"2024-09-16T00:31:23.439285+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:56.777640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.441880+0000","last_change":"2024-09-16T00:30:23.937602+0000","last_active":"2024-09-16T00:31:23.441880+0000","last_peered":"2024-09-16T00:31:23.441880+0000","last_clean":"2024-09-16T00:31:23.441880+0000","last_became_active":"2024-09-16T00:30:23.937261+0000","last_became_peered":"2024-09-16T00:30:23.937261+0000","last_unstale":"2024-09-16T00:31:23.441880+0000","last_undegraded":"2024-09-16T00:31:23.441880+0000","last_fullsized":"2024-09-16T00:31:23.441880+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:24.698758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.440724+0000","last_change":"2024-09-16T00:30:33.605536+0000","last_active":"2024-09-16T00:31:23.440724+0000","last_peered":"2024-09-16T00:31:23.440724+0000","last_clean":"2024-09-16T00:31:23.440724+0000","last_became_active":"2024-09-16T00:30:33.605073+0000","last_became_peered":"2024-09-16T00:30:33.605073+0000","last_unstale":"2024-09-16T00:31:23.440724+0000","last_undegraded":"2024-09-16T00:31:23.440724+0000","last_fullsized":"2024-09-16T00:31:23.440724+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:11.143052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.444986+0000","last_change":"2024-09-16T00:30:26.084507+0000","last_active":"2024-09-16T00:31:23.444986+0000","last_peered":"2024-09-16T00:31:23.444986+0000","last_clean":"2024-09-16T00:31:23.444986+0000","last_became_active":"2024-09-16T00:30:26.084152+0000","last_became_peered":"2024-09-16T00:30:26.084152+0000","last_unstale":"2024-09-16T00:31:23.444986+0000","last_undegraded":"2024-09-16T00:31:23.444986+0000","last_fullsized":"2024-09-16T00:31:23.444986+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:27:21.557409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.445869+0000","last_change":"2024-09-16T00:30:23.932686+0000","last_active":"2024-09-16T00:31:23.445869+0000","last_peered":"2024-09-16T00:31:23.445869+0000","last_clean":"2024-09-16T00:31:23.445869+0000","last_became_active":"2024-09-16T00:30:23.932343+0000","last_became_peered":"2024-09-16T00:30:23.932343+0000","last_unstale":"2024-09-16T00:31:23.445869+0000","last_undegraded":"2024-09-16T00:31:23.445869+0000","last_fullsized":"2024-09-16T00:31:23.445869+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:13:44.185712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":347,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442083+0000","last_change":"2024-09-16T00:29:11.143642+0000","last_active":"2024-09-16T00:31:23.442083+0000","last_peered":"2024-09-16T00:31:23.442083+0000","last_clean":"2024-09-16T00:31:23.442083+0000","last_became_active":"2024-09-16T00:29:11.143227+0000","last_became_peered":"2024-09-16T00:29:11.143227+0000","last_unstale":"2024-09-16T00:31:23.442083+0000","last_undegraded":"2024-09-16T00:31:23.442083+0000","last_fullsized":"2024-09-16T00:31:23.442083+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:25:42.361316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:25:42.361316+0000","last_clean_scrub_stamp":"2024-09-16T00:25:42.361316+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:29:12.740474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":22,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442899+0000","last_change":"2024-09-16T00:30:33.155555+0000","last_active":"2024-09-16T00:31:23.442899+0000","last_peered":"2024-09-16T00:31:23.442899+0000","last_clean":"2024-09-16T00:31:23.442899+0000","last_became_active":"2024-09-16T00:30:33.155170+0000","last_became_peered":"2024-09-16T00:30:33.155170+0000","last_unstale":"2024-09-16T00:31:23.442899+0000","last_undegraded":"2024-09-16T00:31:23.442899+0000","last_fullsized":"2024-09-16T00:31:23.442899+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:34:46.214422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.443059+0000","last_change":"2024-09-16T00:30:26.076003+0000","last_active":"2024-09-16T00:31:23.443059+0000","last_peered":"2024-09-16T00:31:23.443059+0000","last_clean":"2024-09-16T00:31:23.443059+0000","last_became_active":"2024-09-16T00:30:26.075666+0000","last_became_peered":"2024-09-16T00:30:26.075666+0000","last_unstale":"2024-09-16T00:31:23.443059+0000","last_undegraded":"2024-09-16T00:31:23.443059+0000","last_fullsized":"2024-09-16T00:31:23.443059+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:30.113380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.444547+0000","last_change":"2024-09-16T00:30:24.587469+0000","last_active":"2024-09-16T00:31:23.444547+0000","last_peered":"2024-09-16T00:31:23.444547+0000","last_clean":"2024-09-16T00:31:23.444547+0000","last_became_active":"2024-09-16T00:30:24.587049+0000","last_became_peered":"2024-09-16T00:30:24.587049+0000","last_unstale":"2024-09-16T00:31:23.444547+0000","last_undegraded":"2024-09-16T00:31:23.444547+0000","last_fullsized":"2024-09-16T00:31:23.444547+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:17:45.934505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.440438+0000","last_change":"2024-09-16T00:30:33.145921+0000","last_active":"2024-09-16T00:31:23.440438+0000","last_peered":"2024-09-16T00:31:23.440438+0000","last_clean":"2024-09-16T00:31:23.440438+0000","last_became_active":"2024-09-16T00:30:33.145450+0000","last_became_peered":"2024-09-16T00:30:33.145450+0000","last_unstale":"2024-09-16T00:31:23.440438+0000","last_undegraded":"2024-09-16T00:31:23.440438+0000","last_fullsized":"2024-09-16T00:31:23.440438+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:53:51.698686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442375+0000","last_change":"2024-09-16T00:30:26.087940+0000","last_active":"2024-09-16T00:31:23.442375+0000","last_peered":"2024-09-16T00:31:23.442375+0000","last_clean":"2024-09-16T00:31:23.442375+0000","last_became_active":"2024-09-16T00:30:26.087599+0000","last_became_peered":"2024-09-16T00:30:26.087599+0000","last_unstale":"2024-09-16T00:31:23.442375+0000","last_undegraded":"2024-09-16T00:31:23.442375+0000","last_fullsized":"2024-09-16T00:31:23.442375+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:43.934839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.440967+0000","last_change":"2024-09-16T00:30:23.930986+0000","last_active":"2024-09-16T00:31:23.440967+0000","last_peered":"2024-09-16T00:31:23.440967+0000","last_clean":"2024-09-16T00:31:23.440967+0000","last_became_active":"2024-09-16T00:30:23.930588+0000","last_became_peered":"2024-09-16T00:30:23.930588+0000","last_unstale":"2024-09-16T00:31:23.440967+0000","last_undegraded":"2024-09-16T00:31:23.440967+0000","last_fullsized":"2024-09-16T00:31:23.440967+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:04:41.902358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.576963+0000","last_change":"2024-09-16T00:30:33.154894+0000","last_active":"2024-09-16T00:31:23.576963+0000","last_peered":"2024-09-16T00:31:23.576963+0000","last_clean":"2024-09-16T00:31:23.576963+0000","last_became_active":"2024-09-16T00:30:33.154490+0000","last_became_peered":"2024-09-16T00:30:33.154490+0000","last_unstale":"2024-09-16T00:31:23.576963+0000","last_undegraded":"2024-09-16T00:31:23.576963+0000","last_fullsized":"2024-09-16T00:31:23.576963+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:59:53.854212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442565+0000","last_change":"2024-09-16T00:30:26.077069+0000","last_active":"2024-09-16T00:31:23.442565+0000","last_peered":"2024-09-16T00:31:23.442565+0000","last_clean":"2024-09-16T00:31:23.442565+0000","last_became_active":"2024-09-16T00:30:26.076718+0000","last_became_peered":"2024-09-16T00:30:26.076718+0000","last_unstale":"2024-09-16T00:31:23.442565+0000","last_undegraded":"2024-09-16T00:31:23.442565+0000","last_fullsized":"2024-09-16T00:31:23.442565+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:25:37.541704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.439926+0000","last_change":"2024-09-16T00:30:23.932081+0000","last_active":"2024-09-16T00:31:23.439926+0000","last_peered":"2024-09-16T00:31:23.439926+0000","last_clean":"2024-09-16T00:31:23.439926+0000","last_became_active":"2024-09-16T00:30:23.931729+0000","last_became_peered":"2024-09-16T00:30:23.931729+0000","last_unstale":"2024-09-16T00:31:23.439926+0000","last_undegraded":"2024-09-16T00:31:23.439926+0000","last_fullsized":"2024-09-16T00:31:23.439926+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:20:05.178580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.439866+0000","last_change":"2024-09-16T00:30:33.147738+0000","last_active":"2024-09-16T00:31:23.439866+0000","last_peered":"2024-09-16T00:31:23.439866+0000","last_clean":"2024-09-16T00:31:23.439866+0000","last_became_active":"2024-09-16T00:30:33.147326+0000","last_became_peered":"2024-09-16T00:30:33.147326+0000","last_unstale":"2024-09-16T00:31:23.439866+0000","last_undegraded":"2024-09-16T00:31:23.439866+0000","last_fullsized":"2024-09-16T00:31:23.439866+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:44:55.123980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.443143+0000","last_change":"2024-09-16T00:30:26.079312+0000","last_active":"2024-09-16T00:31:23.443143+0000","last_peered":"2024-09-16T00:31:23.443143+0000","last_clean":"2024-09-16T00:31:23.443143+0000","last_became_active":"2024-09-16T00:30:26.078986+0000","last_became_peered":"2024-09-16T00:30:26.078986+0000","last_unstale":"2024-09-16T00:31:23.443143+0000","last_undegraded":"2024-09-16T00:31:23.443143+0000","last_fullsized":"2024-09-16T00:31:23.443143+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:02:06.602850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":17,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.452513+0000","last_change":"2024-09-16T00:31:23.452102+0000","last_active":"2024-09-16T00:31:23.452513+0000","last_peered":"2024-09-16T00:31:23.452513+0000","last_clean":"2024-09-16T00:31:23.452513+0000","last_became_active":"2024-09-16T00:31:23.445974+0000","last_became_peered":"2024-09-16T00:31:23.445974+0000","last_unstale":"2024-09-16T00:31:23.452513+0000","last_undegraded":"2024-09-16T00:31:23.452513+0000","last_fullsized":"2024-09-16T00:31:23.452513+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:52.314494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.439965+0000","last_change":"2024-09-16T00:30:33.149154+0000","last_active":"2024-09-16T00:31:23.439965+0000","last_peered":"2024-09-16T00:31:23.439965+0000","last_clean":"2024-09-16T00:31:23.439965+0000","last_became_active":"2024-09-16T00:30:33.148763+0000","last_became_peered":"2024-09-16T00:30:33.148763+0000","last_unstale":"2024-09-16T00:31:23.439965+0000","last_undegraded":"2024-09-16T00:31:23.439965+0000","last_fullsized":"2024-09-16T00:31:23.439965+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:58:05.884919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.440188+0000","last_change":"2024-09-16T00:30:26.086905+0000","last_active":"2024-09-16T00:31:23.440188+0000","last_peered":"2024-09-16T00:31:23.440188+0000","last_clean":"2024-09-16T00:31:23.440188+0000","last_became_active":"2024-09-16T00:30:26.086519+0000","last_became_peered":"2024-09-16T00:30:26.086519+0000","last_unstale":"2024-09-16T00:31:23.440188+0000","last_undegraded":"2024-09-16T00:31:23.440188+0000","last_fullsized":"2024-09-16T00:31:23.440188+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:51:08.787679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442314+0000","last_change":"2024-09-16T00:30:24.422893+0000","last_active":"2024-09-16T00:31:23.442314+0000","last_peered":"2024-09-16T00:31:23.442314+0000","last_clean":"2024-09-16T00:31:23.442314+0000","last_became_active":"2024-09-16T00:30:24.422073+0000","last_became_peered":"2024-09-16T00:30:24.422073+0000","last_unstale":"2024-09-16T00:31:23.442314+0000","last_undegraded":"2024-09-16T00:31:23.442314+0000","last_fullsized":"2024-09-16T00:31:23.442314+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:15:11.375957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.439835+0000","last_change":"2024-09-16T00:30:33.152582+0000","last_active":"2024-09-16T00:31:23.439835+0000","last_peered":"2024-09-16T00:31:23.439835+0000","last_clean":"2024-09-16T00:31:23.439835+0000","last_became_active":"2024-09-16T00:30:33.152180+0000","last_became_peered":"2024-09-16T00:30:33.152180+0000","last_unstale":"2024-09-16T00:31:23.439835+0000","last_undegraded":"2024-09-16T00:31:23.439835+0000","last_fullsized":"2024-09-16T00:31:23.439835+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:41:10.499391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.441715+0000","last_change":"2024-09-16T00:30:26.086052+0000","last_active":"2024-09-16T00:31:23.441715+0000","last_peered":"2024-09-16T00:31:23.441715+0000","last_clean":"2024-09-16T00:31:23.441715+0000","last_became_active":"2024-09-16T00:30:26.085756+0000","last_became_peered":"2024-09-16T00:30:26.085756+0000","last_unstale":"2024-09-16T00:31:23.441715+0000","last_undegraded":"2024-09-16T00:31:23.441715+0000","last_fullsized":"2024-09-16T00:31:23.441715+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:52.582166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":94,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:56.409939+0000","last_change":"2024-09-16T00:30:23.933528+0000","last_active":"2024-09-16T00:31:56.409939+0000","last_peered":"2024-09-16T00:31:56.409939+0000","last_clean":"2024-09-16T00:31:56.409939+0000","last_became_active":"2024-09-16T00:30:23.933186+0000","last_became_peered":"2024-09-16T00:30:23.933186+0000","last_unstale":"2024-09-16T00:31:56.409939+0000","last_undegraded":"2024-09-16T00:31:56.409939+0000","last_fullsized":"2024-09-16T00:31:56.409939+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:11:00.489676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.578399+0000","last_change":"2024-09-16T00:31:23.578056+0000","last_active":"2024-09-16T00:31:23.578399+0000","last_peered":"2024-09-16T00:31:23.578399+0000","last_clean":"2024-09-16T00:31:23.578399+0000","last_became_active":"2024-09-16T00:31:23.577721+0000","last_became_peered":"2024-09-16T00:31:23.577721+0000","last_unstale":"2024-09-16T00:31:23.578399+0000","last_undegraded":"2024-09-16T00:31:23.578399+0000","last_fullsized":"2024-09-16T00:31:23.578399+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:53.632440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442597+0000","last_change":"2024-09-16T00:30:26.084509+0000","last_active":"2024-09-16T00:31:23.442597+0000","last_peered":"2024-09-16T00:31:23.442597+0000","last_clean":"2024-09-16T00:31:23.442597+0000","last_became_active":"2024-09-16T00:30:26.084188+0000","last_became_peered":"2024-09-16T00:30:26.084188+0000","last_unstale":"2024-09-16T00:31:23.442597+0000","last_undegraded":"2024-09-16T00:31:23.442597+0000","last_fullsized":"2024-09-16T00:31:23.442597+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:21:48.981277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.440467+0000","last_change":"2024-09-16T00:30:23.933499+0000","last_active":"2024-09-16T00:31:23.440467+0000","last_peered":"2024-09-16T00:31:23.440467+0000","last_clean":"2024-09-16T00:31:23.440467+0000","last_became_active":"2024-09-16T00:30:23.933082+0000","last_became_peered":"2024-09-16T00:30:23.933082+0000","last_unstale":"2024-09-16T00:31:23.440467+0000","last_undegraded":"2024-09-16T00:31:23.440467+0000","last_fullsized":"2024-09-16T00:31:23.440467+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:42:18.998433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442466+0000","last_change":"2024-09-16T00:30:33.154748+0000","last_active":"2024-09-16T00:31:23.442466+0000","last_peered":"2024-09-16T00:31:23.442466+0000","last_clean":"2024-09-16T00:31:23.442466+0000","last_became_active":"2024-09-16T00:30:33.154305+0000","last_became_peered":"2024-09-16T00:30:33.154305+0000","last_unstale":"2024-09-16T00:31:23.442466+0000","last_undegraded":"2024-09-16T00:31:23.442466+0000","last_fullsized":"2024-09-16T00:31:23.442466+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:57:34.651844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.439471+0000","last_change":"2024-09-16T00:30:26.080517+0000","last_active":"2024-09-16T00:31:23.439471+0000","last_peered":"2024-09-16T00:31:23.439471+0000","last_clean":"2024-09-16T00:31:23.439471+0000","last_became_active":"2024-09-16T00:30:26.080197+0000","last_became_peered":"2024-09-16T00:30:26.080197+0000","last_unstale":"2024-09-16T00:31:23.439471+0000","last_undegraded":"2024-09-16T00:31:23.439471+0000","last_fullsized":"2024-09-16T00:31:23.439471+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:15.508562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.457335+0000","last_change":"2024-09-16T00:31:23.457047+0000","last_active":"2024-09-16T00:31:23.457335+0000","last_peered":"2024-09-16T00:31:23.457335+0000","last_clean":"2024-09-16T00:31:23.457335+0000","last_became_active":"2024-09-16T00:31:23.447014+0000","last_became_peered":"2024-09-16T00:31:23.447014+0000","last_unstale":"2024-09-16T00:31:23.457335+0000","last_undegraded":"2024-09-16T00:31:23.457335+0000","last_fullsized":"2024-09-16T00:31:23.457335+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:34:42.040228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.440632+0000","last_change":"2024-09-16T00:30:33.145803+0000","last_active":"2024-09-16T00:31:23.440632+0000","last_peered":"2024-09-16T00:31:23.440632+0000","last_clean":"2024-09-16T00:31:23.440632+0000","last_became_active":"2024-09-16T00:30:33.145366+0000","last_became_peered":"2024-09-16T00:30:33.145366+0000","last_unstale":"2024-09-16T00:31:23.440632+0000","last_undegraded":"2024-09-16T00:31:23.440632+0000","last_fullsized":"2024-09-16T00:31:23.440632+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:47:28.689305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.444640+0000","last_change":"2024-09-16T00:30:26.079415+0000","last_active":"2024-09-16T00:31:23.444640+0000","last_peered":"2024-09-16T00:31:23.444640+0000","last_clean":"2024-09-16T00:31:23.444640+0000","last_became_active":"2024-09-16T00:30:26.079075+0000","last_became_peered":"2024-09-16T00:30:26.079075+0000","last_unstale":"2024-09-16T00:31:23.444640+0000","last_undegraded":"2024-09-16T00:31:23.444640+0000","last_fullsized":"2024-09-16T00:31:23.444640+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:18:18.236342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"90'32","reported_seq":186,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:58.413171+0000","last_change":"2024-09-16T00:30:23.933172+0000","last_active":"2024-09-16T00:31:58.413171+0000","last_peered":"2024-09-16T00:31:58.413171+0000","last_clean":"2024-09-16T00:31:58.413171+0000","last_became_active":"2024-09-16T00:30:23.932492+0000","last_became_peered":"2024-09-16T00:30:23.932492+0000","last_unstale":"2024-09-16T00:31:58.413171+0000","last_undegraded":"2024-09-16T00:31:58.413171+0000","last_fullsized":"2024-09-16T00:31:58.413171+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:29.169448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":62076,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":120,"num_read_kb":122,"num_write":33,"num_write_kb":75,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.446248+0000","last_change":"2024-09-16T00:31:23.445673+0000","last_active":"2024-09-16T00:31:23.446248+0000","last_peered":"2024-09-16T00:31:23.446248+0000","last_clean":"2024-09-16T00:31:23.446248+0000","last_became_active":"2024-09-16T00:31:23.445299+0000","last_became_peered":"2024-09-16T00:31:23.445299+0000","last_unstale":"2024-09-16T00:31:23.446248+0000","last_undegraded":"2024-09-16T00:31:23.446248+0000","last_fullsized":"2024-09-16T00:31:23.446248+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:07:44.235340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.440745+0000","last_change":"2024-09-16T00:30:26.085419+0000","last_active":"2024-09-16T00:31:23.440745+0000","last_peered":"2024-09-16T00:31:23.440745+0000","last_clean":"2024-09-16T00:31:23.440745+0000","last_became_active":"2024-09-16T00:30:26.085103+0000","last_became_peered":"2024-09-16T00:30:26.085103+0000","last_unstale":"2024-09-16T00:31:23.440745+0000","last_undegraded":"2024-09-16T00:31:23.440745+0000","last_fullsized":"2024-09-16T00:31:23.440745+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:56:26.352779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":39,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.440913+0000","last_change":"2024-09-16T00:30:24.422646+0000","last_active":"2024-09-16T00:31:23.440913+0000","last_peered":"2024-09-16T00:31:23.440913+0000","last_clean":"2024-09-16T00:31:23.440913+0000","last_became_active":"2024-09-16T00:30:24.422065+0000","last_became_peered":"2024-09-16T00:30:24.422065+0000","last_unstale":"2024-09-16T00:31:23.440913+0000","last_undegraded":"2024-09-16T00:31:23.440913+0000","last_fullsized":"2024-09-16T00:31:23.440913+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:53:12.401613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.445491+0000","last_change":"2024-09-16T00:30:33.147692+0000","last_active":"2024-09-16T00:31:23.445491+0000","last_peered":"2024-09-16T00:31:23.445491+0000","last_clean":"2024-09-16T00:31:23.445491+0000","last_became_active":"2024-09-16T00:30:33.147115+0000","last_became_peered":"2024-09-16T00:30:33.147115+0000","last_unstale":"2024-09-16T00:31:23.445491+0000","last_undegraded":"2024-09-16T00:31:23.445491+0000","last_fullsized":"2024-09-16T00:31:23.445491+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:44:04.536894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.577333+0000","last_change":"2024-09-16T00:30:26.088427+0000","last_active":"2024-09-16T00:31:23.577333+0000","last_peered":"2024-09-16T00:31:23.577333+0000","last_clean":"2024-09-16T00:31:23.577333+0000","last_became_active":"2024-09-16T00:30:26.086454+0000","last_became_peered":"2024-09-16T00:30:26.086454+0000","last_unstale":"2024-09-16T00:31:23.577333+0000","last_undegraded":"2024-09-16T00:31:23.577333+0000","last_fullsized":"2024-09-16T00:31:23.577333+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:54:29.679959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.453484+0000","last_change":"2024-09-16T00:31:23.453484+0000","last_active":"2024-09-16T00:31:23.453484+0000","last_peered":"2024-09-16T00:31:23.453484+0000","last_clean":"2024-09-16T00:31:23.453484+0000","last_became_active":"2024-09-16T00:31:23.446700+0000","last_became_peered":"2024-09-16T00:31:23.446700+0000","last_unstale":"2024-09-16T00:31:23.453484+0000","last_undegraded":"2024-09-16T00:31:23.453484+0000","last_fullsized":"2024-09-16T00:31:23.453484+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:43:26.465657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.441598+0000","last_change":"2024-09-16T00:30:33.156510+0000","last_active":"2024-09-16T00:31:23.441598+0000","last_peered":"2024-09-16T00:31:23.441598+0000","last_clean":"2024-09-16T00:31:23.441598+0000","last_became_active":"2024-09-16T00:30:33.156088+0000","last_became_peered":"2024-09-16T00:30:33.156088+0000","last_unstale":"2024-09-16T00:31:23.441598+0000","last_undegraded":"2024-09-16T00:31:23.441598+0000","last_fullsized":"2024-09-16T00:31:23.441598+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:34.230901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.440674+0000","last_change":"2024-09-16T00:30:26.082598+0000","last_active":"2024-09-16T00:31:23.440674+0000","last_peered":"2024-09-16T00:31:23.440674+0000","last_clean":"2024-09-16T00:31:23.440674+0000","last_became_active":"2024-09-16T00:30:26.082286+0000","last_became_peered":"2024-09-16T00:30:26.082286+0000","last_unstale":"2024-09-16T00:31:23.440674+0000","last_undegraded":"2024-09-16T00:31:23.440674+0000","last_fullsized":"2024-09-16T00:31:23.440674+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:22:18.579149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"88'3","reported_seq":93,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:57.411409+0000","last_change":"2024-09-16T00:30:23.931097+0000","last_active":"2024-09-16T00:31:57.411409+0000","last_peered":"2024-09-16T00:31:57.411409+0000","last_clean":"2024-09-16T00:31:57.411409+0000","last_became_active":"2024-09-16T00:30:23.930719+0000","last_became_peered":"2024-09-16T00:30:23.930719+0000","last_unstale":"2024-09-16T00:31:57.411409+0000","last_undegraded":"2024-09-16T00:31:57.411409+0000","last_fullsized":"2024-09-16T00:31:57.411409+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:13:47.887754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":56,"num_read_kb":56,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.444716+0000","last_change":"2024-09-16T00:30:33.142581+0000","last_active":"2024-09-16T00:31:23.444716+0000","last_peered":"2024-09-16T00:31:23.444716+0000","last_clean":"2024-09-16T00:31:23.444716+0000","last_became_active":"2024-09-16T00:30:33.142263+0000","last_became_peered":"2024-09-16T00:30:33.142263+0000","last_unstale":"2024-09-16T00:31:23.444716+0000","last_undegraded":"2024-09-16T00:31:23.444716+0000","last_fullsized":"2024-09-16T00:31:23.444716+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:10:06.456011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.577032+0000","last_change":"2024-09-16T00:30:33.152084+0000","last_active":"2024-09-16T00:31:23.577032+0000","last_peered":"2024-09-16T00:31:23.577032+0000","last_clean":"2024-09-16T00:31:23.577032+0000","last_became_active":"2024-09-16T00:30:33.151681+0000","last_became_peered":"2024-09-16T00:30:33.151681+0000","last_unstale":"2024-09-16T00:31:23.577032+0000","last_undegraded":"2024-09-16T00:31:23.577032+0000","last_fullsized":"2024-09-16T00:31:23.577032+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:36.966498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.441760+0000","last_change":"2024-09-16T00:30:26.084200+0000","last_active":"2024-09-16T00:31:23.441760+0000","last_peered":"2024-09-16T00:31:23.441760+0000","last_clean":"2024-09-16T00:31:23.441760+0000","last_became_active":"2024-09-16T00:30:26.083858+0000","last_became_peered":"2024-09-16T00:30:26.083858+0000","last_unstale":"2024-09-16T00:31:23.441760+0000","last_undegraded":"2024-09-16T00:31:23.441760+0000","last_fullsized":"2024-09-16T00:31:23.441760+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:42.712624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.440233+0000","last_change":"2024-09-16T00:30:23.932399+0000","last_active":"2024-09-16T00:31:23.440233+0000","last_peered":"2024-09-16T00:31:23.440233+0000","last_clean":"2024-09-16T00:31:23.440233+0000","last_became_active":"2024-09-16T00:30:23.931996+0000","last_became_peered":"2024-09-16T00:30:23.931996+0000","last_unstale":"2024-09-16T00:31:23.440233+0000","last_undegraded":"2024-09-16T00:31:23.440233+0000","last_fullsized":"2024-09-16T00:31:23.440233+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:47:09.634507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442880+0000","last_change":"2024-09-16T00:30:33.154563+0000","last_active":"2024-09-16T00:31:23.442880+0000","last_peered":"2024-09-16T00:31:23.442880+0000","last_clean":"2024-09-16T00:31:23.442880+0000","last_became_active":"2024-09-16T00:30:33.154105+0000","last_became_peered":"2024-09-16T00:30:33.154105+0000","last_unstale":"2024-09-16T00:31:23.442880+0000","last_undegraded":"2024-09-16T00:31:23.442880+0000","last_fullsized":"2024-09-16T00:31:23.442880+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:59.966255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.444616+0000","last_change":"2024-09-16T00:30:26.075126+0000","last_active":"2024-09-16T00:31:23.444616+0000","last_peered":"2024-09-16T00:31:23.444616+0000","last_clean":"2024-09-16T00:31:23.444616+0000","last_became_active":"2024-09-16T00:30:26.074819+0000","last_became_peered":"2024-09-16T00:30:26.074819+0000","last_unstale":"2024-09-16T00:31:23.444616+0000","last_undegraded":"2024-09-16T00:31:23.444616+0000","last_fullsized":"2024-09-16T00:31:23.444616+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:36.996346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.439321+0000","last_change":"2024-09-16T00:30:23.929943+0000","last_active":"2024-09-16T00:31:23.439321+0000","last_peered":"2024-09-16T00:31:23.439321+0000","last_clean":"2024-09-16T00:31:23.439321+0000","last_became_active":"2024-09-16T00:30:23.929578+0000","last_became_peered":"2024-09-16T00:30:23.929578+0000","last_unstale":"2024-09-16T00:31:23.439321+0000","last_undegraded":"2024-09-16T00:31:23.439321+0000","last_fullsized":"2024-09-16T00:31:23.439321+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:05:12.505543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":37,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442749+0000","last_change":"2024-09-16T00:30:23.935758+0000","last_active":"2024-09-16T00:31:23.442749+0000","last_peered":"2024-09-16T00:31:23.442749+0000","last_clean":"2024-09-16T00:31:23.442749+0000","last_became_active":"2024-09-16T00:30:23.935321+0000","last_became_peered":"2024-09-16T00:30:23.935321+0000","last_unstale":"2024-09-16T00:31:23.442749+0000","last_undegraded":"2024-09-16T00:31:23.442749+0000","last_fullsized":"2024-09-16T00:31:23.442749+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:35:40.513435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.440500+0000","last_change":"2024-09-16T00:30:26.086316+0000","last_active":"2024-09-16T00:31:23.440500+0000","last_peered":"2024-09-16T00:31:23.440500+0000","last_clean":"2024-09-16T00:31:23.440500+0000","last_became_active":"2024-09-16T00:30:26.085800+0000","last_became_peered":"2024-09-16T00:30:26.085800+0000","last_unstale":"2024-09-16T00:31:23.440500+0000","last_undegraded":"2024-09-16T00:31:23.440500+0000","last_fullsized":"2024-09-16T00:31:23.440500+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:05:36.277127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.444019+0000","last_change":"2024-09-16T00:30:33.155665+0000","last_active":"2024-09-16T00:31:23.444019+0000","last_peered":"2024-09-16T00:31:23.444019+0000","last_clean":"2024-09-16T00:31:23.444019+0000","last_became_active":"2024-09-16T00:30:33.155253+0000","last_became_peered":"2024-09-16T00:30:33.155253+0000","last_unstale":"2024-09-16T00:31:23.444019+0000","last_undegraded":"2024-09-16T00:31:23.444019+0000","last_fullsized":"2024-09-16T00:31:23.444019+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:37.545232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":98,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:56.410559+0000","last_change":"2024-09-16T00:30:23.934487+0000","last_active":"2024-09-16T00:31:56.410559+0000","last_peered":"2024-09-16T00:31:56.410559+0000","last_clean":"2024-09-16T00:31:56.410559+0000","last_became_active":"2024-09-16T00:30:23.934137+0000","last_became_peered":"2024-09-16T00:30:23.934137+0000","last_unstale":"2024-09-16T00:31:56.410559+0000","last_undegraded":"2024-09-16T00:31:56.410559+0000","last_fullsized":"2024-09-16T00:31:56.410559+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:14:33.088776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1015,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":58,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.443467+0000","last_change":"2024-09-16T00:30:26.077533+0000","last_active":"2024-09-16T00:31:23.443467+0000","last_peered":"2024-09-16T00:31:23.443467+0000","last_clean":"2024-09-16T00:31:23.443467+0000","last_became_active":"2024-09-16T00:30:26.076443+0000","last_became_peered":"2024-09-16T00:30:26.076443+0000","last_unstale":"2024-09-16T00:31:23.443467+0000","last_undegraded":"2024-09-16T00:31:23.443467+0000","last_fullsized":"2024-09-16T00:31:23.443467+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:39:17.495521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":21,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.443290+0000","last_change":"2024-09-16T00:30:33.146123+0000","last_active":"2024-09-16T00:31:23.443290+0000","last_peered":"2024-09-16T00:31:23.443290+0000","last_clean":"2024-09-16T00:31:23.443290+0000","last_became_active":"2024-09-16T00:30:33.145716+0000","last_became_peered":"2024-09-16T00:30:33.145716+0000","last_unstale":"2024-09-16T00:31:23.443290+0000","last_undegraded":"2024-09-16T00:31:23.443290+0000","last_fullsized":"2024-09-16T00:31:23.443290+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:14:27.281367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442159+0000","last_change":"2024-09-16T00:30:26.083155+0000","last_active":"2024-09-16T00:31:23.442159+0000","last_peered":"2024-09-16T00:31:23.442159+0000","last_clean":"2024-09-16T00:31:23.442159+0000","last_became_active":"2024-09-16T00:30:26.082782+0000","last_became_peered":"2024-09-16T00:30:26.082782+0000","last_unstale":"2024-09-16T00:31:23.442159+0000","last_undegraded":"2024-09-16T00:31:23.442159+0000","last_fullsized":"2024-09-16T00:31:23.442159+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:15.929343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.447453+0000","last_change":"2024-09-16T00:31:23.447453+0000","last_active":"2024-09-16T00:31:23.447453+0000","last_peered":"2024-09-16T00:31:23.447453+0000","last_clean":"2024-09-16T00:31:23.447453+0000","last_became_active":"2024-09-16T00:31:23.445507+0000","last_became_peered":"2024-09-16T00:31:23.445507+0000","last_unstale":"2024-09-16T00:31:23.447453+0000","last_undegraded":"2024-09-16T00:31:23.447453+0000","last_fullsized":"2024-09-16T00:31:23.447453+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:18:50.885809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442695+0000","last_change":"2024-09-16T00:30:26.076252+0000","last_active":"2024-09-16T00:31:23.442695+0000","last_peered":"2024-09-16T00:31:23.442695+0000","last_clean":"2024-09-16T00:31:23.442695+0000","last_became_active":"2024-09-16T00:30:26.075922+0000","last_became_peered":"2024-09-16T00:30:26.075922+0000","last_unstale":"2024-09-16T00:31:23.442695+0000","last_undegraded":"2024-09-16T00:31:23.442695+0000","last_fullsized":"2024-09-16T00:31:23.442695+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:28:04.822817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.577020+0000","last_change":"2024-09-16T00:30:26.085123+0000","last_active":"2024-09-16T00:31:23.577020+0000","last_peered":"2024-09-16T00:31:23.577020+0000","last_clean":"2024-09-16T00:31:23.577020+0000","last_became_active":"2024-09-16T00:30:26.084782+0000","last_became_peered":"2024-09-16T00:30:26.084782+0000","last_unstale":"2024-09-16T00:31:23.577020+0000","last_undegraded":"2024-09-16T00:31:23.577020+0000","last_fullsized":"2024-09-16T00:31:23.577020+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:09:33.766340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.439865+0000","last_change":"2024-09-16T00:30:26.077048+0000","last_active":"2024-09-16T00:31:23.439865+0000","last_peered":"2024-09-16T00:31:23.439865+0000","last_clean":"2024-09-16T00:31:23.439865+0000","last_became_active":"2024-09-16T00:30:26.076719+0000","last_became_peered":"2024-09-16T00:30:26.076719+0000","last_unstale":"2024-09-16T00:31:23.439865+0000","last_undegraded":"2024-09-16T00:31:23.439865+0000","last_fullsized":"2024-09-16T00:31:23.439865+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:58.584246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442997+0000","last_change":"2024-09-16T00:30:26.078446+0000","last_active":"2024-09-16T00:31:23.442997+0000","last_peered":"2024-09-16T00:31:23.442997+0000","last_clean":"2024-09-16T00:31:23.442997+0000","last_became_active":"2024-09-16T00:30:26.078145+0000","last_became_peered":"2024-09-16T00:30:26.078145+0000","last_unstale":"2024-09-16T00:31:23.442997+0000","last_undegraded":"2024-09-16T00:31:23.442997+0000","last_fullsized":"2024-09-16T00:31:23.442997+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:52:28.739834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.444338+0000","last_change":"2024-09-16T00:30:26.075875+0000","last_active":"2024-09-16T00:31:23.444338+0000","last_peered":"2024-09-16T00:31:23.444338+0000","last_clean":"2024-09-16T00:31:23.444338+0000","last_became_active":"2024-09-16T00:30:26.075424+0000","last_became_peered":"2024-09-16T00:30:26.075424+0000","last_unstale":"2024-09-16T00:31:23.444338+0000","last_undegraded":"2024-09-16T00:31:23.444338+0000","last_fullsized":"2024-09-16T00:31:23.444338+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:21:07.958507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.445311+0000","last_change":"2024-09-16T00:30:26.077694+0000","last_active":"2024-09-16T00:31:23.445311+0000","last_peered":"2024-09-16T00:31:23.445311+0000","last_clean":"2024-09-16T00:31:23.445311+0000","last_became_active":"2024-09-16T00:30:26.077374+0000","last_became_peered":"2024-09-16T00:30:26.077374+0000","last_unstale":"2024-09-16T00:31:23.445311+0000","last_undegraded":"2024-09-16T00:31:23.445311+0000","last_fullsized":"2024-09-16T00:31:23.445311+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:16:30.822050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.439962+0000","last_change":"2024-09-16T00:30:26.085931+0000","last_active":"2024-09-16T00:31:23.439962+0000","last_peered":"2024-09-16T00:31:23.439962+0000","last_clean":"2024-09-16T00:31:23.439962+0000","last_became_active":"2024-09-16T00:30:26.085420+0000","last_became_peered":"2024-09-16T00:30:26.085420+0000","last_unstale":"2024-09-16T00:31:23.439962+0000","last_undegraded":"2024-09-16T00:31:23.439962+0000","last_fullsized":"2024-09-16T00:31:23.439962+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:20:36.019203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.444885+0000","last_change":"2024-09-16T00:30:26.074217+0000","last_active":"2024-09-16T00:31:23.444885+0000","last_peered":"2024-09-16T00:31:23.444885+0000","last_clean":"2024-09-16T00:31:23.444885+0000","last_became_active":"2024-09-16T00:30:26.073856+0000","last_became_peered":"2024-09-16T00:30:26.073856+0000","last_unstale":"2024-09-16T00:31:23.444885+0000","last_undegraded":"2024-09-16T00:31:23.444885+0000","last_fullsized":"2024-09-16T00:31:23.444885+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:53:57.078465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.440598+0000","last_change":"2024-09-16T00:30:26.084248+0000","last_active":"2024-09-16T00:31:23.440598+0000","last_peered":"2024-09-16T00:31:23.440598+0000","last_clean":"2024-09-16T00:31:23.440598+0000","last_became_active":"2024-09-16T00:30:26.083530+0000","last_became_peered":"2024-09-16T00:30:26.083530+0000","last_unstale":"2024-09-16T00:31:23.440598+0000","last_undegraded":"2024-09-16T00:31:23.440598+0000","last_fullsized":"2024-09-16T00:31:23.440598+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:16:49.483881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.439703+0000","last_change":"2024-09-16T00:30:26.080803+0000","last_active":"2024-09-16T00:31:23.439703+0000","last_peered":"2024-09-16T00:31:23.439703+0000","last_clean":"2024-09-16T00:31:23.439703+0000","last_became_active":"2024-09-16T00:30:26.080482+0000","last_became_peered":"2024-09-16T00:30:26.080482+0000","last_unstale":"2024-09-16T00:31:23.439703+0000","last_undegraded":"2024-09-16T00:31:23.439703+0000","last_fullsized":"2024-09-16T00:31:23.439703+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:35:41.741025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442585+0000","last_change":"2024-09-16T00:30:26.077680+0000","last_active":"2024-09-16T00:31:23.442585+0000","last_peered":"2024-09-16T00:31:23.442585+0000","last_clean":"2024-09-16T00:31:23.442585+0000","last_became_active":"2024-09-16T00:30:26.077371+0000","last_became_peered":"2024-09-16T00:30:26.077371+0000","last_unstale":"2024-09-16T00:31:23.442585+0000","last_undegraded":"2024-09-16T00:31:23.442585+0000","last_fullsized":"2024-09-16T00:31:23.442585+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:31:34.338617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.443453+0000","last_change":"2024-09-16T00:30:26.071531+0000","last_active":"2024-09-16T00:31:23.443453+0000","last_peered":"2024-09-16T00:31:23.443453+0000","last_clean":"2024-09-16T00:31:23.443453+0000","last_became_active":"2024-09-16T00:30:26.071189+0000","last_became_peered":"2024-09-16T00:30:26.071189+0000","last_unstale":"2024-09-16T00:31:23.443453+0000","last_undegraded":"2024-09-16T00:31:23.443453+0000","last_fullsized":"2024-09-16T00:31:23.443453+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:29:18.406272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442215+0000","last_change":"2024-09-16T00:30:26.085398+0000","last_active":"2024-09-16T00:31:23.442215+0000","last_peered":"2024-09-16T00:31:23.442215+0000","last_clean":"2024-09-16T00:31:23.442215+0000","last_became_active":"2024-09-16T00:30:26.085076+0000","last_became_peered":"2024-09-16T00:30:26.085076+0000","last_unstale":"2024-09-16T00:31:23.442215+0000","last_undegraded":"2024-09-16T00:31:23.442215+0000","last_fullsized":"2024-09-16T00:31:23.442215+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:20:20.864660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.443283+0000","last_change":"2024-09-16T00:30:26.080953+0000","last_active":"2024-09-16T00:31:23.443283+0000","last_peered":"2024-09-16T00:31:23.443283+0000","last_clean":"2024-09-16T00:31:23.443283+0000","last_became_active":"2024-09-16T00:30:26.080185+0000","last_became_peered":"2024-09-16T00:30:26.080185+0000","last_unstale":"2024-09-16T00:31:23.443283+0000","last_undegraded":"2024-09-16T00:31:23.443283+0000","last_fullsized":"2024-09-16T00:31:23.443283+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:56.769182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.440107+0000","last_change":"2024-09-16T00:30:26.079320+0000","last_active":"2024-09-16T00:31:23.440107+0000","last_peered":"2024-09-16T00:31:23.440107+0000","last_clean":"2024-09-16T00:31:23.440107+0000","last_became_active":"2024-09-16T00:30:26.078980+0000","last_became_peered":"2024-09-16T00:30:26.078980+0000","last_unstale":"2024-09-16T00:31:23.440107+0000","last_undegraded":"2024-09-16T00:31:23.440107+0000","last_fullsized":"2024-09-16T00:31:23.440107+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:03:15.386657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.576817+0000","last_change":"2024-09-16T00:30:26.084948+0000","last_active":"2024-09-16T00:31:23.576817+0000","last_peered":"2024-09-16T00:31:23.576817+0000","last_clean":"2024-09-16T00:31:23.576817+0000","last_became_active":"2024-09-16T00:30:26.084602+0000","last_became_peered":"2024-09-16T00:30:26.084602+0000","last_unstale":"2024-09-16T00:31:23.576817+0000","last_undegraded":"2024-09-16T00:31:23.576817+0000","last_fullsized":"2024-09-16T00:31:23.576817+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:01.709332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.576775+0000","last_change":"2024-09-16T00:30:26.080491+0000","last_active":"2024-09-16T00:31:23.576775+0000","last_peered":"2024-09-16T00:31:23.576775+0000","last_clean":"2024-09-16T00:31:23.576775+0000","last_became_active":"2024-09-16T00:30:26.080189+0000","last_became_peered":"2024-09-16T00:30:26.080189+0000","last_unstale":"2024-09-16T00:31:23.576775+0000","last_undegraded":"2024-09-16T00:31:23.576775+0000","last_fullsized":"2024-09-16T00:31:23.576775+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:38.358919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.439700+0000","last_change":"2024-09-16T00:30:26.077410+0000","last_active":"2024-09-16T00:31:23.439700+0000","last_peered":"2024-09-16T00:31:23.439700+0000","last_clean":"2024-09-16T00:31:23.439700+0000","last_became_active":"2024-09-16T00:30:26.076498+0000","last_became_peered":"2024-09-16T00:30:26.076498+0000","last_unstale":"2024-09-16T00:31:23.439700+0000","last_undegraded":"2024-09-16T00:31:23.439700+0000","last_fullsized":"2024-09-16T00:31:23.439700+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:54:17.712774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.445826+0000","last_change":"2024-09-16T00:30:26.074500+0000","last_active":"2024-09-16T00:31:23.445826+0000","last_peered":"2024-09-16T00:31:23.445826+0000","last_clean":"2024-09-16T00:31:23.445826+0000","last_became_active":"2024-09-16T00:30:26.074177+0000","last_became_peered":"2024-09-16T00:30:26.074177+0000","last_unstale":"2024-09-16T00:31:23.445826+0000","last_undegraded":"2024-09-16T00:31:23.445826+0000","last_fullsized":"2024-09-16T00:31:23.445826+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:43:13.792231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.443052+0000","last_change":"2024-09-16T00:30:26.078358+0000","last_active":"2024-09-16T00:31:23.443052+0000","last_peered":"2024-09-16T00:31:23.443052+0000","last_clean":"2024-09-16T00:31:23.443052+0000","last_became_active":"2024-09-16T00:30:26.077993+0000","last_became_peered":"2024-09-16T00:30:26.077993+0000","last_unstale":"2024-09-16T00:31:23.443052+0000","last_undegraded":"2024-09-16T00:31:23.443052+0000","last_fullsized":"2024-09-16T00:31:23.443052+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:25:10.711394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":35,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.446836+0000","last_change":"2024-09-16T00:31:23.446836+0000","last_active":"2024-09-16T00:31:23.446836+0000","last_peered":"2024-09-16T00:31:23.446836+0000","last_clean":"2024-09-16T00:31:23.446836+0000","last_became_active":"2024-09-16T00:31:23.446471+0000","last_became_peered":"2024-09-16T00:31:23.446471+0000","last_unstale":"2024-09-16T00:31:23.446836+0000","last_undegraded":"2024-09-16T00:31:23.446836+0000","last_fullsized":"2024-09-16T00:31:23.446836+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:08:34.887618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.441321+0000","last_change":"2024-09-16T00:30:26.086313+0000","last_active":"2024-09-16T00:31:23.441321+0000","last_peered":"2024-09-16T00:31:23.441321+0000","last_clean":"2024-09-16T00:31:23.441321+0000","last_became_active":"2024-09-16T00:30:26.085760+0000","last_became_peered":"2024-09-16T00:30:26.085760+0000","last_unstale":"2024-09-16T00:31:23.441321+0000","last_undegraded":"2024-09-16T00:31:23.441321+0000","last_fullsized":"2024-09-16T00:31:23.441321+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:40:38.236789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.444527+0000","last_change":"2024-09-16T00:30:26.085289+0000","last_active":"2024-09-16T00:31:23.444527+0000","last_peered":"2024-09-16T00:31:23.444527+0000","last_clean":"2024-09-16T00:31:23.444527+0000","last_became_active":"2024-09-16T00:30:26.084995+0000","last_became_peered":"2024-09-16T00:30:26.084995+0000","last_unstale":"2024-09-16T00:31:23.444527+0000","last_undegraded":"2024-09-16T00:31:23.444527+0000","last_fullsized":"2024-09-16T00:31:23.444527+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:45.667892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.443030+0000","last_change":"2024-09-16T00:30:26.088690+0000","last_active":"2024-09-16T00:31:23.443030+0000","last_peered":"2024-09-16T00:31:23.443030+0000","last_clean":"2024-09-16T00:31:23.443030+0000","last_became_active":"2024-09-16T00:30:26.087700+0000","last_became_peered":"2024-09-16T00:30:26.087700+0000","last_unstale":"2024-09-16T00:31:23.443030+0000","last_undegraded":"2024-09-16T00:31:23.443030+0000","last_fullsized":"2024-09-16T00:31:23.443030+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:39:51.445324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442664+0000","last_change":"2024-09-16T00:30:26.080215+0000","last_active":"2024-09-16T00:31:23.442664+0000","last_peered":"2024-09-16T00:31:23.442664+0000","last_clean":"2024-09-16T00:31:23.442664+0000","last_became_active":"2024-09-16T00:30:26.079038+0000","last_became_peered":"2024-09-16T00:30:26.079038+0000","last_unstale":"2024-09-16T00:31:23.442664+0000","last_undegraded":"2024-09-16T00:31:23.442664+0000","last_fullsized":"2024-09-16T00:31:23.442664+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:01.573960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.442533+0000","last_change":"2024-09-16T00:30:26.080966+0000","last_active":"2024-09-16T00:31:23.442533+0000","last_peered":"2024-09-16T00:31:23.442533+0000","last_clean":"2024-09-16T00:31:23.442533+0000","last_became_active":"2024-09-16T00:30:26.080641+0000","last_became_peered":"2024-09-16T00:30:26.080641+0000","last_unstale":"2024-09-16T00:31:23.442533+0000","last_undegraded":"2024-09-16T00:31:23.442533+0000","last_fullsized":"2024-09-16T00:31:23.442533+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:33:01.538169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.445705+0000","last_change":"2024-09-16T00:30:26.072644+0000","last_active":"2024-09-16T00:31:23.445705+0000","last_peered":"2024-09-16T00:31:23.445705+0000","last_clean":"2024-09-16T00:31:23.445705+0000","last_became_active":"2024-09-16T00:30:26.072336+0000","last_became_peered":"2024-09-16T00:30:26.072336+0000","last_unstale":"2024-09-16T00:31:23.445705+0000","last_undegraded":"2024-09-16T00:31:23.445705+0000","last_fullsized":"2024-09-16T00:31:23.445705+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:44:45.760057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.577011+0000","last_change":"2024-09-16T00:30:26.079155+0000","last_active":"2024-09-16T00:31:23.577011+0000","last_peered":"2024-09-16T00:31:23.577011+0000","last_clean":"2024-09-16T00:31:23.577011+0000","last_became_active":"2024-09-16T00:30:26.078820+0000","last_became_peered":"2024-09-16T00:30:26.078820+0000","last_unstale":"2024-09-16T00:31:23.577011+0000","last_undegraded":"2024-09-16T00:31:23.577011+0000","last_fullsized":"2024-09-16T00:31:23.577011+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:35.071143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.577087+0000","last_change":"2024-09-16T00:30:26.081537+0000","last_active":"2024-09-16T00:31:23.577087+0000","last_peered":"2024-09-16T00:31:23.577087+0000","last_clean":"2024-09-16T00:31:23.577087+0000","last_became_active":"2024-09-16T00:30:26.080744+0000","last_became_peered":"2024-09-16T00:30:26.080744+0000","last_unstale":"2024-09-16T00:31:23.577087+0000","last_undegraded":"2024-09-16T00:31:23.577087+0000","last_fullsized":"2024-09-16T00:31:23.577087+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:46:38.179888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-09-16T00:31:23.443231+0000","last_change":"2024-09-16T00:30:26.081066+0000","last_active":"2024-09-16T00:31:23.443231+0000","last_peered":"2024-09-16T00:31:23.443231+0000","last_clean":"2024-09-16T00:31:23.443231+0000","last_became_active":"2024-09-16T00:30:26.080630+0000","last_became_peered":"2024-09-16T00:30:26.080630+0000","last_unstale":"2024-09-16T00:31:23.443231+0000","last_undegraded":"2024-09-16T00:31:23.443231+0000","last_fullsized":"2024-09-16T00:31:23.443231+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:44:42.644792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":66079,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":357,"num_read_kb":249,"num_write":166,"num_write_kb":115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":198237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":142,"ondisk_log_size":142,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514528,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1028096,"data_stored":504478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743462,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32852,"kb_used_data":1668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712300,"statfs":{"total":95995035648,"available":95961395200,"internally_reserved":0,"allocated":1708032,"data_stored":1153766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906987,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1040384,"data_stored":508754,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135920,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36360,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708792,"statfs":{"total":95995035648,"available":95957803008,"internally_reserved":0,"allocated":1032192,"data_stored":504556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397556,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28624,"kb_used_data":1592,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716528,"statfs":{"total":95995035648,"available":95965724672,"internally_reserved":0,"allocated":1630208,"data_stored":1100489,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561082,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1015808,"data_stored":499834,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790015,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28628,"kb_used_data":1600,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716524,"statfs":{"total":95995035648,"available":95965720576,"internally_reserved":0,"allocated":1638400,"data_stored":1096077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953541,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1028096,"data_stored":504484,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182472,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32192,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712960,"statfs":{"total":95995035648,"available":95962071040,"internally_reserved":0,"allocated":1028096,"data_stored":500028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1101824,"data_stored":563529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1081344,"data_stored":561820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1040384,"data_stored":501222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":62076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":63654,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:32:01.422 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-16T00:32:01.423 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-16T00:32:01.423 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-16T00:32:01.423 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph health --format=json 2024-09-16T00:32:01.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:01 smithi122 ceph-mon[35412]: pgmap v339: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T00:32:01.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:01 smithi073 ceph-mon[35254]: pgmap v339: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T00:32:01.683 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:02.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.166+0000 7efca3302640 1 -- 172.21.15.38:0/3225093820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc9c108300 msgr2=0x7efc9c10ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:02.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.166+0000 7efca3302640 1 --2- 172.21.15.38:0/3225093820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc9c108300 0x7efc9c10ec70 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7efc94009cd0 tx=0x7efc9402a8d0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.166+0000 7efca3302640 1 -- 172.21.15.38:0/3225093820 shutdown_connections 2024-09-16T00:32:02.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.166+0000 7efca3302640 1 --2- 172.21.15.38:0/3225093820 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc9c108300 0x7efc9c10ec70 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.166+0000 7efca3302640 1 --2- 172.21.15.38:0/3225093820 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc9c103780 0x7efc9c107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.166+0000 7efca3302640 1 --2- 172.21.15.38:0/3225093820 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc9c102dd0 0x7efc9c1031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.166+0000 7efca3302640 1 -- 172.21.15.38:0/3225093820 >> 172.21.15.38:0/3225093820 conn(0x7efc9c0fc7d0 msgr2=0x7efc9c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:02.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.166+0000 7efca3302640 1 -- 172.21.15.38:0/3225093820 shutdown_connections 2024-09-16T00:32:02.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.166+0000 7efca3302640 1 -- 172.21.15.38:0/3225093820 wait complete. 2024-09-16T00:32:02.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.166+0000 7efca3302640 1 Processor -- start 2024-09-16T00:32:02.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.166+0000 7efca3302640 1 -- start start 2024-09-16T00:32:02.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.167+0000 7efca3302640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc9c102dd0 0x7efc9c198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:02.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.167+0000 7efca3302640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc9c103780 0x7efc9c198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:02.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.167+0000 7efca3302640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc9c108300 0x7efc9c19ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:02.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.167+0000 7efca3302640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c1120d0 con 0x7efc9c102dd0 2024-09-16T00:32:02.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.167+0000 7efca3302640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c111f50 con 0x7efc9c108300 2024-09-16T00:32:02.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.167+0000 7efca3302640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efc9c112250 con 0x7efc9c103780 2024-09-16T00:32:02.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.167+0000 7efca109e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc9c102dd0 0x7efc9c198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:02.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.167+0000 7efca109e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc9c102dd0 0x7efc9c198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35034/0 (socket says 172.21.15.38:35034) 2024-09-16T00:32:02.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.167+0000 7efca109e640 1 -- 172.21.15.38:0/1414444302 learned_addr learned my addr 172.21.15.38:0/1414444302 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:02.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.167+0000 7efca189f640 1 --2- 172.21.15.38:0/1414444302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc9c108300 0x7efc9c19ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:02.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.167+0000 7efca109e640 1 -- 172.21.15.38:0/1414444302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc9c103780 msgr2=0x7efc9c198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:02.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.167+0000 7efca109e640 1 --2- 172.21.15.38:0/1414444302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc9c103780 0x7efc9c198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.167+0000 7efca109e640 1 -- 172.21.15.38:0/1414444302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc9c108300 msgr2=0x7efc9c19ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:02.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.167+0000 7efca109e640 1 --2- 172.21.15.38:0/1414444302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc9c108300 0x7efc9c19ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.167+0000 7efca109e640 1 -- 172.21.15.38:0/1414444302 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc9c19d540 con 0x7efc9c102dd0 2024-09-16T00:32:02.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.167+0000 7efca189f640 1 --2- 172.21.15.38:0/1414444302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc9c108300 0x7efc9c19ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:32:02.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.167+0000 7efca109e640 1 --2- 172.21.15.38:0/1414444302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc9c102dd0 0x7efc9c198590 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7efc9800d770 tx=0x7efc9800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:02.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.167+0000 7efc927fc640 1 -- 172.21.15.38:0/1414444302 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc98017070 con 0x7efc9c102dd0 2024-09-16T00:32:02.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.168+0000 7efca3302640 1 -- 172.21.15.38:0/1414444302 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc9c19d830 con 0x7efc9c102dd0 2024-09-16T00:32:02.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.168+0000 7efc927fc640 1 -- 172.21.15.38:0/1414444302 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efc98005500 con 0x7efc9c102dd0 2024-09-16T00:32:02.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.168+0000 7efca3302640 1 -- 172.21.15.38:0/1414444302 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc9c1a5110 con 0x7efc9c102dd0 2024-09-16T00:32:02.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.168+0000 7efc927fc640 1 -- 172.21.15.38:0/1414444302 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc98020060 con 0x7efc9c102dd0 2024-09-16T00:32:02.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.169+0000 7efca3302640 1 -- 172.21.15.38:0/1414444302 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc9c1031b0 con 0x7efc9c102dd0 2024-09-16T00:32:02.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.170+0000 7efc927fc640 1 -- 172.21.15.38:0/1414444302 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7efc98004da0 con 0x7efc9c102dd0 2024-09-16T00:32:02.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.170+0000 7efc927fc640 1 --2- 172.21.15.38:0/1414444302 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efc880a32d0 0x7efc880a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:02.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.171+0000 7efc927fc640 1 -- 172.21.15.38:0/1414444302 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7efc980d0130 con 0x7efc9c102dd0 2024-09-16T00:32:02.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.171+0000 7efca089d640 1 --2- 172.21.15.38:0/1414444302 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efc880a32d0 0x7efc880a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:02.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.171+0000 7efca089d640 1 --2- 172.21.15.38:0/1414444302 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efc880a32d0 0x7efc880a5790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7efc8c005d80 tx=0x7efc8c005c90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:02.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.174+0000 7efc927fc640 1 -- 172.21.15.38:0/1414444302 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efc9809c9b0 con 0x7efc9c102dd0 2024-09-16T00:32:02.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.389+0000 7efca3302640 1 -- 172.21.15.38:0/1414444302 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7efc9c10a290 con 0x7efc9c102dd0 2024-09-16T00:32:02.390 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:02 smithi038 ceph-mon[30945]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:32:02.390 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.390+0000 7efc927fc640 1 -- 172.21.15.38:0/1414444302 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7efc98096e20 con 0x7efc9c102dd0 2024-09-16T00:32:02.391 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:32:02.391 INFO:teuthology.orchestra.run.smithi038.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-16T00:32:02.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.392+0000 7efca3302640 1 -- 172.21.15.38:0/1414444302 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efc880a32d0 msgr2=0x7efc880a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:02.392 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.392+0000 7efca3302640 1 --2- 172.21.15.38:0/1414444302 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efc880a32d0 0x7efc880a5790 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7efc8c005d80 tx=0x7efc8c005c90 comp rx=0 tx=0).stop 2024-09-16T00:32:02.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.392+0000 7efca3302640 1 -- 172.21.15.38:0/1414444302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc9c102dd0 msgr2=0x7efc9c198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:02.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.392+0000 7efca3302640 1 --2- 172.21.15.38:0/1414444302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc9c102dd0 0x7efc9c198590 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7efc9800d770 tx=0x7efc9800dc30 comp rx=0 tx=0).stop 2024-09-16T00:32:02.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.392+0000 7efca3302640 1 -- 172.21.15.38:0/1414444302 shutdown_connections 2024-09-16T00:32:02.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.392+0000 7efca3302640 1 --2- 172.21.15.38:0/1414444302 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efc880a32d0 0x7efc880a5790 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.392+0000 7efca3302640 1 --2- 172.21.15.38:0/1414444302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc9c108300 0x7efc9c19ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.392+0000 7efca3302640 1 --2- 172.21.15.38:0/1414444302 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efc9c103780 0x7efc9c198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.392+0000 7efca3302640 1 --2- 172.21.15.38:0/1414444302 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efc9c102dd0 0x7efc9c198590 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:02.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.392+0000 7efca3302640 1 -- 172.21.15.38:0/1414444302 >> 172.21.15.38:0/1414444302 conn(0x7efc9c0fc7d0 msgr2=0x7efc9c105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:02.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.392+0000 7efca3302640 1 -- 172.21.15.38:0/1414444302 shutdown_connections 2024-09-16T00:32:02.393 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:02.392+0000 7efca3302640 1 -- 172.21.15.38:0/1414444302 wait complete. 2024-09-16T00:32:02.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:02 smithi122 ceph-mon[35412]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:32:02.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:02 smithi073 ceph-mon[35254]: from='client.15021 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:32:03.102 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-16T00:32:03.102 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:32:03.360 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:03.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:03 smithi038 ceph-mon[30945]: pgmap v340: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T00:32:03.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:03 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/1414444302' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-16T00:32:03.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:03 smithi122 ceph-mon[35412]: pgmap v340: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T00:32:03.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:03 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/1414444302' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-16T00:32:03.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:03 smithi073 ceph-mon[35254]: pgmap v340: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T00:32:03.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:03 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/1414444302' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-16T00:32:03.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:03.842+0000 7fbae65ca640 1 -- 172.21.15.38:0/1959249545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbae01082a0 msgr2=0x7fbae010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:03.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:03.842+0000 7fbae65ca640 1 --2- 172.21.15.38:0/1959249545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbae01082a0 0x7fbae010ec40 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fbad8009cd0 tx=0x7fbad802a8e0 comp rx=0 tx=0).stop 2024-09-16T00:32:03.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:03.842+0000 7fbae65ca640 1 -- 172.21.15.38:0/1959249545 shutdown_connections 2024-09-16T00:32:03.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:03.842+0000 7fbae65ca640 1 --2- 172.21.15.38:0/1959249545 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbae01082a0 0x7fbae010ec40 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:03.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:03.842+0000 7fbae65ca640 1 --2- 172.21.15.38:0/1959249545 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbae01036e0 0x7fbae0107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:03.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:03.842+0000 7fbae65ca640 1 --2- 172.21.15.38:0/1959249545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbae0102d30 0x7fbae0103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:03.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:03.842+0000 7fbae65ca640 1 -- 172.21.15.38:0/1959249545 >> 172.21.15.38:0/1959249545 conn(0x7fbae00fc7d0 msgr2=0x7fbae00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:03.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:03.842+0000 7fbae65ca640 1 -- 172.21.15.38:0/1959249545 shutdown_connections 2024-09-16T00:32:03.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:03.842+0000 7fbae65ca640 1 -- 172.21.15.38:0/1959249545 wait complete. 2024-09-16T00:32:03.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:03.843+0000 7fbae65ca640 1 Processor -- start 2024-09-16T00:32:03.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:03.843+0000 7fbae65ca640 1 -- start start 2024-09-16T00:32:03.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:03.843+0000 7fbae65ca640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbae0102d30 0x7fbae0198540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:03.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:03.843+0000 7fbae65ca640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbae01036e0 0x7fbae0198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:03.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:03.843+0000 7fbae65ca640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbae01082a0 0x7fbae019ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:03.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:03.843+0000 7fbae65ca640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbae01120a0 con 0x7fbae01082a0 2024-09-16T00:32:03.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:03.843+0000 7fbae65ca640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbae0111f20 con 0x7fbae0102d30 2024-09-16T00:32:03.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:03.843+0000 7fbae65ca640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbae0112220 con 0x7fbae01036e0 2024-09-16T00:32:03.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:03.843+0000 7fbae4b67640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbae01082a0 0x7fbae019ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:03.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:03.843+0000 7fbae4b67640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbae01082a0 0x7fbae019ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35056/0 (socket says 172.21.15.38:35056) 2024-09-16T00:32:03.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:03.843+0000 7fbae4b67640 1 -- 172.21.15.38:0/2779050215 learned_addr learned my addr 172.21.15.38:0/2779050215 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:03.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:03.844+0000 7fbadffff640 1 --2- 172.21.15.38:0/2779050215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbae0102d30 0x7fbae0198540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:03.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:03.844+0000 7fbadf7fe640 1 --2- 172.21.15.38:0/2779050215 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbae01036e0 0x7fbae0198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:03.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:03.844+0000 7fbae4b67640 1 -- 172.21.15.38:0/2779050215 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbae01036e0 msgr2=0x7fbae0198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:03.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:03.844+0000 7fbae4b67640 1 --2- 172.21.15.38:0/2779050215 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbae01036e0 0x7fbae0198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:03.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:03.844+0000 7fbae4b67640 1 -- 172.21.15.38:0/2779050215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbae0102d30 msgr2=0x7fbae0198540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:03.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:03.844+0000 7fbae4b67640 1 --2- 172.21.15.38:0/2779050215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbae0102d30 0x7fbae0198540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:03.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:03.844+0000 7fbae4b67640 1 -- 172.21.15.38:0/2779050215 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbae019d4f0 con 0x7fbae01082a0 2024-09-16T00:32:03.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:03.844+0000 7fbae4b67640 1 --2- 172.21.15.38:0/2779050215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbae01082a0 0x7fbae019ce10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbad8005cc0 tx=0x7fbad8003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:03.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:03.844+0000 7fbadd7fa640 1 -- 172.21.15.38:0/2779050215 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8004040 con 0x7fbae01082a0 2024-09-16T00:32:03.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:03.844+0000 7fbae65ca640 1 -- 172.21.15.38:0/2779050215 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbae019d780 con 0x7fbae01082a0 2024-09-16T00:32:03.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:03.844+0000 7fbadd7fa640 1 -- 172.21.15.38:0/2779050215 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbad80046c0 con 0x7fbae01082a0 2024-09-16T00:32:03.871 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:03.844+0000 7fbadd7fa640 1 -- 172.21.15.38:0/2779050215 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbad8046480 con 0x7fbae01082a0 2024-09-16T00:32:03.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:03.844+0000 7fbae65ca640 1 -- 172.21.15.38:0/2779050215 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbae01a50c0 con 0x7fbae01082a0 2024-09-16T00:32:03.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:03.845+0000 7fbae65ca640 1 -- 172.21.15.38:0/2779050215 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbae0103110 con 0x7fbae01082a0 2024-09-16T00:32:03.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:03.846+0000 7fbadd7fa640 1 -- 172.21.15.38:0/2779050215 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fbad80305a0 con 0x7fbae01082a0 2024-09-16T00:32:03.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:03.847+0000 7fbadd7fa640 1 --2- 172.21.15.38:0/2779050215 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fbac80a32d0 0x7fbac80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:03.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:03.847+0000 7fbadffff640 1 --2- 172.21.15.38:0/2779050215 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fbac80a32d0 0x7fbac80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:03.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:03.847+0000 7fbadd7fa640 1 -- 172.21.15.38:0/2779050215 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7fbad802e030 con 0x7fbae01082a0 2024-09-16T00:32:03.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:03.848+0000 7fbadffff640 1 --2- 172.21.15.38:0/2779050215 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fbac80a32d0 0x7fbac80a5790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbad000bbe0 tx=0x7fbad000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:03.872 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:03.850+0000 7fbadd7fa640 1 -- 172.21.15.38:0/2779050215 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbad804b060 con 0x7fbae01082a0 2024-09-16T00:32:04.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:04.060+0000 7fbae65ca640 1 -- 172.21.15.38:0/2779050215 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbae01998a0 con 0x7fbae01082a0 2024-09-16T00:32:04.061 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:04.061+0000 7fbadd7fa640 1 -- 172.21.15.38:0/2779050215 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11909 (secure 0 0 0) 0x7fbae01998a0 con 0x7fbae01082a0 2024-09-16T00:32:04.062 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:32:04.062 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":25,"btime":"2024-09-16T00:31:05:330385+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/367974527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":367974527},{"type":"v1","addr":"172.21.15.38:6835","nonce":367974527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/335077136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":335077136},{"type":"v1","addr":"172.21.15.38:6837","nonce":335077136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/3501211160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3501211160},{"type":"v1","addr":"172.21.15.38:6839","nonce":3501211160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6841/2623130915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":2623130915},{"type":"v1","addr":"172.21.15.38:6841","nonce":2623130915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/412277519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":412277519},{"type":"v1","addr":"172.21.15.122:6833","nonce":412277519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3987234757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3987234757},{"type":"v1","addr":"172.21.15.122:6835","nonce":3987234757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:30:26.764565+0000","modified":"2024-09-16T00:31:05.083019+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24440},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1210328177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1210328177},{"type":"v1","addr":"172.21.15.73:6833","nonce":1210328177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6837/1858299680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1858299680},{"type":"v1","addr":"172.21.15.122:6837","nonce":1858299680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1983975311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1983975311},{"type":"v1","addr":"172.21.15.122:6839","nonce":1983975311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.73:6835/2866363113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2866363113},{"type":"v1","addr":"172.21.15.73:6835","nonce":2866363113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.73:6837/2681081265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2681081265},{"type":"v1","addr":"172.21.15.73:6837","nonce":2681081265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.73:6839/2396224525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2396224525},{"type":"v1","addr":"172.21.15.73:6839","nonce":2396224525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24440,24464,24494]},"id":1}]} 2024-09-16T00:32:04.062 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 25 2024-09-16T00:32:04.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:04.063+0000 7fbae65ca640 1 -- 172.21.15.38:0/2779050215 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fbac80a32d0 msgr2=0x7fbac80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:04.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:04.063+0000 7fbae65ca640 1 --2- 172.21.15.38:0/2779050215 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fbac80a32d0 0x7fbac80a5790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fbad000bbe0 tx=0x7fbad000b390 comp rx=0 tx=0).stop 2024-09-16T00:32:04.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:04.063+0000 7fbae65ca640 1 -- 172.21.15.38:0/2779050215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbae01082a0 msgr2=0x7fbae019ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:04.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:04.063+0000 7fbae65ca640 1 --2- 172.21.15.38:0/2779050215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbae01082a0 0x7fbae019ce10 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbad8005cc0 tx=0x7fbad8003970 comp rx=0 tx=0).stop 2024-09-16T00:32:04.064 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:04.064+0000 7fbae65ca640 1 -- 172.21.15.38:0/2779050215 shutdown_connections 2024-09-16T00:32:04.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:04.064+0000 7fbae65ca640 1 --2- 172.21.15.38:0/2779050215 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fbac80a32d0 0x7fbac80a5790 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:04.064+0000 7fbae65ca640 1 --2- 172.21.15.38:0/2779050215 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbae01082a0 0x7fbae019ce10 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:04.064+0000 7fbae65ca640 1 --2- 172.21.15.38:0/2779050215 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbae01036e0 0x7fbae0198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:04.064+0000 7fbae65ca640 1 --2- 172.21.15.38:0/2779050215 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbae0102d30 0x7fbae0198540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:04.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:04.064+0000 7fbae65ca640 1 -- 172.21.15.38:0/2779050215 >> 172.21.15.38:0/2779050215 conn(0x7fbae00fc7d0 msgr2=0x7fbae0105d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:04.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:04.064+0000 7fbae65ca640 1 -- 172.21.15.38:0/2779050215 shutdown_connections 2024-09-16T00:32:04.065 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:04.064+0000 7fbae65ca640 1 -- 172.21.15.38:0/2779050215 wait complete. 2024-09-16T00:32:04.441 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:04 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/2779050215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:32:04.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:04 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/2779050215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:32:04.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:04 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/2779050215' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:32:04.782 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:32:05.044 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:05.454 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:05 smithi038 ceph-mon[30945]: pgmap v341: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-16T00:32:05.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:05 smithi122 ceph-mon[35412]: pgmap v341: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-16T00:32:05.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:05 smithi073 ceph-mon[35254]: pgmap v341: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-16T00:32:05.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.536+0000 7fd274a6e640 1 -- 172.21.15.38:0/1395575098 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd270106f10 msgr2=0x7fd27010ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:05.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.536+0000 7fd274a6e640 1 --2- 172.21.15.38:0/1395575098 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd270106f10 0x7fd27010ea90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd268009ca0 tx=0x7fd26802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:32:05.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.536+0000 7fd274a6e640 1 -- 172.21.15.38:0/1395575098 shutdown_connections 2024-09-16T00:32:05.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.536+0000 7fd274a6e640 1 --2- 172.21.15.38:0/1395575098 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd270106f10 0x7fd27010ea90 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:05.612 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.536+0000 7fd274a6e640 1 --2- 172.21.15.38:0/1395575098 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd270102790 0x7fd2701067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:05.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.536+0000 7fd274a6e640 1 --2- 172.21.15.38:0/1395575098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd270101de0 0x7fd2701021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:05.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.536+0000 7fd274a6e640 1 -- 172.21.15.38:0/1395575098 >> 172.21.15.38:0/1395575098 conn(0x7fd2700fd8c0 msgr2=0x7fd2700ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:05.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.536+0000 7fd274a6e640 1 -- 172.21.15.38:0/1395575098 shutdown_connections 2024-09-16T00:32:05.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.536+0000 7fd274a6e640 1 -- 172.21.15.38:0/1395575098 wait complete. 2024-09-16T00:32:05.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.537+0000 7fd274a6e640 1 Processor -- start 2024-09-16T00:32:05.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.537+0000 7fd274a6e640 1 -- start start 2024-09-16T00:32:05.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.537+0000 7fd274a6e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd270101de0 0x7fd2700744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:05.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.537+0000 7fd274a6e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd270102790 0x7fd270074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:05.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.537+0000 7fd274a6e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd270106f10 0x7fd270074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:05.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.537+0000 7fd274a6e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd270111f70 con 0x7fd270102790 2024-09-16T00:32:05.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.538+0000 7fd274a6e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd270111df0 con 0x7fd270106f10 2024-09-16T00:32:05.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.538+0000 7fd274a6e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd2701120f0 con 0x7fd270101de0 2024-09-16T00:32:05.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.538+0000 7fd267fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd270102790 0x7fd270074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:05.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.538+0000 7fd267fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd270102790 0x7fd270074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:35084/0 (socket says 172.21.15.38:35084) 2024-09-16T00:32:05.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.538+0000 7fd267fff640 1 -- 172.21.15.38:0/2277287075 learned_addr learned my addr 172.21.15.38:0/2277287075 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:05.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.538+0000 7fd26e59c640 1 --2- 172.21.15.38:0/2277287075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd270101de0 0x7fd2700744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:05.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.538+0000 7fd26ed9d640 1 --2- 172.21.15.38:0/2277287075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd270106f10 0x7fd270074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:05.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.538+0000 7fd267fff640 1 -- 172.21.15.38:0/2277287075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd270101de0 msgr2=0x7fd2700744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:05.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.538+0000 7fd267fff640 1 --2- 172.21.15.38:0/2277287075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd270101de0 0x7fd2700744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:05.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.538+0000 7fd267fff640 1 -- 172.21.15.38:0/2277287075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd270106f10 msgr2=0x7fd270074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:05.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.538+0000 7fd267fff640 1 --2- 172.21.15.38:0/2277287075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd270106f10 0x7fd270074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:05.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.538+0000 7fd267fff640 1 -- 172.21.15.38:0/2277287075 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2701aace0 con 0x7fd270102790 2024-09-16T00:32:05.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.538+0000 7fd267fff640 1 --2- 172.21.15.38:0/2277287075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd270102790 0x7fd270074a30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd25800d770 tx=0x7fd25800db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:05.614 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.538+0000 7fd2677fe640 1 -- 172.21.15.38:0/2277287075 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd258017070 con 0x7fd270102790 2024-09-16T00:32:05.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.538+0000 7fd2677fe640 1 -- 172.21.15.38:0/2277287075 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd258009820 con 0x7fd270102790 2024-09-16T00:32:05.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.538+0000 7fd274a6e640 1 -- 172.21.15.38:0/2277287075 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2701aaf10 con 0x7fd270102790 2024-09-16T00:32:05.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.538+0000 7fd2677fe640 1 -- 172.21.15.38:0/2277287075 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd258004cd0 con 0x7fd270102790 2024-09-16T00:32:05.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.538+0000 7fd274a6e640 1 -- 172.21.15.38:0/2277287075 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2701ab360 con 0x7fd270102790 2024-09-16T00:32:05.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.539+0000 7fd274a6e640 1 -- 172.21.15.38:0/2277287075 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2701032c0 con 0x7fd270102790 2024-09-16T00:32:05.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.542+0000 7fd2677fe640 1 -- 172.21.15.38:0/2277287075 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fd2580099d0 con 0x7fd270102790 2024-09-16T00:32:05.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.542+0000 7fd2677fe640 1 --2- 172.21.15.38:0/2277287075 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd25c0a3350 0x7fd25c0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:05.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.542+0000 7fd26e59c640 1 --2- 172.21.15.38:0/2277287075 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd25c0a3350 0x7fd25c0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:05.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.542+0000 7fd2677fe640 1 -- 172.21.15.38:0/2277287075 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7fd2580cf9d0 con 0x7fd270102790 2024-09-16T00:32:05.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.543+0000 7fd26e59c640 1 --2- 172.21.15.38:0/2277287075 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd25c0a3350 0x7fd25c0a5810 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd26000abe0 tx=0x7fd26000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:05.615 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.546+0000 7fd2677fe640 1 -- 172.21.15.38:0/2277287075 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fd25809c2d0 con 0x7fd270102790 2024-09-16T00:32:05.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.756+0000 7fd274a6e640 1 -- 172.21.15.38:0/2277287075 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd2701021c0 con 0x7fd270102790 2024-09-16T00:32:05.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.757+0000 7fd2677fe640 1 -- 172.21.15.38:0/2277287075 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11909 (secure 0 0 0) 0x7fd2701021c0 con 0x7fd270102790 2024-09-16T00:32:05.758 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:32:05.758 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":25,"btime":"2024-09-16T00:31:05:330385+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/367974527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":367974527},{"type":"v1","addr":"172.21.15.38:6835","nonce":367974527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/335077136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":335077136},{"type":"v1","addr":"172.21.15.38:6837","nonce":335077136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/3501211160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3501211160},{"type":"v1","addr":"172.21.15.38:6839","nonce":3501211160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6841/2623130915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":2623130915},{"type":"v1","addr":"172.21.15.38:6841","nonce":2623130915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/412277519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":412277519},{"type":"v1","addr":"172.21.15.122:6833","nonce":412277519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3987234757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3987234757},{"type":"v1","addr":"172.21.15.122:6835","nonce":3987234757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:30:26.764565+0000","modified":"2024-09-16T00:31:05.083019+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24440},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1210328177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1210328177},{"type":"v1","addr":"172.21.15.73:6833","nonce":1210328177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6837/1858299680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1858299680},{"type":"v1","addr":"172.21.15.122:6837","nonce":1858299680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1983975311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1983975311},{"type":"v1","addr":"172.21.15.122:6839","nonce":1983975311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.73:6835/2866363113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2866363113},{"type":"v1","addr":"172.21.15.73:6835","nonce":2866363113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.73:6837/2681081265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2681081265},{"type":"v1","addr":"172.21.15.73:6837","nonce":2681081265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.73:6839/2396224525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2396224525},{"type":"v1","addr":"172.21.15.73:6839","nonce":2396224525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24440,24464,24494]},"id":1}]} 2024-09-16T00:32:05.758 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 25 2024-09-16T00:32:05.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.759+0000 7fd274a6e640 1 -- 172.21.15.38:0/2277287075 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd25c0a3350 msgr2=0x7fd25c0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:05.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.759+0000 7fd274a6e640 1 --2- 172.21.15.38:0/2277287075 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd25c0a3350 0x7fd25c0a5810 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd26000abe0 tx=0x7fd26000a2d0 comp rx=0 tx=0).stop 2024-09-16T00:32:05.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.759+0000 7fd274a6e640 1 -- 172.21.15.38:0/2277287075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd270102790 msgr2=0x7fd270074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:05.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.759+0000 7fd274a6e640 1 --2- 172.21.15.38:0/2277287075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd270102790 0x7fd270074a30 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd25800d770 tx=0x7fd25800db70 comp rx=0 tx=0).stop 2024-09-16T00:32:05.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.759+0000 7fd274a6e640 1 -- 172.21.15.38:0/2277287075 shutdown_connections 2024-09-16T00:32:05.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.759+0000 7fd274a6e640 1 --2- 172.21.15.38:0/2277287075 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd25c0a3350 0x7fd25c0a5810 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:05.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.759+0000 7fd274a6e640 1 --2- 172.21.15.38:0/2277287075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd270106f10 0x7fd270074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:05.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.759+0000 7fd274a6e640 1 --2- 172.21.15.38:0/2277287075 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd270102790 0x7fd270074a30 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:05.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.759+0000 7fd274a6e640 1 --2- 172.21.15.38:0/2277287075 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd270101de0 0x7fd2700744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:05.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.759+0000 7fd274a6e640 1 -- 172.21.15.38:0/2277287075 >> 172.21.15.38:0/2277287075 conn(0x7fd2700fd8c0 msgr2=0x7fd2700febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:05.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.759+0000 7fd274a6e640 1 -- 172.21.15.38:0/2277287075 shutdown_connections 2024-09-16T00:32:05.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:05.759+0000 7fd274a6e640 1 -- 172.21.15.38:0/2277287075 wait complete. 2024-09-16T00:32:06.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:06 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/2277287075' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:32:06.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:06 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/2277287075' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:32:06.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:06 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/2277287075' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:32:06.580 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:32:06.847 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:07.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.408+0000 7f9c75b99640 1 -- 172.21.15.38:0/1333290664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c701082a0 msgr2=0x7f9c7010ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:07.424 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.408+0000 7f9c75b99640 1 --2- 172.21.15.38:0/1333290664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c701082a0 0x7f9c7010ec40 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f9c68009cd0 tx=0x7f9c6802a8e0 comp rx=0 tx=0).stop 2024-09-16T00:32:07.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.408+0000 7f9c75b99640 1 -- 172.21.15.38:0/1333290664 shutdown_connections 2024-09-16T00:32:07.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.408+0000 7f9c75b99640 1 --2- 172.21.15.38:0/1333290664 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c701082a0 0x7f9c7010ec40 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:07.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.408+0000 7f9c75b99640 1 --2- 172.21.15.38:0/1333290664 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c701036e0 0x7f9c70107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:07.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.408+0000 7f9c75b99640 1 --2- 172.21.15.38:0/1333290664 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c70102d30 0x7f9c70103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:07.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.408+0000 7f9c75b99640 1 -- 172.21.15.38:0/1333290664 >> 172.21.15.38:0/1333290664 conn(0x7f9c700fc7d0 msgr2=0x7f9c700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:07.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.408+0000 7f9c75b99640 1 -- 172.21.15.38:0/1333290664 shutdown_connections 2024-09-16T00:32:07.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.408+0000 7f9c75b99640 1 -- 172.21.15.38:0/1333290664 wait complete. 2024-09-16T00:32:07.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.409+0000 7f9c75b99640 1 Processor -- start 2024-09-16T00:32:07.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.409+0000 7f9c75b99640 1 -- start start 2024-09-16T00:32:07.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.409+0000 7f9c75b99640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c70102d30 0x7f9c70198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:07.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.409+0000 7f9c75b99640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c701036e0 0x7f9c70198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:07.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.409+0000 7f9c75b99640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c701082a0 0x7f9c7019ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:07.425 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.409+0000 7f9c75b99640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9c701120a0 con 0x7f9c701036e0 2024-09-16T00:32:07.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.409+0000 7f9c75b99640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9c70111f20 con 0x7f9c70102d30 2024-09-16T00:32:07.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.409+0000 7f9c75b99640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9c70112220 con 0x7f9c701082a0 2024-09-16T00:32:07.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.409+0000 7f9c6effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c701036e0 0x7f9c70198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:07.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.409+0000 7f9c6effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c701036e0 0x7f9c70198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49606/0 (socket says 172.21.15.38:49606) 2024-09-16T00:32:07.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.409+0000 7f9c6effd640 1 -- 172.21.15.38:0/371246844 learned_addr learned my addr 172.21.15.38:0/371246844 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:07.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.410+0000 7f9c6ffff640 1 --2- 172.21.15.38:0/371246844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c701082a0 0x7f9c7019ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:07.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.410+0000 7f9c6f7fe640 1 --2- 172.21.15.38:0/371246844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c70102d30 0x7f9c70198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:07.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.410+0000 7f9c6effd640 1 -- 172.21.15.38:0/371246844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c701082a0 msgr2=0x7f9c7019ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:07.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.410+0000 7f9c6effd640 1 --2- 172.21.15.38:0/371246844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c701082a0 0x7f9c7019ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:07.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.410+0000 7f9c6effd640 1 -- 172.21.15.38:0/371246844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c70102d30 msgr2=0x7f9c70198530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:07.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.410+0000 7f9c6effd640 1 --2- 172.21.15.38:0/371246844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c70102d30 0x7f9c70198530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:07.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.410+0000 7f9c6effd640 1 -- 172.21.15.38:0/371246844 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c7019d4e0 con 0x7f9c701036e0 2024-09-16T00:32:07.426 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.410+0000 7f9c6ffff640 1 --2- 172.21.15.38:0/371246844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c701082a0 0x7f9c7019ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:07.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.410+0000 7f9c6effd640 1 --2- 172.21.15.38:0/371246844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c701036e0 0x7f9c70198a70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9c6400c970 tx=0x7f9c6400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:07.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.410+0000 7f9c6cff9640 1 -- 172.21.15.38:0/371246844 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c64007bb0 con 0x7f9c701036e0 2024-09-16T00:32:07.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.410+0000 7f9c6cff9640 1 -- 172.21.15.38:0/371246844 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9c64007d50 con 0x7f9c701036e0 2024-09-16T00:32:07.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.410+0000 7f9c75b99640 1 -- 172.21.15.38:0/371246844 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c7019d7d0 con 0x7f9c701036e0 2024-09-16T00:32:07.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.410+0000 7f9c6cff9640 1 -- 172.21.15.38:0/371246844 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c6400f450 con 0x7f9c701036e0 2024-09-16T00:32:07.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.410+0000 7f9c75b99640 1 -- 172.21.15.38:0/371246844 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c701a50b0 con 0x7f9c701036e0 2024-09-16T00:32:07.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.412+0000 7f9c75b99640 1 -- 172.21.15.38:0/371246844 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c70103110 con 0x7f9c701036e0 2024-09-16T00:32:07.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.412+0000 7f9c6cff9640 1 -- 172.21.15.38:0/371246844 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f9c64004f60 con 0x7f9c701036e0 2024-09-16T00:32:07.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.413+0000 7f9c6cff9640 1 --2- 172.21.15.38:0/371246844 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f9c580a32d0 0x7f9c580a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:07.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.413+0000 7f9c6cff9640 1 -- 172.21.15.38:0/371246844 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f9c640cfb90 con 0x7f9c701036e0 2024-09-16T00:32:07.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.413+0000 7f9c6f7fe640 1 --2- 172.21.15.38:0/371246844 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f9c580a32d0 0x7f9c580a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:07.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.414+0000 7f9c6f7fe640 1 --2- 172.21.15.38:0/371246844 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f9c580a32d0 0x7f9c580a5790 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9c60009b60 tx=0x7f9c60005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:07.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.417+0000 7f9c6cff9640 1 -- 172.21.15.38:0/371246844 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9c64014030 con 0x7f9c701036e0 2024-09-16T00:32:07.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:07 smithi038 ceph-mon[30945]: pgmap v342: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:32:07.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:07 smithi073 ceph-mon[35254]: pgmap v342: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:32:07.611 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.611+0000 7f9c75b99640 1 -- 172.21.15.38:0/371246844 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9c7010a2b0 con 0x7f9c701036e0 2024-09-16T00:32:07.613 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.612+0000 7f9c6cff9640 1 -- 172.21.15.38:0/371246844 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24674 (secure 0 0 0) 0x7f9c6409c410 con 0x7f9c701036e0 2024-09-16T00:32:07.613 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:32:07.613 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":90,"fsid":"983b2072-73c1-11ef-bceb-c7b262605968","created":"2024-09-16T00:22:34.897053+0000","modified":"2024-09-16T00:31:23.431474+0000","last_up_change":"2024-09-16T00:29:34.503134+0000","last_in_change":"2024-09-16T00:29:10.135656+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:25:41.931977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:30:22.741889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:30:25.042590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:30:31.554350+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3afc20e3-869f-43a9-a911-b0e8d07de86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6803","nonce":1622028921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6805","nonce":1622028921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6809","nonce":1622028921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6807","nonce":1622028921}]},"public_addr":"172.21.15.38:6803/1622028921","cluster_addr":"172.21.15.38:6805/1622028921","heartbeat_back_addr":"172.21.15.38:6809/1622028921","heartbeat_front_addr":"172.21.15.38:6807/1622028921","state":["exists","up"]},{"osd":1,"uuid":"6d9d7acb-45ed-4543-8afc-6433fc7f096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6801","nonce":3811690336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6803","nonce":3811690336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6807","nonce":3811690336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6805","nonce":3811690336}]},"public_addr":"172.21.15.73:6801/3811690336","cluster_addr":"172.21.15.73:6803/3811690336","heartbeat_back_addr":"172.21.15.73:6807/3811690336","heartbeat_front_addr":"172.21.15.73:6805/3811690336","state":["exists","up"]},{"osd":2,"uuid":"48690f35-99e9-4f45-9bee-487857caa8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6801","nonce":1754741655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6803","nonce":1754741655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6807","nonce":1754741655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6805","nonce":1754741655}]},"public_addr":"172.21.15.122:6801/1754741655","cluster_addr":"172.21.15.122:6803/1754741655","heartbeat_back_addr":"172.21.15.122:6807/1754741655","heartbeat_front_addr":"172.21.15.122:6805/1754741655","state":["exists","up"]},{"osd":3,"uuid":"28ebb4a0-92e7-4f57-82a5-b9b761eca269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6811","nonce":3523190079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6813","nonce":3523190079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6817","nonce":3523190079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6815","nonce":3523190079}]},"public_addr":"172.21.15.38:6811/3523190079","cluster_addr":"172.21.15.38:6813/3523190079","heartbeat_back_addr":"172.21.15.38:6817/3523190079","heartbeat_front_addr":"172.21.15.38:6815/3523190079","state":["exists","up"]},{"osd":4,"uuid":"5ea363f7-7ad8-45ed-b7e4-4b010fe7f8cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6809","nonce":1469927628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6811","nonce":1469927628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6815","nonce":1469927628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6813","nonce":1469927628}]},"public_addr":"172.21.15.73:6809/1469927628","cluster_addr":"172.21.15.73:6811/1469927628","heartbeat_back_addr":"172.21.15.73:6815/1469927628","heartbeat_front_addr":"172.21.15.73:6813/1469927628","state":["exists","up"]},{"osd":5,"uuid":"e3c66d43-dd28-4bb1-a886-62a869be82cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6809","nonce":2954413058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6811","nonce":2954413058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6815","nonce":2954413058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6813","nonce":2954413058}]},"public_addr":"172.21.15.122:6809/2954413058","cluster_addr":"172.21.15.122:6811/2954413058","heartbeat_back_addr":"172.21.15.122:6815/2954413058","heartbeat_front_addr":"172.21.15.122:6813/2954413058","state":["exists","up"]},{"osd":6,"uuid":"211497ac-7c48-4095-bfdb-1d4df0149f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6819","nonce":1415719550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6821","nonce":1415719550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6825","nonce":1415719550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6823","nonce":1415719550}]},"public_addr":"172.21.15.38:6819/1415719550","cluster_addr":"172.21.15.38:6821/1415719550","heartbeat_back_addr":"172.21.15.38:6825/1415719550","heartbeat_front_addr":"172.21.15.38:6823/1415719550","state":["exists","up"]},{"osd":7,"uuid":"06512315-3f5e-44f8-976d-d3f5f1ba00e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6817","nonce":2634361473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6819","nonce":2634361473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6823","nonce":2634361473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6821","nonce":2634361473}]},"public_addr":"172.21.15.73:6817/2634361473","cluster_addr":"172.21.15.73:6819/2634361473","heartbeat_back_addr":"172.21.15.73:6823/2634361473","heartbeat_front_addr":"172.21.15.73:6821/2634361473","state":["exists","up"]},{"osd":8,"uuid":"db703c0a-1f87-46f0-be5f-f826f7f9daac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6817","nonce":1800567633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6819","nonce":1800567633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6823","nonce":1800567633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6821","nonce":1800567633}]},"public_addr":"172.21.15.122:6817/1800567633","cluster_addr":"172.21.15.122:6819/1800567633","heartbeat_back_addr":"172.21.15.122:6823/1800567633","heartbeat_front_addr":"172.21.15.122:6821/1800567633","state":["exists","up"]},{"osd":9,"uuid":"f5384200-b3b4-4936-85e1-fc57a9fe2bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6827","nonce":445727833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6829","nonce":445727833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6833","nonce":445727833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6831","nonce":445727833}]},"public_addr":"172.21.15.38:6827/445727833","cluster_addr":"172.21.15.38:6829/445727833","heartbeat_back_addr":"172.21.15.38:6833/445727833","heartbeat_front_addr":"172.21.15.38:6831/445727833","state":["exists","up"]},{"osd":10,"uuid":"c22883d9-6672-4796-a961-963d21eb37fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6825","nonce":43297139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6827","nonce":43297139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6831","nonce":43297139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6829","nonce":43297139}]},"public_addr":"172.21.15.73:6825/43297139","cluster_addr":"172.21.15.73:6827/43297139","heartbeat_back_addr":"172.21.15.73:6831/43297139","heartbeat_front_addr":"172.21.15.73:6829/43297139","state":["exists","up"]},{"osd":11,"uuid":"9f98e8cb-1bff-4b8b-aa19-56b691387383","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6825","nonce":1886446061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6827","nonce":1886446061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6831","nonce":1886446061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6829","nonce":1886446061}]},"public_addr":"172.21.15.122:6825/1886446061","cluster_addr":"172.21.15.122:6827/1886446061","heartbeat_back_addr":"172.21.15.122:6831/1886446061","heartbeat_front_addr":"172.21.15.122:6829/1886446061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:47.384012+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:12.834488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:38.770055+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:04.800466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:30.808076+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:56.646967+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:27:22.721270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:27:48.510011+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:28:13.922944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:28:40.514947+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:29:06.031052+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:29:31.882062+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6801/1779324916":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/924150447":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/141237849":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/2302081750":"2024-09-17T00:22:56.844212+0000","172.21.15.38:0/2245468594":"2024-09-17T00:22:56.844212+0000","172.21.15.38:6801/583006373":"2024-09-17T00:22:56.844212+0000","172.21.15.38:0/2435492667":"2024-09-17T00:23:21.417483+0000","172.21.15.38:6800/583006373":"2024-09-17T00:22:56.844212+0000","172.21.15.38:6800/1779324916":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/1103516837":"2024-09-17T00:22:56.844212+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:32:07.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.615+0000 7f9c75b99640 1 -- 172.21.15.38:0/371246844 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f9c580a32d0 msgr2=0x7f9c580a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:07.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.615+0000 7f9c75b99640 1 --2- 172.21.15.38:0/371246844 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f9c580a32d0 0x7f9c580a5790 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9c60009b60 tx=0x7f9c60005f90 comp rx=0 tx=0).stop 2024-09-16T00:32:07.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.615+0000 7f9c75b99640 1 -- 172.21.15.38:0/371246844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c701036e0 msgr2=0x7f9c70198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:07.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.615+0000 7f9c75b99640 1 --2- 172.21.15.38:0/371246844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c701036e0 0x7f9c70198a70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9c6400c970 tx=0x7f9c6400ce30 comp rx=0 tx=0).stop 2024-09-16T00:32:07.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.615+0000 7f9c75b99640 1 -- 172.21.15.38:0/371246844 shutdown_connections 2024-09-16T00:32:07.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.615+0000 7f9c75b99640 1 --2- 172.21.15.38:0/371246844 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f9c580a32d0 0x7f9c580a5790 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:07.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.615+0000 7f9c75b99640 1 --2- 172.21.15.38:0/371246844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9c701082a0 0x7f9c7019ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:07.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.615+0000 7f9c75b99640 1 --2- 172.21.15.38:0/371246844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9c701036e0 0x7f9c70198a70 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:07.616 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.615+0000 7f9c75b99640 1 --2- 172.21.15.38:0/371246844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c70102d30 0x7f9c70198530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:07.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.616+0000 7f9c75b99640 1 -- 172.21.15.38:0/371246844 >> 172.21.15.38:0/371246844 conn(0x7f9c700fc7d0 msgr2=0x7f9c70105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:07.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.616+0000 7f9c75b99640 1 -- 172.21.15.38:0/371246844 shutdown_connections 2024-09-16T00:32:07.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:07.616+0000 7f9c75b99640 1 -- 172.21.15.38:0/371246844 wait complete. 2024-09-16T00:32:07.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:07 smithi122 ceph-mon[35412]: pgmap v342: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:32:08.393 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:32:08.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:08 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/371246844' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:32:08.652 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:08.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:08 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/371246844' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:32:08.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:08 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/371246844' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:32:09.176 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.150+0000 7fac20c4f640 1 -- 172.21.15.38:0/1728835989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac1c102d40 msgr2=0x7fac1c103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:09.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.150+0000 7fac20c4f640 1 --2- 172.21.15.38:0/1728835989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac1c102d40 0x7fac1c103120 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fac0c009900 tx=0x7fac0c02a480 comp rx=0 tx=0).stop 2024-09-16T00:32:09.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.150+0000 7fac20c4f640 1 -- 172.21.15.38:0/1728835989 shutdown_connections 2024-09-16T00:32:09.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.150+0000 7fac20c4f640 1 --2- 172.21.15.38:0/1728835989 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac1c10c170 0x7fac1c10e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.150+0000 7fac20c4f640 1 --2- 172.21.15.38:0/1728835989 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fac1c103660 0x7fac1c10bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.150+0000 7fac20c4f640 1 --2- 172.21.15.38:0/1728835989 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac1c102d40 0x7fac1c103120 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.150+0000 7fac20c4f640 1 -- 172.21.15.38:0/1728835989 >> 172.21.15.38:0/1728835989 conn(0x7fac1c0fc7a0 msgr2=0x7fac1c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:09.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.150+0000 7fac20c4f640 1 -- 172.21.15.38:0/1728835989 shutdown_connections 2024-09-16T00:32:09.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.150+0000 7fac20c4f640 1 -- 172.21.15.38:0/1728835989 wait complete. 2024-09-16T00:32:09.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.151+0000 7fac20c4f640 1 Processor -- start 2024-09-16T00:32:09.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.151+0000 7fac20c4f640 1 -- start start 2024-09-16T00:32:09.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.151+0000 7fac20c4f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fac1c102d40 0x7fac1c19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:09.177 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.151+0000 7fac20c4f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac1c103660 0x7fac1c19ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:09.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.151+0000 7fac20c4f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac1c10c170 0x7fac1c19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:09.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.151+0000 7fac20c4f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c111f70 con 0x7fac1c10c170 2024-09-16T00:32:09.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.151+0000 7fac20c4f640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c111df0 con 0x7fac1c102d40 2024-09-16T00:32:09.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.151+0000 7fac20c4f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fac1c1120f0 con 0x7fac1c103660 2024-09-16T00:32:09.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.151+0000 7fac1ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac1c10c170 0x7fac1c19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:09.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.151+0000 7fac1ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac1c10c170 0x7fac1c19efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49624/0 (socket says 172.21.15.38:49624) 2024-09-16T00:32:09.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.151+0000 7fac1ad9d640 1 -- 172.21.15.38:0/432667844 learned_addr learned my addr 172.21.15.38:0/432667844 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:09.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.152+0000 7fac1a59c640 1 --2- 172.21.15.38:0/432667844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fac1c102d40 0x7fac1c19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:09.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.152+0000 7fac19d9b640 1 --2- 172.21.15.38:0/432667844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac1c103660 0x7fac1c19ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:09.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.152+0000 7fac1ad9d640 1 -- 172.21.15.38:0/432667844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac1c103660 msgr2=0x7fac1c19ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:09.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.152+0000 7fac1ad9d640 1 --2- 172.21.15.38:0/432667844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac1c103660 0x7fac1c19ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.152+0000 7fac1ad9d640 1 -- 172.21.15.38:0/432667844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fac1c102d40 msgr2=0x7fac1c19a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:09.178 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.152+0000 7fac1ad9d640 1 --2- 172.21.15.38:0/432667844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fac1c102d40 0x7fac1c19a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.152+0000 7fac1ad9d640 1 -- 172.21.15.38:0/432667844 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac1c19f690 con 0x7fac1c10c170 2024-09-16T00:32:09.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.152+0000 7fac1ad9d640 1 --2- 172.21.15.38:0/432667844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac1c10c170 0x7fac1c19efb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fac1400eaa0 tx=0x7fac1400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:09.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.152+0000 7fac0b7fe640 1 -- 172.21.15.38:0/432667844 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac1400cdb0 con 0x7fac1c10c170 2024-09-16T00:32:09.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.152+0000 7fac20c4f640 1 -- 172.21.15.38:0/432667844 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac1c19f980 con 0x7fac1c10c170 2024-09-16T00:32:09.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.152+0000 7fac0b7fe640 1 -- 172.21.15.38:0/432667844 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fac14005020 con 0x7fac1c10c170 2024-09-16T00:32:09.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.152+0000 7fac0b7fe640 1 -- 172.21.15.38:0/432667844 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac140116b0 con 0x7fac1c10c170 2024-09-16T00:32:09.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.152+0000 7fac20c4f640 1 -- 172.21.15.38:0/432667844 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac1c1a7260 con 0x7fac1c10c170 2024-09-16T00:32:09.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.153+0000 7fac20c4f640 1 -- 172.21.15.38:0/432667844 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac1c103120 con 0x7fac1c10c170 2024-09-16T00:32:09.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.155+0000 7fac0b7fe640 1 -- 172.21.15.38:0/432667844 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fac14005560 con 0x7fac1c10c170 2024-09-16T00:32:09.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.155+0000 7fac0b7fe640 1 --2- 172.21.15.38:0/432667844 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fac040a32d0 0x7fac040a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:09.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.156+0000 7fac0b7fe640 1 -- 172.21.15.38:0/432667844 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7fac140d02f0 con 0x7fac1c10c170 2024-09-16T00:32:09.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.156+0000 7fac1a59c640 1 --2- 172.21.15.38:0/432667844 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fac040a32d0 0x7fac040a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:09.179 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.157+0000 7fac1a59c640 1 --2- 172.21.15.38:0/432667844 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fac040a32d0 0x7fac040a5790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fac0c009230 tx=0x7fac0c0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:09.180 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.159+0000 7fac0b7fe640 1 -- 172.21.15.38:0/432667844 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fac14016030 con 0x7fac1c10c170 2024-09-16T00:32:09.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.372+0000 7fac20c4f640 1 -- 172.21.15.38:0/432667844 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fac1c19ba40 con 0x7fac1c10c170 2024-09-16T00:32:09.373 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:09 smithi038 ceph-mon[30945]: pgmap v343: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:32:09.374 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.373+0000 7fac0b7fe640 1 -- 172.21.15.38:0/432667844 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11909 (secure 0 0 0) 0x7fac1c19ba40 con 0x7fac1c10c170 2024-09-16T00:32:09.374 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:32:09.374 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":25,"btime":"2024-09-16T00:31:05:330385+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/367974527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":367974527},{"type":"v1","addr":"172.21.15.38:6835","nonce":367974527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/335077136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":335077136},{"type":"v1","addr":"172.21.15.38:6837","nonce":335077136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/3501211160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3501211160},{"type":"v1","addr":"172.21.15.38:6839","nonce":3501211160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6841/2623130915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":2623130915},{"type":"v1","addr":"172.21.15.38:6841","nonce":2623130915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/412277519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":412277519},{"type":"v1","addr":"172.21.15.122:6833","nonce":412277519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3987234757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3987234757},{"type":"v1","addr":"172.21.15.122:6835","nonce":3987234757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:30:26.764565+0000","modified":"2024-09-16T00:31:05.083019+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24440},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1210328177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1210328177},{"type":"v1","addr":"172.21.15.73:6833","nonce":1210328177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6837/1858299680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1858299680},{"type":"v1","addr":"172.21.15.122:6837","nonce":1858299680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1983975311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1983975311},{"type":"v1","addr":"172.21.15.122:6839","nonce":1983975311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.73:6835/2866363113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2866363113},{"type":"v1","addr":"172.21.15.73:6835","nonce":2866363113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.73:6837/2681081265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2681081265},{"type":"v1","addr":"172.21.15.73:6837","nonce":2681081265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.73:6839/2396224525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2396224525},{"type":"v1","addr":"172.21.15.73:6839","nonce":2396224525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24440,24464,24494]},"id":1}]} 2024-09-16T00:32:09.374 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 25 2024-09-16T00:32:09.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.376+0000 7fac20c4f640 1 -- 172.21.15.38:0/432667844 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fac040a32d0 msgr2=0x7fac040a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:09.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.376+0000 7fac20c4f640 1 --2- 172.21.15.38:0/432667844 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fac040a32d0 0x7fac040a5790 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fac0c009230 tx=0x7fac0c0058d0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.376+0000 7fac20c4f640 1 -- 172.21.15.38:0/432667844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac1c10c170 msgr2=0x7fac1c19efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:09.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.376+0000 7fac20c4f640 1 --2- 172.21.15.38:0/432667844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac1c10c170 0x7fac1c19efb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fac1400eaa0 tx=0x7fac1400ef60 comp rx=0 tx=0).stop 2024-09-16T00:32:09.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.376+0000 7fac20c4f640 1 -- 172.21.15.38:0/432667844 shutdown_connections 2024-09-16T00:32:09.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.376+0000 7fac20c4f640 1 --2- 172.21.15.38:0/432667844 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fac040a32d0 0x7fac040a5790 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.376+0000 7fac20c4f640 1 --2- 172.21.15.38:0/432667844 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fac1c10c170 0x7fac1c19efb0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.376 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.376+0000 7fac20c4f640 1 --2- 172.21.15.38:0/432667844 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fac1c103660 0x7fac1c19ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.376+0000 7fac20c4f640 1 --2- 172.21.15.38:0/432667844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fac1c102d40 0x7fac1c19a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:09.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.376+0000 7fac20c4f640 1 -- 172.21.15.38:0/432667844 >> 172.21.15.38:0/432667844 conn(0x7fac1c0fc7a0 msgr2=0x7fac1c0fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:09.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.376+0000 7fac20c4f640 1 -- 172.21.15.38:0/432667844 shutdown_connections 2024-09-16T00:32:09.377 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:09.376+0000 7fac20c4f640 1 -- 172.21.15.38:0/432667844 wait complete. 2024-09-16T00:32:09.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:09 smithi073 ceph-mon[35254]: pgmap v343: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:32:09.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:09 smithi122 ceph-mon[35412]: pgmap v343: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:32:10.062 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-16T00:30:26.764565+0000', 'modified': '2024-09-16T00:31:05.083019+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24494, 'mds_1': 24464, 'mds_2': 24440}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24416': {'gid': 24416, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.73:6833/1210328177', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6832', 'nonce': 1210328177}, {'type': 'v1', 'addr': '172.21.15.73:6833', 'nonce': 1210328177}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24421': {'gid': 24421, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.122:6837/1858299680', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6836', 'nonce': 1858299680}, {'type': 'v1', 'addr': '172.21.15.122:6837', 'nonce': 1858299680}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.122:6839/1983975311', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6838', 'nonce': 1983975311}, {'type': 'v1', 'addr': '172.21.15.122:6839', 'nonce': 1983975311}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'e', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.73:6835/2866363113', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6834', 'nonce': 2866363113}, {'type': 'v1', 'addr': '172.21.15.73:6835', 'nonce': 2866363113}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24464': {'gid': 24464, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.73:6837/2681081265', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6836', 'nonce': 2681081265}, {'type': 'v1', 'addr': '172.21.15.73:6837', 'nonce': 2681081265}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.73:6839/2396224525', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6838', 'nonce': 2396224525}, {'type': 'v1', 'addr': '172.21.15.73:6839', 'nonce': 2396224525}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24494, 'qdb_cluster': [24440, 24464, 24494]} 2024-09-16T00:32:10.063 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-16T00:32:10.063 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.e status 2024-09-16T00:32:10.323 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:10.445 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:10 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/432667844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:32:10.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:10 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/432667844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:32:10.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:10 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/432667844' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:32:10.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.794+0000 7fa9ee8d9640 1 -- 172.21.15.38:0/712916491 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9e8102570 msgr2=0x7fa9e81029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:10.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.794+0000 7fa9ee8d9640 1 --2- 172.21.15.38:0/712916491 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9e8102570 0x7fa9e81029b0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc009900 tx=0x7fa9dc02a480 comp rx=0 tx=0).stop 2024-09-16T00:32:10.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.794+0000 7fa9ee8d9640 1 -- 172.21.15.38:0/712916491 shutdown_connections 2024-09-16T00:32:10.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.794+0000 7fa9ee8d9640 1 --2- 172.21.15.38:0/712916491 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9e8102ef0 0x7fa9e810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:10.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.794+0000 7fa9ee8d9640 1 --2- 172.21.15.38:0/712916491 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9e8102570 0x7fa9e81029b0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:10.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.794+0000 7fa9ee8d9640 1 --2- 172.21.15.38:0/712916491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9e8100e20 0x7fa9e8101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:10.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.794+0000 7fa9ee8d9640 1 -- 172.21.15.38:0/712916491 >> 172.21.15.38:0/712916491 conn(0x7fa9e80fc7d0 msgr2=0x7fa9e80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:10.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.794+0000 7fa9ee8d9640 1 -- 172.21.15.38:0/712916491 shutdown_connections 2024-09-16T00:32:10.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.794+0000 7fa9ee8d9640 1 -- 172.21.15.38:0/712916491 wait complete. 2024-09-16T00:32:10.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.795+0000 7fa9ee8d9640 1 Processor -- start 2024-09-16T00:32:10.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.795+0000 7fa9ee8d9640 1 -- start start 2024-09-16T00:32:10.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.795+0000 7fa9ee8d9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9e8100e20 0x7fa9e8198460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:10.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.795+0000 7fa9ee8d9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9e8102570 0x7fa9e81989a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:10.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.795+0000 7fa9ee8d9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9e8102ef0 0x7fa9e819cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:10.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.795+0000 7fa9ee8d9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e81120a0 con 0x7fa9e8100e20 2024-09-16T00:32:10.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.795+0000 7fa9ee8d9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e8111f20 con 0x7fa9e8102570 2024-09-16T00:32:10.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.795+0000 7fa9ee8d9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e8112220 con 0x7fa9e8102ef0 2024-09-16T00:32:10.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.795+0000 7fa9e7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9e8100e20 0x7fa9e8198460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:10.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.795+0000 7fa9e7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9e8100e20 0x7fa9e8198460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49636/0 (socket says 172.21.15.38:49636) 2024-09-16T00:32:10.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.795+0000 7fa9e7fff640 1 -- 172.21.15.38:0/2939898376 learned_addr learned my addr 172.21.15.38:0/2939898376 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:10.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.795+0000 7fa9e77fe640 1 --2- 172.21.15.38:0/2939898376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9e8102570 0x7fa9e81989a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:10.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.795+0000 7fa9ece76640 1 --2- 172.21.15.38:0/2939898376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9e8102ef0 0x7fa9e819cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:10.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.795+0000 7fa9e7fff640 1 -- 172.21.15.38:0/2939898376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9e8102ef0 msgr2=0x7fa9e819cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:10.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.795+0000 7fa9e7fff640 1 --2- 172.21.15.38:0/2939898376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9e8102ef0 0x7fa9e819cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:10.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.795+0000 7fa9e7fff640 1 -- 172.21.15.38:0/2939898376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9e8102570 msgr2=0x7fa9e81989a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:10.823 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.795+0000 7fa9e7fff640 1 --2- 172.21.15.38:0/2939898376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9e8102570 0x7fa9e81989a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:10.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.795+0000 7fa9e7fff640 1 -- 172.21.15.38:0/2939898376 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e819d410 con 0x7fa9e8100e20 2024-09-16T00:32:10.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.796+0000 7fa9e7fff640 1 --2- 172.21.15.38:0/2939898376 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9e8100e20 0x7fa9e8198460 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa9d800c890 tx=0x7fa9d800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:10.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.796+0000 7fa9e57fa640 1 -- 172.21.15.38:0/2939898376 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8004220 con 0x7fa9e8100e20 2024-09-16T00:32:10.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.796+0000 7fa9ee8d9640 1 -- 172.21.15.38:0/2939898376 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9e819d700 con 0x7fa9e8100e20 2024-09-16T00:32:10.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.796+0000 7fa9e57fa640 1 -- 172.21.15.38:0/2939898376 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9d80043c0 con 0x7fa9e8100e20 2024-09-16T00:32:10.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.796+0000 7fa9e57fa640 1 -- 172.21.15.38:0/2939898376 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d8004a00 con 0x7fa9e8100e20 2024-09-16T00:32:10.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.796+0000 7fa9ee8d9640 1 -- 172.21.15.38:0/2939898376 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9e81a4fe0 con 0x7fa9e8100e20 2024-09-16T00:32:10.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.798+0000 7fa9e57fa640 1 -- 172.21.15.38:0/2939898376 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fa9d80079b0 con 0x7fa9e8100e20 2024-09-16T00:32:10.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.798+0000 7fa9f0cb2740 1 Processor -- start 2024-09-16T00:32:10.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.799+0000 7fa9f0cb2740 1 -- start start 2024-09-16T00:32:10.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.799+0000 7fa9e57fa640 1 --2- 172.21.15.38:0/2939898376 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fa9d00a32d0 0x7fa9d00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:10.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.799+0000 7fa9f0cb2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f6007b8b0 0x561f6007bc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:10.824 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.799+0000 7fa9f0cb2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f6007c1d0 0x561f6008aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:10.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.799+0000 7fa9f0cb2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f6008afc0 0x561f6008d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:10.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.799+0000 7fa9f0cb2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561f5ffe2090 con 0x561f6007b8b0 2024-09-16T00:32:10.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.799+0000 7fa9f0cb2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561f5ff89a70 con 0x561f6007c1d0 2024-09-16T00:32:10.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.799+0000 7fa9f0cb2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561f5ffde050 con 0x561f6008afc0 2024-09-16T00:32:10.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.799+0000 7fa9ece76640 1 --2- 172.21.15.38:0/2939898376 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fa9d00a32d0 0x7fa9d00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:10.825 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.799+0000 7fa9e7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f6007b8b0 0x561f6007bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:10.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.799+0000 7fa9e7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f6007b8b0 0x561f6007bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49652/0 (socket says 172.21.15.38:49652) 2024-09-16T00:32:10.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.799+0000 7fa9e7fff640 1 -- 172.21.15.38:0/360764709 learned_addr learned my addr 172.21.15.38:0/360764709 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:10.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.799+0000 7fa9e57fa640 1 -- 172.21.15.38:0/2939898376 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7fa9d80cfae0 con 0x7fa9e8100e20 2024-09-16T00:32:10.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.799+0000 7fa9e77fe640 1 --2- 172.21.15.38:0/360764709 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f6007c1d0 0x561f6008aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:10.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.799+0000 7fa9e7fff640 1 -- 172.21.15.38:0/360764709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f6008afc0 msgr2=0x561f6008d3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:10.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.799+0000 7fa9e7fff640 1 --2- 172.21.15.38:0/360764709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f6008afc0 0x561f6008d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:10.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.799+0000 7fa9e7fff640 1 -- 172.21.15.38:0/360764709 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f6007c1d0 msgr2=0x561f6008aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:10.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.799+0000 7fa9e7fff640 1 --2- 172.21.15.38:0/360764709 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f6007c1d0 0x561f6008aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:10.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.799+0000 7fa9e7fff640 1 -- 172.21.15.38:0/360764709 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f5ff96130 con 0x561f6007b8b0 2024-09-16T00:32:10.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.799+0000 7fa9ece76640 1 --2- 172.21.15.38:0/2939898376 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fa9d00a32d0 0x7fa9d00a5790 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0008dd0 tx=0x7fa9e0008750 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:10.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.800+0000 7fa9e7fff640 1 --2- 172.21.15.38:0/360764709 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f6007b8b0 0x561f6007bc90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa9d80cd180 tx=0x7fa9d8005300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6130eec62256032c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:10.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.800+0000 7fa9ce7fc640 1 -- 172.21.15.38:0/360764709 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d801d070 con 0x561f6007b8b0 2024-09-16T00:32:10.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.800+0000 7fa9ce7fc640 1 -- 172.21.15.38:0/360764709 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9d809c360 con 0x561f6007b8b0 2024-09-16T00:32:10.826 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.800+0000 7fa9ce7fc640 1 -- 172.21.15.38:0/360764709 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d80a48d0 con 0x561f6007b8b0 2024-09-16T00:32:10.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.800+0000 7fa9f0cb2740 1 -- 172.21.15.38:0/360764709 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f6007b8b0 msgr2=0x561f6007bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:10.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.800+0000 7fa9f0cb2740 1 --2- 172.21.15.38:0/360764709 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f6007b8b0 0x561f6007bc90 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fa9d80cd180 tx=0x7fa9d8005300 comp rx=0 tx=0).stop 2024-09-16T00:32:10.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.800+0000 7fa9f0cb2740 1 -- 172.21.15.38:0/360764709 shutdown_connections 2024-09-16T00:32:10.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.800+0000 7fa9f0cb2740 1 --2- 172.21.15.38:0/360764709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f6008afc0 0x561f6008d3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:10.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.800+0000 7fa9f0cb2740 1 --2- 172.21.15.38:0/360764709 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f6007c1d0 0x561f6008aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:10.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.800+0000 7fa9f0cb2740 1 --2- 172.21.15.38:0/360764709 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f6007b8b0 0x561f6007bc90 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:10.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.800+0000 7fa9f0cb2740 1 -- 172.21.15.38:0/360764709 >> 172.21.15.38:0/360764709 conn(0x561f60077f60 msgr2=0x561f60078340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:10.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.801+0000 7fa9f0cb2740 1 -- 172.21.15.38:0/360764709 shutdown_connections 2024-09-16T00:32:10.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.801+0000 7fa9f0cb2740 1 -- 172.21.15.38:0/360764709 wait complete. 2024-09-16T00:32:10.827 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.801+0000 7fa9f0cb2740 1 -- start start 2024-09-16T00:32:10.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.801+0000 7fa9f0cb2740 1 Processor -- start 2024-09-16T00:32:10.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.801+0000 7fa9f0cb2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f6007b8b0 0x561f6008a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:10.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.801+0000 7fa9f0cb2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f6007c1d0 0x561f60087c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:10.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.801+0000 7fa9f0cb2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f6008afc0 0x561f600881c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:10.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.801+0000 7fa9f0cb2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561f6008db90 con 0x561f6007b8b0 2024-09-16T00:32:10.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.801+0000 7fa9f0cb2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561f5ff9cf20 con 0x561f6008afc0 2024-09-16T00:32:10.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.801+0000 7fa9f0cb2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561f6008dd10 con 0x561f6007c1d0 2024-09-16T00:32:10.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.801+0000 7fa9f0cb2740 10 client.0 _ll_register_callbacks cb 0x561f5ff66030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:10.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.801+0000 7fa9e7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f6007b8b0 0x561f6008a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:10.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.802+0000 7fa9e7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f6007b8b0 0x561f6008a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49664/0 (socket says 172.21.15.38:49664) 2024-09-16T00:32:10.828 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.802+0000 7fa9e7fff640 1 -- 172.21.15.38:0/3584862665 learned_addr learned my addr 172.21.15.38:0/3584862665 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:10.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.802+0000 7fa9e77fe640 1 --2- 172.21.15.38:0/3584862665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f6007c1d0 0x561f60087c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:10.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.802+0000 7fa9ece76640 1 --2- 172.21.15.38:0/3584862665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f6008afc0 0x561f600881c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:10.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.802+0000 7fa9e7fff640 1 -- 172.21.15.38:0/3584862665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f6007c1d0 msgr2=0x561f60087c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:10.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.802+0000 7fa9e7fff640 1 --2- 172.21.15.38:0/3584862665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f6007c1d0 0x561f60087c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:10.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.802+0000 7fa9e7fff640 1 -- 172.21.15.38:0/3584862665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f6008afc0 msgr2=0x561f600881c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:10.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.802+0000 7fa9e7fff640 1 --2- 172.21.15.38:0/3584862665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f6008afc0 0x561f600881c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:10.829 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.802+0000 7fa9e7fff640 1 -- 172.21.15.38:0/3584862665 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f5fd532b0 con 0x561f6007b8b0 2024-09-16T00:32:10.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.802+0000 7fa9ece76640 1 --2- 172.21.15.38:0/3584862665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f6008afc0 0x561f600881c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:10.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.802+0000 7fa9e7fff640 1 --2- 172.21.15.38:0/3584862665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f6007b8b0 0x561f6008a910 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8098630 tx=0x7fa9d80ae1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d4740feecdba665 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:10.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.802+0000 7fa9c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:32:10.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.802+0000 7fa9c77fe640 1 -- 172.21.15.38:0/3584862665 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d801d050 con 0x561f6007b8b0 2024-09-16T00:32:10.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.802+0000 7fa9c77fe640 1 -- 172.21.15.38:0/3584862665 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7fa9d80ae760 con 0x561f6007b8b0 2024-09-16T00:32:10.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.802+0000 7fa9f0cb2740 1 -- 172.21.15.38:0/3584862665 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9d809c360 con 0x561f6007b8b0 2024-09-16T00:32:10.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.802+0000 7fa9c77fe640 1 -- 172.21.15.38:0/3584862665 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa9d80ae570 con 0x561f6007b8b0 2024-09-16T00:32:10.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.802+0000 7fa9f0cb2740 10 client.15063 fetch_fsmap learned FSMap version 25 2024-09-16T00:32:10.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.802+0000 7fa9f0cb2740 1 -- 172.21.15.38:0/3584862665 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561f600995a0 con 0x561f6007b8b0 2024-09-16T00:32:10.830 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.803+0000 7fa9c77fe640 1 -- 172.21.15.38:0/3584862665 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9d809c360 con 0x561f6007b8b0 2024-09-16T00:32:10.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.803+0000 7fa9c77fe640 1 -- 172.21.15.38:0/3584862665 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa9d80b4030 con 0x561f6007b8b0 2024-09-16T00:32:10.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.803+0000 7fa9f0cb2740 10 client.15063 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:32:10.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.803+0000 7fa9f0cb2740 10 client.15063 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-16T00:32:10.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.803+0000 7fa9f0cb2740 20 client.15063 populate_metadata read hostname 'smithi038' 2024-09-16T00:32:10.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.803+0000 7fa9f0cb2740 1 --2- 172.21.15.38:0/3584862665 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x561f6009b6a0 0x561f6009cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:10.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.803+0000 7fa9f0cb2740 4 client.15063 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:32:10.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.803+0000 7fa9f0cb2740 1 -- 172.21.15.38:0/3584862665 --> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f5ffef600 con 0x561f6009b6a0 2024-09-16T00:32:10.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.803+0000 7fa9e77fe640 1 --2- 172.21.15.38:0/3584862665 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x561f6009b6a0 0x561f6009cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:10.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.803+0000 7fa9c77fe640 10 client.15063 ms_handle_connect on v2:172.21.15.73:6834/2866363113 2024-09-16T00:32:10.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.803+0000 7fa9e77fe640 1 --2- 172.21.15.38:0/3584862665 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x561f6009b6a0 0x561f6009cb10 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=80d492f984aa7850 server_cookie=1daf7292dc58b798 in_seq=0 out_seq=0 2024-09-16T00:32:10.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.804+0000 7fa9c77fe640 1 -- 172.21.15.38:0/3584862665 <== mds.2 v2:172.21.15.73:6834/2866363113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561f5ffef600 con 0x561f6009b6a0 2024-09-16T00:32:10.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.804+0000 7fa9c77fe640 10 client.15063 handle_command_reply: tid=0 2024-09-16T00:32:10.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.804+0000 7fa9f0cb2740 1 client.15063 shutdown 2024-09-16T00:32:10.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.804+0000 7fa9f0cb2740 1 -- 172.21.15.38:0/3584862665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f6007b8b0 msgr2=0x561f6008a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:10.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.804+0000 7fa9f0cb2740 1 --2- 172.21.15.38:0/3584862665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f6007b8b0 0x561f6008a910 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8098630 tx=0x7fa9d80ae1f0 comp rx=0 tx=0).stop 2024-09-16T00:32:10.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.805+0000 7fa9f0cb2740 1 -- 172.21.15.38:0/3584862665 shutdown_connections 2024-09-16T00:32:10.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.805+0000 7fa9f0cb2740 1 --2- 172.21.15.38:0/3584862665 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x561f6009b6a0 0x561f6009cb10 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:10.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.805+0000 7fa9f0cb2740 1 --2- 172.21.15.38:0/3584862665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f6008afc0 0x561f600881c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:10.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.805+0000 7fa9f0cb2740 1 --2- 172.21.15.38:0/3584862665 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f6007c1d0 0x561f60087c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:10.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.805+0000 7fa9f0cb2740 1 --2- 172.21.15.38:0/3584862665 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f6007b8b0 0x561f6008a910 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:10.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.805+0000 7fa9c77fe640 0 client.15063 ms_handle_reset on v2:172.21.15.73:6834/2866363113 2024-09-16T00:32:10.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.805+0000 7fa9f0cb2740 1 -- 172.21.15.38:0/3584862665 >> 172.21.15.38:0/3584862665 conn(0x561f60077f60 msgr2=0x561f6007dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:10.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.805+0000 7fa9f0cb2740 1 -- 172.21.15.38:0/3584862665 shutdown_connections 2024-09-16T00:32:10.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.805+0000 7fa9f0cb2740 1 -- 172.21.15.38:0/3584862665 wait complete. 2024-09-16T00:32:10.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.805+0000 7fa9f0cb2740 20 client.15063 trim_cache size 0 max 16384 2024-09-16T00:32:10.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.823+0000 7fa9f0cb2740 1 Processor -- start 2024-09-16T00:32:10.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.823+0000 7fa9f0cb2740 1 -- start start 2024-09-16T00:32:10.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.823+0000 7fa9f0cb2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f6007b8b0 0x561f600881b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:10.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.823+0000 7fa9f0cb2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f6007c1d0 0x561f6009b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:10.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.823+0000 7fa9f0cb2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f6008afc0 0x561f60083130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:10.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.823+0000 7fa9f0cb2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561f6009ae50 con 0x561f6008afc0 2024-09-16T00:32:10.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.823+0000 7fa9f0cb2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561f6009b520 con 0x561f6007c1d0 2024-09-16T00:32:10.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.823+0000 7fa9f0cb2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561f6009bbf0 con 0x561f6007b8b0 2024-09-16T00:32:10.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.823+0000 7fa9ece76640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f6008afc0 0x561f60083130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:10.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.823+0000 7fa9ece76640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f6008afc0 0x561f60083130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49674/0 (socket says 172.21.15.38:49674) 2024-09-16T00:32:10.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.823+0000 7fa9ece76640 1 -- 172.21.15.38:0/1504552644 learned_addr learned my addr 172.21.15.38:0/1504552644 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:10.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.823+0000 7fa9e77fe640 1 --2- 172.21.15.38:0/1504552644 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f6007c1d0 0x561f6009b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:10.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.823+0000 7fa9e7fff640 1 --2- 172.21.15.38:0/1504552644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f6007b8b0 0x561f600881b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:10.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.823+0000 7fa9ece76640 1 -- 172.21.15.38:0/1504552644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f6007b8b0 msgr2=0x561f600881b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:10.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.823+0000 7fa9ece76640 1 --2- 172.21.15.38:0/1504552644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f6007b8b0 0x561f600881b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:10.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.823+0000 7fa9ece76640 1 -- 172.21.15.38:0/1504552644 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f6007c1d0 msgr2=0x561f6009b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:10.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.823+0000 7fa9ece76640 1 --2- 172.21.15.38:0/1504552644 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f6007c1d0 0x561f6009b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:10.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.823+0000 7fa9ece76640 1 -- 172.21.15.38:0/1504552644 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f6008ac80 con 0x561f6008afc0 2024-09-16T00:32:10.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.823+0000 7fa9e7fff640 1 --2- 172.21.15.38:0/1504552644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f6007b8b0 0x561f600881b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:32:10.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.824+0000 7fa9ece76640 1 --2- 172.21.15.38:0/1504552644 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f6008afc0 0x561f60083130 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00115f0 tx=0x7fa9e0012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1384f97cb053109 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:10.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.824+0000 7fa9cdffb640 1 -- 172.21.15.38:0/1504552644 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e0023070 con 0x561f6008afc0 2024-09-16T00:32:10.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.824+0000 7fa9cdffb640 1 -- 172.21.15.38:0/1504552644 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9e001f430 con 0x561f6008afc0 2024-09-16T00:32:10.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.824+0000 7fa9cdffb640 1 -- 172.21.15.38:0/1504552644 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9e001e5f0 con 0x561f6008afc0 2024-09-16T00:32:10.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.824+0000 7fa9f0cb2740 1 -- 172.21.15.38:0/1504552644 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f6008afc0 msgr2=0x561f60083130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:10.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.824+0000 7fa9f0cb2740 1 --2- 172.21.15.38:0/1504552644 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f6008afc0 0x561f60083130 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa9e00115f0 tx=0x7fa9e0012000 comp rx=0 tx=0).stop 2024-09-16T00:32:10.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.824+0000 7fa9f0cb2740 1 -- 172.21.15.38:0/1504552644 shutdown_connections 2024-09-16T00:32:10.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.824+0000 7fa9f0cb2740 1 --2- 172.21.15.38:0/1504552644 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f6008afc0 0x561f60083130 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:10.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.824+0000 7fa9f0cb2740 1 --2- 172.21.15.38:0/1504552644 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f6007c1d0 0x561f6009b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:10.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.824+0000 7fa9f0cb2740 1 --2- 172.21.15.38:0/1504552644 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f6007b8b0 0x561f600881b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:10.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.824+0000 7fa9f0cb2740 1 -- 172.21.15.38:0/1504552644 >> 172.21.15.38:0/1504552644 conn(0x561f60077f60 msgr2=0x561f60082a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:10.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.824+0000 7fa9f0cb2740 1 -- 172.21.15.38:0/1504552644 shutdown_connections 2024-09-16T00:32:10.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.824+0000 7fa9f0cb2740 1 -- 172.21.15.38:0/1504552644 wait complete. 2024-09-16T00:32:10.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.825+0000 7fa9f0cb2740 1 -- start start 2024-09-16T00:32:10.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.825+0000 7fa9f0cb2740 1 Processor -- start 2024-09-16T00:32:10.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.825+0000 7fa9f0cb2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f6007b8b0 0x561f6009e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:10.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.825+0000 7fa9f0cb2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f6007c1d0 0x561f6009e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:10.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.825+0000 7fa9f0cb2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f6008afc0 0x561f60103da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:10.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.825+0000 7fa9f0cb2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561f6009c990 con 0x561f6007b8b0 2024-09-16T00:32:10.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.825+0000 7fa9f0cb2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561f6009c2c0 con 0x561f6008afc0 2024-09-16T00:32:10.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.825+0000 7fa9f0cb2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561f6009d060 con 0x561f6007c1d0 2024-09-16T00:32:10.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.825+0000 7fa9f0cb2740 10 client.0 _ll_register_callbacks cb 0x7fa9d809ba30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:10.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.825+0000 7fa9e7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f6007b8b0 0x561f6009e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:10.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.825+0000 7fa9e7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f6007b8b0 0x561f6009e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49688/0 (socket says 172.21.15.38:49688) 2024-09-16T00:32:10.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.825+0000 7fa9e7fff640 1 -- 172.21.15.38:0/1756208876 learned_addr learned my addr 172.21.15.38:0/1756208876 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:10.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.826+0000 7fa9e77fe640 1 --2- 172.21.15.38:0/1756208876 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f6007c1d0 0x561f6009e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:10.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.826+0000 7fa9ece76640 1 --2- 172.21.15.38:0/1756208876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f6008afc0 0x561f60103da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:10.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.826+0000 7fa9e7fff640 1 -- 172.21.15.38:0/1756208876 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f6007c1d0 msgr2=0x561f6009e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:10.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.826+0000 7fa9e7fff640 1 --2- 172.21.15.38:0/1756208876 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f6007c1d0 0x561f6009e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:10.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.826+0000 7fa9e7fff640 1 -- 172.21.15.38:0/1756208876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f6008afc0 msgr2=0x561f60103da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:10.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.826+0000 7fa9e7fff640 1 --2- 172.21.15.38:0/1756208876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f6008afc0 0x561f60103da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:10.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.826+0000 7fa9e7fff640 1 -- 172.21.15.38:0/1756208876 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f601042e0 con 0x561f6007b8b0 2024-09-16T00:32:10.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.826+0000 7fa9e7fff640 1 --2- 172.21.15.38:0/1756208876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f6007b8b0 0x561f6009e0b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa9d80980c0 tx=0x7fa9d8097900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb6a822fcd6b515f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:10.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.826+0000 7fa9c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:32:10.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.826+0000 7fa9c77fe640 1 -- 172.21.15.38:0/1756208876 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9d801d070 con 0x561f6007b8b0 2024-09-16T00:32:10.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.826+0000 7fa9c77fe640 1 -- 172.21.15.38:0/1756208876 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7fa9d809b0b0 con 0x561f6007b8b0 2024-09-16T00:32:10.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.826+0000 7fa9f0cb2740 1 -- 172.21.15.38:0/1756208876 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9e001f430 con 0x561f6007b8b0 2024-09-16T00:32:10.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.826+0000 7fa9c77fe640 1 -- 172.21.15.38:0/1756208876 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fa9d80b4e60 con 0x561f6007b8b0 2024-09-16T00:32:10.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.826+0000 7fa9f0cb2740 10 client.15069 fetch_fsmap learned FSMap version 25 2024-09-16T00:32:10.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.826+0000 7fa9f0cb2740 1 -- 172.21.15.38:0/1756208876 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x561f601053a0 con 0x561f6007b8b0 2024-09-16T00:32:10.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.826+0000 7fa9c77fe640 1 -- 172.21.15.38:0/1756208876 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa9e001f430 con 0x561f6007b8b0 2024-09-16T00:32:10.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.827+0000 7fa9c77fe640 1 -- 172.21.15.38:0/1756208876 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fa9d80b4070 con 0x561f6007b8b0 2024-09-16T00:32:10.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.827+0000 7fa9f0cb2740 10 client.15069 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:32:10.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.827+0000 7fa9f0cb2740 10 client.15069 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-16T00:32:10.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.827+0000 7fa9f0cb2740 20 client.15069 populate_metadata read hostname 'smithi038' 2024-09-16T00:32:10.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.827+0000 7fa9f0cb2740 1 --2- 172.21.15.38:0/1756208876 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x561f60109d30 0x561f60108a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:10.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.827+0000 7fa9f0cb2740 4 client.15069 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:32:10.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.827+0000 7fa9f0cb2740 1 -- 172.21.15.38:0/1756208876 --> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] -- command(tid 0: {"prefix": "status"}) -- 0x561f5ffe9b10 con 0x561f60109d30 2024-09-16T00:32:10.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.827+0000 7fa9e77fe640 1 --2- 172.21.15.38:0/1756208876 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x561f60109d30 0x561f60108a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:10.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.827+0000 7fa9e77fe640 1 --2- 172.21.15.38:0/1756208876 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x561f60109d30 0x561f60108a40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=db25e08ebadf879e server_cookie=6a0bdbfb3900bc93 in_seq=0 out_seq=0 2024-09-16T00:32:10.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.827+0000 7fa9c77fe640 10 client.15069 ms_handle_connect on v2:172.21.15.73:6834/2866363113 2024-09-16T00:32:10.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.828+0000 7fa9c77fe640 1 -- 172.21.15.38:0/1756208876 <== mds.2 v2:172.21.15.73:6834/2866363113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x561f5ffe9b10 con 0x561f60109d30 2024-09-16T00:32:10.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.828+0000 7fa9c77fe640 10 client.15069 handle_command_reply: tid=0 2024-09-16T00:32:10.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.828+0000 7fa9f0cb2740 1 client.15069 shutdown 2024-09-16T00:32:10.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.828+0000 7fa9f0cb2740 1 -- 172.21.15.38:0/1756208876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f6007b8b0 msgr2=0x561f6009e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:10.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.828+0000 7fa9f0cb2740 1 --2- 172.21.15.38:0/1756208876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f6007b8b0 0x561f6009e0b0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fa9d80980c0 tx=0x7fa9d8097900 comp rx=0 tx=0).stop 2024-09-16T00:32:10.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.828+0000 7fa9f0cb2740 1 -- 172.21.15.38:0/1756208876 shutdown_connections 2024-09-16T00:32:10.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.828+0000 7fa9f0cb2740 1 --2- 172.21.15.38:0/1756208876 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x561f60109d30 0x561f60108a40 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:10.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.828+0000 7fa9f0cb2740 1 --2- 172.21.15.38:0/1756208876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f6008afc0 0x561f60103da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:10.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.828+0000 7fa9f0cb2740 1 --2- 172.21.15.38:0/1756208876 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f6007c1d0 0x561f6009e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:10.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.828+0000 7fa9f0cb2740 1 --2- 172.21.15.38:0/1756208876 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f6007b8b0 0x561f6009e0b0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:10.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.828+0000 7fa9c77fe640 0 client.15069 ms_handle_reset on v2:172.21.15.73:6834/2866363113 2024-09-16T00:32:10.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.828+0000 7fa9f0cb2740 1 -- 172.21.15.38:0/1756208876 >> 172.21.15.38:0/1756208876 conn(0x561f60077f60 msgr2=0x561f60083240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:10.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.828+0000 7fa9f0cb2740 1 -- 172.21.15.38:0/1756208876 shutdown_connections 2024-09-16T00:32:10.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.828+0000 7fa9f0cb2740 1 -- 172.21.15.38:0/1756208876 wait complete. 2024-09-16T00:32:10.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.828+0000 7fa9f0cb2740 20 client.15069 trim_cache size 0 max 16384 2024-09-16T00:32:10.850 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-16T00:32:10.850 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "983b2072-73c1-11ef-bceb-c7b262605968", 2024-09-16T00:32:10.850 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 2, 2024-09-16T00:32:10.850 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24440, 2024-09-16T00:32:10.850 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-16T00:32:10.850 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-16T00:32:10.850 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-16T00:32:10.850 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 68.523015135999998, 2024-09-16T00:32:10.851 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 25, 2024-09-16T00:32:10.851 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 90, 2024-09-16T00:32:10.851 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 88, 2024-09-16T00:32:10.851 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 134.103513152 2024-09-16T00:32:10.851 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-16T00:32:10.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.828+0000 7fa9ee8d9640 1 -- 172.21.15.38:0/2939898376 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fa9d00a32d0 msgr2=0x7fa9d00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:10.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.828+0000 7fa9ee8d9640 1 --2- 172.21.15.38:0/2939898376 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fa9d00a32d0 0x7fa9d00a5790 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa9e0008dd0 tx=0x7fa9e0008750 comp rx=0 tx=0).stop 2024-09-16T00:32:10.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.828+0000 7fa9ee8d9640 1 -- 172.21.15.38:0/2939898376 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9e8100e20 msgr2=0x7fa9e8198460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:10.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.828+0000 7fa9ee8d9640 1 --2- 172.21.15.38:0/2939898376 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9e8100e20 0x7fa9e8198460 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fa9d800c890 tx=0x7fa9d800cd50 comp rx=0 tx=0).stop 2024-09-16T00:32:10.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.829+0000 7fa9ee8d9640 1 -- 172.21.15.38:0/2939898376 shutdown_connections 2024-09-16T00:32:10.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.829+0000 7fa9ee8d9640 1 --2- 172.21.15.38:0/2939898376 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fa9d00a32d0 0x7fa9d00a5790 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:10.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.829+0000 7fa9ee8d9640 1 --2- 172.21.15.38:0/2939898376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa9e8102ef0 0x7fa9e819cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:10.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.829+0000 7fa9ee8d9640 1 --2- 172.21.15.38:0/2939898376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9e8102570 0x7fa9e81989a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:10.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.829+0000 7fa9ee8d9640 1 --2- 172.21.15.38:0/2939898376 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9e8100e20 0x7fa9e8198460 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:10.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.829+0000 7fa9ee8d9640 1 -- 172.21.15.38:0/2939898376 >> 172.21.15.38:0/2939898376 conn(0x7fa9e80fc7d0 msgr2=0x7fa9e80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:10.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.829+0000 7fa9ee8d9640 1 -- 172.21.15.38:0/2939898376 shutdown_connections 2024-09-16T00:32:10.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:10.829+0000 7fa9ee8d9640 1 -- 172.21.15.38:0/2939898376 wait complete. 2024-09-16T00:32:11.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:11 smithi038 ceph-mon[30945]: pgmap v344: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T00:32:11.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:11 smithi073 ceph-mon[35254]: pgmap v344: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T00:32:11.647 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-16T00:32:11.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:11 smithi122 ceph-mon[35412]: pgmap v344: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T00:32:11.904 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:12.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.422+0000 7f65b1cca640 1 -- 172.21.15.38:0/3847401084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65ac102570 msgr2=0x7f65ac1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:12.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.422+0000 7f65b1cca640 1 --2- 172.21.15.38:0/3847401084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65ac102570 0x7f65ac1029b0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f6594009ca0 tx=0x7f659402a950 comp rx=0 tx=0).stop 2024-09-16T00:32:12.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.422+0000 7f65b1cca640 1 -- 172.21.15.38:0/3847401084 shutdown_connections 2024-09-16T00:32:12.440 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.422+0000 7f65b1cca640 1 --2- 172.21.15.38:0/3847401084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65ac102ef0 0x7f65ac10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:12.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.422+0000 7f65b1cca640 1 --2- 172.21.15.38:0/3847401084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65ac102570 0x7f65ac1029b0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:12.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.422+0000 7f65b1cca640 1 --2- 172.21.15.38:0/3847401084 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65ac100e20 0x7f65ac101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:12.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.422+0000 7f65b1cca640 1 -- 172.21.15.38:0/3847401084 >> 172.21.15.38:0/3847401084 conn(0x7f65ac0fc7d0 msgr2=0x7f65ac0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:12.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.422+0000 7f65b1cca640 1 -- 172.21.15.38:0/3847401084 shutdown_connections 2024-09-16T00:32:12.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.422+0000 7f65b1cca640 1 -- 172.21.15.38:0/3847401084 wait complete. 2024-09-16T00:32:12.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.422+0000 7f65b1cca640 1 Processor -- start 2024-09-16T00:32:12.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.422+0000 7f65b1cca640 1 -- start start 2024-09-16T00:32:12.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.423+0000 7f65b1cca640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65ac100e20 0x7f65ac198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:12.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.423+0000 7f65b1cca640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65ac102570 0x7f65ac198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:12.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.423+0000 7f65b1cca640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65ac102ef0 0x7f65ac19cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:12.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.423+0000 7f65b1cca640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac110f80 con 0x7f65ac102ef0 2024-09-16T00:32:12.441 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.423+0000 7f65b1cca640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac110e00 con 0x7f65ac100e20 2024-09-16T00:32:12.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.423+0000 7f65b1cca640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f65ac111100 con 0x7f65ac102570 2024-09-16T00:32:12.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.423+0000 7f65abfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65ac102ef0 0x7f65ac19cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:12.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.423+0000 7f65abfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65ac102ef0 0x7f65ac19cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49712/0 (socket says 172.21.15.38:49712) 2024-09-16T00:32:12.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.423+0000 7f65abfff640 1 -- 172.21.15.38:0/3006697969 learned_addr learned my addr 172.21.15.38:0/3006697969 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:12.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.423+0000 7f65ab7fe640 1 --2- 172.21.15.38:0/3006697969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65ac100e20 0x7f65ac198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:12.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.423+0000 7f65aaffd640 1 --2- 172.21.15.38:0/3006697969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65ac102570 0x7f65ac198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:12.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.423+0000 7f65abfff640 1 -- 172.21.15.38:0/3006697969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65ac102570 msgr2=0x7f65ac198950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:12.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.423+0000 7f65abfff640 1 --2- 172.21.15.38:0/3006697969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65ac102570 0x7f65ac198950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:12.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.423+0000 7f65abfff640 1 -- 172.21.15.38:0/3006697969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65ac100e20 msgr2=0x7f65ac198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:12.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.423+0000 7f65abfff640 1 --2- 172.21.15.38:0/3006697969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65ac100e20 0x7f65ac198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:12.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.423+0000 7f65abfff640 1 -- 172.21.15.38:0/3006697969 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65ac19d3c0 con 0x7f65ac102ef0 2024-09-16T00:32:12.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.423+0000 7f65abfff640 1 --2- 172.21.15.38:0/3006697969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65ac102ef0 0x7f65ac19cce0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f659c00c970 tx=0x7f659c00cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:12.442 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.424+0000 7f65a8ff9640 1 -- 172.21.15.38:0/3006697969 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c007bb0 con 0x7f65ac102ef0 2024-09-16T00:32:12.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.424+0000 7f65a8ff9640 1 -- 172.21.15.38:0/3006697969 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f659c007d50 con 0x7f65ac102ef0 2024-09-16T00:32:12.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.424+0000 7f65b1cca640 1 -- 172.21.15.38:0/3006697969 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65ac19d6b0 con 0x7f65ac102ef0 2024-09-16T00:32:12.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.424+0000 7f65a8ff9640 1 -- 172.21.15.38:0/3006697969 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f659c018770 con 0x7f65ac102ef0 2024-09-16T00:32:12.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.424+0000 7f65b1cca640 1 -- 172.21.15.38:0/3006697969 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65ac1a4f90 con 0x7f65ac102ef0 2024-09-16T00:32:12.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.426+0000 7f65a8ff9640 1 -- 172.21.15.38:0/3006697969 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f659c00f450 con 0x7f65ac102ef0 2024-09-16T00:32:12.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.426+0000 7f65b40a3740 1 Processor -- start 2024-09-16T00:32:12.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.426+0000 7f65b40a3740 1 -- start start 2024-09-16T00:32:12.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.427+0000 7f65b40a3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56089fc419d0 0x56089fc41db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:12.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.427+0000 7f65b40a3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56089fc422f0 0x56089fc4c670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:12.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.427+0000 7f65b40a3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56089fc4cbb0 0x56089fc4efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:12.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.427+0000 7f65a8ff9640 1 --2- 172.21.15.38:0/3006697969 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f65980a32d0 0x7f65980a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:12.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.427+0000 7f65b40a3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56089fba8090 con 0x56089fc419d0 2024-09-16T00:32:12.443 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.427+0000 7f65b40a3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56089fb4fa70 con 0x56089fc4cbb0 2024-09-16T00:32:12.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.427+0000 7f65b40a3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56089fba4050 con 0x56089fc422f0 2024-09-16T00:32:12.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.427+0000 7f65aaffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56089fc419d0 0x56089fc41db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:12.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.427+0000 7f65aaffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56089fc419d0 0x56089fc41db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49720/0 (socket says 172.21.15.38:49720) 2024-09-16T00:32:12.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.427+0000 7f65aaffd640 1 -- 172.21.15.38:0/709811707 learned_addr learned my addr 172.21.15.38:0/709811707 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:12.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.427+0000 7f65ab7fe640 1 --2- 172.21.15.38:0/709811707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56089fc422f0 0x56089fc4c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:12.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.427+0000 7f65a8ff9640 1 -- 172.21.15.38:0/3006697969 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f659c0cfe20 con 0x7f65ac102ef0 2024-09-16T00:32:12.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.427+0000 7f65abfff640 1 --2- 172.21.15.38:0/3006697969 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f65980a32d0 0x7f65980a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:12.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.427+0000 7f65aaffd640 1 --2- 172.21.15.38:0/709811707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56089fc4cbb0 0x56089fc4efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:12.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.427+0000 7f65aaffd640 1 -- 172.21.15.38:0/709811707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56089fc422f0 msgr2=0x56089fc4c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:12.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.427+0000 7f65aaffd640 1 --2- 172.21.15.38:0/709811707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56089fc422f0 0x56089fc4c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:12.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.427+0000 7f65aaffd640 1 -- 172.21.15.38:0/709811707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56089fc4cbb0 msgr2=0x56089fc4efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:12.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.427+0000 7f65aaffd640 1 --2- 172.21.15.38:0/709811707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56089fc4cbb0 0x56089fc4efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:12.444 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.427+0000 7f65aaffd640 1 -- 172.21.15.38:0/709811707 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56089fb5c130 con 0x56089fc419d0 2024-09-16T00:32:12.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.427+0000 7f65ab7fe640 1 --2- 172.21.15.38:0/709811707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56089fc422f0 0x56089fc4c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:32:12.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.427+0000 7f65aaffd640 1 --2- 172.21.15.38:0/709811707 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56089fc419d0 0x56089fc41db0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f65940093d0 tx=0x7f6594005fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3843eea706fa926 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:12.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.427+0000 7f65abfff640 1 --2- 172.21.15.38:0/3006697969 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f65980a32d0 0x7f65980a5790 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f659c09bed0 tx=0x7f659c00fba0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:12.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.428+0000 7f65a1ffb640 1 -- 172.21.15.38:0/709811707 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6594034040 con 0x56089fc419d0 2024-09-16T00:32:12.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.428+0000 7f65a1ffb640 1 -- 172.21.15.38:0/709811707 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6594044040 con 0x56089fc419d0 2024-09-16T00:32:12.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.428+0000 7f65a1ffb640 1 -- 172.21.15.38:0/709811707 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65940047e0 con 0x56089fc419d0 2024-09-16T00:32:12.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.428+0000 7f65b40a3740 1 -- 172.21.15.38:0/709811707 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56089fc419d0 msgr2=0x56089fc41db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:12.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.428+0000 7f65b40a3740 1 --2- 172.21.15.38:0/709811707 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56089fc419d0 0x56089fc41db0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f65940093d0 tx=0x7f6594005fa0 comp rx=0 tx=0).stop 2024-09-16T00:32:12.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.428+0000 7f65b40a3740 1 -- 172.21.15.38:0/709811707 shutdown_connections 2024-09-16T00:32:12.446 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.428+0000 7f65b40a3740 1 --2- 172.21.15.38:0/709811707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56089fc4cbb0 0x56089fc4efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:12.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.428+0000 7f65b40a3740 1 --2- 172.21.15.38:0/709811707 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56089fc422f0 0x56089fc4c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:12.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.428+0000 7f65b40a3740 1 --2- 172.21.15.38:0/709811707 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56089fc419d0 0x56089fc41db0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:12.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.428+0000 7f65b40a3740 1 -- 172.21.15.38:0/709811707 >> 172.21.15.38:0/709811707 conn(0x56089fc3df60 msgr2=0x56089fc3e340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:12.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.428+0000 7f65b40a3740 1 -- 172.21.15.38:0/709811707 shutdown_connections 2024-09-16T00:32:12.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.429+0000 7f65b40a3740 1 -- 172.21.15.38:0/709811707 wait complete. 2024-09-16T00:32:12.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.429+0000 7f65b40a3740 1 -- start start 2024-09-16T00:32:12.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.429+0000 7f65b40a3740 1 Processor -- start 2024-09-16T00:32:12.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.429+0000 7f65b40a3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56089fc419d0 0x56089fc4c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:12.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.429+0000 7f65b40a3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56089fc422f0 0x56089fc497c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:12.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.429+0000 7f65b40a3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56089fc4cbb0 0x56089fc49d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:12.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.429+0000 7f65b40a3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56089fc4f780 con 0x56089fc422f0 2024-09-16T00:32:12.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.429+0000 7f65b40a3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56089fb62f20 con 0x56089fc4cbb0 2024-09-16T00:32:12.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.429+0000 7f65b40a3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56089fc4f900 con 0x56089fc419d0 2024-09-16T00:32:12.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.429+0000 7f65b40a3740 10 client.0 _ll_register_callbacks cb 0x56089fb2c030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:12.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.430+0000 7f65ab7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56089fc422f0 0x56089fc497c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:12.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.430+0000 7f65ab7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56089fc422f0 0x56089fc497c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49734/0 (socket says 172.21.15.38:49734) 2024-09-16T00:32:12.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.430+0000 7f65ab7fe640 1 -- 172.21.15.38:0/2264158332 learned_addr learned my addr 172.21.15.38:0/2264158332 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:12.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.430+0000 7f65aaffd640 1 --2- 172.21.15.38:0/2264158332 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56089fc419d0 0x56089fc4c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:12.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.430+0000 7f65aaffd640 1 --2- 172.21.15.38:0/2264158332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56089fc4cbb0 0x56089fc49d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:12.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.430+0000 7f65ab7fe640 1 -- 172.21.15.38:0/2264158332 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56089fc419d0 msgr2=0x56089fc4c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:12.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.430+0000 7f65ab7fe640 1 --2- 172.21.15.38:0/2264158332 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56089fc419d0 0x56089fc4c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:12.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.430+0000 7f65ab7fe640 1 -- 172.21.15.38:0/2264158332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56089fc4cbb0 msgr2=0x56089fc49d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:12.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.430+0000 7f65ab7fe640 1 --2- 172.21.15.38:0/2264158332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56089fc4cbb0 0x56089fc49d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:12.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.430+0000 7f65ab7fe640 1 -- 172.21.15.38:0/2264158332 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56089f9192b0 con 0x56089fc422f0 2024-09-16T00:32:12.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.430+0000 7f65aaffd640 1 --2- 172.21.15.38:0/2264158332 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56089fc419d0 0x56089fc4c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:12.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.430+0000 7f65ab7fe640 1 --2- 172.21.15.38:0/2264158332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56089fc422f0 0x56089fc497c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f65a4010a50 tx=0x7f65a4010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99074c6b78398216 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:12.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.430+0000 7f658affd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:32:12.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.430+0000 7f658affd640 1 -- 172.21.15.38:0/2264158332 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a401e060 con 0x56089fc422f0 2024-09-16T00:32:12.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.430+0000 7f658affd640 1 -- 172.21.15.38:0/2264158332 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f65a400fbd0 con 0x56089fc422f0 2024-09-16T00:32:12.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.430+0000 7f65b40a3740 1 -- 172.21.15.38:0/2264158332 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6594044040 con 0x56089fc422f0 2024-09-16T00:32:12.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.430+0000 7f658affd640 1 -- 172.21.15.38:0/2264158332 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f65a4025050 con 0x56089fc422f0 2024-09-16T00:32:12.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.430+0000 7f65b40a3740 10 client.15081 fetch_fsmap learned FSMap version 25 2024-09-16T00:32:12.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.430+0000 7f65b40a3740 1 -- 172.21.15.38:0/2264158332 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56089fc5b160 con 0x56089fc422f0 2024-09-16T00:32:12.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.431+0000 7f658affd640 1 -- 172.21.15.38:0/2264158332 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6594044040 con 0x56089fc422f0 2024-09-16T00:32:12.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.431+0000 7f658affd640 1 -- 172.21.15.38:0/2264158332 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f65a4018400 con 0x56089fc422f0 2024-09-16T00:32:12.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.431+0000 7f65b40a3740 10 client.15081 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:32:12.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.431+0000 7f65b40a3740 10 client.15081 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-16T00:32:12.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.431+0000 7f65b40a3740 20 client.15081 populate_metadata read hostname 'smithi038' 2024-09-16T00:32:12.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.431+0000 7f65b40a3740 1 --2- 172.21.15.38:0/2264158332 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x56089fc5d770 0x56089fc5ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:12.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.431+0000 7f65b40a3740 4 client.15081 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:32:12.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.431+0000 7f65b40a3740 1 -- 172.21.15.38:0/2264158332 --> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56089fbb5600 con 0x56089fc5d770 2024-09-16T00:32:12.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.431+0000 7f65aaffd640 1 --2- 172.21.15.38:0/2264158332 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x56089fc5d770 0x56089fc5ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:12.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.432+0000 7f65aaffd640 1 --2- 172.21.15.38:0/2264158332 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x56089fc5d770 0x56089fc5ebe0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6e806c82a8a7ded0 server_cookie=12bbf15096d047a3 in_seq=0 out_seq=0 2024-09-16T00:32:12.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.432+0000 7f658affd640 10 client.15081 ms_handle_connect on v2:172.21.15.73:6836/2681081265 2024-09-16T00:32:12.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.433+0000 7f658affd640 1 -- 172.21.15.38:0/2264158332 <== mds.1 v2:172.21.15.73:6836/2681081265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56089fbb5600 con 0x56089fc5d770 2024-09-16T00:32:12.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.433+0000 7f658affd640 10 client.15081 handle_command_reply: tid=0 2024-09-16T00:32:12.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.433+0000 7f65b40a3740 1 client.15081 shutdown 2024-09-16T00:32:12.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.433+0000 7f65b40a3740 1 -- 172.21.15.38:0/2264158332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56089fc422f0 msgr2=0x56089fc497c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:12.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.433+0000 7f65b40a3740 1 --2- 172.21.15.38:0/2264158332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56089fc422f0 0x56089fc497c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f65a4010a50 tx=0x7f65a4010f10 comp rx=0 tx=0).stop 2024-09-16T00:32:12.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.433+0000 7f65b40a3740 1 -- 172.21.15.38:0/2264158332 shutdown_connections 2024-09-16T00:32:12.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.433+0000 7f65b40a3740 1 --2- 172.21.15.38:0/2264158332 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56089fc4cbb0 0x56089fc49d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:12.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.433+0000 7f65b40a3740 1 --2- 172.21.15.38:0/2264158332 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x56089fc5d770 0x56089fc5ebe0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:12.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.433+0000 7f65b40a3740 1 --2- 172.21.15.38:0/2264158332 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56089fc422f0 0x56089fc497c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:12.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.433+0000 7f65b40a3740 1 --2- 172.21.15.38:0/2264158332 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56089fc419d0 0x56089fc4c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:12.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.433+0000 7f65b40a3740 1 -- 172.21.15.38:0/2264158332 >> 172.21.15.38:0/2264158332 conn(0x56089fc3df60 msgr2=0x56089fc43c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:12.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.433+0000 7f658affd640 0 client.15081 ms_handle_reset on v2:172.21.15.73:6836/2681081265 2024-09-16T00:32:12.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.433+0000 7f65b40a3740 1 -- 172.21.15.38:0/2264158332 shutdown_connections 2024-09-16T00:32:12.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.433+0000 7f65b40a3740 1 -- 172.21.15.38:0/2264158332 wait complete. 2024-09-16T00:32:12.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.433+0000 7f65b40a3740 20 client.15081 trim_cache size 0 max 16384 2024-09-16T00:32:12.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.451+0000 7f65b40a3740 1 Processor -- start 2024-09-16T00:32:12.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.451+0000 7f65b40a3740 1 -- start start 2024-09-16T00:32:12.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.451+0000 7f65b40a3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56089fc419d0 0x56089fc5ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:12.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.451+0000 7f65b40a3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56089fc422f0 0x56089fc66e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:12.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.451+0000 7f65b40a3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56089fc4cbb0 0x56089fc673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:12.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.451+0000 7f65b40a3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56089fc5cf20 con 0x56089fc422f0 2024-09-16T00:32:12.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.451+0000 7f65b40a3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56089fc5d5f0 con 0x56089fc419d0 2024-09-16T00:32:12.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.451+0000 7f65b40a3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56089fc5dcc0 con 0x56089fc4cbb0 2024-09-16T00:32:12.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.451+0000 7f65ab7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56089fc422f0 0x56089fc66e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:12.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.452+0000 7f65ab7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56089fc422f0 0x56089fc66e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49740/0 (socket says 172.21.15.38:49740) 2024-09-16T00:32:12.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.452+0000 7f65ab7fe640 1 -- 172.21.15.38:0/2948397343 learned_addr learned my addr 172.21.15.38:0/2948397343 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:12.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.452+0000 7f65aaffd640 1 --2- 172.21.15.38:0/2948397343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56089fc4cbb0 0x56089fc673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:12.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.452+0000 7f65aaffd640 1 --2- 172.21.15.38:0/2948397343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56089fc419d0 0x56089fc5ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:12.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.452+0000 7f65ab7fe640 1 -- 172.21.15.38:0/2948397343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56089fc4cbb0 msgr2=0x56089fc673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:12.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.452+0000 7f65ab7fe640 1 --2- 172.21.15.38:0/2948397343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56089fc4cbb0 0x56089fc673d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:12.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.452+0000 7f65ab7fe640 1 -- 172.21.15.38:0/2948397343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56089fc419d0 msgr2=0x56089fc5ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:12.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.452+0000 7f65ab7fe640 1 --2- 172.21.15.38:0/2948397343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56089fc419d0 0x56089fc5ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:12.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.452+0000 7f65ab7fe640 1 -- 172.21.15.38:0/2948397343 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56089fc4a800 con 0x56089fc422f0 2024-09-16T00:32:12.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.452+0000 7f65ab7fe640 1 --2- 172.21.15.38:0/2948397343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56089fc422f0 0x56089fc66e90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f65a40187d0 tx=0x7f65a400a6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e60752c03ffb45 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:12.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.452+0000 7f65a17fa640 1 -- 172.21.15.38:0/2948397343 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a402f070 con 0x56089fc422f0 2024-09-16T00:32:12.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.452+0000 7f65a17fa640 1 -- 172.21.15.38:0/2948397343 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65a4005550 con 0x56089fc422f0 2024-09-16T00:32:12.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.452+0000 7f65a17fa640 1 -- 172.21.15.38:0/2948397343 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a400fdf0 con 0x56089fc422f0 2024-09-16T00:32:12.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.453+0000 7f65b40a3740 1 -- 172.21.15.38:0/2948397343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56089fc422f0 msgr2=0x56089fc66e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:12.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.453+0000 7f65b40a3740 1 --2- 172.21.15.38:0/2948397343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56089fc422f0 0x56089fc66e90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f65a40187d0 tx=0x7f65a400a6e0 comp rx=0 tx=0).stop 2024-09-16T00:32:12.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.453+0000 7f65b40a3740 1 -- 172.21.15.38:0/2948397343 shutdown_connections 2024-09-16T00:32:12.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.453+0000 7f65b40a3740 1 --2- 172.21.15.38:0/2948397343 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56089fc4cbb0 0x56089fc673d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:12.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.453+0000 7f65b40a3740 1 --2- 172.21.15.38:0/2948397343 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56089fc422f0 0x56089fc66e90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:12.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.453+0000 7f65b40a3740 1 --2- 172.21.15.38:0/2948397343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56089fc419d0 0x56089fc5ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:12.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.453+0000 7f65b40a3740 1 -- 172.21.15.38:0/2948397343 >> 172.21.15.38:0/2948397343 conn(0x56089fc3df60 msgr2=0x56089fc48090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:12.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.453+0000 7f65b40a3740 1 -- 172.21.15.38:0/2948397343 shutdown_connections 2024-09-16T00:32:12.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.453+0000 7f65b40a3740 1 -- 172.21.15.38:0/2948397343 wait complete. 2024-09-16T00:32:12.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.453+0000 7f65b40a3740 1 -- start start 2024-09-16T00:32:12.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.454+0000 7f65b40a3740 1 Processor -- start 2024-09-16T00:32:12.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.454+0000 7f65b40a3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56089fc419d0 0x56089fc461d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:12.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.454+0000 7f65b40a3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56089fc422f0 0x56089fc46710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:12.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.454+0000 7f65b40a3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56089fc4cbb0 0x56089fc65e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:12.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.454+0000 7f65b40a3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56089fc5ea60 con 0x56089fc422f0 2024-09-16T00:32:12.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.454+0000 7f65b40a3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56089fc5e390 con 0x56089fc419d0 2024-09-16T00:32:12.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.454+0000 7f65b40a3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56089fc5f130 con 0x56089fc4cbb0 2024-09-16T00:32:12.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.454+0000 7f65b40a3740 10 client.0 _ll_register_callbacks cb 0x7f65a400a800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:12.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.454+0000 7f65aaffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56089fc419d0 0x56089fc461d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:12.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.454+0000 7f65aaffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56089fc419d0 0x56089fc461d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.38:55856/0 (socket says 172.21.15.38:55856) 2024-09-16T00:32:12.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.454+0000 7f65aaffd640 1 -- 172.21.15.38:0/2206018997 learned_addr learned my addr 172.21.15.38:0/2206018997 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:12.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.454+0000 7f65ab7fe640 1 --2- 172.21.15.38:0/2206018997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56089fc422f0 0x56089fc46710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:12.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.454+0000 7f65aaffd640 1 --2- 172.21.15.38:0/2206018997 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56089fc4cbb0 0x56089fc65e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:12.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.455+0000 7f65ab7fe640 1 -- 172.21.15.38:0/2206018997 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56089fc4cbb0 msgr2=0x56089fc65e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:12.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.455+0000 7f65ab7fe640 1 --2- 172.21.15.38:0/2206018997 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56089fc4cbb0 0x56089fc65e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:12.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.455+0000 7f65ab7fe640 1 -- 172.21.15.38:0/2206018997 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56089fc419d0 msgr2=0x56089fc461d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:12.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.455+0000 7f65ab7fe640 1 --2- 172.21.15.38:0/2206018997 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56089fc419d0 0x56089fc461d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:12.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.455+0000 7f65ab7fe640 1 -- 172.21.15.38:0/2206018997 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56089fc66370 con 0x56089fc422f0 2024-09-16T00:32:12.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.455+0000 7f65aaffd640 1 --2- 172.21.15.38:0/2206018997 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56089fc419d0 0x56089fc461d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:32:12.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.455+0000 7f65ab7fe640 1 --2- 172.21.15.38:0/2206018997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56089fc422f0 0x56089fc46710 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f65a4018a10 tx=0x7f65a4005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6f2ce63e1d041e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:12.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.455+0000 7f658affd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:32:12.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.455+0000 7f658affd640 1 -- 172.21.15.38:0/2206018997 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65a402f070 con 0x56089fc422f0 2024-09-16T00:32:12.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.455+0000 7f658affd640 1 -- 172.21.15.38:0/2206018997 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f65a400fbd0 con 0x56089fc422f0 2024-09-16T00:32:12.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.455+0000 7f65b40a3740 1 -- 172.21.15.38:0/2206018997 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65a4005550 con 0x56089fc422f0 2024-09-16T00:32:12.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.455+0000 7f658affd640 1 -- 172.21.15.38:0/2206018997 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f65a401e040 con 0x56089fc422f0 2024-09-16T00:32:12.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.455+0000 7f65b40a3740 10 client.15087 fetch_fsmap learned FSMap version 25 2024-09-16T00:32:12.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.455+0000 7f65b40a3740 1 -- 172.21.15.38:0/2206018997 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x56089fc675d0 con 0x56089fc422f0 2024-09-16T00:32:12.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.455+0000 7f658affd640 1 -- 172.21.15.38:0/2206018997 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f65a4005550 con 0x56089fc422f0 2024-09-16T00:32:12.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.455+0000 7f658affd640 1 -- 172.21.15.38:0/2206018997 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f65a4033400 con 0x56089fc422f0 2024-09-16T00:32:12.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.456+0000 7f65b40a3740 10 client.15087 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:32:12.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.456+0000 7f65b40a3740 10 client.15087 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-16T00:32:12.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.456+0000 7f65b40a3740 20 client.15087 populate_metadata read hostname 'smithi038' 2024-09-16T00:32:12.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.456+0000 7f65b40a3740 1 --2- 172.21.15.38:0/2206018997 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x56089fccc050 0x56089fccad60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:12.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.456+0000 7f65b40a3740 4 client.15087 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:32:12.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.456+0000 7f65b40a3740 1 -- 172.21.15.38:0/2206018997 --> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] -- command(tid 0: {"prefix": "status"}) -- 0x56089fbafb10 con 0x56089fccc050 2024-09-16T00:32:12.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.456+0000 7f65aaffd640 1 --2- 172.21.15.38:0/2206018997 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x56089fccc050 0x56089fccad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:12.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.456+0000 7f65aaffd640 1 --2- 172.21.15.38:0/2206018997 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x56089fccc050 0x56089fccad60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=45dabed420e2f388 server_cookie=1680d0a46c00d5c9 in_seq=0 out_seq=0 2024-09-16T00:32:12.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.456+0000 7f658affd640 10 client.15087 ms_handle_connect on v2:172.21.15.73:6836/2681081265 2024-09-16T00:32:12.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.457+0000 7f658affd640 1 -- 172.21.15.38:0/2206018997 <== mds.1 v2:172.21.15.73:6836/2681081265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x56089fbafb10 con 0x56089fccc050 2024-09-16T00:32:12.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.457+0000 7f658affd640 10 client.15087 handle_command_reply: tid=0 2024-09-16T00:32:12.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.457+0000 7f65b40a3740 1 client.15087 shutdown 2024-09-16T00:32:12.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.457+0000 7f65b40a3740 1 -- 172.21.15.38:0/2206018997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56089fc422f0 msgr2=0x56089fc46710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:12.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.457+0000 7f65b40a3740 1 --2- 172.21.15.38:0/2206018997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56089fc422f0 0x56089fc46710 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f65a4018a10 tx=0x7f65a4005780 comp rx=0 tx=0).stop 2024-09-16T00:32:12.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.457+0000 7f65b40a3740 1 -- 172.21.15.38:0/2206018997 shutdown_connections 2024-09-16T00:32:12.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.457+0000 7f65b40a3740 1 --2- 172.21.15.38:0/2206018997 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56089fc4cbb0 0x56089fc65e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:12.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.457+0000 7f65b40a3740 1 --2- 172.21.15.38:0/2206018997 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x56089fccc050 0x56089fccad60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:12.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.457+0000 7f65b40a3740 1 --2- 172.21.15.38:0/2206018997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56089fc422f0 0x56089fc46710 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:12.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.457+0000 7f65b40a3740 1 --2- 172.21.15.38:0/2206018997 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56089fc419d0 0x56089fc461d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:12.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.457+0000 7f65b40a3740 1 -- 172.21.15.38:0/2206018997 >> 172.21.15.38:0/2206018997 conn(0x56089fc3df60 msgr2=0x56089fc43420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:12.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.457+0000 7f658affd640 0 client.15087 ms_handle_reset on v2:172.21.15.73:6836/2681081265 2024-09-16T00:32:12.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.457+0000 7f65b40a3740 1 -- 172.21.15.38:0/2206018997 shutdown_connections 2024-09-16T00:32:12.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.457+0000 7f65b40a3740 1 -- 172.21.15.38:0/2206018997 wait complete. 2024-09-16T00:32:12.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.457+0000 7f65b40a3740 20 client.15087 trim_cache size 0 max 16384 2024-09-16T00:32:12.465 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-16T00:32:12.465 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "983b2072-73c1-11ef-bceb-c7b262605968", 2024-09-16T00:32:12.466 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 1, 2024-09-16T00:32:12.466 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24464, 2024-09-16T00:32:12.466 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-16T00:32:12.466 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-16T00:32:12.466 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-16T00:32:12.466 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 71.164213664000002, 2024-09-16T00:32:12.466 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 25, 2024-09-16T00:32:12.467 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 90, 2024-09-16T00:32:12.467 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 88, 2024-09-16T00:32:12.467 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 127.80134502 2024-09-16T00:32:12.467 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-16T00:32:12.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.457+0000 7f65b1cca640 1 -- 172.21.15.38:0/3006697969 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f65980a32d0 msgr2=0x7f65980a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:12.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.458+0000 7f65b1cca640 1 --2- 172.21.15.38:0/3006697969 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f65980a32d0 0x7f65980a5790 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f659c09bed0 tx=0x7f659c00fba0 comp rx=0 tx=0).stop 2024-09-16T00:32:12.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.458+0000 7f65b1cca640 1 -- 172.21.15.38:0/3006697969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65ac102ef0 msgr2=0x7f65ac19cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:12.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.458+0000 7f65b1cca640 1 --2- 172.21.15.38:0/3006697969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65ac102ef0 0x7f65ac19cce0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f659c00c970 tx=0x7f659c00cd70 comp rx=0 tx=0).stop 2024-09-16T00:32:12.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.458+0000 7f65b1cca640 1 -- 172.21.15.38:0/3006697969 shutdown_connections 2024-09-16T00:32:12.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.458+0000 7f65b1cca640 1 --2- 172.21.15.38:0/3006697969 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f65980a32d0 0x7f65980a5790 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:12.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.458+0000 7f65b1cca640 1 --2- 172.21.15.38:0/3006697969 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f65ac102ef0 0x7f65ac19cce0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:12.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.458+0000 7f65b1cca640 1 --2- 172.21.15.38:0/3006697969 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f65ac102570 0x7f65ac198950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:12.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.458+0000 7f65b1cca640 1 --2- 172.21.15.38:0/3006697969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65ac100e20 0x7f65ac198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:12.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.458+0000 7f65b1cca640 1 -- 172.21.15.38:0/3006697969 >> 172.21.15.38:0/3006697969 conn(0x7f65ac0fc7d0 msgr2=0x7f65ac06a150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:12.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.458+0000 7f65b1cca640 1 -- 172.21.15.38:0/3006697969 shutdown_connections 2024-09-16T00:32:12.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:12.458+0000 7f65b1cca640 1 -- 172.21.15.38:0/3006697969 wait complete. 2024-09-16T00:32:13.228 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-16T00:32:13.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:13 smithi038 ceph-mon[30945]: pgmap v345: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T00:32:13.489 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:13.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:13 smithi073 ceph-mon[35254]: pgmap v345: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T00:32:13.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:13 smithi122 ceph-mon[35412]: pgmap v345: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T00:32:14.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.983+0000 7f9d6f252640 1 -- 172.21.15.38:0/209647015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d68102e00 msgr2=0x7f9d681031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:14.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.983+0000 7f9d6f252640 1 --2- 172.21.15.38:0/209647015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d68102e00 0x7f9d681031e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f9d64009900 tx=0x7f9d6402a480 comp rx=0 tx=0).stop 2024-09-16T00:32:14.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.983+0000 7f9d6f252640 1 -- 172.21.15.38:0/209647015 shutdown_connections 2024-09-16T00:32:14.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.983+0000 7f9d6f252640 1 --2- 172.21.15.38:0/209647015 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d6810c1a0 0x7f9d6810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.983+0000 7f9d6f252640 1 --2- 172.21.15.38:0/209647015 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d68103720 0x7f9d6810bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.983+0000 7f9d6f252640 1 --2- 172.21.15.38:0/209647015 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d68102e00 0x7f9d681031e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.983+0000 7f9d6f252640 1 -- 172.21.15.38:0/209647015 >> 172.21.15.38:0/209647015 conn(0x7f9d680fc7a0 msgr2=0x7f9d680febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:14.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.983+0000 7f9d6f252640 1 -- 172.21.15.38:0/209647015 shutdown_connections 2024-09-16T00:32:14.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.983+0000 7f9d6f252640 1 -- 172.21.15.38:0/209647015 wait complete. 2024-09-16T00:32:14.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.984+0000 7f9d6f252640 1 Processor -- start 2024-09-16T00:32:14.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.984+0000 7f9d6f252640 1 -- start start 2024-09-16T00:32:14.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.984+0000 7f9d6f252640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d68102e00 0x7f9d681984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:14.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.984+0000 7f9d6f252640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d68103720 0x7f9d68198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:14.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.984+0000 7f9d6f252640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d6810c1a0 0x7f9d6819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:14.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.984+0000 7f9d6f252640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9d68111f70 con 0x7f9d68103720 2024-09-16T00:32:14.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.984+0000 7f9d6f252640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9d68111df0 con 0x7f9d6810c1a0 2024-09-16T00:32:14.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.984+0000 7f9d6f252640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9d681120f0 con 0x7f9d68102e00 2024-09-16T00:32:14.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.984+0000 7f9d5ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d68103720 0x7f9d68198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:14.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.984+0000 7f9d5ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d68103720 0x7f9d68198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49754/0 (socket says 172.21.15.38:49754) 2024-09-16T00:32:14.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.984+0000 7f9d5ffff640 1 -- 172.21.15.38:0/4182703575 learned_addr learned my addr 172.21.15.38:0/4182703575 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:14.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.984+0000 7f9d6d7ef640 1 --2- 172.21.15.38:0/4182703575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d6810c1a0 0x7f9d6819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:14.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.984+0000 7f9d6cfee640 1 --2- 172.21.15.38:0/4182703575 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d68102e00 0x7f9d681984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:14.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.985+0000 7f9d5ffff640 1 -- 172.21.15.38:0/4182703575 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d68102e00 msgr2=0x7f9d681984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:14.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.985+0000 7f9d5ffff640 1 --2- 172.21.15.38:0/4182703575 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d68102e00 0x7f9d681984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.985+0000 7f9d5ffff640 1 -- 172.21.15.38:0/4182703575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d6810c1a0 msgr2=0x7f9d6819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:14.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.985+0000 7f9d5ffff640 1 --2- 172.21.15.38:0/4182703575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d6810c1a0 0x7f9d6819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.985+0000 7f9d5ffff640 1 -- 172.21.15.38:0/4182703575 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d6819d480 con 0x7f9d68103720 2024-09-16T00:32:14.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.985+0000 7f9d5ffff640 1 --2- 172.21.15.38:0/4182703575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d68103720 0x7f9d68198a10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9d5800c890 tx=0x7f9d5800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:14.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.985+0000 7f9d5dffb640 1 -- 172.21.15.38:0/4182703575 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d58004220 con 0x7f9d68103720 2024-09-16T00:32:14.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.985+0000 7f9d5dffb640 1 -- 172.21.15.38:0/4182703575 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d580043c0 con 0x7f9d68103720 2024-09-16T00:32:14.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.985+0000 7f9d6f252640 1 -- 172.21.15.38:0/4182703575 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d6819d770 con 0x7f9d68103720 2024-09-16T00:32:14.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.985+0000 7f9d5dffb640 1 -- 172.21.15.38:0/4182703575 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d580049e0 con 0x7f9d68103720 2024-09-16T00:32:14.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.985+0000 7f9d6f252640 1 -- 172.21.15.38:0/4182703575 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d681a5050 con 0x7f9d68103720 2024-09-16T00:32:14.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.987+0000 7f9d5dffb640 1 -- 172.21.15.38:0/4182703575 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f9d580079b0 con 0x7f9d68103720 2024-09-16T00:32:14.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.988+0000 7f9d7162b740 1 Processor -- start 2024-09-16T00:32:14.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.988+0000 7f9d7162b740 1 -- start start 2024-09-16T00:32:14.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.988+0000 7f9d5dffb640 1 --2- 172.21.15.38:0/4182703575 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f9d500a3350 0x7f9d500a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:14.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.988+0000 7f9d7162b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3aee6f9d0 0x55f3aee6fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:14.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.988+0000 7f9d7162b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3aee702f0 0x55f3aee7a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:14.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.988+0000 7f9d7162b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aee7abb0 0x55f3aee7cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:14.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.988+0000 7f9d7162b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f3aedd6090 con 0x55f3aee6f9d0 2024-09-16T00:32:14.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.988+0000 7f9d7162b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f3aed7da70 con 0x55f3aee702f0 2024-09-16T00:32:14.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.988+0000 7f9d7162b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f3aedd2050 con 0x55f3aee7abb0 2024-09-16T00:32:14.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.988+0000 7f9d6cfee640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3aee6f9d0 0x55f3aee6fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:14.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.988+0000 7f9d5dffb640 1 -- 172.21.15.38:0/4182703575 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f9d580cfe60 con 0x7f9d68103720 2024-09-16T00:32:14.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.988+0000 7f9d6d7ef640 1 --2- 172.21.15.38:0/4182703575 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f9d500a3350 0x7f9d500a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:14.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.988+0000 7f9d6cfee640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3aee6f9d0 0x55f3aee6fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49766/0 (socket says 172.21.15.38:49766) 2024-09-16T00:32:14.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.988+0000 7f9d6cfee640 1 -- 172.21.15.38:0/571393057 learned_addr learned my addr 172.21.15.38:0/571393057 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:14.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.988+0000 7f9d5ffff640 1 --2- 172.21.15.38:0/571393057 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3aee702f0 0x55f3aee7a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:14.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.988+0000 7f9d6d7ef640 1 --2- 172.21.15.38:0/571393057 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aee7abb0 0x55f3aee7cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:14.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.989+0000 7f9d6cfee640 1 -- 172.21.15.38:0/571393057 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aee7abb0 msgr2=0x55f3aee7cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:14.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.989+0000 7f9d6cfee640 1 --2- 172.21.15.38:0/571393057 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aee7abb0 0x55f3aee7cfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.989+0000 7f9d6cfee640 1 -- 172.21.15.38:0/571393057 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3aee702f0 msgr2=0x55f3aee7a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:14.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.989+0000 7f9d6cfee640 1 --2- 172.21.15.38:0/571393057 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3aee702f0 0x55f3aee7a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.989+0000 7f9d6cfee640 1 -- 172.21.15.38:0/571393057 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3aed8a130 con 0x55f3aee6f9d0 2024-09-16T00:32:14.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.989+0000 7f9d6d7ef640 1 --2- 172.21.15.38:0/4182703575 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f9d500a3350 0x7f9d500a5810 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9d60005fd0 tx=0x7f9d60009f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:14.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.989+0000 7f9d6cfee640 1 --2- 172.21.15.38:0/571393057 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3aee6f9d0 0x55f3aee6fdb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9d64033000 tx=0x7f9d64032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8eb78c9f2e31343 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:14.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.989+0000 7f9d4effd640 1 -- 172.21.15.38:0/571393057 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6403f070 con 0x55f3aee6f9d0 2024-09-16T00:32:14.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.989+0000 7f9d4effd640 1 -- 172.21.15.38:0/571393057 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d640317b0 con 0x55f3aee6f9d0 2024-09-16T00:32:14.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.989+0000 7f9d4effd640 1 -- 172.21.15.38:0/571393057 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d64035070 con 0x55f3aee6f9d0 2024-09-16T00:32:14.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.990+0000 7f9d7162b740 1 -- 172.21.15.38:0/571393057 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3aee6f9d0 msgr2=0x55f3aee6fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:14.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.990+0000 7f9d7162b740 1 --2- 172.21.15.38:0/571393057 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3aee6f9d0 0x55f3aee6fdb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f9d64033000 tx=0x7f9d64032ce0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.990+0000 7f9d7162b740 1 -- 172.21.15.38:0/571393057 shutdown_connections 2024-09-16T00:32:14.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.990+0000 7f9d7162b740 1 --2- 172.21.15.38:0/571393057 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aee7abb0 0x55f3aee7cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.990+0000 7f9d7162b740 1 --2- 172.21.15.38:0/571393057 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3aee702f0 0x55f3aee7a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.990+0000 7f9d7162b740 1 --2- 172.21.15.38:0/571393057 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3aee6f9d0 0x55f3aee6fdb0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.990+0000 7f9d7162b740 1 -- 172.21.15.38:0/571393057 >> 172.21.15.38:0/571393057 conn(0x55f3aee6bf60 msgr2=0x55f3aee6c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:14.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.990+0000 7f9d7162b740 1 -- 172.21.15.38:0/571393057 shutdown_connections 2024-09-16T00:32:14.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.990+0000 7f9d7162b740 1 -- 172.21.15.38:0/571393057 wait complete. 2024-09-16T00:32:14.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.990+0000 7f9d7162b740 1 -- start start 2024-09-16T00:32:14.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.990+0000 7f9d7162b740 1 Processor -- start 2024-09-16T00:32:14.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.991+0000 7f9d7162b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3aee6f9d0 0x55f3aee7a410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:14.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.991+0000 7f9d7162b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aee702f0 0x55f3aee77760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:14.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.991+0000 7f9d7162b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3aee7abb0 0x55f3aee77ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:14.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.991+0000 7f9d7162b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f3aee7d780 con 0x55f3aee7abb0 2024-09-16T00:32:14.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.991+0000 7f9d7162b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f3aed90f20 con 0x55f3aee6f9d0 2024-09-16T00:32:14.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.991+0000 7f9d7162b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f3aee7d900 con 0x55f3aee702f0 2024-09-16T00:32:14.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.991+0000 7f9d7162b740 10 client.0 _ll_register_callbacks cb 0x55f3aed5a030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:14.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.991+0000 7f9d6d7ef640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3aee7abb0 0x55f3aee77ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:14.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.991+0000 7f9d6d7ef640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3aee7abb0 0x55f3aee77ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49778/0 (socket says 172.21.15.38:49778) 2024-09-16T00:32:14.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.991+0000 7f9d6d7ef640 1 -- 172.21.15.38:0/3545034760 learned_addr learned my addr 172.21.15.38:0/3545034760 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:14.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.991+0000 7f9d6cfee640 1 --2- 172.21.15.38:0/3545034760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3aee6f9d0 0x55f3aee7a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:14.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.991+0000 7f9d5ffff640 1 --2- 172.21.15.38:0/3545034760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aee702f0 0x55f3aee77760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:14.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.991+0000 7f9d6d7ef640 1 -- 172.21.15.38:0/3545034760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aee702f0 msgr2=0x55f3aee77760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:14.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.991+0000 7f9d6d7ef640 1 --2- 172.21.15.38:0/3545034760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aee702f0 0x55f3aee77760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.991+0000 7f9d6d7ef640 1 -- 172.21.15.38:0/3545034760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3aee6f9d0 msgr2=0x55f3aee7a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:14.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.991+0000 7f9d6d7ef640 1 --2- 172.21.15.38:0/3545034760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3aee6f9d0 0x55f3aee7a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.991+0000 7f9d6d7ef640 1 -- 172.21.15.38:0/3545034760 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3aeb472b0 con 0x55f3aee7abb0 2024-09-16T00:32:14.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.991+0000 7f9d6d7ef640 1 --2- 172.21.15.38:0/3545034760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3aee7abb0 0x55f3aee77ca0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9d60012df0 tx=0x7f9d60004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abcbd1b9f53f8b2d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:14.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.991+0000 7f9d47fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:32:14.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.992+0000 7f9d47fff640 1 -- 172.21.15.38:0/3545034760 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d60020670 con 0x55f3aee7abb0 2024-09-16T00:32:14.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.992+0000 7f9d47fff640 1 -- 172.21.15.38:0/3545034760 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f9d60020810 con 0x55f3aee7abb0 2024-09-16T00:32:14.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.992+0000 7f9d7162b740 1 -- 172.21.15.38:0/3545034760 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d640317b0 con 0x55f3aee7abb0 2024-09-16T00:32:14.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.992+0000 7f9d47fff640 1 -- 172.21.15.38:0/3545034760 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9d60031050 con 0x55f3aee7abb0 2024-09-16T00:32:14.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.992+0000 7f9d7162b740 10 client.15099 fetch_fsmap learned FSMap version 25 2024-09-16T00:32:14.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.992+0000 7f9d7162b740 1 -- 172.21.15.38:0/3545034760 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f3aee89070 con 0x55f3aee7abb0 2024-09-16T00:32:14.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.992+0000 7f9d47fff640 1 -- 172.21.15.38:0/3545034760 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d640317b0 con 0x55f3aee7abb0 2024-09-16T00:32:14.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.992+0000 7f9d47fff640 1 -- 172.21.15.38:0/3545034760 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f9d6001e040 con 0x55f3aee7abb0 2024-09-16T00:32:14.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.992+0000 7f9d7162b740 10 client.15099 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:32:14.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.992+0000 7f9d7162b740 10 client.15099 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T00:32:14.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.992+0000 7f9d7162b740 20 client.15099 populate_metadata read hostname 'smithi038' 2024-09-16T00:32:14.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.992+0000 7f9d7162b740 1 --2- 172.21.15.38:0/3545034760 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55f3aee8b6f0 0x55f3aee8cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:14.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.992+0000 7f9d7162b740 4 client.15099 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:32:14.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.992+0000 7f9d7162b740 1 -- 172.21.15.38:0/3545034760 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3aede3600 con 0x55f3aee8b6f0 2024-09-16T00:32:14.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.993+0000 7f9d6cfee640 1 --2- 172.21.15.38:0/3545034760 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55f3aee8b6f0 0x55f3aee8cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:14.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.993+0000 7f9d47fff640 10 client.15099 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:32:14.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.993+0000 7f9d6cfee640 1 --2- 172.21.15.38:0/3545034760 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55f3aee8b6f0 0x55f3aee8cb60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82b03340f12b23bd server_cookie=23e104d88486d82f in_seq=0 out_seq=0 2024-09-16T00:32:14.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.994+0000 7f9d47fff640 1 -- 172.21.15.38:0/3545034760 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55f3aede3600 con 0x55f3aee8b6f0 2024-09-16T00:32:14.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.994+0000 7f9d47fff640 10 client.15099 handle_command_reply: tid=0 2024-09-16T00:32:14.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.994+0000 7f9d7162b740 1 client.15099 shutdown 2024-09-16T00:32:14.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.994+0000 7f9d7162b740 1 -- 172.21.15.38:0/3545034760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3aee7abb0 msgr2=0x55f3aee77ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:14.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.994+0000 7f9d7162b740 1 --2- 172.21.15.38:0/3545034760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3aee7abb0 0x55f3aee77ca0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f9d60012df0 tx=0x7f9d60004ab0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.994+0000 7f9d7162b740 1 -- 172.21.15.38:0/3545034760 shutdown_connections 2024-09-16T00:32:14.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.994+0000 7f9d7162b740 1 --2- 172.21.15.38:0/3545034760 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55f3aee8b6f0 0x55f3aee8cb60 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.994+0000 7f9d7162b740 1 --2- 172.21.15.38:0/3545034760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3aee7abb0 0x55f3aee77ca0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.994+0000 7f9d7162b740 1 --2- 172.21.15.38:0/3545034760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aee702f0 0x55f3aee77760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.994+0000 7f9d7162b740 1 --2- 172.21.15.38:0/3545034760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3aee6f9d0 0x55f3aee7a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.994+0000 7f9d47fff640 0 client.15099 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:32:14.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.994+0000 7f9d7162b740 1 -- 172.21.15.38:0/3545034760 >> 172.21.15.38:0/3545034760 conn(0x55f3aee6bf60 msgr2=0x55f3aee71c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:14.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.994+0000 7f9d7162b740 1 -- 172.21.15.38:0/3545034760 shutdown_connections 2024-09-16T00:32:14.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.994+0000 7f9d7162b740 1 -- 172.21.15.38:0/3545034760 wait complete. 2024-09-16T00:32:14.018 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:13.994+0000 7f9d7162b740 20 client.15099 trim_cache size 0 max 16384 2024-09-16T00:32:14.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.012+0000 7f9d7162b740 1 Processor -- start 2024-09-16T00:32:14.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.012+0000 7f9d7162b740 1 -- start start 2024-09-16T00:32:14.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.012+0000 7f9d7162b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aee6f9d0 0x55f3aee76ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:14.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.012+0000 7f9d7162b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3aee702f0 0x55f3aee76030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:14.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.012+0000 7f9d7162b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3aee7abb0 0x55f3aee76570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:14.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.012+0000 7f9d7162b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f3aee8aea0 con 0x55f3aee702f0 2024-09-16T00:32:14.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.012+0000 7f9d7162b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f3aee8b570 con 0x55f3aee7abb0 2024-09-16T00:32:14.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.012+0000 7f9d7162b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f3aee8bc40 con 0x55f3aee6f9d0 2024-09-16T00:32:14.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.012+0000 7f9d5ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3aee702f0 0x55f3aee76030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:14.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.012+0000 7f9d5ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3aee702f0 0x55f3aee76030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49790/0 (socket says 172.21.15.38:49790) 2024-09-16T00:32:14.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.012+0000 7f9d5ffff640 1 -- 172.21.15.38:0/3457618517 learned_addr learned my addr 172.21.15.38:0/3457618517 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:14.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.013+0000 7f9d6d7ef640 1 --2- 172.21.15.38:0/3457618517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3aee7abb0 0x55f3aee76570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:14.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.013+0000 7f9d6cfee640 1 --2- 172.21.15.38:0/3457618517 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aee6f9d0 0x55f3aee76ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:14.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.013+0000 7f9d5ffff640 1 -- 172.21.15.38:0/3457618517 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aee6f9d0 msgr2=0x55f3aee76ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:14.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.013+0000 7f9d5ffff640 1 --2- 172.21.15.38:0/3457618517 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aee6f9d0 0x55f3aee76ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.013+0000 7f9d5ffff640 1 -- 172.21.15.38:0/3457618517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3aee7abb0 msgr2=0x55f3aee76570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:14.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.013+0000 7f9d5ffff640 1 --2- 172.21.15.38:0/3457618517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3aee7abb0 0x55f3aee76570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.013+0000 7f9d5ffff640 1 -- 172.21.15.38:0/3457618517 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3aee7a780 con 0x55f3aee702f0 2024-09-16T00:32:14.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.013+0000 7f9d6d7ef640 1 --2- 172.21.15.38:0/3457618517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3aee7abb0 0x55f3aee76570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:14.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.013+0000 7f9d5ffff640 1 --2- 172.21.15.38:0/3457618517 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3aee702f0 0x55f3aee76030 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9d58007590 tx=0x7f9d5809b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0e59ec382a0a13e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:14.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.013+0000 7f9d4e7fc640 1 -- 172.21.15.38:0/3457618517 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d5801d040 con 0x55f3aee702f0 2024-09-16T00:32:14.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.013+0000 7f9d4e7fc640 1 -- 172.21.15.38:0/3457618517 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d5809bf80 con 0x55f3aee702f0 2024-09-16T00:32:14.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.013+0000 7f9d4e7fc640 1 -- 172.21.15.38:0/3457618517 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d580b5540 con 0x55f3aee702f0 2024-09-16T00:32:14.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.014+0000 7f9d7162b740 1 -- 172.21.15.38:0/3457618517 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3aee702f0 msgr2=0x55f3aee76030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:14.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.014+0000 7f9d7162b740 1 --2- 172.21.15.38:0/3457618517 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3aee702f0 0x55f3aee76030 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f9d58007590 tx=0x7f9d5809b9a0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.014+0000 7f9d7162b740 1 -- 172.21.15.38:0/3457618517 shutdown_connections 2024-09-16T00:32:14.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.014+0000 7f9d7162b740 1 --2- 172.21.15.38:0/3457618517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3aee7abb0 0x55f3aee76570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.014+0000 7f9d7162b740 1 --2- 172.21.15.38:0/3457618517 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3aee702f0 0x55f3aee76030 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.014+0000 7f9d7162b740 1 --2- 172.21.15.38:0/3457618517 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aee6f9d0 0x55f3aee76ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.014+0000 7f9d7162b740 1 -- 172.21.15.38:0/3457618517 >> 172.21.15.38:0/3457618517 conn(0x55f3aee6bf60 msgr2=0x55f3aee6c340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:14.025 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.014+0000 7f9d7162b740 1 -- 172.21.15.38:0/3457618517 shutdown_connections 2024-09-16T00:32:14.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.014+0000 7f9d7162b740 1 -- 172.21.15.38:0/3457618517 wait complete. 2024-09-16T00:32:14.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.014+0000 7f9d7162b740 1 -- start start 2024-09-16T00:32:14.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.014+0000 7f9d7162b740 1 Processor -- start 2024-09-16T00:32:14.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.015+0000 7f9d7162b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3aee6f9d0 0x55f3aee95330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:14.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.015+0000 7f9d7162b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aee702f0 0x55f3aee90660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:14.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.015+0000 7f9d7162b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3aee7abb0 0x55f3aee90ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:14.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.015+0000 7f9d7162b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55f3aee8c9e0 con 0x55f3aee6f9d0 2024-09-16T00:32:14.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.015+0000 7f9d7162b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55f3aee8c310 con 0x55f3aee7abb0 2024-09-16T00:32:14.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.015+0000 7f9d7162b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55f3aee8d0b0 con 0x55f3aee702f0 2024-09-16T00:32:14.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.015+0000 7f9d7162b740 10 client.0 _ll_register_callbacks cb 0x7f9d60012510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:14.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.015+0000 7f9d6cfee640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3aee6f9d0 0x55f3aee95330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:14.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.015+0000 7f9d6cfee640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3aee6f9d0 0x55f3aee95330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49792/0 (socket says 172.21.15.38:49792) 2024-09-16T00:32:14.026 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.015+0000 7f9d6cfee640 1 -- 172.21.15.38:0/2644020834 learned_addr learned my addr 172.21.15.38:0/2644020834 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:14.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.015+0000 7f9d5ffff640 1 --2- 172.21.15.38:0/2644020834 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aee702f0 0x55f3aee90660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:14.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.015+0000 7f9d6d7ef640 1 --2- 172.21.15.38:0/2644020834 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3aee7abb0 0x55f3aee90ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:14.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.015+0000 7f9d6cfee640 1 -- 172.21.15.38:0/2644020834 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aee702f0 msgr2=0x55f3aee90660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:14.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.015+0000 7f9d6cfee640 1 --2- 172.21.15.38:0/2644020834 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aee702f0 0x55f3aee90660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.015+0000 7f9d6cfee640 1 -- 172.21.15.38:0/2644020834 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3aee7abb0 msgr2=0x55f3aee90ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:14.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.015+0000 7f9d6cfee640 1 --2- 172.21.15.38:0/2644020834 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3aee7abb0 0x55f3aee90ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.015+0000 7f9d6cfee640 1 -- 172.21.15.38:0/2644020834 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3aee91160 con 0x55f3aee6f9d0 2024-09-16T00:32:14.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.015+0000 7f9d6d7ef640 1 --2- 172.21.15.38:0/2644020834 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3aee7abb0 0x55f3aee90ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:14.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.015+0000 7f9d6cfee640 1 --2- 172.21.15.38:0/2644020834 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3aee6f9d0 0x55f3aee95330 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9d64031550 tx=0x7f9d64005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92c7d54ab0132a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:14.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.015+0000 7f9d47fff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:32:14.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.016+0000 7f9d47fff640 1 -- 172.21.15.38:0/2644020834 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d6403f040 con 0x55f3aee6f9d0 2024-09-16T00:32:14.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.016+0000 7f9d47fff640 1 -- 172.21.15.38:0/2644020834 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f9d640055e0 con 0x55f3aee6f9d0 2024-09-16T00:32:14.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.016+0000 7f9d7162b740 1 -- 172.21.15.38:0/2644020834 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d5809bf80 con 0x55f3aee6f9d0 2024-09-16T00:32:14.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.016+0000 7f9d47fff640 1 -- 172.21.15.38:0/2644020834 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f9d64035040 con 0x55f3aee6f9d0 2024-09-16T00:32:14.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.016+0000 7f9d7162b740 10 client.15105 fetch_fsmap learned FSMap version 25 2024-09-16T00:32:14.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.016+0000 7f9d7162b740 1 -- 172.21.15.38:0/2644020834 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55f3aeef5bd0 con 0x55f3aee6f9d0 2024-09-16T00:32:14.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.016+0000 7f9d47fff640 1 -- 172.21.15.38:0/2644020834 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9d5809bf80 con 0x55f3aee6f9d0 2024-09-16T00:32:14.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.016+0000 7f9d47fff640 1 -- 172.21.15.38:0/2644020834 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f9d64003bb0 con 0x55f3aee6f9d0 2024-09-16T00:32:14.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.016+0000 7f9d7162b740 10 client.15105 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:32:14.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.016+0000 7f9d7162b740 10 client.15105 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T00:32:14.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.016+0000 7f9d7162b740 20 client.15105 populate_metadata read hostname 'smithi038' 2024-09-16T00:32:14.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.016+0000 7f9d7162b740 1 --2- 172.21.15.38:0/2644020834 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55f3aeefa360 0x55f3aeef9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:14.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.016+0000 7f9d7162b740 4 client.15105 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:32:14.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.016+0000 7f9d7162b740 1 -- 172.21.15.38:0/2644020834 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "status"}) -- 0x55f3aedddb10 con 0x55f3aeefa360 2024-09-16T00:32:14.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.016+0000 7f9d5ffff640 1 --2- 172.21.15.38:0/2644020834 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55f3aeefa360 0x55f3aeef9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:14.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.017+0000 7f9d5ffff640 1 --2- 172.21.15.38:0/2644020834 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55f3aeefa360 0x55f3aeef9070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a81c75ab288a527 server_cookie=fb95fe217a6a6e62 in_seq=0 out_seq=0 2024-09-16T00:32:14.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.017+0000 7f9d47fff640 10 client.15105 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:32:14.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.017+0000 7f9d47fff640 1 -- 172.21.15.38:0/2644020834 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+321 (crc 0 0 0) 0x55f3aedddb10 con 0x55f3aeefa360 2024-09-16T00:32:14.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.017+0000 7f9d47fff640 10 client.15105 handle_command_reply: tid=0 2024-09-16T00:32:14.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.017+0000 7f9d7162b740 1 client.15105 shutdown 2024-09-16T00:32:14.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.017+0000 7f9d7162b740 1 -- 172.21.15.38:0/2644020834 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3aee6f9d0 msgr2=0x55f3aee95330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:14.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.017+0000 7f9d7162b740 1 --2- 172.21.15.38:0/2644020834 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3aee6f9d0 0x55f3aee95330 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9d64031550 tx=0x7f9d64005a60 comp rx=0 tx=0).stop 2024-09-16T00:32:14.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.017+0000 7f9d7162b740 1 -- 172.21.15.38:0/2644020834 shutdown_connections 2024-09-16T00:32:14.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.017+0000 7f9d7162b740 1 --2- 172.21.15.38:0/2644020834 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55f3aee7abb0 0x55f3aee90ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.017+0000 7f9d7162b740 1 --2- 172.21.15.38:0/2644020834 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55f3aee702f0 0x55f3aee90660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.017+0000 7f9d7162b740 1 --2- 172.21.15.38:0/2644020834 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55f3aeefa360 0x55f3aeef9070 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.017+0000 7f9d7162b740 1 --2- 172.21.15.38:0/2644020834 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55f3aee6f9d0 0x55f3aee95330 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.017+0000 7f9d7162b740 1 -- 172.21.15.38:0/2644020834 >> 172.21.15.38:0/2644020834 conn(0x55f3aee6bf60 msgr2=0x55f3aee75ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:14.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.017+0000 7f9d47fff640 0 client.15105 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:32:14.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.018+0000 7f9d7162b740 1 -- 172.21.15.38:0/2644020834 shutdown_connections 2024-09-16T00:32:14.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.018+0000 7f9d7162b740 1 -- 172.21.15.38:0/2644020834 wait complete. 2024-09-16T00:32:14.030 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.018+0000 7f9d7162b740 20 client.15105 trim_cache size 0 max 16384 2024-09-16T00:32:14.030 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-16T00:32:14.030 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "983b2072-73c1-11ef-bceb-c7b262605968", 2024-09-16T00:32:14.030 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-09-16T00:32:14.030 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24494, 2024-09-16T00:32:14.030 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-16T00:32:14.031 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-16T00:32:14.031 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-16T00:32:14.031 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 106.95513586, 2024-09-16T00:32:14.031 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 25, 2024-09-16T00:32:14.031 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 90, 2024-09-16T00:32:14.031 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 88, 2024-09-16T00:32:14.031 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 116.873685061 2024-09-16T00:32:14.031 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-16T00:32:14.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.018+0000 7f9d6f252640 1 -- 172.21.15.38:0/4182703575 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f9d500a3350 msgr2=0x7f9d500a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:14.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.018+0000 7f9d6f252640 1 --2- 172.21.15.38:0/4182703575 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f9d500a3350 0x7f9d500a5810 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9d60005fd0 tx=0x7f9d60009f90 comp rx=0 tx=0).stop 2024-09-16T00:32:14.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.018+0000 7f9d6f252640 1 -- 172.21.15.38:0/4182703575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d68103720 msgr2=0x7f9d68198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:14.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.018+0000 7f9d6f252640 1 --2- 172.21.15.38:0/4182703575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d68103720 0x7f9d68198a10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f9d5800c890 tx=0x7f9d5800cd50 comp rx=0 tx=0).stop 2024-09-16T00:32:14.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.018+0000 7f9d6f252640 1 -- 172.21.15.38:0/4182703575 shutdown_connections 2024-09-16T00:32:14.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.018+0000 7f9d6f252640 1 --2- 172.21.15.38:0/4182703575 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f9d500a3350 0x7f9d500a5810 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.018+0000 7f9d6f252640 1 --2- 172.21.15.38:0/4182703575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d6810c1a0 0x7f9d6819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.018+0000 7f9d6f252640 1 --2- 172.21.15.38:0/4182703575 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9d68103720 0x7f9d68198a10 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.018+0000 7f9d6f252640 1 --2- 172.21.15.38:0/4182703575 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9d68102e00 0x7f9d681984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:14.032 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.018+0000 7f9d6f252640 1 -- 172.21.15.38:0/4182703575 >> 172.21.15.38:0/4182703575 conn(0x7f9d680fc7a0 msgr2=0x7f9d680fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:14.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.018+0000 7f9d6f252640 1 -- 172.21.15.38:0/4182703575 shutdown_connections 2024-09-16T00:32:14.033 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:14.018+0000 7f9d6f252640 1 -- 172.21.15.38:0/4182703575 wait complete. 2024-09-16T00:32:14.867 INFO:tasks.cephadm:Setup complete, yielding 2024-09-16T00:32:14.867 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T00:32:14.878 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi038.front.sepia.ceph.com 2024-09-16T00:32:14.879 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- bash -c 'ceph orch status' 2024-09-16T00:32:15.143 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:15.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:15 smithi073 ceph-mon[35254]: pgmap v346: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-16T00:32:15.551 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:15 smithi038 ceph-mon[30945]: pgmap v346: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-16T00:32:15.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.634+0000 7f1431c9b640 1 -- 172.21.15.38:0/203631189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f142c102e00 msgr2=0x7f142c103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:15.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.634+0000 7f1431c9b640 1 --2- 172.21.15.38:0/203631189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f142c102e00 0x7f142c103280 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f1420009900 tx=0x7f142002a410 comp rx=0 tx=0).stop 2024-09-16T00:32:15.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.634+0000 7f1431c9b640 1 -- 172.21.15.38:0/203631189 shutdown_connections 2024-09-16T00:32:15.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.634+0000 7f1431c9b640 1 --2- 172.21.15.38:0/203631189 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f142c1037c0 0x7f142c10a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.634+0000 7f1431c9b640 1 --2- 172.21.15.38:0/203631189 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f142c102e00 0x7f142c103280 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.634+0000 7f1431c9b640 1 --2- 172.21.15.38:0/203631189 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f142c101c00 0x7f142c102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.634+0000 7f1431c9b640 1 -- 172.21.15.38:0/203631189 >> 172.21.15.38:0/203631189 conn(0x7f142c0fd390 msgr2=0x7f142c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:15.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.634+0000 7f1431c9b640 1 -- 172.21.15.38:0/203631189 shutdown_connections 2024-09-16T00:32:15.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.634+0000 7f1431c9b640 1 -- 172.21.15.38:0/203631189 wait complete. 2024-09-16T00:32:15.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.635+0000 7f1431c9b640 1 Processor -- start 2024-09-16T00:32:15.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.635+0000 7f1431c9b640 1 -- start start 2024-09-16T00:32:15.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.635+0000 7f1431c9b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f142c101c00 0x7f142c19a090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:15.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.635+0000 7f1431c9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f142c102e00 0x7f142c19a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:15.661 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.635+0000 7f1431c9b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f142c1037c0 0x7f142c1a1650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:15.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.635+0000 7f1431c9b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f142c10d880 con 0x7f142c102e00 2024-09-16T00:32:15.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.635+0000 7f1431c9b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f142c10d700 con 0x7f142c1037c0 2024-09-16T00:32:15.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.635+0000 7f1431c9b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f142c10da00 con 0x7f142c101c00 2024-09-16T00:32:15.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.635+0000 7f142affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f142c102e00 0x7f142c19a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:15.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.636+0000 7f142affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f142c102e00 0x7f142c19a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:49808/0 (socket says 172.21.15.38:49808) 2024-09-16T00:32:15.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.636+0000 7f142affd640 1 -- 172.21.15.38:0/471544816 learned_addr learned my addr 172.21.15.38:0/471544816 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:15.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.636+0000 7f142b7fe640 1 --2- 172.21.15.38:0/471544816 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f142c101c00 0x7f142c19a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:15.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.636+0000 7f142bfff640 1 --2- 172.21.15.38:0/471544816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f142c1037c0 0x7f142c1a1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:15.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.636+0000 7f142affd640 1 -- 172.21.15.38:0/471544816 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f142c101c00 msgr2=0x7f142c19a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:15.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.636+0000 7f142affd640 1 --2- 172.21.15.38:0/471544816 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f142c101c00 0x7f142c19a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.636+0000 7f142affd640 1 -- 172.21.15.38:0/471544816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f142c1037c0 msgr2=0x7f142c1a1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:15.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.636+0000 7f142affd640 1 --2- 172.21.15.38:0/471544816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f142c1037c0 0x7f142c1a1650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.636+0000 7f142affd640 1 -- 172.21.15.38:0/471544816 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f142c1a1cc0 con 0x7f142c102e00 2024-09-16T00:32:15.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.636+0000 7f142affd640 1 --2- 172.21.15.38:0/471544816 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f142c102e00 0x7f142c19a5d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1420009230 tx=0x7f1420031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:15.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.636+0000 7f1428ff9640 1 -- 172.21.15.38:0/471544816 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142003f030 con 0x7f142c102e00 2024-09-16T00:32:15.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.636+0000 7f1431c9b640 1 -- 172.21.15.38:0/471544816 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f142c1a1f50 con 0x7f142c102e00 2024-09-16T00:32:15.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.636+0000 7f1428ff9640 1 -- 172.21.15.38:0/471544816 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1420003ad0 con 0x7f142c102e00 2024-09-16T00:32:15.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.636+0000 7f1428ff9640 1 -- 172.21.15.38:0/471544816 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1420035070 con 0x7f142c102e00 2024-09-16T00:32:15.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.636+0000 7f1431c9b640 1 -- 172.21.15.38:0/471544816 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f142c1a23b0 con 0x7f142c102e00 2024-09-16T00:32:15.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.637+0000 7f1431c9b640 1 -- 172.21.15.38:0/471544816 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f142c1a25e0 con 0x7f142c102e00 2024-09-16T00:32:15.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.638+0000 7f1428ff9640 1 -- 172.21.15.38:0/471544816 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f1420042030 con 0x7f142c102e00 2024-09-16T00:32:15.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.639+0000 7f1428ff9640 1 --2- 172.21.15.38:0/471544816 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f14140a3300 0x7f14140a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:15.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.639+0000 7f1428ff9640 1 -- 172.21.15.38:0/471544816 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f14200f07a0 con 0x7f142c102e00 2024-09-16T00:32:15.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.639+0000 7f142b7fe640 1 --2- 172.21.15.38:0/471544816 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f14140a3300 0x7f14140a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:15.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.640+0000 7f142b7fe640 1 --2- 172.21.15.38:0/471544816 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f14140a3300 0x7f14140a57c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f142c102c60 tx=0x7f141c005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:15.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.643+0000 7f1428ff9640 1 -- 172.21.15.38:0/471544816 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f14200bd020 con 0x7f142c102e00 2024-09-16T00:32:15.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:15 smithi122 ceph-mon[35412]: pgmap v346: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-16T00:32:15.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.834+0000 7f1431c9b640 1 -- 172.21.15.38:0/471544816 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f142c1a27b0 con 0x7f14140a3300 2024-09-16T00:32:15.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.835+0000 7f1428ff9640 1 -- 172.21.15.38:0/471544816 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f142c1a27b0 con 0x7f14140a3300 2024-09-16T00:32:15.836 INFO:teuthology.orchestra.run.smithi038.stdout:Backend: cephadm 2024-09-16T00:32:15.836 INFO:teuthology.orchestra.run.smithi038.stdout:Available: Yes 2024-09-16T00:32:15.836 INFO:teuthology.orchestra.run.smithi038.stdout:Paused: No 2024-09-16T00:32:15.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.838+0000 7f1431c9b640 1 -- 172.21.15.38:0/471544816 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f14140a3300 msgr2=0x7f14140a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:15.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.838+0000 7f1431c9b640 1 --2- 172.21.15.38:0/471544816 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f14140a3300 0x7f14140a57c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f142c102c60 tx=0x7f141c005f90 comp rx=0 tx=0).stop 2024-09-16T00:32:15.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.838+0000 7f1431c9b640 1 -- 172.21.15.38:0/471544816 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f142c102e00 msgr2=0x7f142c19a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:15.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.838+0000 7f1431c9b640 1 --2- 172.21.15.38:0/471544816 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f142c102e00 0x7f142c19a5d0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f1420009230 tx=0x7f1420031b60 comp rx=0 tx=0).stop 2024-09-16T00:32:15.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.838+0000 7f1431c9b640 1 -- 172.21.15.38:0/471544816 shutdown_connections 2024-09-16T00:32:15.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.838+0000 7f1431c9b640 1 --2- 172.21.15.38:0/471544816 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f14140a3300 0x7f14140a57c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.838+0000 7f1431c9b640 1 --2- 172.21.15.38:0/471544816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f142c1037c0 0x7f142c1a1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.838+0000 7f1431c9b640 1 --2- 172.21.15.38:0/471544816 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f142c102e00 0x7f142c19a5d0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.838+0000 7f1431c9b640 1 --2- 172.21.15.38:0/471544816 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f142c101c00 0x7f142c19a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:15.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.838+0000 7f1431c9b640 1 -- 172.21.15.38:0/471544816 >> 172.21.15.38:0/471544816 conn(0x7f142c0fd390 msgr2=0x7f142c074420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:15.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.838+0000 7f1431c9b640 1 -- 172.21.15.38:0/471544816 shutdown_connections 2024-09-16T00:32:15.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:15.838+0000 7f1431c9b640 1 -- 172.21.15.38:0/471544816 wait complete. 2024-09-16T00:32:16.669 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ps' 2024-09-16T00:32:16.931 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:17.338 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:17 smithi038 ceph-mon[30945]: pgmap v347: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:32:17.338 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:17 smithi038 ceph-mon[30945]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:17.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.420+0000 7fc747049640 1 -- 172.21.15.38:0/554604761 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc740101c00 msgr2=0x7fc740102000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:17.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.420+0000 7fc747049640 1 --2- 172.21.15.38:0/554604761 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc740101c00 0x7fc740102000 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc734009900 tx=0x7fc73402a410 comp rx=0 tx=0).stop 2024-09-16T00:32:17.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.420+0000 7fc747049640 1 -- 172.21.15.38:0/554604761 shutdown_connections 2024-09-16T00:32:17.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.420+0000 7fc747049640 1 --2- 172.21.15.38:0/554604761 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc7401037c0 0x7fc74010a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:17.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.420+0000 7fc747049640 1 --2- 172.21.15.38:0/554604761 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc740102e00 0x7fc740103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:17.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.420+0000 7fc747049640 1 --2- 172.21.15.38:0/554604761 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc740101c00 0x7fc740102000 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:17.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.420+0000 7fc747049640 1 -- 172.21.15.38:0/554604761 >> 172.21.15.38:0/554604761 conn(0x7fc7400fd390 msgr2=0x7fc7400ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:17.447 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.421+0000 7fc747049640 1 -- 172.21.15.38:0/554604761 shutdown_connections 2024-09-16T00:32:17.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.421+0000 7fc747049640 1 -- 172.21.15.38:0/554604761 wait complete. 2024-09-16T00:32:17.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.421+0000 7fc747049640 1 Processor -- start 2024-09-16T00:32:17.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.421+0000 7fc747049640 1 -- start start 2024-09-16T00:32:17.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.421+0000 7fc747049640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc740101c00 0x7fc740197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:17.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.421+0000 7fc747049640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc740102e00 0x7fc740198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:17.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.421+0000 7fc747049640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7401037c0 0x7fc74019f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:17.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.421+0000 7fc747049640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fc74010d890 con 0x7fc7401037c0 2024-09-16T00:32:17.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.421+0000 7fc747049640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fc74010d710 con 0x7fc740101c00 2024-09-16T00:32:17.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.421+0000 7fc747049640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fc74010da10 con 0x7fc740102e00 2024-09-16T00:32:17.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.422+0000 7fc7455e6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7401037c0 0x7fc74019f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:17.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.422+0000 7fc7455e6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7401037c0 0x7fc74019f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53390/0 (socket says 172.21.15.38:53390) 2024-09-16T00:32:17.448 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.422+0000 7fc7455e6640 1 -- 172.21.15.38:0/3108766661 learned_addr learned my addr 172.21.15.38:0/3108766661 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:17.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.422+0000 7fc744de5640 1 --2- 172.21.15.38:0/3108766661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc740101c00 0x7fc740197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:17.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.422+0000 7fc73ffff640 1 --2- 172.21.15.38:0/3108766661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc740102e00 0x7fc740198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:17.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.422+0000 7fc7455e6640 1 -- 172.21.15.38:0/3108766661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc740102e00 msgr2=0x7fc740198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:17.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.422+0000 7fc7455e6640 1 --2- 172.21.15.38:0/3108766661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc740102e00 0x7fc740198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:17.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.422+0000 7fc7455e6640 1 -- 172.21.15.38:0/3108766661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc740101c00 msgr2=0x7fc740197ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:17.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.422+0000 7fc7455e6640 1 --2- 172.21.15.38:0/3108766661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc740101c00 0x7fc740197ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:17.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.422+0000 7fc7455e6640 1 -- 172.21.15.38:0/3108766661 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc74019fb00 con 0x7fc7401037c0 2024-09-16T00:32:17.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.422+0000 7fc7455e6640 1 --2- 172.21.15.38:0/3108766661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7401037c0 0x7fc74019f490 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc73800eaa0 tx=0x7fc73800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:17.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.422+0000 7fc73dffb640 1 -- 172.21.15.38:0/3108766661 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc73800cdb0 con 0x7fc7401037c0 2024-09-16T00:32:17.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.422+0000 7fc73dffb640 1 -- 172.21.15.38:0/3108766661 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fc738005020 con 0x7fc7401037c0 2024-09-16T00:32:17.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.422+0000 7fc747049640 1 -- 172.21.15.38:0/3108766661 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc74019fdf0 con 0x7fc7401037c0 2024-09-16T00:32:17.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.422+0000 7fc73dffb640 1 -- 172.21.15.38:0/3108766661 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7380116b0 con 0x7fc7401037c0 2024-09-16T00:32:17.449 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.423+0000 7fc747049640 1 -- 172.21.15.38:0/3108766661 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7401a02a0 con 0x7fc7401037c0 2024-09-16T00:32:17.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.424+0000 7fc747049640 1 -- 172.21.15.38:0/3108766661 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc740108fa0 con 0x7fc7401037c0 2024-09-16T00:32:17.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.425+0000 7fc73dffb640 1 -- 172.21.15.38:0/3108766661 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7fc738005560 con 0x7fc7401037c0 2024-09-16T00:32:17.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.425+0000 7fc73dffb640 1 --2- 172.21.15.38:0/3108766661 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fc7300a3300 0x7fc7300a57c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:17.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.426+0000 7fc744de5640 1 --2- 172.21.15.38:0/3108766661 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fc7300a3300 0x7fc7300a57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:17.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.426+0000 7fc73dffb640 1 -- 172.21.15.38:0/3108766661 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7fc7380cff90 con 0x7fc7401037c0 2024-09-16T00:32:17.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.426+0000 7fc744de5640 1 --2- 172.21.15.38:0/3108766661 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fc7300a3300 0x7fc7300a57c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc734005ab0 tx=0x7fc734000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:17.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.430+0000 7fc73dffb640 1 -- 172.21.15.38:0/3108766661 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fc738016030 con 0x7fc7401037c0 2024-09-16T00:32:17.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:17 smithi073 ceph-mon[35254]: pgmap v347: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:32:17.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:17 smithi073 ceph-mon[35254]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:17.617 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.616+0000 7fc747049640 1 -- 172.21.15.38:0/3108766661 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fc740062630 con 0x7fc7300a3300 2024-09-16T00:32:17.629 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.629+0000 7fc73dffb640 1 -- 172.21.15.38:0/3108766661 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4061 (secure 0 0 0) 0x7fc740062630 con 0x7fc7300a3300 2024-09-16T00:32:17.630 INFO:teuthology.orchestra.run.smithi038.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-16T00:32:17.630 INFO:teuthology.orchestra.run.smithi038.stdout:mds.a smithi038 running (2m) 51s ago 2m 14.4M - 19.1.1-326-g8921c8ef f1455bd0219c 12df256a6ebe 2024-09-16T00:32:17.630 INFO:teuthology.orchestra.run.smithi038.stdout:mds.b smithi073 running (2m) 53s ago 2m 29.0M - 19.1.1-326-g8921c8ef f1455bd0219c 47c89ec1eb8f 2024-09-16T00:32:17.630 INFO:teuthology.orchestra.run.smithi038.stdout:mds.c smithi122 running (2m) 53s ago 2m 14.8M - 19.1.1-326-g8921c8ef f1455bd0219c 6b38590c3049 2024-09-16T00:32:17.630 INFO:teuthology.orchestra.run.smithi038.stdout:mds.d smithi038 running (2m) 51s ago 2m 14.4M - 19.1.1-326-g8921c8ef f1455bd0219c 250b747ee394 2024-09-16T00:32:17.630 INFO:teuthology.orchestra.run.smithi038.stdout:mds.e smithi073 running (2m) 53s ago 2m 16.4M - 19.1.1-326-g8921c8ef f1455bd0219c 2355052d8723 2024-09-16T00:32:17.630 INFO:teuthology.orchestra.run.smithi038.stdout:mds.f smithi122 running (2m) 53s ago 2m 16.0M - 19.1.1-326-g8921c8ef f1455bd0219c f3f7a5fa1fd3 2024-09-16T00:32:17.630 INFO:teuthology.orchestra.run.smithi038.stdout:mds.g smithi038 running (2m) 51s ago 2m 13.6M - 19.1.1-326-g8921c8ef f1455bd0219c f715123d24f8 2024-09-16T00:32:17.630 INFO:teuthology.orchestra.run.smithi038.stdout:mds.h smithi073 running (2m) 53s ago 2m 19.2M - 19.1.1-326-g8921c8ef f1455bd0219c 90fff75c7a54 2024-09-16T00:32:17.630 INFO:teuthology.orchestra.run.smithi038.stdout:mds.i smithi122 running (2m) 53s ago 2m 16.8M - 19.1.1-326-g8921c8ef f1455bd0219c 0fa3ecb62b42 2024-09-16T00:32:17.630 INFO:teuthology.orchestra.run.smithi038.stdout:mds.j smithi038 running (2m) 51s ago 2m 14.1M - 19.1.1-326-g8921c8ef f1455bd0219c 7b4923415ebd 2024-09-16T00:32:17.631 INFO:teuthology.orchestra.run.smithi038.stdout:mds.k smithi073 running (2m) 53s ago 2m 17.5M - 19.1.1-326-g8921c8ef f1455bd0219c 9d7f1d729e1f 2024-09-16T00:32:17.631 INFO:teuthology.orchestra.run.smithi038.stdout:mds.l smithi122 running (2m) 53s ago 2m 14.4M - 19.1.1-326-g8921c8ef f1455bd0219c 4a9bfdbf120d 2024-09-16T00:32:17.631 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.x smithi038 *:9283,8765 running (9m) 51s ago 9m 509M - 19.1.1-326-g8921c8ef f1455bd0219c d898ac20a9d1 2024-09-16T00:32:17.631 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.y smithi073 *:8443,8765 running (7m) 53s ago 7m 442M - 19.1.1-326-g8921c8ef f1455bd0219c 33aa75fcc7b0 2024-09-16T00:32:17.631 INFO:teuthology.orchestra.run.smithi038.stdout:mgr.z smithi122 *:8443,8765 running (7m) 53s ago 7m 443M - 19.1.1-326-g8921c8ef f1455bd0219c b280204398a6 2024-09-16T00:32:17.631 INFO:teuthology.orchestra.run.smithi038.stdout:mon.a smithi038 running (9m) 51s ago 9m 51.4M 2048M 19.1.1-326-g8921c8ef f1455bd0219c e8fd9f4da33c 2024-09-16T00:32:17.631 INFO:teuthology.orchestra.run.smithi038.stdout:mon.b smithi073 running (8m) 53s ago 8m 41.3M 2048M 19.1.1-326-g8921c8ef f1455bd0219c 98333bd91265 2024-09-16T00:32:17.631 INFO:teuthology.orchestra.run.smithi038.stdout:mon.c smithi122 running (8m) 53s ago 8m 40.7M 2048M 19.1.1-326-g8921c8ef f1455bd0219c ae3c68d8f84e 2024-09-16T00:32:17.631 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 smithi038 running (7m) 51s ago 7m 53.7M 4273M 19.1.1-326-g8921c8ef f1455bd0219c f6234478898f 2024-09-16T00:32:17.631 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 smithi073 running (7m) 53s ago 7m 46.7M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 5301a16e1095 2024-09-16T00:32:17.631 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 smithi122 running (6m) 53s ago 6m 46.7M 4271M 19.1.1-326-g8921c8ef f1455bd0219c 1783ff12716c 2024-09-16T00:32:17.631 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 smithi038 running (6m) 51s ago 6m 53.3M 4273M 19.1.1-326-g8921c8ef f1455bd0219c 4b92370e8325 2024-09-16T00:32:17.631 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 smithi073 running (5m) 53s ago 5m 50.5M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 2918aab07d64 2024-09-16T00:32:17.631 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 smithi122 running (5m) 53s ago 5m 46.0M 4271M 19.1.1-326-g8921c8ef f1455bd0219c fea16b1b88fe 2024-09-16T00:32:17.632 INFO:teuthology.orchestra.run.smithi038.stdout:osd.6 smithi038 running (5m) 51s ago 5m 50.5M 4273M 19.1.1-326-g8921c8ef f1455bd0219c 4b65e2980991 2024-09-16T00:32:17.632 INFO:teuthology.orchestra.run.smithi038.stdout:osd.7 smithi073 running (4m) 53s ago 4m 44.5M 4272M 19.1.1-326-g8921c8ef f1455bd0219c e35b8391b6db 2024-09-16T00:32:17.632 INFO:teuthology.orchestra.run.smithi038.stdout:osd.8 smithi122 running (4m) 53s ago 4m 53.5M 4271M 19.1.1-326-g8921c8ef f1455bd0219c 731b66bd4090 2024-09-16T00:32:17.632 INFO:teuthology.orchestra.run.smithi038.stdout:osd.9 smithi038 running (3m) 51s ago 3m 46.0M 4273M 19.1.1-326-g8921c8ef f1455bd0219c fcdd85ecf9a6 2024-09-16T00:32:17.632 INFO:teuthology.orchestra.run.smithi038.stdout:osd.10 smithi073 running (3m) 53s ago 3m 51.5M 4272M 19.1.1-326-g8921c8ef f1455bd0219c 8b803c705ab4 2024-09-16T00:32:17.632 INFO:teuthology.orchestra.run.smithi038.stdout:osd.11 smithi122 running (2m) 53s ago 2m 50.8M 4271M 19.1.1-326-g8921c8ef f1455bd0219c ea2f03571f35 2024-09-16T00:32:17.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.631+0000 7fc747049640 1 -- 172.21.15.38:0/3108766661 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fc7300a3300 msgr2=0x7fc7300a57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:17.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.631+0000 7fc747049640 1 --2- 172.21.15.38:0/3108766661 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fc7300a3300 0x7fc7300a57c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc734005ab0 tx=0x7fc734000bc0 comp rx=0 tx=0).stop 2024-09-16T00:32:17.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.631+0000 7fc747049640 1 -- 172.21.15.38:0/3108766661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7401037c0 msgr2=0x7fc74019f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:17.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.631+0000 7fc747049640 1 --2- 172.21.15.38:0/3108766661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7401037c0 0x7fc74019f490 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc73800eaa0 tx=0x7fc73800ef60 comp rx=0 tx=0).stop 2024-09-16T00:32:17.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.631+0000 7fc747049640 1 -- 172.21.15.38:0/3108766661 shutdown_connections 2024-09-16T00:32:17.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.631+0000 7fc747049640 1 --2- 172.21.15.38:0/3108766661 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fc7300a3300 0x7fc7300a57c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:17.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.631+0000 7fc747049640 1 --2- 172.21.15.38:0/3108766661 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fc7401037c0 0x7fc74019f490 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:17.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.631+0000 7fc747049640 1 --2- 172.21.15.38:0/3108766661 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fc740102e00 0x7fc740198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:17.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.631+0000 7fc747049640 1 --2- 172.21.15.38:0/3108766661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc740101c00 0x7fc740197ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:17.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.631+0000 7fc747049640 1 -- 172.21.15.38:0/3108766661 >> 172.21.15.38:0/3108766661 conn(0x7fc7400fd390 msgr2=0x7fc740106170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:17.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.631+0000 7fc747049640 1 -- 172.21.15.38:0/3108766661 shutdown_connections 2024-09-16T00:32:17.633 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:17.631+0000 7fc747049640 1 -- 172.21.15.38:0/3108766661 wait complete. 2024-09-16T00:32:17.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:17 smithi122 ceph-mon[35412]: pgmap v347: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:32:17.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:17 smithi122 ceph-mon[35412]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:18.365 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- bash -c 'ceph orch ls' 2024-09-16T00:32:18.620 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:19.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.115+0000 7f7aa6b01640 1 -- 172.21.15.38:0/2441259910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7aa01037c0 msgr2=0x7f7aa010a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:19.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.115+0000 7f7aa6b01640 1 --2- 172.21.15.38:0/2441259910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7aa01037c0 0x7f7aa010a050 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f7a98009ca0 tx=0x7f7a9802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:32:19.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.115+0000 7f7aa6b01640 1 -- 172.21.15.38:0/2441259910 shutdown_connections 2024-09-16T00:32:19.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.115+0000 7f7aa6b01640 1 --2- 172.21.15.38:0/2441259910 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7aa01037c0 0x7f7aa010a050 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:19.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.115+0000 7f7aa6b01640 1 --2- 172.21.15.38:0/2441259910 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7aa0102e00 0x7f7aa0103280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:19.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.115+0000 7f7aa6b01640 1 --2- 172.21.15.38:0/2441259910 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7aa0101c00 0x7f7aa0102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:19.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.115+0000 7f7aa6b01640 1 -- 172.21.15.38:0/2441259910 >> 172.21.15.38:0/2441259910 conn(0x7f7aa00fd390 msgr2=0x7f7aa00ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:19.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.115+0000 7f7aa6b01640 1 -- 172.21.15.38:0/2441259910 shutdown_connections 2024-09-16T00:32:19.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.115+0000 7f7aa6b01640 1 -- 172.21.15.38:0/2441259910 wait complete. 2024-09-16T00:32:19.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.116+0000 7f7aa6b01640 1 Processor -- start 2024-09-16T00:32:19.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.116+0000 7f7aa6b01640 1 -- start start 2024-09-16T00:32:19.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.116+0000 7f7aa6b01640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7aa0101c00 0x7f7aa0075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:19.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.116+0000 7f7aa6b01640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7aa0102e00 0x7f7aa00745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:19.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.116+0000 7f7aa6b01640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7aa01037c0 0x7f7aa0074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:19.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.116+0000 7f7aa6b01640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa010d8d0 con 0x7f7aa0101c00 2024-09-16T00:32:19.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.116+0000 7f7aa6b01640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa010d750 con 0x7f7aa0102e00 2024-09-16T00:32:19.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.116+0000 7f7aa6b01640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa010da50 con 0x7f7aa01037c0 2024-09-16T00:32:19.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.116+0000 7f7aa489d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7aa0101c00 0x7f7aa0075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:19.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.116+0000 7f7aa489d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7aa0101c00 0x7f7aa0075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53400/0 (socket says 172.21.15.38:53400) 2024-09-16T00:32:19.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.116+0000 7f7aa489d640 1 -- 172.21.15.38:0/2126537939 learned_addr learned my addr 172.21.15.38:0/2126537939 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:19.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.117+0000 7f7aa509e640 1 --2- 172.21.15.38:0/2126537939 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7aa01037c0 0x7f7aa0074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:19.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.117+0000 7f7a97fff640 1 --2- 172.21.15.38:0/2126537939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7aa0102e00 0x7f7aa00745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:19.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.117+0000 7f7aa489d640 1 -- 172.21.15.38:0/2126537939 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7aa01037c0 msgr2=0x7f7aa0074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:19.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.117+0000 7f7aa489d640 1 --2- 172.21.15.38:0/2126537939 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7aa01037c0 0x7f7aa0074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:19.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.117+0000 7f7aa489d640 1 -- 172.21.15.38:0/2126537939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7aa0102e00 msgr2=0x7f7aa00745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:19.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.117+0000 7f7aa489d640 1 --2- 172.21.15.38:0/2126537939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7aa0102e00 0x7f7aa00745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:19.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.117+0000 7f7aa489d640 1 -- 172.21.15.38:0/2126537939 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aa00753e0 con 0x7f7aa0101c00 2024-09-16T00:32:19.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.117+0000 7f7a97fff640 1 --2- 172.21.15.38:0/2126537939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7aa0102e00 0x7f7aa00745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:32:19.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.117+0000 7f7aa489d640 1 --2- 172.21.15.38:0/2126537939 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7aa0101c00 0x7f7aa0075f90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c00c970 tx=0x7f7a9c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:19.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.117+0000 7f7a95ffb640 1 -- 172.21.15.38:0/2126537939 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c004220 con 0x7f7aa0101c00 2024-09-16T00:32:19.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.117+0000 7f7a95ffb640 1 -- 172.21.15.38:0/2126537939 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7a9c0043c0 con 0x7f7aa0101c00 2024-09-16T00:32:19.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.117+0000 7f7aa6b01640 1 -- 172.21.15.38:0/2126537939 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aa01a1fd0 con 0x7f7aa0101c00 2024-09-16T00:32:19.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.117+0000 7f7a95ffb640 1 -- 172.21.15.38:0/2126537939 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c00f450 con 0x7f7aa0101c00 2024-09-16T00:32:19.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.117+0000 7f7aa6b01640 1 -- 172.21.15.38:0/2126537939 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aa01a2430 con 0x7f7aa0101c00 2024-09-16T00:32:19.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.118+0000 7f7aa6b01640 1 -- 172.21.15.38:0/2126537939 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7aa0108f80 con 0x7f7aa0101c00 2024-09-16T00:32:19.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.119+0000 7f7a95ffb640 1 -- 172.21.15.38:0/2126537939 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f7a9c0079b0 con 0x7f7aa0101c00 2024-09-16T00:32:19.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.120+0000 7f7a95ffb640 1 --2- 172.21.15.38:0/2126537939 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f7a880a32d0 0x7f7a880a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:19.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.120+0000 7f7a95ffb640 1 -- 172.21.15.38:0/2126537939 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f7a9c0cfc30 con 0x7f7aa0101c00 2024-09-16T00:32:19.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.120+0000 7f7a97fff640 1 --2- 172.21.15.38:0/2126537939 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f7a880a32d0 0x7f7a880a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:19.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.121+0000 7f7a97fff640 1 --2- 172.21.15.38:0/2126537939 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f7a880a32d0 0x7f7a880a5790 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0075d10 tx=0x7f7a90005e50 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:19.136 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.124+0000 7f7a95ffb640 1 -- 172.21.15.38:0/2126537939 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7a9c014030 con 0x7f7aa0101c00 2024-09-16T00:32:19.308 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.308+0000 7f7aa6b01640 1 -- 172.21.15.38:0/2126537939 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f7aa0062630 con 0x7f7a880a32d0 2024-09-16T00:32:19.309 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:19 smithi038 ceph-mon[30945]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:32:19.309 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:19 smithi038 ceph-mon[30945]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:19.316 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.316+0000 7f7a95ffb640 1 -- 172.21.15.38:0/2126537939 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f7aa0062630 con 0x7f7a880a32d0 2024-09-16T00:32:19.316 INFO:teuthology.orchestra.run.smithi038.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-16T00:32:19.316 INFO:teuthology.orchestra.run.smithi038.stdout:mds.all 12/12 54s ago 2m smithi038=a;smithi038=d;smithi038=g;smithi038=j;smithi073=b;smithi073=e;smithi073=h;smithi073=k;smithi122=c;smithi122=f;smithi122=i;smithi122=l;count:12 2024-09-16T00:32:19.316 INFO:teuthology.orchestra.run.smithi038.stdout:mgr 3/3 54s ago 8m smithi038=x;smithi073=y;smithi122=z;count:3 2024-09-16T00:32:19.316 INFO:teuthology.orchestra.run.smithi038.stdout:mon 3/3 54s ago 8m smithi038:172.21.15.38=a;smithi073:172.21.15.73=b;smithi122:172.21.15.122=c;count:3 2024-09-16T00:32:19.316 INFO:teuthology.orchestra.run.smithi038.stdout:osd 12 54s ago - 2024-09-16T00:32:19.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.318+0000 7f7aa6b01640 1 -- 172.21.15.38:0/2126537939 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f7a880a32d0 msgr2=0x7f7a880a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:19.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.318+0000 7f7aa6b01640 1 --2- 172.21.15.38:0/2126537939 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f7a880a32d0 0x7f7a880a5790 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0075d10 tx=0x7f7a90005e50 comp rx=0 tx=0).stop 2024-09-16T00:32:19.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.318+0000 7f7aa6b01640 1 -- 172.21.15.38:0/2126537939 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7aa0101c00 msgr2=0x7f7aa0075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:19.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.318+0000 7f7aa6b01640 1 --2- 172.21.15.38:0/2126537939 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7aa0101c00 0x7f7aa0075f90 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c00c970 tx=0x7f7a9c00ce30 comp rx=0 tx=0).stop 2024-09-16T00:32:19.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.318+0000 7f7aa6b01640 1 -- 172.21.15.38:0/2126537939 shutdown_connections 2024-09-16T00:32:19.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.318+0000 7f7aa6b01640 1 --2- 172.21.15.38:0/2126537939 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f7a880a32d0 0x7f7a880a5790 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:19.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.318+0000 7f7aa6b01640 1 --2- 172.21.15.38:0/2126537939 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7aa01037c0 0x7f7aa0074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:19.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.318+0000 7f7aa6b01640 1 --2- 172.21.15.38:0/2126537939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7aa0102e00 0x7f7aa00745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:19.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.318+0000 7f7aa6b01640 1 --2- 172.21.15.38:0/2126537939 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7aa0101c00 0x7f7aa0075f90 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:19.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.318+0000 7f7aa6b01640 1 -- 172.21.15.38:0/2126537939 >> 172.21.15.38:0/2126537939 conn(0x7f7aa00fd390 msgr2=0x7f7aa0106150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:19.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.318+0000 7f7aa6b01640 1 -- 172.21.15.38:0/2126537939 shutdown_connections 2024-09-16T00:32:19.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:19.318+0000 7f7aa6b01640 1 -- 172.21.15.38:0/2126537939 wait complete. 2024-09-16T00:32:19.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:19 smithi073 ceph-mon[35254]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:32:19.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:19 smithi073 ceph-mon[35254]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:19.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:19 smithi122 ceph-mon[35412]: pgmap v348: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:32:19.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:19 smithi122 ceph-mon[35412]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:20.221 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- bash -c 'ceph orch host ls' 2024-09-16T00:32:20.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:20 smithi038 ceph-mon[30945]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:20.480 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:20.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:20 smithi073 ceph-mon[35254]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:20.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:20 smithi122 ceph-mon[35412]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:20.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:20.977+0000 7f637732d640 1 -- 172.21.15.38:0/698157562 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6370101cd0 msgr2=0x7f6370102150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:20.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:20.977+0000 7f637732d640 1 --2- 172.21.15.38:0/698157562 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6370101cd0 0x7f6370102150 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6360009900 tx=0x7f636002a410 comp rx=0 tx=0).stop 2024-09-16T00:32:20.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:20.977+0000 7f637732d640 1 -- 172.21.15.38:0/698157562 shutdown_connections 2024-09-16T00:32:20.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:20.977+0000 7f637732d640 1 --2- 172.21.15.38:0/698157562 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6370102690 0x7f6370108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:20.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:20.977+0000 7f637732d640 1 --2- 172.21.15.38:0/698157562 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6370101cd0 0x7f6370102150 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:20.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:20.977+0000 7f637732d640 1 --2- 172.21.15.38:0/698157562 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6370100ad0 0x7f6370100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:20.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:20.977+0000 7f637732d640 1 -- 172.21.15.38:0/698157562 >> 172.21.15.38:0/698157562 conn(0x7f63700fc280 msgr2=0x7f63700fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:20.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:20.977+0000 7f637732d640 1 -- 172.21.15.38:0/698157562 shutdown_connections 2024-09-16T00:32:20.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:20.977+0000 7f637732d640 1 -- 172.21.15.38:0/698157562 wait complete. 2024-09-16T00:32:20.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:20.977+0000 7f637732d640 1 Processor -- start 2024-09-16T00:32:20.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:20.977+0000 7f637732d640 1 -- start start 2024-09-16T00:32:20.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:20.978+0000 7f637732d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6370100ad0 0x7f637019a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:20.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:20.978+0000 7f637732d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6370101cd0 0x7f637019a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:20.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:20.978+0000 7f637732d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6370102690 0x7f63701a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:20.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:20.978+0000 7f637732d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f637010d8c0 con 0x7f6370101cd0 2024-09-16T00:32:20.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:20.978+0000 7f637732d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f637010d740 con 0x7f6370100ad0 2024-09-16T00:32:20.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:20.978+0000 7f637732d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f637010da40 con 0x7f6370102690 2024-09-16T00:32:20.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:20.978+0000 7f63748c8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6370101cd0 0x7f637019a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:20.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:20.978+0000 7f63748c8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6370101cd0 0x7f637019a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53416/0 (socket says 172.21.15.38:53416) 2024-09-16T00:32:20.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:20.978+0000 7f63748c8640 1 -- 172.21.15.38:0/3913367115 learned_addr learned my addr 172.21.15.38:0/3913367115 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:20.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:20.978+0000 7f63750c9640 1 --2- 172.21.15.38:0/3913367115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6370100ad0 0x7f637019a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:20.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:20.978+0000 7f63758ca640 1 --2- 172.21.15.38:0/3913367115 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6370102690 0x7f63701a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:20.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:20.978+0000 7f63748c8640 1 -- 172.21.15.38:0/3913367115 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6370102690 msgr2=0x7f63701a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:20.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:20.978+0000 7f63748c8640 1 --2- 172.21.15.38:0/3913367115 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6370102690 0x7f63701a16c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:20.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:20.978+0000 7f63748c8640 1 -- 172.21.15.38:0/3913367115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6370100ad0 msgr2=0x7f637019a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:20.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:20.978+0000 7f63748c8640 1 --2- 172.21.15.38:0/3913367115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6370100ad0 0x7f637019a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:20.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:20.978+0000 7f63748c8640 1 -- 172.21.15.38:0/3913367115 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63701a1d30 con 0x7f6370101cd0 2024-09-16T00:32:20.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:20.978+0000 7f63758ca640 1 --2- 172.21.15.38:0/3913367115 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6370102690 0x7f63701a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:32:20.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:20.978+0000 7f63748c8640 1 --2- 172.21.15.38:0/3913367115 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6370101cd0 0x7f637019a640 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6360009230 tx=0x7f6360004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:20.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:20.978+0000 7f63667fc640 1 -- 172.21.15.38:0/3913367115 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f636003f030 con 0x7f6370101cd0 2024-09-16T00:32:20.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:20.978+0000 7f63667fc640 1 -- 172.21.15.38:0/3913367115 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6360003ce0 con 0x7f6370101cd0 2024-09-16T00:32:20.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:20.978+0000 7f637732d640 1 -- 172.21.15.38:0/3913367115 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63701a1fc0 con 0x7f6370101cd0 2024-09-16T00:32:20.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:20.978+0000 7f63667fc640 1 -- 172.21.15.38:0/3913367115 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6360035070 con 0x7f6370101cd0 2024-09-16T00:32:20.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:20.979+0000 7f637732d640 1 -- 172.21.15.38:0/3913367115 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63701a24a0 con 0x7f6370101cd0 2024-09-16T00:32:20.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:20.979+0000 7f637732d640 1 -- 172.21.15.38:0/3913367115 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63701a26d0 con 0x7f6370101cd0 2024-09-16T00:32:20.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:20.981+0000 7f63667fc640 1 -- 172.21.15.38:0/3913367115 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 17) ==== 138406+0+0 (secure 0 0 0) 0x7f6360042030 con 0x7f6370101cd0 2024-09-16T00:32:20.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:20.982+0000 7f63667fc640 1 --2- 172.21.15.38:0/3913367115 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f635c0a3380 0x7f635c0a5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:20.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:20.982+0000 7f63750c9640 1 --2- 172.21.15.38:0/3913367115 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f635c0a3380 0x7f635c0a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:20.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:20.982+0000 7f63667fc640 1 -- 172.21.15.38:0/3913367115 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f63600f0910 con 0x7f6370101cd0 2024-09-16T00:32:20.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:20.983+0000 7f63750c9640 1 --2- 172.21.15.38:0/3913367115 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f635c0a3380 0x7f635c0a5840 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f636c0058c0 tx=0x7f636c005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:20.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:20.985+0000 7f63667fc640 1 -- 172.21.15.38:0/3913367115 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f63600bd190 con 0x7f6370101cd0 2024-09-16T00:32:21.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:21.167+0000 7f637732d640 1 -- 172.21.15.38:0/3913367115 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f6370108b90 con 0x7f635c0a3380 2024-09-16T00:32:21.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:21.168+0000 7f63667fc640 1 -- 172.21.15.38:0/3913367115 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f6370108b90 con 0x7f635c0a3380 2024-09-16T00:32:21.168 INFO:teuthology.orchestra.run.smithi038.stdout:HOST ADDR LABELS STATUS 2024-09-16T00:32:21.169 INFO:teuthology.orchestra.run.smithi038.stdout:smithi038 172.21.15.38 2024-09-16T00:32:21.169 INFO:teuthology.orchestra.run.smithi038.stdout:smithi073 172.21.15.73 2024-09-16T00:32:21.169 INFO:teuthology.orchestra.run.smithi038.stdout:smithi122 172.21.15.122 2024-09-16T00:32:21.169 INFO:teuthology.orchestra.run.smithi038.stdout:3 hosts in cluster 2024-09-16T00:32:21.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:21.170+0000 7f637732d640 1 -- 172.21.15.38:0/3913367115 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f635c0a3380 msgr2=0x7f635c0a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:21.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:21.170+0000 7f637732d640 1 --2- 172.21.15.38:0/3913367115 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f635c0a3380 0x7f635c0a5840 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f636c0058c0 tx=0x7f636c005850 comp rx=0 tx=0).stop 2024-09-16T00:32:21.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:21.170+0000 7f637732d640 1 -- 172.21.15.38:0/3913367115 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6370101cd0 msgr2=0x7f637019a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:21.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:21.170+0000 7f637732d640 1 --2- 172.21.15.38:0/3913367115 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6370101cd0 0x7f637019a640 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f6360009230 tx=0x7f6360004430 comp rx=0 tx=0).stop 2024-09-16T00:32:21.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:21.170+0000 7f637732d640 1 -- 172.21.15.38:0/3913367115 shutdown_connections 2024-09-16T00:32:21.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:21.170+0000 7f637732d640 1 --2- 172.21.15.38:0/3913367115 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f635c0a3380 0x7f635c0a5840 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:21.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:21.170+0000 7f637732d640 1 --2- 172.21.15.38:0/3913367115 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6370102690 0x7f63701a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:21.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:21.170+0000 7f637732d640 1 --2- 172.21.15.38:0/3913367115 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6370101cd0 0x7f637019a640 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:21.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:21.170+0000 7f637732d640 1 --2- 172.21.15.38:0/3913367115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6370100ad0 0x7f637019a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:21.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:21.170+0000 7f637732d640 1 -- 172.21.15.38:0/3913367115 >> 172.21.15.38:0/3913367115 conn(0x7f63700fc280 msgr2=0x7f6370107470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:21.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:21.171+0000 7f637732d640 1 -- 172.21.15.38:0/3913367115 shutdown_connections 2024-09-16T00:32:21.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:21.171+0000 7f637732d640 1 -- 172.21.15.38:0/3913367115 wait complete. 2024-09-16T00:32:21.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:21 smithi038 ceph-mon[30945]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T00:32:21.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:21 smithi073 ceph-mon[35254]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T00:32:21.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:21 smithi122 ceph-mon[35412]: pgmap v349: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T00:32:21.961 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- bash -c 'ceph orch device ls' 2024-09-16T00:32:22.217 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:22.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:22 smithi073 ceph-mon[35254]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:22.620 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:22 smithi038 ceph-mon[30945]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:22.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:22 smithi122 ceph-mon[35412]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:22.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.705+0000 7f020bb9a640 1 -- 172.21.15.38:0/122338475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0204102690 msgr2=0x7f0204108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:22.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.705+0000 7f020bb9a640 1 --2- 172.21.15.38:0/122338475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0204102690 0x7f0204108f20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f01fc009ca0 tx=0x7f01fc02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:32:22.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.705+0000 7f020bb9a640 1 -- 172.21.15.38:0/122338475 shutdown_connections 2024-09-16T00:32:22.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.705+0000 7f020bb9a640 1 --2- 172.21.15.38:0/122338475 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0204102690 0x7f0204108f20 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:22.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.705+0000 7f020bb9a640 1 --2- 172.21.15.38:0/122338475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0204101cd0 0x7f0204102150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:22.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.705+0000 7f020bb9a640 1 --2- 172.21.15.38:0/122338475 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0204100ad0 0x7f0204100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:22.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.705+0000 7f020bb9a640 1 -- 172.21.15.38:0/122338475 >> 172.21.15.38:0/122338475 conn(0x7f02040fc280 msgr2=0x7f02040fe6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:22.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.705+0000 7f020bb9a640 1 -- 172.21.15.38:0/122338475 shutdown_connections 2024-09-16T00:32:22.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.705+0000 7f020bb9a640 1 -- 172.21.15.38:0/122338475 wait complete. 2024-09-16T00:32:22.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.706+0000 7f020bb9a640 1 Processor -- start 2024-09-16T00:32:22.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.706+0000 7f020bb9a640 1 -- start start 2024-09-16T00:32:22.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.706+0000 7f020bb9a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0204100ad0 0x7f020406aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:22.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.706+0000 7f020bb9a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0204101cd0 0x7f020419cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:22.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.706+0000 7f020bb9a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0204102690 0x7f020419f280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:22.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.706+0000 7f020bb9a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f020410c7b0 con 0x7f0204102690 2024-09-16T00:32:22.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.706+0000 7f020bb9a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f020410c630 con 0x7f0204100ad0 2024-09-16T00:32:22.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.706+0000 7f020bb9a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f020410c930 con 0x7f0204101cd0 2024-09-16T00:32:22.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.706+0000 7f020a137640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0204102690 0x7f020419f280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:22.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.707+0000 7f020a137640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0204102690 0x7f020419f280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53434/0 (socket says 172.21.15.38:53434) 2024-09-16T00:32:22.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.707+0000 7f020a137640 1 -- 172.21.15.38:0/1089988565 learned_addr learned my addr 172.21.15.38:0/1089988565 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:22.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.707+0000 7f0209135640 1 --2- 172.21.15.38:0/1089988565 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0204101cd0 0x7f020419cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:22.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.707+0000 7f0209936640 1 --2- 172.21.15.38:0/1089988565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0204100ad0 0x7f020406aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:22.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.707+0000 7f020a137640 1 -- 172.21.15.38:0/1089988565 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0204101cd0 msgr2=0x7f020419cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:22.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.707+0000 7f020a137640 1 --2- 172.21.15.38:0/1089988565 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0204101cd0 0x7f020419cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:22.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.707+0000 7f020a137640 1 -- 172.21.15.38:0/1089988565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0204100ad0 msgr2=0x7f020406aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:22.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.707+0000 7f020a137640 1 --2- 172.21.15.38:0/1089988565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0204100ad0 0x7f020406aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:22.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.707+0000 7f020a137640 1 -- 172.21.15.38:0/1089988565 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f020419fb90 con 0x7f0204102690 2024-09-16T00:32:22.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.707+0000 7f0209936640 1 --2- 172.21.15.38:0/1089988565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0204100ad0 0x7f020406aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:22.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.707+0000 7f020a137640 1 --2- 172.21.15.38:0/1089988565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0204102690 0x7f020419f280 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f01fc005e30 tx=0x7f01fc031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:22.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.707+0000 7f01faffd640 1 -- 172.21.15.38:0/1089988565 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01fc003b70 con 0x7f0204102690 2024-09-16T00:32:22.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.707+0000 7f01faffd640 1 -- 172.21.15.38:0/1089988565 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f01fc003d10 con 0x7f0204102690 2024-09-16T00:32:22.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.707+0000 7f020bb9a640 1 -- 172.21.15.38:0/1089988565 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f020419fdc0 con 0x7f0204102690 2024-09-16T00:32:22.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.707+0000 7f01faffd640 1 -- 172.21.15.38:0/1089988565 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01fc02fd30 con 0x7f0204102690 2024-09-16T00:32:22.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.707+0000 7f020bb9a640 1 -- 172.21.15.38:0/1089988565 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02041a0350 con 0x7f0204102690 2024-09-16T00:32:22.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.708+0000 7f020bb9a640 1 -- 172.21.15.38:0/1089988565 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0204107ef0 con 0x7f0204102690 2024-09-16T00:32:22.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.709+0000 7f01faffd640 1 -- 172.21.15.38:0/1089988565 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f01fc004050 con 0x7f0204102690 2024-09-16T00:32:22.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.710+0000 7f01faffd640 1 --2- 172.21.15.38:0/1089988565 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f01f00a3240 0x7f01f00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:22.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.710+0000 7f0209936640 1 --2- 172.21.15.38:0/1089988565 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f01f00a3240 0x7f01f00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:22.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.710+0000 7f01faffd640 1 -- 172.21.15.38:0/1089988565 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9071+0+0 (secure 0 0 0) 0x7f01fc0f10e0 con 0x7f0204102690 2024-09-16T00:32:22.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.711+0000 7f0209936640 1 --2- 172.21.15.38:0/1089988565 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f01f00a3240 0x7f01f00a5700 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f01f40058c0 tx=0x7f01f4005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:22.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.714+0000 7f01faffd640 1 -- 172.21.15.38:0/1089988565 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f01fc0bda60 con 0x7f0204102690 2024-09-16T00:32:22.898 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.898+0000 7f020bb9a640 1 -- 172.21.15.38:0/1089988565 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f0204062630 con 0x7f01f00a3240 2024-09-16T00:32:22.900 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.900+0000 7f01faffd640 1 -- 172.21.15.38:0/1089988565 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f0204062630 con 0x7f01f00a3240 2024-09-16T00:32:22.901 INFO:teuthology.orchestra.run.smithi038.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-16T00:32:22.901 INFO:teuthology.orchestra.run.smithi038.stdout:smithi038 /dev/nvme0n1 ssd INTEL_SSDPEDKE040T7_BTLE736103BB4P0KGN 3726G No 3m ago Has a FileSystem, LVM detected 2024-09-16T00:32:22.901 INFO:teuthology.orchestra.run.smithi038.stdout:smithi073 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040115400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-09-16T00:32:22.901 INFO:teuthology.orchestra.run.smithi038.stdout:smithi122 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204010U400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-16T00:32:22.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.903+0000 7f020bb9a640 1 -- 172.21.15.38:0/1089988565 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f01f00a3240 msgr2=0x7f01f00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:22.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.903+0000 7f020bb9a640 1 --2- 172.21.15.38:0/1089988565 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f01f00a3240 0x7f01f00a5700 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f01f40058c0 tx=0x7f01f4005850 comp rx=0 tx=0).stop 2024-09-16T00:32:22.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.903+0000 7f020bb9a640 1 -- 172.21.15.38:0/1089988565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0204102690 msgr2=0x7f020419f280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:22.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.903+0000 7f020bb9a640 1 --2- 172.21.15.38:0/1089988565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0204102690 0x7f020419f280 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f01fc005e30 tx=0x7f01fc031ee0 comp rx=0 tx=0).stop 2024-09-16T00:32:22.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.903+0000 7f020bb9a640 1 -- 172.21.15.38:0/1089988565 shutdown_connections 2024-09-16T00:32:22.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.903+0000 7f020bb9a640 1 --2- 172.21.15.38:0/1089988565 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f01f00a3240 0x7f01f00a5700 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:22.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.903+0000 7f020bb9a640 1 --2- 172.21.15.38:0/1089988565 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0204102690 0x7f020419f280 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:22.904 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.903+0000 7f020bb9a640 1 --2- 172.21.15.38:0/1089988565 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0204101cd0 0x7f020419cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:22.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.903+0000 7f020bb9a640 1 --2- 172.21.15.38:0/1089988565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0204100ad0 0x7f020406aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:22.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.903+0000 7f020bb9a640 1 -- 172.21.15.38:0/1089988565 >> 172.21.15.38:0/1089988565 conn(0x7f02040fc280 msgr2=0x7f02041050c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:22.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.904+0000 7f020bb9a640 1 -- 172.21.15.38:0/1089988565 shutdown_connections 2024-09-16T00:32:22.905 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:22.904+0000 7f020bb9a640 1 -- 172.21.15.38:0/1089988565 wait complete. 2024-09-16T00:32:23.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:23 smithi073 ceph-mon[35254]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T00:32:23.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:23 smithi073 ceph-mon[35254]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-16T00:32:23.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:23 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]: dispatch 2024-09-16T00:32:23.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:23 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-09-16T00:32:23.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:23 smithi038 ceph-mon[30945]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T00:32:23.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:23 smithi038 ceph-mon[30945]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-16T00:32:23.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:23 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]: dispatch 2024-09-16T00:32:23.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:23 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-09-16T00:32:23.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:23 smithi122 ceph-mon[35412]: pgmap v350: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T00:32:23.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:23 smithi122 ceph-mon[35412]: mgrmap e18: x(active, since 9m), standbys: z, y 2024-09-16T00:32:23.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:23 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]: dispatch 2024-09-16T00:32:23.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:23 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-09-16T00:32:23.748 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-16T00:32:23.757 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi038.front.sepia.ceph.com 2024-09-16T00:32:23.758 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- bash -c 'ceph fs dump' 2024-09-16T00:32:24.017 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:24.424 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:24 smithi038 ceph-mon[30945]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:24.424 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:24 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]': finished 2024-09-16T00:32:24.424 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:24 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-09-16T00:32:24.424 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:24 smithi038 ceph-mon[30945]: osdmap e91: 12 total, 12 up, 12 in 2024-09-16T00:32:24.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.507+0000 7f2238963640 1 -- 172.21.15.38:0/1389939001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22341026d0 msgr2=0x7f2234108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:24.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.507+0000 7f2238963640 1 --2- 172.21.15.38:0/1389939001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22341026d0 0x7f2234108f60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f222c009ca0 tx=0x7f222c02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:32:24.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.507+0000 7f2238963640 1 -- 172.21.15.38:0/1389939001 shutdown_connections 2024-09-16T00:32:24.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.507+0000 7f2238963640 1 --2- 172.21.15.38:0/1389939001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f22341026d0 0x7f2234108f60 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:24.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.507+0000 7f2238963640 1 --2- 172.21.15.38:0/1389939001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2234101d10 0x7f2234102190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:24.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.507+0000 7f2238963640 1 --2- 172.21.15.38:0/1389939001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2234100b10 0x7f2234100f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:24.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.507+0000 7f2238963640 1 -- 172.21.15.38:0/1389939001 >> 172.21.15.38:0/1389939001 conn(0x7f22340fc2a0 msgr2=0x7f22340fe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:24.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.507+0000 7f2238963640 1 -- 172.21.15.38:0/1389939001 shutdown_connections 2024-09-16T00:32:24.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.507+0000 7f2238963640 1 -- 172.21.15.38:0/1389939001 wait complete. 2024-09-16T00:32:24.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.508+0000 7f2238963640 1 Processor -- start 2024-09-16T00:32:24.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.508+0000 7f2238963640 1 -- start start 2024-09-16T00:32:24.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.508+0000 7f2238963640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2234100b10 0x7f223406b330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:24.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.508+0000 7f2238963640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2234101d10 0x7f223419cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:24.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.508+0000 7f2238963640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22341026d0 0x7f223419f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:24.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.508+0000 7f2238963640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f223410c810 con 0x7f2234101d10 2024-09-16T00:32:24.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.508+0000 7f2238963640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f223410c690 con 0x7f2234100b10 2024-09-16T00:32:24.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.508+0000 7f2238963640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f223410c990 con 0x7f22341026d0 2024-09-16T00:32:24.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.508+0000 7f2231d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2234101d10 0x7f223419cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:24.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.508+0000 7f2231d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2234101d10 0x7f223419cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53444/0 (socket says 172.21.15.38:53444) 2024-09-16T00:32:24.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.508+0000 7f2231d9b640 1 -- 172.21.15.38:0/2418054547 learned_addr learned my addr 172.21.15.38:0/2418054547 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:24.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.508+0000 7f2232d9d640 1 --2- 172.21.15.38:0/2418054547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22341026d0 0x7f223419f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:24.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.508+0000 7f223259c640 1 --2- 172.21.15.38:0/2418054547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2234100b10 0x7f223406b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:24.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.508+0000 7f2231d9b640 1 -- 172.21.15.38:0/2418054547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22341026d0 msgr2=0x7f223419f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:24.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.508+0000 7f2231d9b640 1 --2- 172.21.15.38:0/2418054547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22341026d0 0x7f223419f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:24.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.508+0000 7f2231d9b640 1 -- 172.21.15.38:0/2418054547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2234100b10 msgr2=0x7f223406b330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:24.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.508+0000 7f2231d9b640 1 --2- 172.21.15.38:0/2418054547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2234100b10 0x7f223406b330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:24.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.508+0000 7f2231d9b640 1 -- 172.21.15.38:0/2418054547 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f223419fb60 con 0x7f2234101d10 2024-09-16T00:32:24.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.508+0000 7f2232d9d640 1 --2- 172.21.15.38:0/2418054547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22341026d0 0x7f223419f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:32:24.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.509+0000 7f2231d9b640 1 --2- 172.21.15.38:0/2418054547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2234101d10 0x7f223419cc50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f222800d950 tx=0x7f222800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:24.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.509+0000 7f22237fe640 1 -- 172.21.15.38:0/2418054547 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2228017070 con 0x7f2234101d10 2024-09-16T00:32:24.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.509+0000 7f22237fe640 1 -- 172.21.15.38:0/2418054547 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2228009c30 con 0x7f2234101d10 2024-09-16T00:32:24.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.509+0000 7f2238963640 1 -- 172.21.15.38:0/2418054547 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f223419fe50 con 0x7f2234101d10 2024-09-16T00:32:24.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.509+0000 7f22237fe640 1 -- 172.21.15.38:0/2418054547 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f222800f6b0 con 0x7f2234101d10 2024-09-16T00:32:24.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.509+0000 7f2238963640 1 -- 172.21.15.38:0/2418054547 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22341a0380 con 0x7f2234101d10 2024-09-16T00:32:24.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.510+0000 7f2238963640 1 -- 172.21.15.38:0/2418054547 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2234107eb0 con 0x7f2234101d10 2024-09-16T00:32:24.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.511+0000 7f22237fe640 1 -- 172.21.15.38:0/2418054547 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f22280109d0 con 0x7f2234101d10 2024-09-16T00:32:24.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.512+0000 7f22237fe640 1 --2- 172.21.15.38:0/2418054547 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f221c0a3240 0x7f221c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:24.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.512+0000 7f22237fe640 1 -- 172.21.15.38:0/2418054547 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f22280cf950 con 0x7f2234101d10 2024-09-16T00:32:24.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.512+0000 7f223259c640 1 --2- 172.21.15.38:0/2418054547 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f221c0a3240 0x7f221c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:24.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.512+0000 7f223259c640 1 --2- 172.21.15.38:0/2418054547 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f221c0a3240 0x7f221c0a5700 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f223406b870 tx=0x7f2224008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:24.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.516+0000 7f22237fe640 1 -- 172.21.15.38:0/2418054547 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2228009de0 con 0x7f2234101d10 2024-09-16T00:32:24.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:24 smithi122 ceph-mon[35412]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:24.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:24 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]': finished 2024-09-16T00:32:24.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:24 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-09-16T00:32:24.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:24 smithi122 ceph-mon[35412]: osdmap e91: 12 total, 12 up, 12 in 2024-09-16T00:32:24.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.732+0000 7f2238963640 1 -- 172.21.15.38:0/2418054547 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f2234062e90 con 0x7f2234101d10 2024-09-16T00:32:24.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.732+0000 7f22237fe640 1 -- 172.21.15.38:0/2418054547 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) ==== 76+0+3086 (secure 0 0 0) 0x7f222809c2f0 con 0x7f2234101d10 2024-09-16T00:32:24.733 INFO:teuthology.orchestra.run.smithi038.stdout:e25 2024-09-16T00:32:24.733 INFO:teuthology.orchestra.run.smithi038.stdout:btime 2024-09-16T00:31:05:330385+0000 2024-09-16T00:32:24.733 INFO:teuthology.orchestra.run.smithi038.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-16T00:32:24.733 INFO:teuthology.orchestra.run.smithi038.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-16T00:32:24.733 INFO:teuthology.orchestra.run.smithi038.stdout:legacy client fscid: 1 2024-09-16T00:32:24.733 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:32:24.734 INFO:teuthology.orchestra.run.smithi038.stdout:Filesystem 'cephfs' (1) 2024-09-16T00:32:24.734 INFO:teuthology.orchestra.run.smithi038.stdout:fs_name cephfs 2024-09-16T00:32:24.734 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 25 2024-09-16T00:32:24.734 INFO:teuthology.orchestra.run.smithi038.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-16T00:32:24.734 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-09-16T00:30:26.764565+0000 2024-09-16T00:32:24.734 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-09-16T00:31:05.083019+0000 2024-09-16T00:32:24.734 INFO:teuthology.orchestra.run.smithi038.stdout:tableserver 0 2024-09-16T00:32:24.734 INFO:teuthology.orchestra.run.smithi038.stdout:root 0 2024-09-16T00:32:24.734 INFO:teuthology.orchestra.run.smithi038.stdout:session_timeout 300 2024-09-16T00:32:24.734 INFO:teuthology.orchestra.run.smithi038.stdout:session_autoclose 300 2024-09-16T00:32:24.734 INFO:teuthology.orchestra.run.smithi038.stdout:max_file_size 1099511627776 2024-09-16T00:32:24.734 INFO:teuthology.orchestra.run.smithi038.stdout:max_xattr_size 65536 2024-09-16T00:32:24.734 INFO:teuthology.orchestra.run.smithi038.stdout:required_client_features {} 2024-09-16T00:32:24.734 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure 0 2024-09-16T00:32:24.734 INFO:teuthology.orchestra.run.smithi038.stdout:last_failure_osd_epoch 0 2024-09-16T00:32:24.734 INFO:teuthology.orchestra.run.smithi038.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-16T00:32:24.735 INFO:teuthology.orchestra.run.smithi038.stdout:max_mds 3 2024-09-16T00:32:24.735 INFO:teuthology.orchestra.run.smithi038.stdout:in 0,1,2 2024-09-16T00:32:24.735 INFO:teuthology.orchestra.run.smithi038.stdout:up {0=24494,1=24464,2=24440} 2024-09-16T00:32:24.735 INFO:teuthology.orchestra.run.smithi038.stdout:failed 2024-09-16T00:32:24.735 INFO:teuthology.orchestra.run.smithi038.stdout:damaged 2024-09-16T00:32:24.735 INFO:teuthology.orchestra.run.smithi038.stdout:stopped 2024-09-16T00:32:24.735 INFO:teuthology.orchestra.run.smithi038.stdout:data_pools [3,4] 2024-09-16T00:32:24.735 INFO:teuthology.orchestra.run.smithi038.stdout:metadata_pool 2 2024-09-16T00:32:24.735 INFO:teuthology.orchestra.run.smithi038.stdout:inline_data disabled 2024-09-16T00:32:24.735 INFO:teuthology.orchestra.run.smithi038.stdout:balancer 2024-09-16T00:32:24.735 INFO:teuthology.orchestra.run.smithi038.stdout:bal_rank_mask -1 2024-09-16T00:32:24.735 INFO:teuthology.orchestra.run.smithi038.stdout:standby_count_wanted 0 2024-09-16T00:32:24.735 INFO:teuthology.orchestra.run.smithi038.stdout:qdb_cluster leader: 24494 members: 24440,24464,24494 2024-09-16T00:32:24.735 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.b{0:24494} state up:active seq 4 addr [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:32:24.735 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.i{0:24436} state up:standby-replay seq 1 addr [v2:172.21.15.122:6838/1983975311,v1:172.21.15.122:6839/1983975311] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:32:24.736 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.h{1:24464} state up:active seq 16 addr [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:32:24.736 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.c{1:24421} state up:standby-replay seq 1 addr [v2:172.21.15.122:6836/1858299680,v1:172.21.15.122:6837/1858299680] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:32:24.736 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.e{2:24440} state up:active seq 18 addr [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:32:24.736 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.k{2:24416} state up:standby-replay seq 1 addr [v2:172.21.15.73:6832/1210328177,v1:172.21.15.73:6833/1210328177] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:32:24.736 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:32:24.736 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:32:24.736 INFO:teuthology.orchestra.run.smithi038.stdout:Standby daemons: 2024-09-16T00:32:24.736 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:32:24.736 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.j{-1:14535} state up:standby seq 1 addr [v2:172.21.15.38:6834/367974527,v1:172.21.15.38:6835/367974527] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:32:24.736 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.d{-1:14547} state up:standby seq 1 addr [v2:172.21.15.38:6836/335077136,v1:172.21.15.38:6837/335077136] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:32:24.736 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.g{-1:14604} state up:standby seq 1 addr [v2:172.21.15.38:6838/3501211160,v1:172.21.15.38:6839/3501211160] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:32:24.736 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.a{-1:14616} state up:standby seq 1 addr [v2:172.21.15.38:6840/2623130915,v1:172.21.15.38:6841/2623130915] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:32:24.736 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.l{-1:24388} state up:standby seq 1 addr [v2:172.21.15.122:6832/412277519,v1:172.21.15.122:6833/412277519] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:32:24.737 INFO:teuthology.orchestra.run.smithi038.stdout:[mds.f{-1:24415} state up:standby seq 1 addr [v2:172.21.15.122:6834/3987234757,v1:172.21.15.122:6835/3987234757] compat {c=[1],r=[1],i=[1fff]}] 2024-09-16T00:32:24.737 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 25 2024-09-16T00:32:24.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.735+0000 7f2238963640 1 -- 172.21.15.38:0/2418054547 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f221c0a3240 msgr2=0x7f221c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:24.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.735+0000 7f2238963640 1 --2- 172.21.15.38:0/2418054547 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f221c0a3240 0x7f221c0a5700 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f223406b870 tx=0x7f2224008040 comp rx=0 tx=0).stop 2024-09-16T00:32:24.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.735+0000 7f2238963640 1 -- 172.21.15.38:0/2418054547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2234101d10 msgr2=0x7f223419cc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:24.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.735+0000 7f2238963640 1 --2- 172.21.15.38:0/2418054547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2234101d10 0x7f223419cc50 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f222800d950 tx=0x7f222800de10 comp rx=0 tx=0).stop 2024-09-16T00:32:24.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.735+0000 7f2238963640 1 -- 172.21.15.38:0/2418054547 shutdown_connections 2024-09-16T00:32:24.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.735+0000 7f2238963640 1 --2- 172.21.15.38:0/2418054547 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f221c0a3240 0x7f221c0a5700 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:24.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.735+0000 7f2238963640 1 --2- 172.21.15.38:0/2418054547 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f22341026d0 0x7f223419f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:24.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.735+0000 7f2238963640 1 --2- 172.21.15.38:0/2418054547 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2234101d10 0x7f223419cc50 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:24.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.735+0000 7f2238963640 1 --2- 172.21.15.38:0/2418054547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2234100b10 0x7f223406b330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:24.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.735+0000 7f2238963640 1 -- 172.21.15.38:0/2418054547 >> 172.21.15.38:0/2418054547 conn(0x7f22340fc2a0 msgr2=0x7f2234105080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:24.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.735+0000 7f2238963640 1 -- 172.21.15.38:0/2418054547 shutdown_connections 2024-09-16T00:32:24.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:24.735+0000 7f2238963640 1 -- 172.21.15.38:0/2418054547 wait complete. 2024-09-16T00:32:24.793 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:24 smithi073 ceph-mon[35254]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:24.793 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:24 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [5, 8]}]': finished 2024-09-16T00:32:24.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:24 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-09-16T00:32:24.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:24 smithi073 ceph-mon[35254]: osdmap e91: 12 total, 12 up, 12 in 2024-09-16T00:32:25.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:25 smithi038 ceph-mon[30945]: pgmap v352: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-16T00:32:25.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:25 smithi038 ceph-mon[30945]: osdmap e92: 12 total, 12 up, 12 in 2024-09-16T00:32:25.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:25 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/2418054547' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:32:25.576 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- bash -c 'ceph osd dump' 2024-09-16T00:32:25.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:25 smithi122 ceph-mon[35412]: pgmap v352: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-16T00:32:25.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:25 smithi122 ceph-mon[35412]: osdmap e92: 12 total, 12 up, 12 in 2024-09-16T00:32:25.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:25 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/2418054547' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:32:25.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:25 smithi073 ceph-mon[35254]: pgmap v352: 129 pgs: 129 active+clean; 641 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2024-09-16T00:32:25.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:25 smithi073 ceph-mon[35254]: osdmap e92: 12 total, 12 up, 12 in 2024-09-16T00:32:25.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:25 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/2418054547' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-09-16T00:32:25.832 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:26.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.417+0000 7fa69b9ae640 1 -- 172.21.15.38:0/2431495993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa694102e00 msgr2=0x7fa694103280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:26.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.417+0000 7fa69b9ae640 1 --2- 172.21.15.38:0/2431495993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa694102e00 0x7fa694103280 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fa684009900 tx=0x7fa68402a410 comp rx=0 tx=0).stop 2024-09-16T00:32:26.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.417+0000 7fa69b9ae640 1 -- 172.21.15.38:0/2431495993 shutdown_connections 2024-09-16T00:32:26.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.417+0000 7fa69b9ae640 1 --2- 172.21.15.38:0/2431495993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6941037c0 0x7fa69410a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:26.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.417+0000 7fa69b9ae640 1 --2- 172.21.15.38:0/2431495993 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa694102e00 0x7fa694103280 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:26.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.417+0000 7fa69b9ae640 1 --2- 172.21.15.38:0/2431495993 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa694101c00 0x7fa694102000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:26.427 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.417+0000 7fa69b9ae640 1 -- 172.21.15.38:0/2431495993 >> 172.21.15.38:0/2431495993 conn(0x7fa6940fd390 msgr2=0x7fa6940ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:26.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.417+0000 7fa69b9ae640 1 -- 172.21.15.38:0/2431495993 shutdown_connections 2024-09-16T00:32:26.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.417+0000 7fa69b9ae640 1 -- 172.21.15.38:0/2431495993 wait complete. 2024-09-16T00:32:26.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.418+0000 7fa69b9ae640 1 Processor -- start 2024-09-16T00:32:26.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.418+0000 7fa69b9ae640 1 -- start start 2024-09-16T00:32:26.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.418+0000 7fa69b9ae640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa694101c00 0x7fa694075f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:26.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.418+0000 7fa69b9ae640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa694102e00 0x7fa6940745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:26.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.418+0000 7fa69b9ae640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6941037c0 0x7fa694074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:26.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.418+0000 7fa69b9ae640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa69410d900 con 0x7fa694101c00 2024-09-16T00:32:26.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.418+0000 7fa69b9ae640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa69410d780 con 0x7fa694102e00 2024-09-16T00:32:26.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.418+0000 7fa69b9ae640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa69410da80 con 0x7fa6941037c0 2024-09-16T00:32:26.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.418+0000 7fa69974a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa694101c00 0x7fa694075f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:26.428 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.418+0000 7fa69974a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa694101c00 0x7fa694075f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53708/0 (socket says 172.21.15.38:53708) 2024-09-16T00:32:26.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.418+0000 7fa69974a640 1 -- 172.21.15.38:0/4088453833 learned_addr learned my addr 172.21.15.38:0/4088453833 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:26.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.418+0000 7fa698f49640 1 --2- 172.21.15.38:0/4088453833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa694102e00 0x7fa6940745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:26.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.418+0000 7fa699f4b640 1 --2- 172.21.15.38:0/4088453833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6941037c0 0x7fa694074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:26.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.419+0000 7fa69974a640 1 -- 172.21.15.38:0/4088453833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6941037c0 msgr2=0x7fa694074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:26.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.419+0000 7fa69974a640 1 --2- 172.21.15.38:0/4088453833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6941037c0 0x7fa694074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:26.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.419+0000 7fa69974a640 1 -- 172.21.15.38:0/4088453833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa694102e00 msgr2=0x7fa6940745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:26.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.419+0000 7fa69974a640 1 --2- 172.21.15.38:0/4088453833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa694102e00 0x7fa6940745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:26.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.419+0000 7fa69974a640 1 -- 172.21.15.38:0/4088453833 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6940753e0 con 0x7fa694101c00 2024-09-16T00:32:26.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.419+0000 7fa699f4b640 1 --2- 172.21.15.38:0/4088453833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6941037c0 0x7fa694074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:26.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.419+0000 7fa69974a640 1 --2- 172.21.15.38:0/4088453833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa694101c00 0x7fa694075f90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa69000b450 tx=0x7fa69000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:26.429 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.419+0000 7fa68a7fc640 1 -- 172.21.15.38:0/4088453833 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa690015020 con 0x7fa694101c00 2024-09-16T00:32:26.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.419+0000 7fa68a7fc640 1 -- 172.21.15.38:0/4088453833 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa690011d70 con 0x7fa694101c00 2024-09-16T00:32:26.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.419+0000 7fa69b9ae640 1 -- 172.21.15.38:0/4088453833 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6941a2110 con 0x7fa694101c00 2024-09-16T00:32:26.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.419+0000 7fa68a7fc640 1 -- 172.21.15.38:0/4088453833 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa69000f340 con 0x7fa694101c00 2024-09-16T00:32:26.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.419+0000 7fa69b9ae640 1 -- 172.21.15.38:0/4088453833 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6941a26f0 con 0x7fa694101c00 2024-09-16T00:32:26.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.420+0000 7fa69b9ae640 1 -- 172.21.15.38:0/4088453833 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa694109050 con 0x7fa694101c00 2024-09-16T00:32:26.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.421+0000 7fa68a7fc640 1 -- 172.21.15.38:0/4088453833 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fa690018020 con 0x7fa694101c00 2024-09-16T00:32:26.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.422+0000 7fa68a7fc640 1 --2- 172.21.15.38:0/4088453833 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fa6800a32c0 0x7fa6800a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:26.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.422+0000 7fa698f49640 1 --2- 172.21.15.38:0/4088453833 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fa6800a32c0 0x7fa6800a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:26.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.422+0000 7fa68a7fc640 1 -- 172.21.15.38:0/4088453833 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7fa6900cf670 con 0x7fa694101c00 2024-09-16T00:32:26.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.422+0000 7fa698f49640 1 --2- 172.21.15.38:0/4088453833 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fa6800a32c0 0x7fa6800a5780 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa6840314e0 tx=0x7fa684000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:26.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.426+0000 7fa68a7fc640 1 -- 172.21.15.38:0/4088453833 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa69001f020 con 0x7fa694101c00 2024-09-16T00:32:26.618 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.618+0000 7fa69b9ae640 1 -- 172.21.15.38:0/4088453833 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fa694062df0 con 0x7fa694101c00 2024-09-16T00:32:26.619 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.619+0000 7fa68a7fc640 1 -- 172.21.15.38:0/4088453833 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v92) ==== 56+0+5615 (secure 0 0 0) 0x7fa69000d040 con 0x7fa694101c00 2024-09-16T00:32:26.620 INFO:teuthology.orchestra.run.smithi038.stdout:epoch 92 2024-09-16T00:32:26.620 INFO:teuthology.orchestra.run.smithi038.stdout:fsid 983b2072-73c1-11ef-bceb-c7b262605968 2024-09-16T00:32:26.620 INFO:teuthology.orchestra.run.smithi038.stdout:created 2024-09-16T00:22:34.897053+0000 2024-09-16T00:32:26.620 INFO:teuthology.orchestra.run.smithi038.stdout:modified 2024-09-16T00:32:24.296695+0000 2024-09-16T00:32:26.620 INFO:teuthology.orchestra.run.smithi038.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-16T00:32:26.620 INFO:teuthology.orchestra.run.smithi038.stdout:crush_version 27 2024-09-16T00:32:26.621 INFO:teuthology.orchestra.run.smithi038.stdout:full_ratio 0.95 2024-09-16T00:32:26.621 INFO:teuthology.orchestra.run.smithi038.stdout:backfillfull_ratio 0.9 2024-09-16T00:32:26.621 INFO:teuthology.orchestra.run.smithi038.stdout:nearfull_ratio 0.85 2024-09-16T00:32:26.621 INFO:teuthology.orchestra.run.smithi038.stdout:require_min_compat_client luminous 2024-09-16T00:32:26.621 INFO:teuthology.orchestra.run.smithi038.stdout:min_compat_client luminous 2024-09-16T00:32:26.621 INFO:teuthology.orchestra.run.smithi038.stdout:require_osd_release squid 2024-09-16T00:32:26.621 INFO:teuthology.orchestra.run.smithi038.stdout:stretch_mode_enabled false 2024-09-16T00:32:26.621 INFO:teuthology.orchestra.run.smithi038.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-16T00:32:26.621 INFO:teuthology.orchestra.run.smithi038.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-16T00:32:26.621 INFO:teuthology.orchestra.run.smithi038.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 82 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-09-16T00:32:26.621 INFO:teuthology.orchestra.run.smithi038.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 88 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-09-16T00:32:26.621 INFO:teuthology.orchestra.run.smithi038.stdout:max_osd 12 2024-09-16T00:32:26.622 INFO:teuthology.orchestra.run.smithi038.stdout:osd.0 up in weight 1 up_from 8 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.38:6802/1622028921,v1:172.21.15.38:6803/1622028921] [v2:172.21.15.38:6804/1622028921,v1:172.21.15.38:6805/1622028921] exists,up 3afc20e3-869f-43a9-a911-b0e8d07de86f 2024-09-16T00:32:26.622 INFO:teuthology.orchestra.run.smithi038.stdout:osd.1 up in weight 1 up_from 13 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6800/3811690336,v1:172.21.15.73:6801/3811690336] [v2:172.21.15.73:6802/3811690336,v1:172.21.15.73:6803/3811690336] exists,up 6d9d7acb-45ed-4543-8afc-6433fc7f096f 2024-09-16T00:32:26.622 INFO:teuthology.orchestra.run.smithi038.stdout:osd.2 up in weight 1 up_from 18 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6800/1754741655,v1:172.21.15.122:6801/1754741655] [v2:172.21.15.122:6802/1754741655,v1:172.21.15.122:6803/1754741655] exists,up 48690f35-99e9-4f45-9bee-487857caa8b8 2024-09-16T00:32:26.622 INFO:teuthology.orchestra.run.smithi038.stdout:osd.3 up in weight 1 up_from 25 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.38:6810/3523190079,v1:172.21.15.38:6811/3523190079] [v2:172.21.15.38:6812/3523190079,v1:172.21.15.38:6813/3523190079] exists,up 28ebb4a0-92e7-4f57-82a5-b9b761eca269 2024-09-16T00:32:26.622 INFO:teuthology.orchestra.run.smithi038.stdout:osd.4 up in weight 1 up_from 32 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6808/1469927628,v1:172.21.15.73:6809/1469927628] [v2:172.21.15.73:6810/1469927628,v1:172.21.15.73:6811/1469927628] exists,up 5ea363f7-7ad8-45ed-b7e4-4b010fe7f8cc 2024-09-16T00:32:26.622 INFO:teuthology.orchestra.run.smithi038.stdout:osd.5 up in weight 1 up_from 37 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6808/2954413058,v1:172.21.15.122:6809/2954413058] [v2:172.21.15.122:6810/2954413058,v1:172.21.15.122:6811/2954413058] exists,up e3c66d43-dd28-4bb1-a886-62a869be82cb 2024-09-16T00:32:26.622 INFO:teuthology.orchestra.run.smithi038.stdout:osd.6 up in weight 1 up_from 44 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.38:6818/1415719550,v1:172.21.15.38:6819/1415719550] [v2:172.21.15.38:6820/1415719550,v1:172.21.15.38:6821/1415719550] exists,up 211497ac-7c48-4095-bfdb-1d4df0149f57 2024-09-16T00:32:26.622 INFO:teuthology.orchestra.run.smithi038.stdout:osd.7 up in weight 1 up_from 49 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6816/2634361473,v1:172.21.15.73:6817/2634361473] [v2:172.21.15.73:6818/2634361473,v1:172.21.15.73:6819/2634361473] exists,up 06512315-3f5e-44f8-976d-d3f5f1ba00e3 2024-09-16T00:32:26.622 INFO:teuthology.orchestra.run.smithi038.stdout:osd.8 up in weight 1 up_from 57 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6816/1800567633,v1:172.21.15.122:6817/1800567633] [v2:172.21.15.122:6818/1800567633,v1:172.21.15.122:6819/1800567633] exists,up db703c0a-1f87-46f0-be5f-f826f7f9daac 2024-09-16T00:32:26.622 INFO:teuthology.orchestra.run.smithi038.stdout:osd.9 up in weight 1 up_from 64 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.38:6826/445727833,v1:172.21.15.38:6827/445727833] [v2:172.21.15.38:6828/445727833,v1:172.21.15.38:6829/445727833] exists,up f5384200-b3b4-4936-85e1-fc57a9fe2bb8 2024-09-16T00:32:26.622 INFO:teuthology.orchestra.run.smithi038.stdout:osd.10 up in weight 1 up_from 69 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6824/43297139,v1:172.21.15.73:6825/43297139] [v2:172.21.15.73:6826/43297139,v1:172.21.15.73:6827/43297139] exists,up c22883d9-6672-4796-a961-963d21eb37fc 2024-09-16T00:32:26.622 INFO:teuthology.orchestra.run.smithi038.stdout:osd.11 up in weight 1 up_from 76 up_thru 91 down_at 0 last_clean_interval [0,0) [v2:172.21.15.122:6824/1886446061,v1:172.21.15.122:6825/1886446061] [v2:172.21.15.122:6826/1886446061,v1:172.21.15.122:6827/1886446061] exists,up 9f98e8cb-1bff-4b8b-aa19-56b691387383 2024-09-16T00:32:26.623 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 2.f [8,11] 2024-09-16T00:32:26.623 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 2.13 [2,11] 2024-09-16T00:32:26.623 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 2.16 [2,4] 2024-09-16T00:32:26.623 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 3.7 [3,9] 2024-09-16T00:32:26.623 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 3.1d [4,8,5,9] 2024-09-16T00:32:26.623 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 3.21 [4,6] 2024-09-16T00:32:26.623 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 3.36 [4,9] 2024-09-16T00:32:26.623 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 4.f [5,8] 2024-09-16T00:32:26.623 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 4.12 [9,10] 2024-09-16T00:32:26.623 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 4.14 [9,8] 2024-09-16T00:32:26.623 INFO:teuthology.orchestra.run.smithi038.stdout:pg_upmap_items 4.17 [9,8] 2024-09-16T00:32:26.623 INFO:teuthology.orchestra.run.smithi038.stdout:blocklist 172.21.15.38:6801/1779324916 expires 2024-09-17T00:23:21.417483+0000 2024-09-16T00:32:26.624 INFO:teuthology.orchestra.run.smithi038.stdout:blocklist 172.21.15.38:0/924150447 expires 2024-09-17T00:23:21.417483+0000 2024-09-16T00:32:26.624 INFO:teuthology.orchestra.run.smithi038.stdout:blocklist 172.21.15.38:0/141237849 expires 2024-09-17T00:23:21.417483+0000 2024-09-16T00:32:26.624 INFO:teuthology.orchestra.run.smithi038.stdout:blocklist 172.21.15.38:0/2302081750 expires 2024-09-17T00:22:56.844212+0000 2024-09-16T00:32:26.624 INFO:teuthology.orchestra.run.smithi038.stdout:blocklist 172.21.15.38:0/2245468594 expires 2024-09-17T00:22:56.844212+0000 2024-09-16T00:32:26.624 INFO:teuthology.orchestra.run.smithi038.stdout:blocklist 172.21.15.38:6801/583006373 expires 2024-09-17T00:22:56.844212+0000 2024-09-16T00:32:26.624 INFO:teuthology.orchestra.run.smithi038.stdout:blocklist 172.21.15.38:0/2435492667 expires 2024-09-17T00:23:21.417483+0000 2024-09-16T00:32:26.624 INFO:teuthology.orchestra.run.smithi038.stdout:blocklist 172.21.15.38:6800/583006373 expires 2024-09-17T00:22:56.844212+0000 2024-09-16T00:32:26.624 INFO:teuthology.orchestra.run.smithi038.stdout:blocklist 172.21.15.38:6800/1779324916 expires 2024-09-17T00:23:21.417483+0000 2024-09-16T00:32:26.624 INFO:teuthology.orchestra.run.smithi038.stdout:blocklist 172.21.15.38:0/1103516837 expires 2024-09-17T00:22:56.844212+0000 2024-09-16T00:32:26.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.621+0000 7fa69b9ae640 1 -- 172.21.15.38:0/4088453833 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fa6800a32c0 msgr2=0x7fa6800a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:26.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.621+0000 7fa69b9ae640 1 --2- 172.21.15.38:0/4088453833 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fa6800a32c0 0x7fa6800a5780 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fa6840314e0 tx=0x7fa684000bc0 comp rx=0 tx=0).stop 2024-09-16T00:32:26.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.621+0000 7fa69b9ae640 1 -- 172.21.15.38:0/4088453833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa694101c00 msgr2=0x7fa694075f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:26.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.621+0000 7fa69b9ae640 1 --2- 172.21.15.38:0/4088453833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa694101c00 0x7fa694075f90 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa69000b450 tx=0x7fa69000b910 comp rx=0 tx=0).stop 2024-09-16T00:32:26.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.622+0000 7fa69b9ae640 1 -- 172.21.15.38:0/4088453833 shutdown_connections 2024-09-16T00:32:26.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.622+0000 7fa69b9ae640 1 --2- 172.21.15.38:0/4088453833 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fa6800a32c0 0x7fa6800a5780 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:26.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.622+0000 7fa69b9ae640 1 --2- 172.21.15.38:0/4088453833 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6941037c0 0x7fa694074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:26.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.622+0000 7fa69b9ae640 1 --2- 172.21.15.38:0/4088453833 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa694102e00 0x7fa6940745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:26.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.622+0000 7fa69b9ae640 1 --2- 172.21.15.38:0/4088453833 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa694101c00 0x7fa694075f90 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:26.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.622+0000 7fa69b9ae640 1 -- 172.21.15.38:0/4088453833 >> 172.21.15.38:0/4088453833 conn(0x7fa6940fd390 msgr2=0x7fa694106220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:26.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.622+0000 7fa69b9ae640 1 -- 172.21.15.38:0/4088453833 shutdown_connections 2024-09-16T00:32:26.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:26.622+0000 7fa69b9ae640 1 -- 172.21.15.38:0/4088453833 wait complete. 2024-09-16T00:32:27.341 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-16T00:32:27.354 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:32:27.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:27 smithi122 ceph-mon[35412]: pgmap v354: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 8 op/s 2024-09-16T00:32:27.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:27 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:32:27.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:27 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/4088453833' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-16T00:32:27.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:27 smithi073 ceph-mon[35254]: pgmap v354: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 8 op/s 2024-09-16T00:32:27.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:27 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:32:27.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:27 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/4088453833' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-16T00:32:27.670 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:27 smithi038 ceph-mon[30945]: pgmap v354: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 639 B/s wr, 8 op/s 2024-09-16T00:32:27.670 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:27 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:32:27.670 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:27 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/4088453833' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-16T00:32:28.136 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:28.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.641+0000 7feabb052640 1 -- 172.21.15.38:0/1308044251 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feab4106f10 msgr2=0x7feab410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:28.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.641+0000 7feabb052640 1 --2- 172.21.15.38:0/1308044251 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feab4106f10 0x7feab410ea90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7feaac009ca0 tx=0x7feaac02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:32:28.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.641+0000 7feabb052640 1 -- 172.21.15.38:0/1308044251 shutdown_connections 2024-09-16T00:32:28.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.641+0000 7feabb052640 1 --2- 172.21.15.38:0/1308044251 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feab4106f10 0x7feab410ea90 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:28.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.641+0000 7feabb052640 1 --2- 172.21.15.38:0/1308044251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feab4102790 0x7feab41067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:28.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.641+0000 7feabb052640 1 --2- 172.21.15.38:0/1308044251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feab4101de0 0x7feab41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:28.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.641+0000 7feabb052640 1 -- 172.21.15.38:0/1308044251 >> 172.21.15.38:0/1308044251 conn(0x7feab40fd8c0 msgr2=0x7feab40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:28.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.641+0000 7feabb052640 1 -- 172.21.15.38:0/1308044251 shutdown_connections 2024-09-16T00:32:28.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.641+0000 7feabb052640 1 -- 172.21.15.38:0/1308044251 wait complete. 2024-09-16T00:32:28.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.642+0000 7feabb052640 1 Processor -- start 2024-09-16T00:32:28.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.642+0000 7feabb052640 1 -- start start 2024-09-16T00:32:28.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.642+0000 7feabb052640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feab4101de0 0x7feab40744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:28.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.642+0000 7feabb052640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feab4102790 0x7feab4074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:28.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.642+0000 7feabb052640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feab4106f10 0x7feab4074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:28.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.642+0000 7feabb052640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7feab4111f70 con 0x7feab4102790 2024-09-16T00:32:28.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.642+0000 7feabb052640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7feab4111df0 con 0x7feab4106f10 2024-09-16T00:32:28.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.642+0000 7feabb052640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7feab41120f0 con 0x7feab4101de0 2024-09-16T00:32:28.685 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.642+0000 7feab3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feab4102790 0x7feab4074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:28.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.642+0000 7feab3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feab4102790 0x7feab4074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53728/0 (socket says 172.21.15.38:53728) 2024-09-16T00:32:28.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.642+0000 7feab3fff640 1 -- 172.21.15.38:0/404981027 learned_addr learned my addr 172.21.15.38:0/404981027 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:28.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.643+0000 7feab95ef640 1 --2- 172.21.15.38:0/404981027 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feab4106f10 0x7feab4074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:28.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.643+0000 7feab8dee640 1 --2- 172.21.15.38:0/404981027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feab4101de0 0x7feab40744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:28.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.643+0000 7feab3fff640 1 -- 172.21.15.38:0/404981027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feab4101de0 msgr2=0x7feab40744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:28.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.643+0000 7feab3fff640 1 --2- 172.21.15.38:0/404981027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feab4101de0 0x7feab40744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:28.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.643+0000 7feab3fff640 1 -- 172.21.15.38:0/404981027 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feab4106f10 msgr2=0x7feab4074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:28.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.643+0000 7feab3fff640 1 --2- 172.21.15.38:0/404981027 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feab4106f10 0x7feab4074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:28.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.643+0000 7feab3fff640 1 -- 172.21.15.38:0/404981027 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feab41a6890 con 0x7feab4102790 2024-09-16T00:32:28.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.643+0000 7feab8dee640 1 --2- 172.21.15.38:0/404981027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feab4101de0 0x7feab40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:28.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.643+0000 7feab95ef640 1 --2- 172.21.15.38:0/404981027 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feab4106f10 0x7feab4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:28.686 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.643+0000 7feab3fff640 1 --2- 172.21.15.38:0/404981027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feab4102790 0x7feab4074a30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fea9c00e9f0 tx=0x7fea9c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:28.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.643+0000 7feab1ffb640 1 -- 172.21.15.38:0/404981027 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c00cd80 con 0x7feab4102790 2024-09-16T00:32:28.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.643+0000 7feab1ffb640 1 -- 172.21.15.38:0/404981027 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fea9c010910 con 0x7feab4102790 2024-09-16T00:32:28.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.643+0000 7feabb052640 1 -- 172.21.15.38:0/404981027 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feab41a6b80 con 0x7feab4102790 2024-09-16T00:32:28.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.643+0000 7feab1ffb640 1 -- 172.21.15.38:0/404981027 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea9c018990 con 0x7feab4102790 2024-09-16T00:32:28.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.643+0000 7feabb052640 1 -- 172.21.15.38:0/404981027 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feab41a7110 con 0x7feab4102790 2024-09-16T00:32:28.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.644+0000 7feabb052640 1 -- 172.21.15.38:0/404981027 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feab41021c0 con 0x7feab4102790 2024-09-16T00:32:28.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.646+0000 7feab1ffb640 1 -- 172.21.15.38:0/404981027 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fea9c010ac0 con 0x7feab4102790 2024-09-16T00:32:28.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.646+0000 7feab1ffb640 1 --2- 172.21.15.38:0/404981027 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7feaa00a32c0 0x7feaa00a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:28.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.646+0000 7feab8dee640 1 --2- 172.21.15.38:0/404981027 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7feaa00a32c0 0x7feaa00a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:28.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.646+0000 7feab1ffb640 1 -- 172.21.15.38:0/404981027 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7fea9c0cffe0 con 0x7feab4102790 2024-09-16T00:32:28.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.647+0000 7feab8dee640 1 --2- 172.21.15.38:0/404981027 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7feaa00a32c0 0x7feaa00a5780 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7feaa4009c20 tx=0x7feaa4005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:28.687 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.650+0000 7feab1ffb640 1 -- 172.21.15.38:0/404981027 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fea9c09c960 con 0x7feab4102790 2024-09-16T00:32:28.886 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.886+0000 7feabb052640 1 -- 172.21.15.38:0/404981027 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feab4109bc0 con 0x7feab4102790 2024-09-16T00:32:28.887 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.887+0000 7feab1ffb640 1 -- 172.21.15.38:0/404981027 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11909 (secure 0 0 0) 0x7fea9c09c260 con 0x7feab4102790 2024-09-16T00:32:28.888 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:32:28.888 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":25,"btime":"2024-09-16T00:31:05:330385+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/367974527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":367974527},{"type":"v1","addr":"172.21.15.38:6835","nonce":367974527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/335077136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":335077136},{"type":"v1","addr":"172.21.15.38:6837","nonce":335077136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/3501211160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3501211160},{"type":"v1","addr":"172.21.15.38:6839","nonce":3501211160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6841/2623130915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":2623130915},{"type":"v1","addr":"172.21.15.38:6841","nonce":2623130915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/412277519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":412277519},{"type":"v1","addr":"172.21.15.122:6833","nonce":412277519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3987234757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3987234757},{"type":"v1","addr":"172.21.15.122:6835","nonce":3987234757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:30:26.764565+0000","modified":"2024-09-16T00:31:05.083019+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24440},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1210328177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1210328177},{"type":"v1","addr":"172.21.15.73:6833","nonce":1210328177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6837/1858299680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1858299680},{"type":"v1","addr":"172.21.15.122:6837","nonce":1858299680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1983975311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1983975311},{"type":"v1","addr":"172.21.15.122:6839","nonce":1983975311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.73:6835/2866363113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2866363113},{"type":"v1","addr":"172.21.15.73:6835","nonce":2866363113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.73:6837/2681081265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2681081265},{"type":"v1","addr":"172.21.15.73:6837","nonce":2681081265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.73:6839/2396224525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2396224525},{"type":"v1","addr":"172.21.15.73:6839","nonce":2396224525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24440,24464,24494]},"id":1}]} 2024-09-16T00:32:28.888 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 25 2024-09-16T00:32:28.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.889+0000 7feabb052640 1 -- 172.21.15.38:0/404981027 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7feaa00a32c0 msgr2=0x7feaa00a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:28.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.889+0000 7feabb052640 1 --2- 172.21.15.38:0/404981027 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7feaa00a32c0 0x7feaa00a5780 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7feaa4009c20 tx=0x7feaa4005f90 comp rx=0 tx=0).stop 2024-09-16T00:32:28.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.889+0000 7feabb052640 1 -- 172.21.15.38:0/404981027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feab4102790 msgr2=0x7feab4074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:28.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.889+0000 7feabb052640 1 --2- 172.21.15.38:0/404981027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feab4102790 0x7feab4074a30 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fea9c00e9f0 tx=0x7fea9c00eeb0 comp rx=0 tx=0).stop 2024-09-16T00:32:28.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.889+0000 7feabb052640 1 -- 172.21.15.38:0/404981027 shutdown_connections 2024-09-16T00:32:28.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.889+0000 7feabb052640 1 --2- 172.21.15.38:0/404981027 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7feaa00a32c0 0x7feaa00a5780 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:28.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.889+0000 7feabb052640 1 --2- 172.21.15.38:0/404981027 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feab4106f10 0x7feab4074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:28.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.889+0000 7feabb052640 1 --2- 172.21.15.38:0/404981027 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7feab4102790 0x7feab4074a30 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:28.890 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.889+0000 7feabb052640 1 --2- 172.21.15.38:0/404981027 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7feab4101de0 0x7feab40744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:28.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.889+0000 7feabb052640 1 -- 172.21.15.38:0/404981027 >> 172.21.15.38:0/404981027 conn(0x7feab40fd8c0 msgr2=0x7feab40feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:28.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.890+0000 7feabb052640 1 -- 172.21.15.38:0/404981027 shutdown_connections 2024-09-16T00:32:28.891 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:28.890+0000 7feabb052640 1 -- 172.21.15.38:0/404981027 wait complete. 2024-09-16T00:32:29.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:29 smithi038 ceph-mon[30945]: pgmap v355: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 639 B/s wr, 5 op/s 2024-09-16T00:32:29.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:29 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:32:29.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:29 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:32:29.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:29 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:32:29.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:29 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:32:29.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:29 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/404981027' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:32:29.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:29 smithi073 ceph-mon[35254]: pgmap v355: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 639 B/s wr, 5 op/s 2024-09-16T00:32:29.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:29 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:32:29.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:29 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:32:29.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:29 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:32:29.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:29 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:32:29.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:29 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/404981027' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:32:29.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:29 smithi122 ceph-mon[35412]: pgmap v355: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 639 B/s wr, 5 op/s 2024-09-16T00:32:29.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:29 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:32:29.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:29 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:32:29.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:29 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:32:29.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:29 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:32:29.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:29 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/404981027' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:32:29.586 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:32:29.870 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:30.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.364+0000 7fb6cae44640 1 -- 172.21.15.38:0/3672021698 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6c4102570 msgr2=0x7fb6c41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:30.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.364+0000 7fb6cae44640 1 --2- 172.21.15.38:0/3672021698 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6c4102570 0x7fb6c41029b0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0009900 tx=0x7fb6b002a480 comp rx=0 tx=0).stop 2024-09-16T00:32:30.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.364+0000 7fb6cae44640 1 -- 172.21.15.38:0/3672021698 shutdown_connections 2024-09-16T00:32:30.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.364+0000 7fb6cae44640 1 --2- 172.21.15.38:0/3672021698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6c4102ef0 0x7fb6c410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:30.395 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.364+0000 7fb6cae44640 1 --2- 172.21.15.38:0/3672021698 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6c4102570 0x7fb6c41029b0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:30.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.364+0000 7fb6cae44640 1 --2- 172.21.15.38:0/3672021698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6c4100e20 0x7fb6c4101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:30.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.364+0000 7fb6cae44640 1 -- 172.21.15.38:0/3672021698 >> 172.21.15.38:0/3672021698 conn(0x7fb6c40fc7d0 msgr2=0x7fb6c40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:30.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.364+0000 7fb6cae44640 1 -- 172.21.15.38:0/3672021698 shutdown_connections 2024-09-16T00:32:30.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.364+0000 7fb6cae44640 1 -- 172.21.15.38:0/3672021698 wait complete. 2024-09-16T00:32:30.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.365+0000 7fb6cae44640 1 Processor -- start 2024-09-16T00:32:30.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.365+0000 7fb6cae44640 1 -- start start 2024-09-16T00:32:30.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.365+0000 7fb6cae44640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6c4100e20 0x7fb6c4198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:30.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.365+0000 7fb6cae44640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6c4102570 0x7fb6c4198960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:30.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.365+0000 7fb6cae44640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6c4102ef0 0x7fb6c419ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:30.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.365+0000 7fb6cae44640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c41120a0 con 0x7fb6c4102570 2024-09-16T00:32:30.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.365+0000 7fb6cae44640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c4111f20 con 0x7fb6c4100e20 2024-09-16T00:32:30.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.365+0000 7fb6cae44640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb6c4112220 con 0x7fb6c4102ef0 2024-09-16T00:32:30.396 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.365+0000 7fb6c3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6c4102570 0x7fb6c4198960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:30.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.365+0000 7fb6c3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6c4102570 0x7fb6c4198960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53754/0 (socket says 172.21.15.38:53754) 2024-09-16T00:32:30.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.365+0000 7fb6c3fff640 1 -- 172.21.15.38:0/385688980 learned_addr learned my addr 172.21.15.38:0/385688980 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:30.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.366+0000 7fb6c8be0640 1 --2- 172.21.15.38:0/385688980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6c4100e20 0x7fb6c4198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:30.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.366+0000 7fb6c93e1640 1 --2- 172.21.15.38:0/385688980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6c4102ef0 0x7fb6c419ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:30.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.366+0000 7fb6c3fff640 1 -- 172.21.15.38:0/385688980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6c4102ef0 msgr2=0x7fb6c419ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:30.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.366+0000 7fb6c3fff640 1 --2- 172.21.15.38:0/385688980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6c4102ef0 0x7fb6c419ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:30.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.366+0000 7fb6c3fff640 1 -- 172.21.15.38:0/385688980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6c4100e20 msgr2=0x7fb6c4198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:30.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.366+0000 7fb6c3fff640 1 --2- 172.21.15.38:0/385688980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6c4100e20 0x7fb6c4198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:30.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.366+0000 7fb6c3fff640 1 -- 172.21.15.38:0/385688980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6c419d3d0 con 0x7fb6c4102570 2024-09-16T00:32:30.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.366+0000 7fb6c93e1640 1 --2- 172.21.15.38:0/385688980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6c4102ef0 0x7fb6c419ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:30.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.366+0000 7fb6c3fff640 1 --2- 172.21.15.38:0/385688980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6c4102570 0x7fb6c4198960 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0006010 tx=0x7fb6b0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:30.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.366+0000 7fb6c1ffb640 1 -- 172.21.15.38:0/385688980 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b003f070 con 0x7fb6c4102570 2024-09-16T00:32:30.397 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.366+0000 7fb6c1ffb640 1 -- 172.21.15.38:0/385688980 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb6b0031c30 con 0x7fb6c4102570 2024-09-16T00:32:30.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.366+0000 7fb6c1ffb640 1 -- 172.21.15.38:0/385688980 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6b0035030 con 0x7fb6c4102570 2024-09-16T00:32:30.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.366+0000 7fb6cae44640 1 -- 172.21.15.38:0/385688980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6c419d660 con 0x7fb6c4102570 2024-09-16T00:32:30.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.367+0000 7fb6cae44640 1 -- 172.21.15.38:0/385688980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6c419daf0 con 0x7fb6c4102570 2024-09-16T00:32:30.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.368+0000 7fb6cae44640 1 -- 172.21.15.38:0/385688980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6c4101200 con 0x7fb6c4102570 2024-09-16T00:32:30.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.369+0000 7fb6c1ffb640 1 -- 172.21.15.38:0/385688980 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fb6b0042070 con 0x7fb6c4102570 2024-09-16T00:32:30.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.370+0000 7fb6c1ffb640 1 --2- 172.21.15.38:0/385688980 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb6b40a3240 0x7fb6b40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:30.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.370+0000 7fb6c8be0640 1 --2- 172.21.15.38:0/385688980 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb6b40a3240 0x7fb6b40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:30.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.370+0000 7fb6c1ffb640 1 -- 172.21.15.38:0/385688980 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7fb6b00f0550 con 0x7fb6c4102570 2024-09-16T00:32:30.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.371+0000 7fb6c8be0640 1 --2- 172.21.15.38:0/385688980 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb6b40a3240 0x7fb6b40a5700 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb6b800bbe0 tx=0x7fb6b800b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:30.398 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.375+0000 7fb6c1ffb640 1 -- 172.21.15.38:0/385688980 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb6b0031de0 con 0x7fb6c4102570 2024-09-16T00:32:30.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.601+0000 7fb6cae44640 1 -- 172.21.15.38:0/385688980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6c41a5000 con 0x7fb6c4102570 2024-09-16T00:32:30.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.602+0000 7fb6c1ffb640 1 -- 172.21.15.38:0/385688980 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) ==== 94+0+11909 (secure 0 0 0) 0x7fb6b00bce90 con 0x7fb6c4102570 2024-09-16T00:32:30.602 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:32:30.603 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":25,"btime":"2024-09-16T00:31:05:330385+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/367974527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":367974527},{"type":"v1","addr":"172.21.15.38:6835","nonce":367974527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/335077136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":335077136},{"type":"v1","addr":"172.21.15.38:6837","nonce":335077136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/3501211160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3501211160},{"type":"v1","addr":"172.21.15.38:6839","nonce":3501211160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6841/2623130915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":2623130915},{"type":"v1","addr":"172.21.15.38:6841","nonce":2623130915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/412277519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":412277519},{"type":"v1","addr":"172.21.15.122:6833","nonce":412277519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3987234757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3987234757},{"type":"v1","addr":"172.21.15.122:6835","nonce":3987234757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:30:26.764565+0000","modified":"2024-09-16T00:31:05.083019+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24440},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1210328177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1210328177},{"type":"v1","addr":"172.21.15.73:6833","nonce":1210328177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6837/1858299680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1858299680},{"type":"v1","addr":"172.21.15.122:6837","nonce":1858299680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1983975311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1983975311},{"type":"v1","addr":"172.21.15.122:6839","nonce":1983975311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.73:6835/2866363113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2866363113},{"type":"v1","addr":"172.21.15.73:6835","nonce":2866363113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.73:6837/2681081265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2681081265},{"type":"v1","addr":"172.21.15.73:6837","nonce":2681081265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.73:6839/2396224525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2396224525},{"type":"v1","addr":"172.21.15.73:6839","nonce":2396224525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24440,24464,24494]},"id":1}]} 2024-09-16T00:32:30.603 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 25 2024-09-16T00:32:30.604 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.604+0000 7fb6cae44640 1 -- 172.21.15.38:0/385688980 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb6b40a3240 msgr2=0x7fb6b40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:30.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.604+0000 7fb6cae44640 1 --2- 172.21.15.38:0/385688980 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb6b40a3240 0x7fb6b40a5700 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb6b800bbe0 tx=0x7fb6b800b390 comp rx=0 tx=0).stop 2024-09-16T00:32:30.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.604+0000 7fb6cae44640 1 -- 172.21.15.38:0/385688980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6c4102570 msgr2=0x7fb6c4198960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:30.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.604+0000 7fb6cae44640 1 --2- 172.21.15.38:0/385688980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6c4102570 0x7fb6c4198960 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0006010 tx=0x7fb6b0003c10 comp rx=0 tx=0).stop 2024-09-16T00:32:30.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.604+0000 7fb6cae44640 1 -- 172.21.15.38:0/385688980 shutdown_connections 2024-09-16T00:32:30.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.604+0000 7fb6cae44640 1 --2- 172.21.15.38:0/385688980 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb6b40a3240 0x7fb6b40a5700 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:30.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.604+0000 7fb6cae44640 1 --2- 172.21.15.38:0/385688980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb6c4102ef0 0x7fb6c419ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:30.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.604+0000 7fb6cae44640 1 --2- 172.21.15.38:0/385688980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb6c4102570 0x7fb6c4198960 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:30.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.604+0000 7fb6cae44640 1 --2- 172.21.15.38:0/385688980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6c4100e20 0x7fb6c4198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:30.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.604+0000 7fb6cae44640 1 -- 172.21.15.38:0/385688980 >> 172.21.15.38:0/385688980 conn(0x7fb6c40fc7d0 msgr2=0x7fb6c40febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:30.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.604+0000 7fb6cae44640 1 -- 172.21.15.38:0/385688980 shutdown_connections 2024-09-16T00:32:30.605 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:30.604+0000 7fb6cae44640 1 -- 172.21.15.38:0/385688980 wait complete. 2024-09-16T00:32:30.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:30 smithi038 ceph-mon[30945]: pgmap v356: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 639 B/s wr, 7 op/s 2024-09-16T00:32:30.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:30 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:32:30.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:30 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:32:30.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:30 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:30.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:30 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:32:30.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:30 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:32:30.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:30 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/385688980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:32:31.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:30 smithi122 ceph-mon[35412]: pgmap v356: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 639 B/s wr, 7 op/s 2024-09-16T00:32:31.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:30 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:32:31.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:30 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:32:31.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:30 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:31.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:30 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:32:31.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:30 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:32:31.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:30 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/385688980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:32:31.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:30 smithi073 ceph-mon[35254]: pgmap v356: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 639 B/s wr, 7 op/s 2024-09-16T00:32:31.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:30 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:32:31.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:30 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:32:31.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:30 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:32:31.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:30 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:32:31.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:30 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:32:31.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:30 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/385688980' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:32:31.419 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:32:31.679 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:32.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.172+0000 7f4c787a0640 1 -- 172.21.15.38:0/3938652709 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c70100e20 msgr2=0x7f4c70101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:32.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.172+0000 7f4c787a0640 1 --2- 172.21.15.38:0/3938652709 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c70100e20 0x7f4c70101200 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c009900 tx=0x7f4c6c02a480 comp rx=0 tx=0).stop 2024-09-16T00:32:32.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.172+0000 7f4c787a0640 1 -- 172.21.15.38:0/3938652709 shutdown_connections 2024-09-16T00:32:32.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.172+0000 7f4c787a0640 1 --2- 172.21.15.38:0/3938652709 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c70102ef0 0x7f4c7010da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:32.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.172+0000 7f4c787a0640 1 --2- 172.21.15.38:0/3938652709 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c70102570 0x7f4c701029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:32.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.172+0000 7f4c787a0640 1 --2- 172.21.15.38:0/3938652709 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c70100e20 0x7f4c70101200 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:32.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.172+0000 7f4c787a0640 1 -- 172.21.15.38:0/3938652709 >> 172.21.15.38:0/3938652709 conn(0x7f4c700fc7d0 msgr2=0x7f4c700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:32.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.172+0000 7f4c787a0640 1 -- 172.21.15.38:0/3938652709 shutdown_connections 2024-09-16T00:32:32.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.172+0000 7f4c787a0640 1 -- 172.21.15.38:0/3938652709 wait complete. 2024-09-16T00:32:32.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.172+0000 7f4c787a0640 1 Processor -- start 2024-09-16T00:32:32.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.172+0000 7f4c787a0640 1 -- start start 2024-09-16T00:32:32.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.173+0000 7f4c787a0640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c70100e20 0x7f4c701973e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:32.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.173+0000 7f4c787a0640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c70102570 0x7f4c70197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:32.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.173+0000 7f4c787a0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c70102ef0 0x7f4c7019bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:32.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.173+0000 7f4c787a0640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4c70110f80 con 0x7f4c70102ef0 2024-09-16T00:32:32.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.173+0000 7f4c787a0640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4c70110e00 con 0x7f4c70100e20 2024-09-16T00:32:32.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.173+0000 7f4c787a0640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4c70111100 con 0x7f4c70102570 2024-09-16T00:32:32.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.173+0000 7f4c76d3d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c70102ef0 0x7f4c7019bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:32.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.173+0000 7f4c76d3d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c70102ef0 0x7f4c7019bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53770/0 (socket says 172.21.15.38:53770) 2024-09-16T00:32:32.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.173+0000 7f4c76d3d640 1 -- 172.21.15.38:0/233217355 learned_addr learned my addr 172.21.15.38:0/233217355 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:32.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.173+0000 7f4c75d3b640 1 --2- 172.21.15.38:0/233217355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c70102570 0x7f4c70197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:32.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.173+0000 7f4c7653c640 1 --2- 172.21.15.38:0/233217355 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c70100e20 0x7f4c701973e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:32.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.173+0000 7f4c76d3d640 1 -- 172.21.15.38:0/233217355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c70102570 msgr2=0x7f4c70197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:32.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.173+0000 7f4c76d3d640 1 --2- 172.21.15.38:0/233217355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c70102570 0x7f4c70197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:32.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.173+0000 7f4c76d3d640 1 -- 172.21.15.38:0/233217355 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c70100e20 msgr2=0x7f4c701973e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:32.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.173+0000 7f4c76d3d640 1 --2- 172.21.15.38:0/233217355 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c70100e20 0x7f4c701973e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:32.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.173+0000 7f4c76d3d640 1 -- 172.21.15.38:0/233217355 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c7019c390 con 0x7f4c70102ef0 2024-09-16T00:32:32.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.173+0000 7f4c75d3b640 1 --2- 172.21.15.38:0/233217355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c70102570 0x7f4c70197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:32.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.173+0000 7f4c76d3d640 1 --2- 172.21.15.38:0/233217355 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c70102ef0 0x7f4c7019bcb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4c6800cad0 tx=0x7f4c6800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:32.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.173+0000 7f4c677fe640 1 -- 172.21.15.38:0/233217355 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6800ae20 con 0x7f4c70102ef0 2024-09-16T00:32:32.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.173+0000 7f4c677fe640 1 -- 172.21.15.38:0/233217355 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4c68005020 con 0x7f4c70102ef0 2024-09-16T00:32:32.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.173+0000 7f4c787a0640 1 -- 172.21.15.38:0/233217355 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c7019c680 con 0x7f4c70102ef0 2024-09-16T00:32:32.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.173+0000 7f4c677fe640 1 -- 172.21.15.38:0/233217355 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c6800f6f0 con 0x7f4c70102ef0 2024-09-16T00:32:32.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.173+0000 7f4c787a0640 1 -- 172.21.15.38:0/233217355 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c701a3f60 con 0x7f4c70102ef0 2024-09-16T00:32:32.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.174+0000 7f4c787a0640 1 -- 172.21.15.38:0/233217355 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c701021b0 con 0x7f4c70102ef0 2024-09-16T00:32:32.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.176+0000 7f4c677fe640 1 -- 172.21.15.38:0/233217355 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f4c680051d0 con 0x7f4c70102ef0 2024-09-16T00:32:32.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.176+0000 7f4c677fe640 1 --2- 172.21.15.38:0/233217355 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f4c5c0a3240 0x7f4c5c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:32.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.176+0000 7f4c7653c640 1 --2- 172.21.15.38:0/233217355 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f4c5c0a3240 0x7f4c5c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:32.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.176+0000 7f4c677fe640 1 -- 172.21.15.38:0/233217355 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f4c680cfd80 con 0x7f4c70102ef0 2024-09-16T00:32:32.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.177+0000 7f4c7653c640 1 --2- 172.21.15.38:0/233217355 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f4c5c0a3240 0x7f4c5c0a5700 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c005ab0 tx=0x7f4c6c005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:32.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.180+0000 7f4c677fe640 1 -- 172.21.15.38:0/233217355 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4c68014030 con 0x7f4c70102ef0 2024-09-16T00:32:32.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.357+0000 7f4c787a0640 1 -- 172.21.15.38:0/233217355 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4c70109090 con 0x7f4c70102ef0 2024-09-16T00:32:32.359 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.359+0000 7f4c677fe640 1 -- 172.21.15.38:0/233217355 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24767 (secure 0 0 0) 0x7f4c6809c720 con 0x7f4c70102ef0 2024-09-16T00:32:32.359 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:32:32.359 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":92,"fsid":"983b2072-73c1-11ef-bceb-c7b262605968","created":"2024-09-16T00:22:34.897053+0000","modified":"2024-09-16T00:32:24.296695+0000","last_up_change":"2024-09-16T00:29:34.503134+0000","last_in_change":"2024-09-16T00:29:10.135656+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:25:41.931977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:30:22.741889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:30:25.042590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:30:31.554350+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3afc20e3-869f-43a9-a911-b0e8d07de86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6803","nonce":1622028921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6805","nonce":1622028921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6809","nonce":1622028921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6807","nonce":1622028921}]},"public_addr":"172.21.15.38:6803/1622028921","cluster_addr":"172.21.15.38:6805/1622028921","heartbeat_back_addr":"172.21.15.38:6809/1622028921","heartbeat_front_addr":"172.21.15.38:6807/1622028921","state":["exists","up"]},{"osd":1,"uuid":"6d9d7acb-45ed-4543-8afc-6433fc7f096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6801","nonce":3811690336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6803","nonce":3811690336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6807","nonce":3811690336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6805","nonce":3811690336}]},"public_addr":"172.21.15.73:6801/3811690336","cluster_addr":"172.21.15.73:6803/3811690336","heartbeat_back_addr":"172.21.15.73:6807/3811690336","heartbeat_front_addr":"172.21.15.73:6805/3811690336","state":["exists","up"]},{"osd":2,"uuid":"48690f35-99e9-4f45-9bee-487857caa8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6801","nonce":1754741655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6803","nonce":1754741655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6807","nonce":1754741655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6805","nonce":1754741655}]},"public_addr":"172.21.15.122:6801/1754741655","cluster_addr":"172.21.15.122:6803/1754741655","heartbeat_back_addr":"172.21.15.122:6807/1754741655","heartbeat_front_addr":"172.21.15.122:6805/1754741655","state":["exists","up"]},{"osd":3,"uuid":"28ebb4a0-92e7-4f57-82a5-b9b761eca269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6811","nonce":3523190079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6813","nonce":3523190079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6817","nonce":3523190079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6815","nonce":3523190079}]},"public_addr":"172.21.15.38:6811/3523190079","cluster_addr":"172.21.15.38:6813/3523190079","heartbeat_back_addr":"172.21.15.38:6817/3523190079","heartbeat_front_addr":"172.21.15.38:6815/3523190079","state":["exists","up"]},{"osd":4,"uuid":"5ea363f7-7ad8-45ed-b7e4-4b010fe7f8cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6809","nonce":1469927628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6811","nonce":1469927628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6815","nonce":1469927628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6813","nonce":1469927628}]},"public_addr":"172.21.15.73:6809/1469927628","cluster_addr":"172.21.15.73:6811/1469927628","heartbeat_back_addr":"172.21.15.73:6815/1469927628","heartbeat_front_addr":"172.21.15.73:6813/1469927628","state":["exists","up"]},{"osd":5,"uuid":"e3c66d43-dd28-4bb1-a886-62a869be82cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6809","nonce":2954413058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6811","nonce":2954413058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6815","nonce":2954413058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6813","nonce":2954413058}]},"public_addr":"172.21.15.122:6809/2954413058","cluster_addr":"172.21.15.122:6811/2954413058","heartbeat_back_addr":"172.21.15.122:6815/2954413058","heartbeat_front_addr":"172.21.15.122:6813/2954413058","state":["exists","up"]},{"osd":6,"uuid":"211497ac-7c48-4095-bfdb-1d4df0149f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6819","nonce":1415719550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6821","nonce":1415719550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6825","nonce":1415719550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6823","nonce":1415719550}]},"public_addr":"172.21.15.38:6819/1415719550","cluster_addr":"172.21.15.38:6821/1415719550","heartbeat_back_addr":"172.21.15.38:6825/1415719550","heartbeat_front_addr":"172.21.15.38:6823/1415719550","state":["exists","up"]},{"osd":7,"uuid":"06512315-3f5e-44f8-976d-d3f5f1ba00e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6817","nonce":2634361473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6819","nonce":2634361473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6823","nonce":2634361473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6821","nonce":2634361473}]},"public_addr":"172.21.15.73:6817/2634361473","cluster_addr":"172.21.15.73:6819/2634361473","heartbeat_back_addr":"172.21.15.73:6823/2634361473","heartbeat_front_addr":"172.21.15.73:6821/2634361473","state":["exists","up"]},{"osd":8,"uuid":"db703c0a-1f87-46f0-be5f-f826f7f9daac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6817","nonce":1800567633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6819","nonce":1800567633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6823","nonce":1800567633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6821","nonce":1800567633}]},"public_addr":"172.21.15.122:6817/1800567633","cluster_addr":"172.21.15.122:6819/1800567633","heartbeat_back_addr":"172.21.15.122:6823/1800567633","heartbeat_front_addr":"172.21.15.122:6821/1800567633","state":["exists","up"]},{"osd":9,"uuid":"f5384200-b3b4-4936-85e1-fc57a9fe2bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6827","nonce":445727833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6829","nonce":445727833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6833","nonce":445727833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6831","nonce":445727833}]},"public_addr":"172.21.15.38:6827/445727833","cluster_addr":"172.21.15.38:6829/445727833","heartbeat_back_addr":"172.21.15.38:6833/445727833","heartbeat_front_addr":"172.21.15.38:6831/445727833","state":["exists","up"]},{"osd":10,"uuid":"c22883d9-6672-4796-a961-963d21eb37fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6825","nonce":43297139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6827","nonce":43297139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6831","nonce":43297139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6829","nonce":43297139}]},"public_addr":"172.21.15.73:6825/43297139","cluster_addr":"172.21.15.73:6827/43297139","heartbeat_back_addr":"172.21.15.73:6831/43297139","heartbeat_front_addr":"172.21.15.73:6829/43297139","state":["exists","up"]},{"osd":11,"uuid":"9f98e8cb-1bff-4b8b-aa19-56b691387383","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6825","nonce":1886446061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6827","nonce":1886446061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6831","nonce":1886446061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6829","nonce":1886446061}]},"public_addr":"172.21.15.122:6825/1886446061","cluster_addr":"172.21.15.122:6827/1886446061","heartbeat_back_addr":"172.21.15.122:6831/1886446061","heartbeat_front_addr":"172.21.15.122:6829/1886446061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:47.384012+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:12.834488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:38.770055+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:04.800466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:30.808076+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:56.646967+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:27:22.721270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:27:48.510011+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:28:13.922944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:28:40.514947+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:29:06.031052+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:29:31.882062+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6801/1779324916":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/924150447":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/141237849":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/2302081750":"2024-09-17T00:22:56.844212+0000","172.21.15.38:0/2245468594":"2024-09-17T00:22:56.844212+0000","172.21.15.38:6801/583006373":"2024-09-17T00:22:56.844212+0000","172.21.15.38:0/2435492667":"2024-09-17T00:23:21.417483+0000","172.21.15.38:6800/583006373":"2024-09-17T00:22:56.844212+0000","172.21.15.38:6800/1779324916":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/1103516837":"2024-09-17T00:22:56.844212+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:32:32.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.361+0000 7f4c787a0640 1 -- 172.21.15.38:0/233217355 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f4c5c0a3240 msgr2=0x7f4c5c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:32.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.361+0000 7f4c787a0640 1 --2- 172.21.15.38:0/233217355 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f4c5c0a3240 0x7f4c5c0a5700 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f4c6c005ab0 tx=0x7f4c6c005a20 comp rx=0 tx=0).stop 2024-09-16T00:32:32.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.361+0000 7f4c787a0640 1 -- 172.21.15.38:0/233217355 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c70102ef0 msgr2=0x7f4c7019bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:32.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.361+0000 7f4c787a0640 1 --2- 172.21.15.38:0/233217355 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c70102ef0 0x7f4c7019bcb0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f4c6800cad0 tx=0x7f4c6800cf90 comp rx=0 tx=0).stop 2024-09-16T00:32:32.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.361+0000 7f4c787a0640 1 -- 172.21.15.38:0/233217355 shutdown_connections 2024-09-16T00:32:32.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.361+0000 7f4c787a0640 1 --2- 172.21.15.38:0/233217355 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f4c5c0a3240 0x7f4c5c0a5700 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:32.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.361+0000 7f4c787a0640 1 --2- 172.21.15.38:0/233217355 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4c70102ef0 0x7f4c7019bcb0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:32.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.361+0000 7f4c787a0640 1 --2- 172.21.15.38:0/233217355 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4c70102570 0x7f4c70197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:32.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.361+0000 7f4c787a0640 1 --2- 172.21.15.38:0/233217355 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c70100e20 0x7f4c701973e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:32.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.361+0000 7f4c787a0640 1 -- 172.21.15.38:0/233217355 >> 172.21.15.38:0/233217355 conn(0x7f4c700fc7d0 msgr2=0x7f4c700fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:32.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.361+0000 7f4c787a0640 1 -- 172.21.15.38:0/233217355 shutdown_connections 2024-09-16T00:32:32.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:32.361+0000 7f4c787a0640 1 -- 172.21.15.38:0/233217355 wait complete. 2024-09-16T00:32:33.043 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:32 smithi038 ceph-mon[30945]: pgmap v357: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 496 B/s wr, 6 op/s 2024-09-16T00:32:33.043 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:32 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/233217355' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:32:33.073 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-16T00:30:26.764565+0000', 'modified': '2024-09-16T00:31:05.083019+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24494, 'mds_1': 24464, 'mds_2': 24440}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24416': {'gid': 24416, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.73:6833/1210328177', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6832', 'nonce': 1210328177}, {'type': 'v1', 'addr': '172.21.15.73:6833', 'nonce': 1210328177}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24421': {'gid': 24421, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.122:6837/1858299680', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6836', 'nonce': 1858299680}, {'type': 'v1', 'addr': '172.21.15.122:6837', 'nonce': 1858299680}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.122:6839/1983975311', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6838', 'nonce': 1983975311}, {'type': 'v1', 'addr': '172.21.15.122:6839', 'nonce': 1983975311}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'e', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.73:6835/2866363113', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6834', 'nonce': 2866363113}, {'type': 'v1', 'addr': '172.21.15.73:6835', 'nonce': 2866363113}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24464': {'gid': 24464, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.73:6837/2681081265', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6836', 'nonce': 2681081265}, {'type': 'v1', 'addr': '172.21.15.73:6837', 'nonce': 2681081265}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.73:6839/2396224525', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6838', 'nonce': 2396224525}, {'type': 'v1', 'addr': '172.21.15.73:6839', 'nonce': 2396224525}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24494, 'qdb_cluster': [24440, 24464, 24494]} 2024-09-16T00:32:33.073 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-16T00:32:33.073 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.e status 2024-09-16T00:32:33.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:32 smithi122 ceph-mon[35412]: pgmap v357: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 496 B/s wr, 6 op/s 2024-09-16T00:32:33.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:32 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/233217355' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:32:33.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:32 smithi073 ceph-mon[35254]: pgmap v357: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 496 B/s wr, 6 op/s 2024-09-16T00:32:33.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:32 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/233217355' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:32:33.336 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:33.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.805+0000 7f34d5629640 1 -- 172.21.15.38:0/3107988721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34d0102ef0 msgr2=0x7f34d010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:33.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.805+0000 7f34d5629640 1 --2- 172.21.15.38:0/3107988721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34d0102ef0 0x7f34d010da40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f34c8009900 tx=0x7f34c802a480 comp rx=0 tx=0).stop 2024-09-16T00:32:33.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.805+0000 7f34d5629640 1 -- 172.21.15.38:0/3107988721 shutdown_connections 2024-09-16T00:32:33.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.805+0000 7f34d5629640 1 --2- 172.21.15.38:0/3107988721 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34d0102ef0 0x7f34d010da40 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:33.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.805+0000 7f34d5629640 1 --2- 172.21.15.38:0/3107988721 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34d0102570 0x7f34d01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:33.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.805+0000 7f34d5629640 1 --2- 172.21.15.38:0/3107988721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34d0100e20 0x7f34d0101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:33.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.805+0000 7f34d5629640 1 -- 172.21.15.38:0/3107988721 >> 172.21.15.38:0/3107988721 conn(0x7f34d00fc7d0 msgr2=0x7f34d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:33.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.805+0000 7f34d5629640 1 -- 172.21.15.38:0/3107988721 shutdown_connections 2024-09-16T00:32:33.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.805+0000 7f34d5629640 1 -- 172.21.15.38:0/3107988721 wait complete. 2024-09-16T00:32:33.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.805+0000 7f34d5629640 1 Processor -- start 2024-09-16T00:32:33.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.806+0000 7f34d5629640 1 -- start start 2024-09-16T00:32:33.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.806+0000 7f34d5629640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34d0100e20 0x7f34d0197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:33.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.806+0000 7f34d5629640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34d0102570 0x7f34d01978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:33.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.806+0000 7f34d5629640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34d0102ef0 0x7f34d019bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:33.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.806+0000 7f34d5629640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f34d0110f80 con 0x7f34d0100e20 2024-09-16T00:32:33.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.806+0000 7f34d5629640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f34d0110e00 con 0x7f34d0102ef0 2024-09-16T00:32:33.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.806+0000 7f34d5629640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f34d0111100 con 0x7f34d0102570 2024-09-16T00:32:33.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.806+0000 7f34ceffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34d0100e20 0x7f34d0197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:33.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.806+0000 7f34ceffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34d0100e20 0x7f34d0197360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53788/0 (socket says 172.21.15.38:53788) 2024-09-16T00:32:33.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.806+0000 7f34ceffd640 1 -- 172.21.15.38:0/1380656465 learned_addr learned my addr 172.21.15.38:0/1380656465 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:33.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.806+0000 7f34ce7fc640 1 --2- 172.21.15.38:0/1380656465 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34d0102570 0x7f34d01978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:33.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.806+0000 7f34cf7fe640 1 --2- 172.21.15.38:0/1380656465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34d0102ef0 0x7f34d019bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:33.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.806+0000 7f34ceffd640 1 -- 172.21.15.38:0/1380656465 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34d0102570 msgr2=0x7f34d01978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:33.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.806+0000 7f34ceffd640 1 --2- 172.21.15.38:0/1380656465 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34d0102570 0x7f34d01978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:33.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.806+0000 7f34ceffd640 1 -- 172.21.15.38:0/1380656465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34d0102ef0 msgr2=0x7f34d019bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:33.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.806+0000 7f34ceffd640 1 --2- 172.21.15.38:0/1380656465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34d0102ef0 0x7f34d019bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:33.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.806+0000 7f34ceffd640 1 -- 172.21.15.38:0/1380656465 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34d019c310 con 0x7f34d0100e20 2024-09-16T00:32:33.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.806+0000 7f34cf7fe640 1 --2- 172.21.15.38:0/1380656465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34d0102ef0 0x7f34d019bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:32:33.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.806+0000 7f34ceffd640 1 --2- 172.21.15.38:0/1380656465 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34d0100e20 0x7f34d0197360 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f34c000eab0 tx=0x7f34c000ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:33.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.807+0000 7f34b7fff640 1 -- 172.21.15.38:0/1380656465 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c000ce40 con 0x7f34d0100e20 2024-09-16T00:32:33.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.807+0000 7f34d5629640 1 -- 172.21.15.38:0/1380656465 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34d019c600 con 0x7f34d0100e20 2024-09-16T00:32:33.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.807+0000 7f34b7fff640 1 -- 172.21.15.38:0/1380656465 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34c0011950 con 0x7f34d0100e20 2024-09-16T00:32:33.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.807+0000 7f34b7fff640 1 -- 172.21.15.38:0/1380656465 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c0010ae0 con 0x7f34d0100e20 2024-09-16T00:32:33.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.807+0000 7f34d5629640 1 -- 172.21.15.38:0/1380656465 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34d01a3ee0 con 0x7f34d0100e20 2024-09-16T00:32:33.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.809+0000 7f34b7fff640 1 -- 172.21.15.38:0/1380656465 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f34c0011b00 con 0x7f34d0100e20 2024-09-16T00:32:33.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.809+0000 7f34d7a02740 1 Processor -- start 2024-09-16T00:32:33.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.809+0000 7f34d7a02740 1 -- start start 2024-09-16T00:32:33.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.810+0000 7f34d7a02740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619e911d9d0 0x5619e911ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:33.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.810+0000 7f34d7a02740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5619e911e2f0 0x5619e9128670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:33.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.810+0000 7f34d7a02740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619e9128bb0 0x5619e912afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:33.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.810+0000 7f34d7a02740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5619e9084090 con 0x5619e911e2f0 2024-09-16T00:32:33.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.810+0000 7f34d7a02740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5619e902ba70 con 0x5619e9128bb0 2024-09-16T00:32:33.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.810+0000 7f34d7a02740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5619e9080050 con 0x5619e911d9d0 2024-09-16T00:32:33.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.810+0000 7f34ceffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5619e911e2f0 0x5619e9128670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:33.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.810+0000 7f34cf7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619e911d9d0 0x5619e911ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:33.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.810+0000 7f34ceffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5619e911e2f0 0x5619e9128670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53804/0 (socket says 172.21.15.38:53804) 2024-09-16T00:32:33.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.810+0000 7f34cf7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619e911d9d0 0x5619e911ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.38:38524/0 (socket says 172.21.15.38:38524) 2024-09-16T00:32:33.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.810+0000 7f34ceffd640 1 -- 172.21.15.38:0/2950936526 learned_addr learned my addr 172.21.15.38:0/2950936526 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:33.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.810+0000 7f34ce7fc640 1 --2- 172.21.15.38:0/2950936526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619e9128bb0 0x5619e912afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:33.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.810+0000 7f34b7fff640 1 --2- 172.21.15.38:0/1380656465 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f34b80a3240 0x7f34b80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:33.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.810+0000 7f34ceffd640 1 -- 172.21.15.38:0/2950936526 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619e911d9d0 msgr2=0x5619e911ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:33.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.810+0000 7f34ceffd640 1 --2- 172.21.15.38:0/2950936526 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619e911d9d0 0x5619e911ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:33.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.810+0000 7f34ceffd640 1 -- 172.21.15.38:0/2950936526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619e9128bb0 msgr2=0x5619e912afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:33.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.810+0000 7f34ceffd640 1 --2- 172.21.15.38:0/2950936526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619e9128bb0 0x5619e912afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:33.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.810+0000 7f34ceffd640 1 -- 172.21.15.38:0/2950936526 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619e9038130 con 0x5619e911e2f0 2024-09-16T00:32:33.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.810+0000 7f34cf7fe640 1 --2- 172.21.15.38:0/1380656465 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f34b80a3240 0x7f34b80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:33.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.810+0000 7f34b7fff640 1 -- 172.21.15.38:0/1380656465 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f34c00d04d0 con 0x7f34d0100e20 2024-09-16T00:32:33.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.811+0000 7f34ceffd640 1 --2- 172.21.15.38:0/2950936526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5619e911e2f0 0x5619e9128670 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f34c009c6a0 tx=0x7f34c005c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fded2105be7deac server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:33.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.811+0000 7f34cf7fe640 1 --2- 172.21.15.38:0/1380656465 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f34b80a3240 0x7f34b80a5700 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f34c802a9a0 tx=0x7f34c8009260 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:33.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.811+0000 7f34b57fa640 1 -- 172.21.15.38:0/2950936526 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c009ce10 con 0x5619e911e2f0 2024-09-16T00:32:33.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.811+0000 7f34b57fa640 1 -- 172.21.15.38:0/2950936526 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34c009c710 con 0x5619e911e2f0 2024-09-16T00:32:33.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.811+0000 7f34b57fa640 1 -- 172.21.15.38:0/2950936526 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c00a5380 con 0x5619e911e2f0 2024-09-16T00:32:33.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.811+0000 7f34d7a02740 1 -- 172.21.15.38:0/2950936526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5619e911e2f0 msgr2=0x5619e9128670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:33.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.811+0000 7f34d7a02740 1 --2- 172.21.15.38:0/2950936526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5619e911e2f0 0x5619e9128670 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f34c009c6a0 tx=0x7f34c005c540 comp rx=0 tx=0).stop 2024-09-16T00:32:33.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.812+0000 7f34d7a02740 1 -- 172.21.15.38:0/2950936526 shutdown_connections 2024-09-16T00:32:33.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.812+0000 7f34d7a02740 1 --2- 172.21.15.38:0/2950936526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619e9128bb0 0x5619e912afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:33.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.812+0000 7f34d7a02740 1 --2- 172.21.15.38:0/2950936526 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5619e911e2f0 0x5619e9128670 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:33.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.812+0000 7f34d7a02740 1 --2- 172.21.15.38:0/2950936526 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619e911d9d0 0x5619e911ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:33.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.812+0000 7f34d7a02740 1 -- 172.21.15.38:0/2950936526 >> 172.21.15.38:0/2950936526 conn(0x5619e9119f60 msgr2=0x5619e911a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:33.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.812+0000 7f34d7a02740 1 -- 172.21.15.38:0/2950936526 shutdown_connections 2024-09-16T00:32:33.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.812+0000 7f34d7a02740 1 -- 172.21.15.38:0/2950936526 wait complete. 2024-09-16T00:32:33.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.812+0000 7f34d7a02740 1 -- start start 2024-09-16T00:32:33.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.812+0000 7f34d7a02740 1 Processor -- start 2024-09-16T00:32:33.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.812+0000 7f34d7a02740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619e911d9d0 0x5619e9128430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:33.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.812+0000 7f34d7a02740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619e911e2f0 0x5619e9125780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:33.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.812+0000 7f34d7a02740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5619e9128bb0 0x5619e9125cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:33.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.812+0000 7f34d7a02740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5619e912b720 con 0x5619e9128bb0 2024-09-16T00:32:33.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.812+0000 7f34d7a02740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5619e903ef20 con 0x5619e911d9d0 2024-09-16T00:32:33.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.812+0000 7f34d7a02740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5619e912b8a0 con 0x5619e911e2f0 2024-09-16T00:32:33.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.812+0000 7f34d7a02740 10 client.0 _ll_register_callbacks cb 0x5619e9008030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:33.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.812+0000 7f34cf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5619e9128bb0 0x5619e9125cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:33.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.812+0000 7f34cf7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5619e9128bb0 0x5619e9125cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53812/0 (socket says 172.21.15.38:53812) 2024-09-16T00:32:33.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.812+0000 7f34cf7fe640 1 -- 172.21.15.38:0/366821130 learned_addr learned my addr 172.21.15.38:0/366821130 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:33.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.813+0000 7f34ceffd640 1 --2- 172.21.15.38:0/366821130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619e911d9d0 0x5619e9128430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:33.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.813+0000 7f34ce7fc640 1 --2- 172.21.15.38:0/366821130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619e911e2f0 0x5619e9125780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:33.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.813+0000 7f34cf7fe640 1 -- 172.21.15.38:0/366821130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619e911e2f0 msgr2=0x5619e9125780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:33.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.813+0000 7f34cf7fe640 1 --2- 172.21.15.38:0/366821130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619e911e2f0 0x5619e9125780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:33.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.813+0000 7f34cf7fe640 1 -- 172.21.15.38:0/366821130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619e911d9d0 msgr2=0x5619e9128430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:33.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.813+0000 7f34cf7fe640 1 --2- 172.21.15.38:0/366821130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619e911d9d0 0x5619e9128430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:33.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.813+0000 7f34cf7fe640 1 -- 172.21.15.38:0/366821130 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619e8df52b0 con 0x5619e9128bb0 2024-09-16T00:32:33.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.813+0000 7f34ce7fc640 1 --2- 172.21.15.38:0/366821130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619e911e2f0 0x5619e9125780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:33.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.813+0000 7f34cf7fe640 1 --2- 172.21.15.38:0/366821130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5619e9128bb0 0x5619e9125cc0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f34c80052c0 tx=0x7f34c8031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a44dcd474ea482d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:33.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.813+0000 7f34b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:32:33.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.813+0000 7f34b2ff5640 1 -- 172.21.15.38:0/366821130 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c8035040 con 0x5619e9128bb0 2024-09-16T00:32:33.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.813+0000 7f34b2ff5640 1 -- 172.21.15.38:0/366821130 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f34c8004d50 con 0x5619e9128bb0 2024-09-16T00:32:33.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.813+0000 7f34d7a02740 1 -- 172.21.15.38:0/366821130 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34c009c710 con 0x5619e9128bb0 2024-09-16T00:32:33.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.813+0000 7f34b2ff5640 1 -- 172.21.15.38:0/366821130 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f34c80035f0 con 0x5619e9128bb0 2024-09-16T00:32:33.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.813+0000 7f34d7a02740 10 client.15177 fetch_fsmap learned FSMap version 25 2024-09-16T00:32:33.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.813+0000 7f34d7a02740 1 -- 172.21.15.38:0/366821130 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5619e9137030 con 0x5619e9128bb0 2024-09-16T00:32:33.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.813+0000 7f34b2ff5640 1 -- 172.21.15.38:0/366821130 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34c009c710 con 0x5619e9128bb0 2024-09-16T00:32:33.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.814+0000 7f34b2ff5640 1 -- 172.21.15.38:0/366821130 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f34c8031990 con 0x5619e9128bb0 2024-09-16T00:32:33.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.814+0000 7f34d7a02740 10 client.15177 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:32:33.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.814+0000 7f34d7a02740 10 client.15177 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-16T00:32:33.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.814+0000 7f34d7a02740 20 client.15177 populate_metadata read hostname 'smithi038' 2024-09-16T00:32:33.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.814+0000 7f34d7a02740 1 --2- 172.21.15.38:0/366821130 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x5619e9139730 0x5619e913aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:33.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.814+0000 7f34d7a02740 4 client.15177 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:32:33.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.814+0000 7f34d7a02740 1 -- 172.21.15.38:0/366821130 --> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5619e9091600 con 0x5619e9139730 2024-09-16T00:32:33.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.814+0000 7f34ceffd640 1 --2- 172.21.15.38:0/366821130 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x5619e9139730 0x5619e913aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:33.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.814+0000 7f34b2ff5640 10 client.15177 ms_handle_connect on v2:172.21.15.73:6834/2866363113 2024-09-16T00:32:33.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.814+0000 7f34ceffd640 1 --2- 172.21.15.38:0/366821130 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x5619e9139730 0x5619e913aba0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5db52e36cd0a7ec server_cookie=7e335d04c3743c24 in_seq=0 out_seq=0 2024-09-16T00:32:33.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.815+0000 7f34b2ff5640 1 -- 172.21.15.38:0/366821130 <== mds.2 v2:172.21.15.73:6834/2866363113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5619e9091600 con 0x5619e9139730 2024-09-16T00:32:33.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.815+0000 7f34b2ff5640 10 client.15177 handle_command_reply: tid=0 2024-09-16T00:32:33.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.815+0000 7f34d7a02740 1 client.15177 shutdown 2024-09-16T00:32:33.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.815+0000 7f34d7a02740 1 -- 172.21.15.38:0/366821130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5619e9128bb0 msgr2=0x5619e9125cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:33.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.815+0000 7f34d7a02740 1 --2- 172.21.15.38:0/366821130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5619e9128bb0 0x5619e9125cc0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f34c80052c0 tx=0x7f34c8031400 comp rx=0 tx=0).stop 2024-09-16T00:32:33.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.815+0000 7f34d7a02740 1 -- 172.21.15.38:0/366821130 shutdown_connections 2024-09-16T00:32:33.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.815+0000 7f34d7a02740 1 --2- 172.21.15.38:0/366821130 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x5619e9139730 0x5619e913aba0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:33.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.815+0000 7f34d7a02740 1 --2- 172.21.15.38:0/366821130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5619e9128bb0 0x5619e9125cc0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:33.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.815+0000 7f34d7a02740 1 --2- 172.21.15.38:0/366821130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619e911e2f0 0x5619e9125780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:33.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.815+0000 7f34d7a02740 1 --2- 172.21.15.38:0/366821130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619e911d9d0 0x5619e9128430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:33.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.815+0000 7f34b2ff5640 0 client.15177 ms_handle_reset on v2:172.21.15.73:6834/2866363113 2024-09-16T00:32:33.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.815+0000 7f34d7a02740 1 -- 172.21.15.38:0/366821130 >> 172.21.15.38:0/366821130 conn(0x5619e9119f60 msgr2=0x5619e911fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:33.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.816+0000 7f34d7a02740 1 -- 172.21.15.38:0/366821130 shutdown_connections 2024-09-16T00:32:33.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.816+0000 7f34d7a02740 1 -- 172.21.15.38:0/366821130 wait complete. 2024-09-16T00:32:33.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.816+0000 7f34d7a02740 20 client.15177 trim_cache size 0 max 16384 2024-09-16T00:32:33.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.834+0000 7f34d7a02740 1 Processor -- start 2024-09-16T00:32:33.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.834+0000 7f34d7a02740 1 -- start start 2024-09-16T00:32:33.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.834+0000 7f34d7a02740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619e911d9d0 0x5619e913aba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:33.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.834+0000 7f34d7a02740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5619e911e2f0 0x5619e9125740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:33.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.834+0000 7f34d7a02740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619e9128bb0 0x5619e9125c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:33.845 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.834+0000 7f34d7a02740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5619e9138ee0 con 0x5619e911e2f0 2024-09-16T00:32:33.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.834+0000 7f34d7a02740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5619e91395b0 con 0x5619e9128bb0 2024-09-16T00:32:33.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.834+0000 7f34d7a02740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5619e9139c80 con 0x5619e911d9d0 2024-09-16T00:32:33.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.834+0000 7f34ce7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5619e911e2f0 0x5619e9125740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:33.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.834+0000 7f34ce7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5619e911e2f0 0x5619e9125740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53814/0 (socket says 172.21.15.38:53814) 2024-09-16T00:32:33.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.834+0000 7f34ce7fc640 1 -- 172.21.15.38:0/737810022 learned_addr learned my addr 172.21.15.38:0/737810022 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:33.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.835+0000 7f34ceffd640 1 --2- 172.21.15.38:0/737810022 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619e911d9d0 0x5619e913aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:33.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.835+0000 7f34cf7fe640 1 --2- 172.21.15.38:0/737810022 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619e9128bb0 0x5619e9125c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:33.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.835+0000 7f34ce7fc640 1 -- 172.21.15.38:0/737810022 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619e911d9d0 msgr2=0x5619e913aba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:33.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.835+0000 7f34ce7fc640 1 --2- 172.21.15.38:0/737810022 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619e911d9d0 0x5619e913aba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:33.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.835+0000 7f34ce7fc640 1 -- 172.21.15.38:0/737810022 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619e9128bb0 msgr2=0x5619e9125c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:33.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.835+0000 7f34ce7fc640 1 --2- 172.21.15.38:0/737810022 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619e9128bb0 0x5619e9125c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:33.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.835+0000 7f34ce7fc640 1 -- 172.21.15.38:0/737810022 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5619e911e120 con 0x5619e911e2f0 2024-09-16T00:32:33.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.835+0000 7f34cf7fe640 1 --2- 172.21.15.38:0/737810022 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619e9128bb0 0x5619e9125c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:32:33.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.835+0000 7f34ceffd640 1 --2- 172.21.15.38:0/737810022 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619e911d9d0 0x5619e913aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:32:33.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.835+0000 7f34ce7fc640 1 --2- 172.21.15.38:0/737810022 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5619e911e2f0 0x5619e9125740 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f34c40058c0 tx=0x7f34c4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fce03833d59c133 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:33.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.835+0000 7f34b4ff9640 1 -- 172.21.15.38:0/737810022 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c401e070 con 0x5619e911e2f0 2024-09-16T00:32:33.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.835+0000 7f34b4ff9640 1 -- 172.21.15.38:0/737810022 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34c40035f0 con 0x5619e911e2f0 2024-09-16T00:32:33.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.835+0000 7f34b4ff9640 1 -- 172.21.15.38:0/737810022 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c401a650 con 0x5619e911e2f0 2024-09-16T00:32:33.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.836+0000 7f34d7a02740 1 -- 172.21.15.38:0/737810022 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5619e911e2f0 msgr2=0x5619e9125740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:33.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.836+0000 7f34d7a02740 1 --2- 172.21.15.38:0/737810022 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5619e911e2f0 0x5619e9125740 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f34c40058c0 tx=0x7f34c4011810 comp rx=0 tx=0).stop 2024-09-16T00:32:33.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.836+0000 7f34d7a02740 1 -- 172.21.15.38:0/737810022 shutdown_connections 2024-09-16T00:32:33.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.836+0000 7f34d7a02740 1 --2- 172.21.15.38:0/737810022 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619e9128bb0 0x5619e9125c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:33.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.836+0000 7f34d7a02740 1 --2- 172.21.15.38:0/737810022 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5619e911e2f0 0x5619e9125740 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:33.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.836+0000 7f34d7a02740 1 --2- 172.21.15.38:0/737810022 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619e911d9d0 0x5619e913aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:33.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.836+0000 7f34d7a02740 1 -- 172.21.15.38:0/737810022 >> 172.21.15.38:0/737810022 conn(0x5619e9119f60 msgr2=0x5619e9114ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:33.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.836+0000 7f34d7a02740 1 -- 172.21.15.38:0/737810022 shutdown_connections 2024-09-16T00:32:33.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.836+0000 7f34d7a02740 1 -- 172.21.15.38:0/737810022 wait complete. 2024-09-16T00:32:33.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.836+0000 7f34d7a02740 1 -- start start 2024-09-16T00:32:33.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.837+0000 7f34d7a02740 1 Processor -- start 2024-09-16T00:32:33.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.837+0000 7f34d7a02740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619e911d9d0 0x5619e9143090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:33.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.837+0000 7f34d7a02740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5619e911e2f0 0x5619e91435d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:33.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.837+0000 7f34d7a02740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619e9128bb0 0x5619e913c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:33.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.837+0000 7f34d7a02740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5619e913aa20 con 0x5619e911e2f0 2024-09-16T00:32:33.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.837+0000 7f34d7a02740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5619e913a350 con 0x5619e9128bb0 2024-09-16T00:32:33.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.837+0000 7f34d7a02740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5619e913b0f0 con 0x5619e911d9d0 2024-09-16T00:32:33.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.837+0000 7f34d7a02740 10 client.0 _ll_register_callbacks cb 0x7f34c8009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:33.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.837+0000 7f34ce7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5619e911e2f0 0x5619e91435d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:33.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.837+0000 7f34ce7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5619e911e2f0 0x5619e91435d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53830/0 (socket says 172.21.15.38:53830) 2024-09-16T00:32:33.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.837+0000 7f34ce7fc640 1 -- 172.21.15.38:0/124456829 learned_addr learned my addr 172.21.15.38:0/124456829 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:33.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.837+0000 7f34cf7fe640 1 --2- 172.21.15.38:0/124456829 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619e9128bb0 0x5619e913c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:33.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.837+0000 7f34ceffd640 1 --2- 172.21.15.38:0/124456829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619e911d9d0 0x5619e9143090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:33.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.837+0000 7f34ce7fc640 1 -- 172.21.15.38:0/124456829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619e911d9d0 msgr2=0x5619e9143090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:33.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.837+0000 7f34ce7fc640 1 --2- 172.21.15.38:0/124456829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619e911d9d0 0x5619e9143090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:33.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.837+0000 7f34ce7fc640 1 -- 172.21.15.38:0/124456829 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619e9128bb0 msgr2=0x5619e913c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:33.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.837+0000 7f34ce7fc640 1 --2- 172.21.15.38:0/124456829 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619e9128bb0 0x5619e913c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:33.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.837+0000 7f34ce7fc640 1 -- 172.21.15.38:0/124456829 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5619e913cbc0 con 0x5619e911e2f0 2024-09-16T00:32:33.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.837+0000 7f34ce7fc640 1 --2- 172.21.15.38:0/124456829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5619e911e2f0 0x5619e91435d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f34c4005cb0 tx=0x7f34c40039b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9780c73db130b8df server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:33.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.837+0000 7f34b2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:32:33.849 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.838+0000 7f34b2ff5640 1 -- 172.21.15.38:0/124456829 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34c401e040 con 0x5619e911e2f0 2024-09-16T00:32:33.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.838+0000 7f34d7a02740 1 -- 172.21.15.38:0/124456829 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34c40035f0 con 0x5619e911e2f0 2024-09-16T00:32:33.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.838+0000 7f34b2ff5640 1 -- 172.21.15.38:0/124456829 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f34c4005240 con 0x5619e911e2f0 2024-09-16T00:32:33.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.838+0000 7f34b2ff5640 1 -- 172.21.15.38:0/124456829 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f34c400c8e0 con 0x5619e911e2f0 2024-09-16T00:32:33.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.838+0000 7f34d7a02740 10 client.15183 fetch_fsmap learned FSMap version 25 2024-09-16T00:32:33.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.838+0000 7f34d7a02740 1 -- 172.21.15.38:0/124456829 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x5619e91a58a0 con 0x5619e911e2f0 2024-09-16T00:32:33.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.838+0000 7f34b2ff5640 1 -- 172.21.15.38:0/124456829 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f34c40035f0 con 0x5619e911e2f0 2024-09-16T00:32:33.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.838+0000 7f34b2ff5640 1 -- 172.21.15.38:0/124456829 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f34c402c030 con 0x5619e911e2f0 2024-09-16T00:32:33.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.838+0000 7f34d7a02740 10 client.15183 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:32:33.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.838+0000 7f34d7a02740 10 client.15183 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-16T00:32:33.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.838+0000 7f34d7a02740 20 client.15183 populate_metadata read hostname 'smithi038' 2024-09-16T00:32:33.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.838+0000 7f34d7a02740 1 --2- 172.21.15.38:0/124456829 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x5619e91aa0f0 0x5619e91a8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:33.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.838+0000 7f34d7a02740 4 client.15183 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:32:33.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.838+0000 7f34d7a02740 1 -- 172.21.15.38:0/124456829 --> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] -- command(tid 0: {"prefix": "status"}) -- 0x5619e908bb10 con 0x5619e91aa0f0 2024-09-16T00:32:33.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.839+0000 7f34ceffd640 1 --2- 172.21.15.38:0/124456829 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x5619e91aa0f0 0x5619e91a8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:33.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.839+0000 7f34b2ff5640 10 client.15183 ms_handle_connect on v2:172.21.15.73:6834/2866363113 2024-09-16T00:32:33.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.839+0000 7f34ceffd640 1 --2- 172.21.15.38:0/124456829 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x5619e91aa0f0 0x5619e91a8e00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3263de2b2b8ef06a server_cookie=5d41684fed4a7d61 in_seq=0 out_seq=0 2024-09-16T00:32:33.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.839+0000 7f34b2ff5640 1 -- 172.21.15.38:0/124456829 <== mds.2 v2:172.21.15.73:6834/2866363113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5619e908bb10 con 0x5619e91aa0f0 2024-09-16T00:32:33.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.839+0000 7f34b2ff5640 10 client.15183 handle_command_reply: tid=0 2024-09-16T00:32:33.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.839+0000 7f34d7a02740 1 client.15183 shutdown 2024-09-16T00:32:33.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.839+0000 7f34d7a02740 1 -- 172.21.15.38:0/124456829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5619e911e2f0 msgr2=0x5619e91435d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:33.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.840+0000 7f34d7a02740 1 --2- 172.21.15.38:0/124456829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5619e911e2f0 0x5619e91435d0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f34c4005cb0 tx=0x7f34c40039b0 comp rx=0 tx=0).stop 2024-09-16T00:32:33.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.840+0000 7f34d7a02740 1 -- 172.21.15.38:0/124456829 shutdown_connections 2024-09-16T00:32:33.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.840+0000 7f34d7a02740 1 --2- 172.21.15.38:0/124456829 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x5619e91aa0f0 0x5619e91a8e00 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:33.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.840+0000 7f34d7a02740 1 --2- 172.21.15.38:0/124456829 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5619e9128bb0 0x5619e913c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:33.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.840+0000 7f34d7a02740 1 --2- 172.21.15.38:0/124456829 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5619e911e2f0 0x5619e91435d0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:33.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.840+0000 7f34d7a02740 1 --2- 172.21.15.38:0/124456829 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5619e911d9d0 0x5619e9143090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:33.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.840+0000 7f34d7a02740 1 -- 172.21.15.38:0/124456829 >> 172.21.15.38:0/124456829 conn(0x5619e9119f60 msgr2=0x5619e911a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:33.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.840+0000 7f34b2ff5640 0 client.15183 ms_handle_reset on v2:172.21.15.73:6834/2866363113 2024-09-16T00:32:33.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.840+0000 7f34d7a02740 1 -- 172.21.15.38:0/124456829 shutdown_connections 2024-09-16T00:32:33.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.840+0000 7f34d7a02740 1 -- 172.21.15.38:0/124456829 wait complete. 2024-09-16T00:32:33.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.840+0000 7f34d7a02740 20 client.15183 trim_cache size 0 max 16384 2024-09-16T00:32:33.852 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-16T00:32:33.852 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "983b2072-73c1-11ef-bceb-c7b262605968", 2024-09-16T00:32:33.852 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 2, 2024-09-16T00:32:33.852 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24440, 2024-09-16T00:32:33.853 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-16T00:32:33.853 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-16T00:32:33.853 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-16T00:32:33.853 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 91.534802315999997, 2024-09-16T00:32:33.853 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 25, 2024-09-16T00:32:33.853 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 92, 2024-09-16T00:32:33.853 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 88, 2024-09-16T00:32:33.853 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 157.11530154100001 2024-09-16T00:32:33.853 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-16T00:32:33.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.840+0000 7f34d5629640 1 -- 172.21.15.38:0/1380656465 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f34b80a3240 msgr2=0x7f34b80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:33.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.840+0000 7f34d5629640 1 --2- 172.21.15.38:0/1380656465 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f34b80a3240 0x7f34b80a5700 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f34c802a9a0 tx=0x7f34c8009260 comp rx=0 tx=0).stop 2024-09-16T00:32:33.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.840+0000 7f34d5629640 1 -- 172.21.15.38:0/1380656465 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34d0100e20 msgr2=0x7f34d0197360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:33.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.840+0000 7f34d5629640 1 --2- 172.21.15.38:0/1380656465 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34d0100e20 0x7f34d0197360 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f34c000eab0 tx=0x7f34c000ef70 comp rx=0 tx=0).stop 2024-09-16T00:32:33.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.840+0000 7f34d5629640 1 -- 172.21.15.38:0/1380656465 shutdown_connections 2024-09-16T00:32:33.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.840+0000 7f34d5629640 1 --2- 172.21.15.38:0/1380656465 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f34b80a3240 0x7f34b80a5700 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:33.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.840+0000 7f34d5629640 1 --2- 172.21.15.38:0/1380656465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34d0102ef0 0x7f34d019bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:33.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.840+0000 7f34d5629640 1 --2- 172.21.15.38:0/1380656465 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34d0102570 0x7f34d01978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:33.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.840+0000 7f34d5629640 1 --2- 172.21.15.38:0/1380656465 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34d0100e20 0x7f34d0197360 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:33.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.840+0000 7f34d5629640 1 -- 172.21.15.38:0/1380656465 >> 172.21.15.38:0/1380656465 conn(0x7f34d00fc7d0 msgr2=0x7f34d0106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:33.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.840+0000 7f34d5629640 1 -- 172.21.15.38:0/1380656465 shutdown_connections 2024-09-16T00:32:33.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:33.841+0000 7f34d5629640 1 -- 172.21.15.38:0/1380656465 wait complete. 2024-09-16T00:32:34.571 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-16T00:32:34.829 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:35.099 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:34 smithi038 ceph-mon[30945]: pgmap v358: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 409 B/s wr, 6 op/s 2024-09-16T00:32:35.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:34 smithi122 ceph-mon[35412]: pgmap v358: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 409 B/s wr, 6 op/s 2024-09-16T00:32:35.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:34 smithi073 ceph-mon[35254]: pgmap v358: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 409 B/s wr, 6 op/s 2024-09-16T00:32:35.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.323+0000 7f562bf00640 1 -- 172.21.15.38:0/473080388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5624102ef0 msgr2=0x7f562410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:35.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.323+0000 7f562bf00640 1 --2- 172.21.15.38:0/473080388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5624102ef0 0x7f562410da40 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f561c009ca0 tx=0x7f561c02a950 comp rx=0 tx=0).stop 2024-09-16T00:32:35.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.323+0000 7f562bf00640 1 -- 172.21.15.38:0/473080388 shutdown_connections 2024-09-16T00:32:35.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.323+0000 7f562bf00640 1 --2- 172.21.15.38:0/473080388 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5624102ef0 0x7f562410da40 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:35.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.323+0000 7f562bf00640 1 --2- 172.21.15.38:0/473080388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5624102570 0x7f56241029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:35.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.323+0000 7f562bf00640 1 --2- 172.21.15.38:0/473080388 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5624100e20 0x7f5624101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:35.347 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.323+0000 7f562bf00640 1 -- 172.21.15.38:0/473080388 >> 172.21.15.38:0/473080388 conn(0x7f56240fc7d0 msgr2=0x7f56240febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:35.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.323+0000 7f562bf00640 1 -- 172.21.15.38:0/473080388 shutdown_connections 2024-09-16T00:32:35.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.323+0000 7f562bf00640 1 -- 172.21.15.38:0/473080388 wait complete. 2024-09-16T00:32:35.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.323+0000 7f562bf00640 1 Processor -- start 2024-09-16T00:32:35.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.324+0000 7f562bf00640 1 -- start start 2024-09-16T00:32:35.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.324+0000 7f562bf00640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5624100e20 0x7f56240fcc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:35.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.324+0000 7f562bf00640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5624102570 0x7f56240fd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:35.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.324+0000 7f562bf00640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5624102ef0 0x7f56240fd700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:35.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.324+0000 7f562bf00640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5624110f80 con 0x7f5624100e20 2024-09-16T00:32:35.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.324+0000 7f562bf00640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5624110e00 con 0x7f5624102570 2024-09-16T00:32:35.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.324+0000 7f562bf00640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5624111100 con 0x7f5624102ef0 2024-09-16T00:32:35.348 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.324+0000 7f5629c9c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5624100e20 0x7f56240fcc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:35.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.324+0000 7f5629c9c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5624100e20 0x7f56240fcc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53850/0 (socket says 172.21.15.38:53850) 2024-09-16T00:32:35.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.324+0000 7f5629c9c640 1 -- 172.21.15.38:0/950987602 learned_addr learned my addr 172.21.15.38:0/950987602 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:35.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.324+0000 7f562949b640 1 --2- 172.21.15.38:0/950987602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5624102570 0x7f56240fd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:35.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.324+0000 7f562a49d640 1 --2- 172.21.15.38:0/950987602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5624102ef0 0x7f56240fd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:35.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.324+0000 7f5629c9c640 1 -- 172.21.15.38:0/950987602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5624102ef0 msgr2=0x7f56240fd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:35.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.324+0000 7f5629c9c640 1 --2- 172.21.15.38:0/950987602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5624102ef0 0x7f56240fd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:35.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.324+0000 7f5629c9c640 1 -- 172.21.15.38:0/950987602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5624102570 msgr2=0x7f56240fd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:35.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.324+0000 7f5629c9c640 1 --2- 172.21.15.38:0/950987602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5624102570 0x7f56240fd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:35.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.324+0000 7f5629c9c640 1 -- 172.21.15.38:0/950987602 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56241a68d0 con 0x7f5624100e20 2024-09-16T00:32:35.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.324+0000 7f562a49d640 1 --2- 172.21.15.38:0/950987602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5624102ef0 0x7f56240fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:32:35.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.324+0000 7f562949b640 1 --2- 172.21.15.38:0/950987602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5624102570 0x7f56240fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:32:35.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.324+0000 7f5629c9c640 1 --2- 172.21.15.38:0/950987602 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5624100e20 0x7f56240fcc80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f562000d770 tx=0x7f562000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:35.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.325+0000 7f561affd640 1 -- 172.21.15.38:0/950987602 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5620017070 con 0x7f5624100e20 2024-09-16T00:32:35.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.325+0000 7f562bf00640 1 -- 172.21.15.38:0/950987602 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56241a6bc0 con 0x7f5624100e20 2024-09-16T00:32:35.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.325+0000 7f561affd640 1 -- 172.21.15.38:0/950987602 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5620005500 con 0x7f5624100e20 2024-09-16T00:32:35.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.325+0000 7f561affd640 1 -- 172.21.15.38:0/950987602 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5620004e00 con 0x7f5624100e20 2024-09-16T00:32:35.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.325+0000 7f562bf00640 1 -- 172.21.15.38:0/950987602 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56241a7100 con 0x7f5624100e20 2024-09-16T00:32:35.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.327+0000 7f562e2d9740 1 Processor -- start 2024-09-16T00:32:35.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.327+0000 7f562e2d9740 1 -- start start 2024-09-16T00:32:35.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.327+0000 7f562e2d9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd38f089d0 0x55cd38f08db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:35.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.328+0000 7f562e2d9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd38f092f0 0x55cd38f13670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:35.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.328+0000 7f562e2d9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cd38f13bb0 0x55cd38f15fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:35.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.328+0000 7f561affd640 1 -- 172.21.15.38:0/950987602 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f5620004fa0 con 0x7f5624100e20 2024-09-16T00:32:35.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.328+0000 7f562e2d9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55cd38e6f090 con 0x55cd38f089d0 2024-09-16T00:32:35.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.328+0000 7f562e2d9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55cd38e16a70 con 0x55cd38f13bb0 2024-09-16T00:32:35.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.328+0000 7f562e2d9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cd38e6b050 con 0x55cd38f092f0 2024-09-16T00:32:35.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.328+0000 7f562a49d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd38f089d0 0x55cd38f08db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:35.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.328+0000 7f562a49d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd38f089d0 0x55cd38f08db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53856/0 (socket says 172.21.15.38:53856) 2024-09-16T00:32:35.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.328+0000 7f562a49d640 1 -- 172.21.15.38:0/3595349198 learned_addr learned my addr 172.21.15.38:0/3595349198 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:35.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.328+0000 7f5629c9c640 1 --2- 172.21.15.38:0/3595349198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd38f092f0 0x55cd38f13670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:35.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.328+0000 7f562949b640 1 --2- 172.21.15.38:0/3595349198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cd38f13bb0 0x55cd38f15fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:35.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.328+0000 7f562a49d640 1 -- 172.21.15.38:0/3595349198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd38f092f0 msgr2=0x55cd38f13670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:35.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.328+0000 7f562a49d640 1 --2- 172.21.15.38:0/3595349198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd38f092f0 0x55cd38f13670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:35.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.328+0000 7f562a49d640 1 -- 172.21.15.38:0/3595349198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cd38f13bb0 msgr2=0x55cd38f15fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:35.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.328+0000 7f562a49d640 1 --2- 172.21.15.38:0/3595349198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cd38f13bb0 0x55cd38f15fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:35.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.328+0000 7f562a49d640 1 -- 172.21.15.38:0/3595349198 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd38e23130 con 0x55cd38f089d0 2024-09-16T00:32:35.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.328+0000 7f562949b640 1 --2- 172.21.15.38:0/3595349198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cd38f13bb0 0x55cd38f15fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:35.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.328+0000 7f562a49d640 1 --2- 172.21.15.38:0/3595349198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd38f089d0 0x55cd38f08db0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f561c009880 tx=0x7f561c0098b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=328b1df54c8c009c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:35.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.329+0000 7f561affd640 1 --2- 172.21.15.38:0/950987602 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f56100a32c0 0x7f56100a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:35.351 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.329+0000 7f560ffff640 1 -- 172.21.15.38:0/3595349198 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c034050 con 0x55cd38f089d0 2024-09-16T00:32:35.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.329+0000 7f560ffff640 1 -- 172.21.15.38:0/3595349198 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f561c02fbb0 con 0x55cd38f089d0 2024-09-16T00:32:35.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.329+0000 7f560ffff640 1 -- 172.21.15.38:0/3595349198 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c004500 con 0x55cd38f089d0 2024-09-16T00:32:35.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.329+0000 7f5629c9c640 1 --2- 172.21.15.38:0/950987602 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f56100a32c0 0x7f56100a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:35.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.329+0000 7f561affd640 1 -- 172.21.15.38:0/950987602 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f56200cf6f0 con 0x7f5624100e20 2024-09-16T00:32:35.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.330+0000 7f562e2d9740 1 -- 172.21.15.38:0/3595349198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd38f089d0 msgr2=0x55cd38f08db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:35.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.330+0000 7f5629c9c640 1 --2- 172.21.15.38:0/950987602 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f56100a32c0 0x7f56100a5780 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f56200cd1e0 tx=0x7f5620086040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:35.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.330+0000 7f562e2d9740 1 --2- 172.21.15.38:0/3595349198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd38f089d0 0x55cd38f08db0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f561c009880 tx=0x7f561c0098b0 comp rx=0 tx=0).stop 2024-09-16T00:32:35.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.330+0000 7f562e2d9740 1 -- 172.21.15.38:0/3595349198 shutdown_connections 2024-09-16T00:32:35.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.330+0000 7f562e2d9740 1 --2- 172.21.15.38:0/3595349198 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cd38f13bb0 0x55cd38f15fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:35.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.330+0000 7f562e2d9740 1 --2- 172.21.15.38:0/3595349198 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd38f092f0 0x55cd38f13670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:35.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.330+0000 7f562e2d9740 1 --2- 172.21.15.38:0/3595349198 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd38f089d0 0x55cd38f08db0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:35.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.330+0000 7f562e2d9740 1 -- 172.21.15.38:0/3595349198 >> 172.21.15.38:0/3595349198 conn(0x55cd38f04f60 msgr2=0x55cd38f05340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:35.352 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.330+0000 7f562e2d9740 1 -- 172.21.15.38:0/3595349198 shutdown_connections 2024-09-16T00:32:35.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.330+0000 7f562e2d9740 1 -- 172.21.15.38:0/3595349198 wait complete. 2024-09-16T00:32:35.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.330+0000 7f562e2d9740 1 -- start start 2024-09-16T00:32:35.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.330+0000 7f562e2d9740 1 Processor -- start 2024-09-16T00:32:35.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.331+0000 7f562e2d9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd38f089d0 0x55cd38f13400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:35.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.331+0000 7f562e2d9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cd38f092f0 0x55cd38f10750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:35.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.331+0000 7f562e2d9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd38f13bb0 0x55cd38f10c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:35.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.331+0000 7f562e2d9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55cd38f16780 con 0x55cd38f13bb0 2024-09-16T00:32:35.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.331+0000 7f562e2d9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55cd38e29f20 con 0x55cd38f092f0 2024-09-16T00:32:35.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.331+0000 7f562e2d9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cd38f16900 con 0x55cd38f089d0 2024-09-16T00:32:35.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.331+0000 7f562e2d9740 10 client.0 _ll_register_callbacks cb 0x55cd38df3030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:35.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.331+0000 7f562a49d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd38f13bb0 0x55cd38f10c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:35.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.331+0000 7f562a49d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd38f13bb0 0x55cd38f10c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53864/0 (socket says 172.21.15.38:53864) 2024-09-16T00:32:35.353 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.331+0000 7f562a49d640 1 -- 172.21.15.38:0/4014091592 learned_addr learned my addr 172.21.15.38:0/4014091592 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:35.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.331+0000 7f562949b640 1 --2- 172.21.15.38:0/4014091592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cd38f092f0 0x55cd38f10750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:35.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.331+0000 7f562a49d640 1 --2- 172.21.15.38:0/4014091592 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd38f089d0 0x55cd38f13400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:35.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.331+0000 7f562a49d640 1 -- 172.21.15.38:0/4014091592 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd38f089d0 msgr2=0x55cd38f13400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:35.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.331+0000 7f562a49d640 1 --2- 172.21.15.38:0/4014091592 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd38f089d0 0x55cd38f13400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:35.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.331+0000 7f562a49d640 1 -- 172.21.15.38:0/4014091592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cd38f092f0 msgr2=0x55cd38f10750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:35.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.331+0000 7f562a49d640 1 --2- 172.21.15.38:0/4014091592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cd38f092f0 0x55cd38f10750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:35.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.331+0000 7f562a49d640 1 -- 172.21.15.38:0/4014091592 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd38be02b0 con 0x55cd38f13bb0 2024-09-16T00:32:35.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.332+0000 7f562a49d640 1 --2- 172.21.15.38:0/4014091592 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd38f13bb0 0x55cd38f10c90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f561c009620 tx=0x7f561c0040e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e90b75043a34a056 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:35.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.332+0000 7f560d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:32:35.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.332+0000 7f560d7fa640 1 -- 172.21.15.38:0/4014091592 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c004280 con 0x55cd38f13bb0 2024-09-16T00:32:35.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.332+0000 7f560d7fa640 1 -- 172.21.15.38:0/4014091592 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f561c004420 con 0x55cd38f13bb0 2024-09-16T00:32:35.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.332+0000 7f562e2d9740 1 -- 172.21.15.38:0/4014091592 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f561c02fbb0 con 0x55cd38f13bb0 2024-09-16T00:32:35.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.332+0000 7f560d7fa640 1 -- 172.21.15.38:0/4014091592 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f561c05a050 con 0x55cd38f13bb0 2024-09-16T00:32:35.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.332+0000 7f562e2d9740 10 client.15195 fetch_fsmap learned FSMap version 25 2024-09-16T00:32:35.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.332+0000 7f562e2d9740 1 -- 172.21.15.38:0/4014091592 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cd38f16780 con 0x55cd38f13bb0 2024-09-16T00:32:35.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.332+0000 7f560d7fa640 1 -- 172.21.15.38:0/4014091592 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f561c02fbb0 con 0x55cd38f13bb0 2024-09-16T00:32:35.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.332+0000 7f560d7fa640 1 -- 172.21.15.38:0/4014091592 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f561c041070 con 0x55cd38f13bb0 2024-09-16T00:32:35.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.332+0000 7f562e2d9740 10 client.15195 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:32:35.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.332+0000 7f562e2d9740 10 client.15195 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-16T00:32:35.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.332+0000 7f562e2d9740 20 client.15195 populate_metadata read hostname 'smithi038' 2024-09-16T00:32:35.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.332+0000 7f562e2d9740 1 --2- 172.21.15.38:0/4014091592 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x55cd38f242a0 0x55cd38f25710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:35.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.332+0000 7f562e2d9740 4 client.15195 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:32:35.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.332+0000 7f562e2d9740 1 -- 172.21.15.38:0/4014091592 --> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cd38e7c600 con 0x55cd38f242a0 2024-09-16T00:32:35.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.333+0000 7f562949b640 1 --2- 172.21.15.38:0/4014091592 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x55cd38f242a0 0x55cd38f25710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:35.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.333+0000 7f560d7fa640 10 client.15195 ms_handle_connect on v2:172.21.15.73:6836/2681081265 2024-09-16T00:32:35.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.333+0000 7f562949b640 1 --2- 172.21.15.38:0/4014091592 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x55cd38f242a0 0x55cd38f25710 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f2d2b9bb85f86de9 server_cookie=759d8ef21fb2e8d7 in_seq=0 out_seq=0 2024-09-16T00:32:35.355 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.334+0000 7f560d7fa640 1 -- 172.21.15.38:0/4014091592 <== mds.1 v2:172.21.15.73:6836/2681081265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cd38e7c600 con 0x55cd38f242a0 2024-09-16T00:32:35.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.334+0000 7f560d7fa640 10 client.15195 handle_command_reply: tid=0 2024-09-16T00:32:35.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.334+0000 7f562e2d9740 1 client.15195 shutdown 2024-09-16T00:32:35.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.334+0000 7f562e2d9740 1 -- 172.21.15.38:0/4014091592 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd38f13bb0 msgr2=0x55cd38f10c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:35.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.334+0000 7f562e2d9740 1 --2- 172.21.15.38:0/4014091592 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd38f13bb0 0x55cd38f10c90 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f561c009620 tx=0x7f561c0040e0 comp rx=0 tx=0).stop 2024-09-16T00:32:35.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.334+0000 7f562e2d9740 1 -- 172.21.15.38:0/4014091592 shutdown_connections 2024-09-16T00:32:35.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.334+0000 7f562e2d9740 1 --2- 172.21.15.38:0/4014091592 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x55cd38f242a0 0x55cd38f25710 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:35.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.334+0000 7f562e2d9740 1 --2- 172.21.15.38:0/4014091592 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd38f13bb0 0x55cd38f10c90 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:35.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.334+0000 7f562e2d9740 1 --2- 172.21.15.38:0/4014091592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cd38f092f0 0x55cd38f10750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:35.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.334+0000 7f562e2d9740 1 --2- 172.21.15.38:0/4014091592 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd38f089d0 0x55cd38f13400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:35.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.334+0000 7f562e2d9740 1 -- 172.21.15.38:0/4014091592 >> 172.21.15.38:0/4014091592 conn(0x55cd38f04f60 msgr2=0x55cd38f0ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:35.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.334+0000 7f560d7fa640 0 client.15195 ms_handle_reset on v2:172.21.15.73:6836/2681081265 2024-09-16T00:32:35.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.334+0000 7f562e2d9740 1 -- 172.21.15.38:0/4014091592 shutdown_connections 2024-09-16T00:32:35.356 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.334+0000 7f562e2d9740 1 -- 172.21.15.38:0/4014091592 wait complete. 2024-09-16T00:32:35.357 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.334+0000 7f562e2d9740 20 client.15195 trim_cache size 0 max 16384 2024-09-16T00:32:35.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.353+0000 7f562e2d9740 1 Processor -- start 2024-09-16T00:32:35.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.353+0000 7f562e2d9740 1 -- start start 2024-09-16T00:32:35.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.353+0000 7f562e2d9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd38f089d0 0x55cd38f25710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:35.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.353+0000 7f562e2d9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cd38f092f0 0x55cd38effce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:35.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.353+0000 7f562e2d9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd38f13bb0 0x55cd38f0f020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:35.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.353+0000 7f562e2d9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55cd38f23a50 con 0x55cd38f089d0 2024-09-16T00:32:35.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.353+0000 7f562e2d9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55cd38f24120 con 0x55cd38f092f0 2024-09-16T00:32:35.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.353+0000 7f562e2d9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cd38f247f0 con 0x55cd38f13bb0 2024-09-16T00:32:35.361 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.353+0000 7f562a49d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd38f089d0 0x55cd38f25710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:35.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.353+0000 7f562a49d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd38f089d0 0x55cd38f25710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53866/0 (socket says 172.21.15.38:53866) 2024-09-16T00:32:35.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.353+0000 7f562a49d640 1 -- 172.21.15.38:0/1269065329 learned_addr learned my addr 172.21.15.38:0/1269065329 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:35.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.353+0000 7f562949b640 1 --2- 172.21.15.38:0/1269065329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cd38f092f0 0x55cd38effce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:35.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.354+0000 7f562a49d640 1 --2- 172.21.15.38:0/1269065329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd38f13bb0 0x55cd38f0f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:35.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.354+0000 7f562a49d640 1 -- 172.21.15.38:0/1269065329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd38f13bb0 msgr2=0x55cd38f0f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:35.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.354+0000 7f562a49d640 1 --2- 172.21.15.38:0/1269065329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd38f13bb0 0x55cd38f0f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:35.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.354+0000 7f562a49d640 1 -- 172.21.15.38:0/1269065329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cd38f092f0 msgr2=0x55cd38effce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:35.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.354+0000 7f562a49d640 1 --2- 172.21.15.38:0/1269065329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cd38f092f0 0x55cd38effce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:35.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.354+0000 7f562a49d640 1 -- 172.21.15.38:0/1269065329 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cd38f13020 con 0x55cd38f089d0 2024-09-16T00:32:35.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.354+0000 7f562949b640 1 --2- 172.21.15.38:0/1269065329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cd38f092f0 0x55cd38effce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:32:35.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.354+0000 7f562a49d640 1 --2- 172.21.15.38:0/1269065329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd38f089d0 0x55cd38f25710 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f561c058040 tx=0x7f561c02a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cddd82dcdc024e8a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:35.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.354+0000 7f560f7fe640 1 -- 172.21.15.38:0/1269065329 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c034040 con 0x55cd38f089d0 2024-09-16T00:32:35.362 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.354+0000 7f560f7fe640 1 -- 172.21.15.38:0/1269065329 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f561c049040 con 0x55cd38f089d0 2024-09-16T00:32:35.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.354+0000 7f560f7fe640 1 -- 172.21.15.38:0/1269065329 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c02fb70 con 0x55cd38f089d0 2024-09-16T00:32:35.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.355+0000 7f562e2d9740 1 -- 172.21.15.38:0/1269065329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd38f089d0 msgr2=0x55cd38f25710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:35.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.355+0000 7f562e2d9740 1 --2- 172.21.15.38:0/1269065329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd38f089d0 0x55cd38f25710 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f561c058040 tx=0x7f561c02a950 comp rx=0 tx=0).stop 2024-09-16T00:32:35.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.355+0000 7f562e2d9740 1 -- 172.21.15.38:0/1269065329 shutdown_connections 2024-09-16T00:32:35.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.355+0000 7f562e2d9740 1 --2- 172.21.15.38:0/1269065329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd38f13bb0 0x55cd38f0f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:35.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.355+0000 7f562e2d9740 1 --2- 172.21.15.38:0/1269065329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cd38f092f0 0x55cd38effce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:35.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.355+0000 7f562e2d9740 1 --2- 172.21.15.38:0/1269065329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd38f089d0 0x55cd38f25710 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:35.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.355+0000 7f562e2d9740 1 -- 172.21.15.38:0/1269065329 >> 172.21.15.38:0/1269065329 conn(0x55cd38f04f60 msgr2=0x55cd38f05340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:35.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.355+0000 7f562e2d9740 1 -- 172.21.15.38:0/1269065329 shutdown_connections 2024-09-16T00:32:35.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.355+0000 7f562e2d9740 1 -- 172.21.15.38:0/1269065329 wait complete. 2024-09-16T00:32:35.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.356+0000 7f562e2d9740 1 -- start start 2024-09-16T00:32:35.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.356+0000 7f562e2d9740 1 Processor -- start 2024-09-16T00:32:35.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.356+0000 7f562e2d9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd38f089d0 0x55cd38f2dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:35.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.356+0000 7f562e2d9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cd38f092f0 0x55cd38f2e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:35.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.356+0000 7f562e2d9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd38f13bb0 0x55cd38f27010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:35.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.356+0000 7f562e2d9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55cd38f25590 con 0x55cd38f13bb0 2024-09-16T00:32:35.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.356+0000 7f562e2d9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55cd38f24ec0 con 0x55cd38f092f0 2024-09-16T00:32:35.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.356+0000 7f562e2d9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cd38f25c60 con 0x55cd38f089d0 2024-09-16T00:32:35.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.356+0000 7f562e2d9740 10 client.0 _ll_register_callbacks cb 0x7f56200d5500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:35.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.356+0000 7f562a49d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd38f13bb0 0x55cd38f27010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:35.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.356+0000 7f562a49d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd38f13bb0 0x55cd38f27010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53880/0 (socket says 172.21.15.38:53880) 2024-09-16T00:32:35.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.356+0000 7f562a49d640 1 -- 172.21.15.38:0/4067609779 learned_addr learned my addr 172.21.15.38:0/4067609779 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:35.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.356+0000 7f562a49d640 1 --2- 172.21.15.38:0/4067609779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd38f089d0 0x55cd38f2dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:35.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.357+0000 7f562949b640 1 --2- 172.21.15.38:0/4067609779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cd38f092f0 0x55cd38f2e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:35.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.357+0000 7f562a49d640 1 -- 172.21.15.38:0/4067609779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd38f089d0 msgr2=0x55cd38f2dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:35.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.357+0000 7f562a49d640 1 --2- 172.21.15.38:0/4067609779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd38f089d0 0x55cd38f2dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:35.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.357+0000 7f562a49d640 1 -- 172.21.15.38:0/4067609779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cd38f092f0 msgr2=0x55cd38f2e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:35.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.357+0000 7f562a49d640 1 --2- 172.21.15.38:0/4067609779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cd38f092f0 0x55cd38f2e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:35.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.357+0000 7f562a49d640 1 -- 172.21.15.38:0/4067609779 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cd38f27840 con 0x55cd38f13bb0 2024-09-16T00:32:35.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.357+0000 7f562a49d640 1 --2- 172.21.15.38:0/4067609779 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd38f13bb0 0x55cd38f27010 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f561c02fa80 tx=0x7f561c053040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405eb19e10447a4c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:35.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.357+0000 7f560d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:32:35.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.357+0000 7f560d7fa640 1 -- 172.21.15.38:0/4067609779 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f561c034030 con 0x55cd38f13bb0 2024-09-16T00:32:35.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.357+0000 7f560d7fa640 1 -- 172.21.15.38:0/4067609779 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f561c0308a0 con 0x55cd38f13bb0 2024-09-16T00:32:35.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.357+0000 7f562e2d9740 1 -- 172.21.15.38:0/4067609779 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f561c049040 con 0x55cd38f13bb0 2024-09-16T00:32:35.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.357+0000 7f560d7fa640 1 -- 172.21.15.38:0/4067609779 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7f561c030aa0 con 0x55cd38f13bb0 2024-09-16T00:32:35.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.357+0000 7f562e2d9740 10 client.15201 fetch_fsmap learned FSMap version 25 2024-09-16T00:32:35.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.357+0000 7f562e2d9740 1 -- 172.21.15.38:0/4067609779 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55cd38f2c2a0 con 0x55cd38f13bb0 2024-09-16T00:32:35.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.358+0000 7f560d7fa640 1 -- 172.21.15.38:0/4067609779 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f561c030c40 con 0x55cd38f13bb0 2024-09-16T00:32:35.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.358+0000 7f560d7fa640 1 -- 172.21.15.38:0/4067609779 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7f561c041070 con 0x55cd38f13bb0 2024-09-16T00:32:35.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.358+0000 7f562e2d9740 10 client.15201 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:32:35.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.358+0000 7f562e2d9740 10 client.15201 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-16T00:32:35.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.358+0000 7f562e2d9740 20 client.15201 populate_metadata read hostname 'smithi038' 2024-09-16T00:32:35.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.358+0000 7f562e2d9740 1 --2- 172.21.15.38:0/4067609779 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x55cd38f92c20 0x55cd38f91930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:35.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.358+0000 7f562e2d9740 4 client.15201 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:32:35.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.358+0000 7f562e2d9740 1 -- 172.21.15.38:0/4067609779 --> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] -- command(tid 0: {"prefix": "status"}) -- 0x7f561c049040 con 0x55cd38f92c20 2024-09-16T00:32:35.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.359+0000 7f562949b640 1 --2- 172.21.15.38:0/4067609779 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x55cd38f92c20 0x55cd38f91930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:35.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.359+0000 7f562949b640 1 --2- 172.21.15.38:0/4067609779 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x55cd38f92c20 0x55cd38f91930 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d01e6b60b9c0d15c server_cookie=840e48d97b6c385 in_seq=0 out_seq=0 2024-09-16T00:32:35.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.359+0000 7f560d7fa640 10 client.15201 ms_handle_connect on v2:172.21.15.73:6836/2681081265 2024-09-16T00:32:35.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.359+0000 7f560d7fa640 1 -- 172.21.15.38:0/4067609779 <== mds.1 v2:172.21.15.73:6836/2681081265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x7f561c049040 con 0x55cd38f92c20 2024-09-16T00:32:35.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.359+0000 7f560d7fa640 10 client.15201 handle_command_reply: tid=0 2024-09-16T00:32:35.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.359+0000 7f562e2d9740 1 client.15201 shutdown 2024-09-16T00:32:35.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.359+0000 7f562e2d9740 1 -- 172.21.15.38:0/4067609779 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd38f13bb0 msgr2=0x55cd38f27010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:35.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.359+0000 7f562e2d9740 1 --2- 172.21.15.38:0/4067609779 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd38f13bb0 0x55cd38f27010 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f561c02fa80 tx=0x7f561c053040 comp rx=0 tx=0).stop 2024-09-16T00:32:35.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.360+0000 7f562e2d9740 1 -- 172.21.15.38:0/4067609779 shutdown_connections 2024-09-16T00:32:35.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.360+0000 7f562e2d9740 1 --2- 172.21.15.38:0/4067609779 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x55cd38f92c20 0x55cd38f91930 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:35.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.360+0000 7f562e2d9740 1 --2- 172.21.15.38:0/4067609779 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cd38f13bb0 0x55cd38f27010 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:35.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.360+0000 7f562e2d9740 1 --2- 172.21.15.38:0/4067609779 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cd38f092f0 0x55cd38f2e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:35.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.360+0000 7f560d7fa640 0 client.15201 ms_handle_reset on v2:172.21.15.73:6836/2681081265 2024-09-16T00:32:35.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.360+0000 7f562e2d9740 1 --2- 172.21.15.38:0/4067609779 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cd38f089d0 0x55cd38f2dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:35.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.360+0000 7f562e2d9740 1 -- 172.21.15.38:0/4067609779 >> 172.21.15.38:0/4067609779 conn(0x55cd38f04f60 msgr2=0x55cd38f05340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:35.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.360+0000 7f562e2d9740 1 -- 172.21.15.38:0/4067609779 shutdown_connections 2024-09-16T00:32:35.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.360+0000 7f562e2d9740 1 -- 172.21.15.38:0/4067609779 wait complete. 2024-09-16T00:32:35.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.360+0000 7f562e2d9740 20 client.15201 trim_cache size 0 max 16384 2024-09-16T00:32:35.368 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-16T00:32:35.368 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "983b2072-73c1-11ef-bceb-c7b262605968", 2024-09-16T00:32:35.368 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 1, 2024-09-16T00:32:35.368 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24464, 2024-09-16T00:32:35.368 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-16T00:32:35.368 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-16T00:32:35.368 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-16T00:32:35.368 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 94.066897623000003, 2024-09-16T00:32:35.368 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 25, 2024-09-16T00:32:35.368 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 92, 2024-09-16T00:32:35.368 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 88, 2024-09-16T00:32:35.368 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 150.70402816699999 2024-09-16T00:32:35.369 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-16T00:32:35.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.360+0000 7f562bf00640 1 -- 172.21.15.38:0/950987602 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f56100a32c0 msgr2=0x7f56100a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:35.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.360+0000 7f562bf00640 1 --2- 172.21.15.38:0/950987602 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f56100a32c0 0x7f56100a5780 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f56200cd1e0 tx=0x7f5620086040 comp rx=0 tx=0).stop 2024-09-16T00:32:35.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.360+0000 7f562bf00640 1 -- 172.21.15.38:0/950987602 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5624100e20 msgr2=0x7f56240fcc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:35.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.360+0000 7f562bf00640 1 --2- 172.21.15.38:0/950987602 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5624100e20 0x7f56240fcc80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f562000d770 tx=0x7f562000dc30 comp rx=0 tx=0).stop 2024-09-16T00:32:35.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.361+0000 7f562bf00640 1 -- 172.21.15.38:0/950987602 shutdown_connections 2024-09-16T00:32:35.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.361+0000 7f562bf00640 1 --2- 172.21.15.38:0/950987602 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f56100a32c0 0x7f56100a5780 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:35.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.361+0000 7f562bf00640 1 --2- 172.21.15.38:0/950987602 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5624102ef0 0x7f56240fd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:35.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.361+0000 7f562bf00640 1 --2- 172.21.15.38:0/950987602 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5624102570 0x7f56240fd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:35.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.361+0000 7f562bf00640 1 --2- 172.21.15.38:0/950987602 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5624100e20 0x7f56240fcc80 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:35.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.361+0000 7f562bf00640 1 -- 172.21.15.38:0/950987602 >> 172.21.15.38:0/950987602 conn(0x7f56240fc7d0 msgr2=0x7f562406a110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:35.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.361+0000 7f562bf00640 1 -- 172.21.15.38:0/950987602 shutdown_connections 2024-09-16T00:32:35.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:35.361+0000 7f562bf00640 1 -- 172.21.15.38:0/950987602 wait complete. 2024-09-16T00:32:36.181 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-16T00:32:36.441 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:36.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.927+0000 7fed335c9640 1 -- 172.21.15.38:0/2785520298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed2c1027a0 msgr2=0x7fed2c1067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:36.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.927+0000 7fed335c9640 1 --2- 172.21.15.38:0/2785520298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed2c1027a0 0x7fed2c1067f0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fed1c009900 tx=0x7fed1c02a410 comp rx=0 tx=0).stop 2024-09-16T00:32:36.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.927+0000 7fed335c9640 1 -- 172.21.15.38:0/2785520298 shutdown_connections 2024-09-16T00:32:36.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.927+0000 7fed335c9640 1 --2- 172.21.15.38:0/2785520298 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed2c106f20 0x7fed2c10eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:36.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.927+0000 7fed335c9640 1 --2- 172.21.15.38:0/2785520298 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed2c1027a0 0x7fed2c1067f0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:36.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.927+0000 7fed335c9640 1 --2- 172.21.15.38:0/2785520298 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fed2c101df0 0x7fed2c1021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:36.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.927+0000 7fed335c9640 1 -- 172.21.15.38:0/2785520298 >> 172.21.15.38:0/2785520298 conn(0x7fed2c0fd8f0 msgr2=0x7fed2c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:36.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.927+0000 7fed335c9640 1 -- 172.21.15.38:0/2785520298 shutdown_connections 2024-09-16T00:32:36.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.927+0000 7fed335c9640 1 -- 172.21.15.38:0/2785520298 wait complete. 2024-09-16T00:32:36.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.928+0000 7fed335c9640 1 Processor -- start 2024-09-16T00:32:36.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.928+0000 7fed335c9640 1 -- start start 2024-09-16T00:32:36.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.928+0000 7fed335c9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed2c101df0 0x7fed2c198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:36.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.928+0000 7fed335c9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fed2c1027a0 0x7fed2c198980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:36.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.928+0000 7fed335c9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed2c106f20 0x7fed2c19cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:36.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.928+0000 7fed335c9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c111f80 con 0x7fed2c106f20 2024-09-16T00:32:36.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.928+0000 7fed335c9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c111e00 con 0x7fed2c1027a0 2024-09-16T00:32:36.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.928+0000 7fed335c9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fed2c112100 con 0x7fed2c101df0 2024-09-16T00:32:36.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.928+0000 7fed31b66640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed2c106f20 0x7fed2c19cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:36.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.928+0000 7fed31b66640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed2c106f20 0x7fed2c19cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46888/0 (socket says 172.21.15.38:46888) 2024-09-16T00:32:36.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.928+0000 7fed31b66640 1 -- 172.21.15.38:0/3736373139 learned_addr learned my addr 172.21.15.38:0/3736373139 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:36.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.929+0000 7fed31365640 1 --2- 172.21.15.38:0/3736373139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed2c101df0 0x7fed2c198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:36.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.929+0000 7fed30b64640 1 --2- 172.21.15.38:0/3736373139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fed2c1027a0 0x7fed2c198980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:36.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.929+0000 7fed31b66640 1 -- 172.21.15.38:0/3736373139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed2c101df0 msgr2=0x7fed2c198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:36.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.929+0000 7fed31b66640 1 --2- 172.21.15.38:0/3736373139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed2c101df0 0x7fed2c198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:36.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.929+0000 7fed31b66640 1 -- 172.21.15.38:0/3736373139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fed2c1027a0 msgr2=0x7fed2c198980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:36.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.929+0000 7fed31b66640 1 --2- 172.21.15.38:0/3736373139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fed2c1027a0 0x7fed2c198980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:36.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.929+0000 7fed31b66640 1 -- 172.21.15.38:0/3736373139 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2c19d3f0 con 0x7fed2c106f20 2024-09-16T00:32:36.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.929+0000 7fed31b66640 1 --2- 172.21.15.38:0/3736373139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed2c106f20 0x7fed2c19cd10 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fed2400eb10 tx=0x7fed2400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:36.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.929+0000 7fed227fc640 1 -- 172.21.15.38:0/3736373139 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed24004b20 con 0x7fed2c106f20 2024-09-16T00:32:36.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.929+0000 7fed335c9640 1 -- 172.21.15.38:0/3736373139 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed2c19d6e0 con 0x7fed2c106f20 2024-09-16T00:32:36.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.929+0000 7fed227fc640 1 -- 172.21.15.38:0/3736373139 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed24004cc0 con 0x7fed2c106f20 2024-09-16T00:32:36.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.929+0000 7fed227fc640 1 -- 172.21.15.38:0/3736373139 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed24010760 con 0x7fed2c106f20 2024-09-16T00:32:36.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.929+0000 7fed335c9640 1 -- 172.21.15.38:0/3736373139 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed2c1a4fc0 con 0x7fed2c106f20 2024-09-16T00:32:36.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.932+0000 7fed359a2740 1 Processor -- start 2024-09-16T00:32:36.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.932+0000 7fed227fc640 1 -- 172.21.15.38:0/3736373139 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fed24005020 con 0x7fed2c106f20 2024-09-16T00:32:36.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.932+0000 7fed359a2740 1 -- start start 2024-09-16T00:32:36.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.933+0000 7fed359a2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43f0ed9d0 0x55e43f0eddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:36.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.933+0000 7fed359a2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e43f0ee2f0 0x55e43f0f8670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:36.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.933+0000 7fed359a2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e43f0f8bb0 0x55e43f0fafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:36.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.933+0000 7fed359a2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e43f054090 con 0x55e43f0f8bb0 2024-09-16T00:32:36.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.933+0000 7fed359a2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e43effba70 con 0x55e43f0ee2f0 2024-09-16T00:32:36.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.933+0000 7fed359a2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e43f050050 con 0x55e43f0ed9d0 2024-09-16T00:32:36.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.933+0000 7fed30b64640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e43f0f8bb0 0x55e43f0fafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:36.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.933+0000 7fed30b64640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e43f0f8bb0 0x55e43f0fafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46894/0 (socket says 172.21.15.38:46894) 2024-09-16T00:32:36.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.933+0000 7fed30b64640 1 -- 172.21.15.38:0/99203771 learned_addr learned my addr 172.21.15.38:0/99203771 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:36.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.933+0000 7fed31365640 1 --2- 172.21.15.38:0/99203771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e43f0ee2f0 0x55e43f0f8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:36.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.933+0000 7fed30b64640 1 --2- 172.21.15.38:0/99203771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43f0ed9d0 0x55e43f0eddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:36.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.933+0000 7fed30b64640 1 -- 172.21.15.38:0/99203771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43f0ed9d0 msgr2=0x55e43f0eddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:36.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.933+0000 7fed30b64640 1 --2- 172.21.15.38:0/99203771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43f0ed9d0 0x55e43f0eddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:36.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.933+0000 7fed30b64640 1 -- 172.21.15.38:0/99203771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e43f0ee2f0 msgr2=0x55e43f0f8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:36.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.933+0000 7fed30b64640 1 --2- 172.21.15.38:0/99203771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e43f0ee2f0 0x55e43f0f8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:36.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.933+0000 7fed30b64640 1 -- 172.21.15.38:0/99203771 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e43f008130 con 0x55e43f0f8bb0 2024-09-16T00:32:36.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.933+0000 7fed30b64640 1 --2- 172.21.15.38:0/99203771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e43f0f8bb0 0x55e43f0fafa0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fed1c000c00 tx=0x7fed1c0314e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ff0b6eec93286af server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:36.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.933+0000 7fed227fc640 1 --2- 172.21.15.38:0/3736373139 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fed180a3240 0x7fed180a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:36.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.934+0000 7fed31365640 1 --2- 172.21.15.38:0/3736373139 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fed180a3240 0x7fed180a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:36.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.934+0000 7fed177fe640 1 -- 172.21.15.38:0/99203771 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c035070 con 0x55e43f0f8bb0 2024-09-16T00:32:36.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.934+0000 7fed227fc640 1 -- 172.21.15.38:0/3736373139 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7fed240d02d0 con 0x7fed2c106f20 2024-09-16T00:32:36.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.934+0000 7fed177fe640 1 -- 172.21.15.38:0/99203771 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed1c004030 con 0x55e43f0f8bb0 2024-09-16T00:32:36.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.934+0000 7fed177fe640 1 -- 172.21.15.38:0/99203771 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c0055f0 con 0x55e43f0f8bb0 2024-09-16T00:32:36.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.934+0000 7fed31365640 1 --2- 172.21.15.38:0/3736373139 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fed180a3240 0x7fed180a5700 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x55e43f0eadc0 tx=0x7fed2800b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:36.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.934+0000 7fed359a2740 1 -- 172.21.15.38:0/99203771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e43f0f8bb0 msgr2=0x55e43f0fafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:36.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.934+0000 7fed359a2740 1 --2- 172.21.15.38:0/99203771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e43f0f8bb0 0x55e43f0fafa0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fed1c000c00 tx=0x7fed1c0314e0 comp rx=0 tx=0).stop 2024-09-16T00:32:36.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.935+0000 7fed359a2740 1 -- 172.21.15.38:0/99203771 shutdown_connections 2024-09-16T00:32:36.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.935+0000 7fed359a2740 1 --2- 172.21.15.38:0/99203771 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e43f0f8bb0 0x55e43f0fafa0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:36.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.935+0000 7fed359a2740 1 --2- 172.21.15.38:0/99203771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e43f0ee2f0 0x55e43f0f8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:36.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.935+0000 7fed359a2740 1 --2- 172.21.15.38:0/99203771 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43f0ed9d0 0x55e43f0eddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:36.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.935+0000 7fed359a2740 1 -- 172.21.15.38:0/99203771 >> 172.21.15.38:0/99203771 conn(0x55e43f0e9f60 msgr2=0x55e43f0ea340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:36.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.935+0000 7fed359a2740 1 -- 172.21.15.38:0/99203771 shutdown_connections 2024-09-16T00:32:36.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.935+0000 7fed359a2740 1 -- 172.21.15.38:0/99203771 wait complete. 2024-09-16T00:32:36.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.935+0000 7fed359a2740 1 -- start start 2024-09-16T00:32:36.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.935+0000 7fed359a2740 1 Processor -- start 2024-09-16T00:32:36.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.936+0000 7fed359a2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e43f0ed9d0 0x55e43f0f8420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:36.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.936+0000 7fed359a2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43f0ee2f0 0x55e43f0f5770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:36.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.936+0000 7fed359a2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e43f0f8bb0 0x55e43f0f5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:36.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.936+0000 7fed359a2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e43f0fb780 con 0x55e43f0f8bb0 2024-09-16T00:32:36.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.936+0000 7fed359a2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e43f00ef20 con 0x55e43f0ed9d0 2024-09-16T00:32:36.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.936+0000 7fed359a2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e43f0fb900 con 0x55e43f0ee2f0 2024-09-16T00:32:36.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.936+0000 7fed359a2740 10 client.0 _ll_register_callbacks cb 0x55e43efd8030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:36.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.936+0000 7fed31b66640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e43f0f8bb0 0x55e43f0f5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:36.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.936+0000 7fed31b66640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e43f0f8bb0 0x55e43f0f5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46906/0 (socket says 172.21.15.38:46906) 2024-09-16T00:32:36.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.936+0000 7fed31b66640 1 -- 172.21.15.38:0/1227052156 learned_addr learned my addr 172.21.15.38:0/1227052156 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:36.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.936+0000 7fed30b64640 1 --2- 172.21.15.38:0/1227052156 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43f0ee2f0 0x55e43f0f5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:36.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.936+0000 7fed31365640 1 --2- 172.21.15.38:0/1227052156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e43f0ed9d0 0x55e43f0f8420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:36.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.936+0000 7fed31b66640 1 -- 172.21.15.38:0/1227052156 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43f0ee2f0 msgr2=0x55e43f0f5770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:36.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.936+0000 7fed31b66640 1 --2- 172.21.15.38:0/1227052156 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43f0ee2f0 0x55e43f0f5770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:36.951 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.936+0000 7fed31b66640 1 -- 172.21.15.38:0/1227052156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e43f0ed9d0 msgr2=0x55e43f0f8420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:36.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.936+0000 7fed31b66640 1 --2- 172.21.15.38:0/1227052156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e43f0ed9d0 0x55e43f0f8420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:36.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.936+0000 7fed31b66640 1 -- 172.21.15.38:0/1227052156 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e43edc52b0 con 0x55e43f0f8bb0 2024-09-16T00:32:36.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.936+0000 7fed31365640 1 --2- 172.21.15.38:0/1227052156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e43f0ed9d0 0x55e43f0f8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:36.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.936+0000 7fed31b66640 1 --2- 172.21.15.38:0/1227052156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e43f0f8bb0 0x55e43f0f5cb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fed24000f80 tx=0x7fed2409c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbe2fcd0f6fed554 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:36.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.936+0000 7fed14ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:32:36.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.936+0000 7fed14ff9640 1 -- 172.21.15.38:0/1227052156 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed240b5060 con 0x55e43f0f8bb0 2024-09-16T00:32:36.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.936+0000 7fed14ff9640 1 -- 172.21.15.38:0/1227052156 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7fed24005510 con 0x55e43f0f8bb0 2024-09-16T00:32:36.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.936+0000 7fed359a2740 1 -- 172.21.15.38:0/1227052156 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed1c004030 con 0x55e43f0f8bb0 2024-09-16T00:32:36.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.937+0000 7fed14ff9640 1 -- 172.21.15.38:0/1227052156 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fed2409c570 con 0x55e43f0f8bb0 2024-09-16T00:32:36.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.937+0000 7fed359a2740 10 client.15213 fetch_fsmap learned FSMap version 25 2024-09-16T00:32:36.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.937+0000 7fed359a2740 1 -- 172.21.15.38:0/1227052156 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e43f107070 con 0x55e43f0f8bb0 2024-09-16T00:32:36.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.937+0000 7fed14ff9640 1 -- 172.21.15.38:0/1227052156 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed1c004030 con 0x55e43f0f8bb0 2024-09-16T00:32:36.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.937+0000 7fed14ff9640 1 -- 172.21.15.38:0/1227052156 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fed240982c0 con 0x55e43f0f8bb0 2024-09-16T00:32:36.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.937+0000 7fed359a2740 10 client.15213 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:32:36.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.937+0000 7fed359a2740 10 client.15213 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T00:32:36.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.937+0000 7fed359a2740 20 client.15213 populate_metadata read hostname 'smithi038' 2024-09-16T00:32:36.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.937+0000 7fed359a2740 1 --2- 172.21.15.38:0/1227052156 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55e43f1095e0 0x55e43f10aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:36.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.937+0000 7fed359a2740 4 client.15213 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:32:36.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.937+0000 7fed359a2740 1 -- 172.21.15.38:0/1227052156 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e43f061600 con 0x55e43f1095e0 2024-09-16T00:32:36.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.937+0000 7fed31365640 1 --2- 172.21.15.38:0/1227052156 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55e43f1095e0 0x55e43f10aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:36.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.938+0000 7fed31365640 1 --2- 172.21.15.38:0/1227052156 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55e43f1095e0 0x55e43f10aa50 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4448d16f006f7df server_cookie=197e3321f2a589a1 in_seq=0 out_seq=0 2024-09-16T00:32:36.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.938+0000 7fed14ff9640 10 client.15213 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:32:36.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.938+0000 7fed14ff9640 1 -- 172.21.15.38:0/1227052156 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55e43f061600 con 0x55e43f1095e0 2024-09-16T00:32:36.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.938+0000 7fed14ff9640 10 client.15213 handle_command_reply: tid=0 2024-09-16T00:32:36.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.939+0000 7fed359a2740 1 client.15213 shutdown 2024-09-16T00:32:36.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.939+0000 7fed359a2740 1 -- 172.21.15.38:0/1227052156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e43f0f8bb0 msgr2=0x55e43f0f5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:36.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.939+0000 7fed359a2740 1 --2- 172.21.15.38:0/1227052156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e43f0f8bb0 0x55e43f0f5cb0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fed24000f80 tx=0x7fed2409c400 comp rx=0 tx=0).stop 2024-09-16T00:32:36.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.939+0000 7fed359a2740 1 -- 172.21.15.38:0/1227052156 shutdown_connections 2024-09-16T00:32:36.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.939+0000 7fed359a2740 1 --2- 172.21.15.38:0/1227052156 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55e43f1095e0 0x55e43f10aa50 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:36.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.939+0000 7fed359a2740 1 --2- 172.21.15.38:0/1227052156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e43f0f8bb0 0x55e43f0f5cb0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:36.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.939+0000 7fed359a2740 1 --2- 172.21.15.38:0/1227052156 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43f0ee2f0 0x55e43f0f5770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:36.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.939+0000 7fed14ff9640 0 client.15213 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:32:36.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.939+0000 7fed359a2740 1 --2- 172.21.15.38:0/1227052156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e43f0ed9d0 0x55e43f0f8420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:36.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.939+0000 7fed359a2740 1 -- 172.21.15.38:0/1227052156 >> 172.21.15.38:0/1227052156 conn(0x55e43f0e9f60 msgr2=0x55e43f0efc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:36.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.939+0000 7fed359a2740 1 -- 172.21.15.38:0/1227052156 shutdown_connections 2024-09-16T00:32:36.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.939+0000 7fed359a2740 1 -- 172.21.15.38:0/1227052156 wait complete. 2024-09-16T00:32:36.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.939+0000 7fed359a2740 20 client.15213 trim_cache size 0 max 16384 2024-09-16T00:32:36.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.957+0000 7fed359a2740 1 Processor -- start 2024-09-16T00:32:36.958 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.957+0000 7fed359a2740 1 -- start start 2024-09-16T00:32:36.958 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:36 smithi038 ceph-mon[30945]: pgmap v359: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-09-16T00:32:36.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.958+0000 7fed359a2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e43f0ed9d0 0x55e43f108f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:36.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.958+0000 7fed359a2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43f0ee2f0 0x55e43f10aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:36.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.958+0000 7fed359a2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e43f0f8bb0 0x55e43f0f5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:36.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.958+0000 7fed359a2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e43f108d90 con 0x55e43f0f8bb0 2024-09-16T00:32:36.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.958+0000 7fed359a2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e43f109460 con 0x55e43f0ed9d0 2024-09-16T00:32:36.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.958+0000 7fed359a2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e43f109b30 con 0x55e43f0ee2f0 2024-09-16T00:32:36.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.958+0000 7fed31b66640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e43f0f8bb0 0x55e43f0f5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:36.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.958+0000 7fed31b66640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e43f0f8bb0 0x55e43f0f5030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46908/0 (socket says 172.21.15.38:46908) 2024-09-16T00:32:36.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.958+0000 7fed31b66640 1 -- 172.21.15.38:0/1898466329 learned_addr learned my addr 172.21.15.38:0/1898466329 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:36.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.958+0000 7fed31365640 1 --2- 172.21.15.38:0/1898466329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e43f0ed9d0 0x55e43f108f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:36.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.958+0000 7fed30b64640 1 --2- 172.21.15.38:0/1898466329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43f0ee2f0 0x55e43f10aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:36.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.958+0000 7fed31b66640 1 -- 172.21.15.38:0/1898466329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43f0ee2f0 msgr2=0x55e43f10aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:36.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.958+0000 7fed31b66640 1 --2- 172.21.15.38:0/1898466329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43f0ee2f0 0x55e43f10aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:36.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.958+0000 7fed31b66640 1 -- 172.21.15.38:0/1898466329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e43f0ed9d0 msgr2=0x55e43f108f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:36.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.958+0000 7fed31b66640 1 --2- 172.21.15.38:0/1898466329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e43f0ed9d0 0x55e43f108f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:36.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.958+0000 7fed31b66640 1 -- 172.21.15.38:0/1898466329 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e43f0ee120 con 0x55e43f0f8bb0 2024-09-16T00:32:36.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.958+0000 7fed31365640 1 --2- 172.21.15.38:0/1898466329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e43f0ed9d0 0x55e43f108f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:32:36.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.959+0000 7fed31b66640 1 --2- 172.21.15.38:0/1898466329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e43f0f8bb0 0x55e43f0f5030 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fed2409c270 tx=0x7fed2409aa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c49f7d72740e41 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:36.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.959+0000 7fed16ffd640 1 -- 172.21.15.38:0/1898466329 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed240b5040 con 0x55e43f0f8bb0 2024-09-16T00:32:36.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.959+0000 7fed16ffd640 1 -- 172.21.15.38:0/1898466329 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed240a51e0 con 0x55e43f0f8bb0 2024-09-16T00:32:36.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.959+0000 7fed16ffd640 1 -- 172.21.15.38:0/1898466329 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed240982c0 con 0x55e43f0f8bb0 2024-09-16T00:32:36.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.959+0000 7fed359a2740 1 -- 172.21.15.38:0/1898466329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e43f0f8bb0 msgr2=0x55e43f0f5030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:36.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.959+0000 7fed359a2740 1 --2- 172.21.15.38:0/1898466329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e43f0f8bb0 0x55e43f0f5030 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fed2409c270 tx=0x7fed2409aa80 comp rx=0 tx=0).stop 2024-09-16T00:32:36.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.960+0000 7fed359a2740 1 -- 172.21.15.38:0/1898466329 shutdown_connections 2024-09-16T00:32:36.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.960+0000 7fed359a2740 1 --2- 172.21.15.38:0/1898466329 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e43f0f8bb0 0x55e43f0f5030 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:36.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.960+0000 7fed359a2740 1 --2- 172.21.15.38:0/1898466329 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43f0ee2f0 0x55e43f10aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:36.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.960+0000 7fed359a2740 1 --2- 172.21.15.38:0/1898466329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e43f0ed9d0 0x55e43f108f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:36.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.960+0000 7fed359a2740 1 -- 172.21.15.38:0/1898466329 >> 172.21.15.38:0/1898466329 conn(0x55e43f0e9f60 msgr2=0x55e43f0ea340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:36.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.960+0000 7fed359a2740 1 -- 172.21.15.38:0/1898466329 shutdown_connections 2024-09-16T00:32:36.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.960+0000 7fed359a2740 1 -- 172.21.15.38:0/1898466329 wait complete. 2024-09-16T00:32:36.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.960+0000 7fed359a2740 1 -- start start 2024-09-16T00:32:36.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.960+0000 7fed359a2740 1 Processor -- start 2024-09-16T00:32:36.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.960+0000 7fed359a2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43f0ed9d0 0x55e43f110bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:36.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.960+0000 7fed359a2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e43f0ee2f0 0x55e43f111110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:36.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.960+0000 7fed359a2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e43f0f8bb0 0x55e43f10c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:36.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.960+0000 7fed359a2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55e43f10a8d0 con 0x55e43f0f8bb0 2024-09-16T00:32:36.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.960+0000 7fed359a2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55e43f10a200 con 0x55e43f0ee2f0 2024-09-16T00:32:36.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.961+0000 7fed359a2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55e43f10afa0 con 0x55e43f0ed9d0 2024-09-16T00:32:36.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.961+0000 7fed359a2740 10 client.0 _ll_register_callbacks cb 0x7fed24098810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:36.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.961+0000 7fed31365640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43f0ed9d0 0x55e43f110bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:36.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.961+0000 7fed31b66640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e43f0f8bb0 0x55e43f10c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:36.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.961+0000 7fed31b66640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e43f0f8bb0 0x55e43f10c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46914/0 (socket says 172.21.15.38:46914) 2024-09-16T00:32:36.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.961+0000 7fed31365640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43f0ed9d0 0x55e43f110bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.38:59322/0 (socket says 172.21.15.38:59322) 2024-09-16T00:32:36.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.961+0000 7fed31b66640 1 -- 172.21.15.38:0/3627650309 learned_addr learned my addr 172.21.15.38:0/3627650309 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:36.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.961+0000 7fed30b64640 1 --2- 172.21.15.38:0/3627650309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e43f0ee2f0 0x55e43f111110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:36.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.961+0000 7fed31b66640 1 -- 172.21.15.38:0/3627650309 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43f0ed9d0 msgr2=0x55e43f110bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:36.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.961+0000 7fed31b66640 1 --2- 172.21.15.38:0/3627650309 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43f0ed9d0 0x55e43f110bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:36.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.961+0000 7fed31b66640 1 -- 172.21.15.38:0/3627650309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e43f0ee2f0 msgr2=0x55e43f111110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:36.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.961+0000 7fed31b66640 1 --2- 172.21.15.38:0/3627650309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e43f0ee2f0 0x55e43f111110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:36.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.961+0000 7fed31b66640 1 -- 172.21.15.38:0/3627650309 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e43f10c860 con 0x55e43f0f8bb0 2024-09-16T00:32:36.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.961+0000 7fed30b64640 1 --2- 172.21.15.38:0/3627650309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e43f0ee2f0 0x55e43f111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:32:36.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.961+0000 7fed31b66640 1 --2- 172.21.15.38:0/3627650309 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e43f0f8bb0 0x55e43f10c1a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fed2409a820 tx=0x7fed2400efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98634586d94895e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:36.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.961+0000 7fed14ff9640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:32:36.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.961+0000 7fed14ff9640 1 -- 172.21.15.38:0/3627650309 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed240b5040 con 0x55e43f0f8bb0 2024-09-16T00:32:36.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.961+0000 7fed14ff9640 1 -- 172.21.15.38:0/3627650309 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7fed24005510 con 0x55e43f0f8bb0 2024-09-16T00:32:36.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.961+0000 7fed359a2740 1 -- 172.21.15.38:0/3627650309 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed240a51e0 con 0x55e43f0f8bb0 2024-09-16T00:32:36.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.962+0000 7fed14ff9640 1 -- 172.21.15.38:0/3627650309 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) ==== 24+0+0 (secure 0 0 0) 0x7fed2409bed0 con 0x55e43f0f8bb0 2024-09-16T00:32:36.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.962+0000 7fed359a2740 10 client.15219 fetch_fsmap learned FSMap version 25 2024-09-16T00:32:36.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.962+0000 7fed359a2740 1 -- 172.21.15.38:0/3627650309 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) -- 0x55e43f10c860 con 0x55e43f0f8bb0 2024-09-16T00:32:36.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.962+0000 7fed14ff9640 1 -- 172.21.15.38:0/3627650309 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fed240a51e0 con 0x55e43f0f8bb0 2024-09-16T00:32:36.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.962+0000 7fed14ff9640 1 -- 172.21.15.38:0/3627650309 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 25) ==== 8354+0+0 (secure 0 0 0) 0x7fed240a1ab0 con 0x55e43f0f8bb0 2024-09-16T00:32:36.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.962+0000 7fed359a2740 10 client.15219 fetch_fsmap finished waiting for FSMap version 25 2024-09-16T00:32:36.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.962+0000 7fed359a2740 10 client.15219 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T00:32:36.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.962+0000 7fed359a2740 20 client.15219 populate_metadata read hostname 'smithi038' 2024-09-16T00:32:36.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.962+0000 7fed359a2740 1 --2- 172.21.15.38:0/3627650309 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55e43f177a90 0x55e43f1767a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:36.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.962+0000 7fed359a2740 4 client.15219 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:32:36.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.962+0000 7fed359a2740 1 -- 172.21.15.38:0/3627650309 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "status"}) -- 0x55e43f05bb10 con 0x55e43f177a90 2024-09-16T00:32:36.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.962+0000 7fed31365640 1 --2- 172.21.15.38:0/3627650309 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55e43f177a90 0x55e43f1767a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:36.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.963+0000 7fed14ff9640 10 client.15219 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:32:36.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.963+0000 7fed31365640 1 --2- 172.21.15.38:0/3627650309 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55e43f177a90 0x55e43f1767a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a4c1f766e1d1528 server_cookie=37414170dd5a79a4 in_seq=0 out_seq=0 2024-09-16T00:32:36.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.963+0000 7fed14ff9640 1 -- 172.21.15.38:0/3627650309 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55e43f05bb10 con 0x55e43f177a90 2024-09-16T00:32:36.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.963+0000 7fed14ff9640 10 client.15219 handle_command_reply: tid=0 2024-09-16T00:32:36.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.963+0000 7fed359a2740 1 client.15219 shutdown 2024-09-16T00:32:36.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.963+0000 7fed359a2740 1 -- 172.21.15.38:0/3627650309 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e43f0f8bb0 msgr2=0x55e43f10c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:36.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.963+0000 7fed359a2740 1 --2- 172.21.15.38:0/3627650309 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e43f0f8bb0 0x55e43f10c1a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fed2409a820 tx=0x7fed2400efd0 comp rx=0 tx=0).stop 2024-09-16T00:32:36.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.963+0000 7fed359a2740 1 -- 172.21.15.38:0/3627650309 shutdown_connections 2024-09-16T00:32:36.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.963+0000 7fed359a2740 1 --2- 172.21.15.38:0/3627650309 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55e43f177a90 0x55e43f1767a0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:36.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.963+0000 7fed359a2740 1 --2- 172.21.15.38:0/3627650309 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55e43f0f8bb0 0x55e43f10c1a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:36.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.963+0000 7fed359a2740 1 --2- 172.21.15.38:0/3627650309 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55e43f0ee2f0 0x55e43f111110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:36.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.963+0000 7fed359a2740 1 --2- 172.21.15.38:0/3627650309 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55e43f0ed9d0 0x55e43f110bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:36.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.963+0000 7fed14ff9640 0 client.15219 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:32:36.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.963+0000 7fed359a2740 1 -- 172.21.15.38:0/3627650309 >> 172.21.15.38:0/3627650309 conn(0x55e43f0e9f60 msgr2=0x55e43f0f05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:36.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.964+0000 7fed359a2740 1 -- 172.21.15.38:0/3627650309 shutdown_connections 2024-09-16T00:32:36.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.964+0000 7fed359a2740 1 -- 172.21.15.38:0/3627650309 wait complete. 2024-09-16T00:32:36.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.964+0000 7fed359a2740 20 client.15219 trim_cache size 0 max 16384 2024-09-16T00:32:36.971 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-16T00:32:36.971 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "983b2072-73c1-11ef-bceb-c7b262605968", 2024-09-16T00:32:36.971 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-09-16T00:32:36.971 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24494, 2024-09-16T00:32:36.971 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-16T00:32:36.971 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-16T00:32:36.971 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-16T00:32:36.971 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 129.901066012, 2024-09-16T00:32:36.971 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 25, 2024-09-16T00:32:36.971 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 92, 2024-09-16T00:32:36.972 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 88, 2024-09-16T00:32:36.972 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 139.81961447699999 2024-09-16T00:32:36.972 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-16T00:32:36.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.964+0000 7fed335c9640 1 -- 172.21.15.38:0/3736373139 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fed180a3240 msgr2=0x7fed180a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:36.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.964+0000 7fed335c9640 1 --2- 172.21.15.38:0/3736373139 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fed180a3240 0x7fed180a5700 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x55e43f0eadc0 tx=0x7fed2800b410 comp rx=0 tx=0).stop 2024-09-16T00:32:36.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.964+0000 7fed335c9640 1 -- 172.21.15.38:0/3736373139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed2c106f20 msgr2=0x7fed2c19cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:36.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.964+0000 7fed335c9640 1 --2- 172.21.15.38:0/3736373139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed2c106f20 0x7fed2c19cd10 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fed2400eb10 tx=0x7fed2400c2d0 comp rx=0 tx=0).stop 2024-09-16T00:32:36.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.964+0000 7fed335c9640 1 -- 172.21.15.38:0/3736373139 shutdown_connections 2024-09-16T00:32:36.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.964+0000 7fed335c9640 1 --2- 172.21.15.38:0/3736373139 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fed180a3240 0x7fed180a5700 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:36.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.964+0000 7fed335c9640 1 --2- 172.21.15.38:0/3736373139 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fed2c106f20 0x7fed2c19cd10 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:36.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.964+0000 7fed335c9640 1 --2- 172.21.15.38:0/3736373139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fed2c1027a0 0x7fed2c198980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:36.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.964+0000 7fed335c9640 1 --2- 172.21.15.38:0/3736373139 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fed2c101df0 0x7fed2c198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:36.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.964+0000 7fed335c9640 1 -- 172.21.15.38:0/3736373139 >> 172.21.15.38:0/3736373139 conn(0x7fed2c0fd8f0 msgr2=0x7fed2c0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:36.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.965+0000 7fed335c9640 1 -- 172.21.15.38:0/3736373139 shutdown_connections 2024-09-16T00:32:36.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:36.965+0000 7fed335c9640 1 -- 172.21.15.38:0/3736373139 wait complete. 2024-09-16T00:32:37.096 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:36 smithi073 ceph-mon[35254]: pgmap v359: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-09-16T00:32:37.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:36 smithi122 ceph-mon[35412]: pgmap v359: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-09-16T00:32:37.666 INFO:teuthology.run_tasks:Running task sequential... 2024-09-16T00:32:37.675 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-09-16T00:32:37.675 INFO:teuthology.task.sequential:In sequential, running task print... 2024-09-16T00:32:37.675 INFO:teuthology.task.print:Enabling mgr modules 2024-09-16T00:32:37.676 INFO:teuthology.run_tasks:Running task kclient... 2024-09-16T00:32:37.694 INFO:tasks.kclient:Mounting kernel clients... 2024-09-16T00:32:37.694 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-16T00:32:37.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:32:37.695 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-09-16T00:32:37.713 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-09-16T00:32:37.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:32:37.714 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-09-16T00:32:37.784 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:32:37.784 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link delete ceph-brx 2024-09-16T00:32:37.882 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-09-16T00:32:37.882 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-16T00:32:37.882 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T00:32:37.882 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs ls 2024-09-16T00:32:38.141 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:38.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.639+0000 7efcdeee5640 1 -- 172.21.15.38:0/322276560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcd8101de0 msgr2=0x7efcd81021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:38.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.639+0000 7efcdeee5640 1 --2- 172.21.15.38:0/322276560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcd8101de0 0x7efcd81021c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7efccc009900 tx=0x7efccc02a410 comp rx=0 tx=0).stop 2024-09-16T00:32:38.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.639+0000 7efcdeee5640 1 -- 172.21.15.38:0/322276560 shutdown_connections 2024-09-16T00:32:38.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.639+0000 7efcdeee5640 1 --2- 172.21.15.38:0/322276560 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efcd8106f10 0x7efcd810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:38.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.639+0000 7efcdeee5640 1 --2- 172.21.15.38:0/322276560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efcd8102790 0x7efcd81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:38.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.639+0000 7efcdeee5640 1 --2- 172.21.15.38:0/322276560 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcd8101de0 0x7efcd81021c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:38.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.639+0000 7efcdeee5640 1 -- 172.21.15.38:0/322276560 >> 172.21.15.38:0/322276560 conn(0x7efcd80fd8c0 msgr2=0x7efcd80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:38.664 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.639+0000 7efcdeee5640 1 -- 172.21.15.38:0/322276560 shutdown_connections 2024-09-16T00:32:38.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.639+0000 7efcdeee5640 1 -- 172.21.15.38:0/322276560 wait complete. 2024-09-16T00:32:38.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.639+0000 7efcdeee5640 1 Processor -- start 2024-09-16T00:32:38.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.639+0000 7efcdeee5640 1 -- start start 2024-09-16T00:32:38.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.639+0000 7efcdeee5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efcd8101de0 0x7efcd81984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:38.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.639+0000 7efcdeee5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efcd8102790 0x7efcd8198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:38.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.639+0000 7efcdeee5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcd8106f10 0x7efcd819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:38.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.639+0000 7efcdeee5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efcd8111f70 con 0x7efcd8106f10 2024-09-16T00:32:38.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.640+0000 7efcdeee5640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efcd8111df0 con 0x7efcd8101de0 2024-09-16T00:32:38.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.640+0000 7efcdeee5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efcd81120f0 con 0x7efcd8102790 2024-09-16T00:32:38.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.640+0000 7efcdd482640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcd8106f10 0x7efcd819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:38.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.640+0000 7efcdd482640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcd8106f10 0x7efcd819cda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46930/0 (socket says 172.21.15.38:46930) 2024-09-16T00:32:38.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.640+0000 7efcdd482640 1 -- 172.21.15.38:0/2526901817 learned_addr learned my addr 172.21.15.38:0/2526901817 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:38.665 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.640+0000 7efcdcc81640 1 --2- 172.21.15.38:0/2526901817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efcd8101de0 0x7efcd81984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:38.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.640+0000 7efcd7fff640 1 --2- 172.21.15.38:0/2526901817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efcd8102790 0x7efcd8198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:38.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.640+0000 7efcdd482640 1 -- 172.21.15.38:0/2526901817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efcd8102790 msgr2=0x7efcd8198a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:38.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.640+0000 7efcdd482640 1 --2- 172.21.15.38:0/2526901817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efcd8102790 0x7efcd8198a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:38.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.640+0000 7efcdd482640 1 -- 172.21.15.38:0/2526901817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efcd8101de0 msgr2=0x7efcd81984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:38.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.640+0000 7efcdd482640 1 --2- 172.21.15.38:0/2526901817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efcd8101de0 0x7efcd81984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:38.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.640+0000 7efcdd482640 1 -- 172.21.15.38:0/2526901817 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd819d480 con 0x7efcd8106f10 2024-09-16T00:32:38.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.640+0000 7efcdd482640 1 --2- 172.21.15.38:0/2526901817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcd8106f10 0x7efcd819cda0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7efcd0009fd0 tx=0x7efcd000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:38.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.640+0000 7efcd5ffb640 1 -- 172.21.15.38:0/2526901817 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd0017070 con 0x7efcd8106f10 2024-09-16T00:32:38.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.640+0000 7efcd5ffb640 1 -- 172.21.15.38:0/2526901817 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efcd0003e50 con 0x7efcd8106f10 2024-09-16T00:32:38.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.640+0000 7efcdeee5640 1 -- 172.21.15.38:0/2526901817 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcd819d770 con 0x7efcd8106f10 2024-09-16T00:32:38.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.640+0000 7efcd5ffb640 1 -- 172.21.15.38:0/2526901817 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcd00106c0 con 0x7efcd8106f10 2024-09-16T00:32:38.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.641+0000 7efcdeee5640 1 -- 172.21.15.38:0/2526901817 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcd81a5050 con 0x7efcd8106f10 2024-09-16T00:32:38.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.641+0000 7efcdeee5640 1 -- 172.21.15.38:0/2526901817 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcd81021c0 con 0x7efcd8106f10 2024-09-16T00:32:38.666 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.643+0000 7efcd5ffb640 1 -- 172.21.15.38:0/2526901817 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7efcd0003970 con 0x7efcd8106f10 2024-09-16T00:32:38.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.643+0000 7efcd5ffb640 1 --2- 172.21.15.38:0/2526901817 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efcc80a3240 0x7efcc80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:38.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.643+0000 7efcdcc81640 1 --2- 172.21.15.38:0/2526901817 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efcc80a3240 0x7efcc80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:38.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.643+0000 7efcd5ffb640 1 -- 172.21.15.38:0/2526901817 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7efcd00cf950 con 0x7efcd8106f10 2024-09-16T00:32:38.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.644+0000 7efcdcc81640 1 --2- 172.21.15.38:0/2526901817 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efcc80a3240 0x7efcc80a5700 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efccc009230 tx=0x7efccc0058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:38.667 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.648+0000 7efcd5ffb640 1 -- 172.21.15.38:0/2526901817 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efcd009bca0 con 0x7efcd8106f10 2024-09-16T00:32:38.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.860+0000 7efcdeee5640 1 -- 172.21.15.38:0/2526901817 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7efcd810a0f0 con 0x7efcd8106f10 2024-09-16T00:32:38.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.860+0000 7efcd5ffb640 1 -- 172.21.15.38:0/2526901817 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) ==== 53+0+88 (secure 0 0 0) 0x7efcd009ffa0 con 0x7efcd8106f10 2024-09-16T00:32:38.861 INFO:teuthology.orchestra.run.smithi038.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-09-16T00:32:38.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.863+0000 7efcdeee5640 1 -- 172.21.15.38:0/2526901817 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efcc80a3240 msgr2=0x7efcc80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:38.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.863+0000 7efcdeee5640 1 --2- 172.21.15.38:0/2526901817 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efcc80a3240 0x7efcc80a5700 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7efccc009230 tx=0x7efccc0058d0 comp rx=0 tx=0).stop 2024-09-16T00:32:38.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.863+0000 7efcdeee5640 1 -- 172.21.15.38:0/2526901817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcd8106f10 msgr2=0x7efcd819cda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:38.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.863+0000 7efcdeee5640 1 --2- 172.21.15.38:0/2526901817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcd8106f10 0x7efcd819cda0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7efcd0009fd0 tx=0x7efcd000b260 comp rx=0 tx=0).stop 2024-09-16T00:32:38.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.863+0000 7efcdeee5640 1 -- 172.21.15.38:0/2526901817 shutdown_connections 2024-09-16T00:32:38.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.863+0000 7efcdeee5640 1 --2- 172.21.15.38:0/2526901817 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efcc80a3240 0x7efcc80a5700 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:38.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.863+0000 7efcdeee5640 1 --2- 172.21.15.38:0/2526901817 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcd8106f10 0x7efcd819cda0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:38.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.863+0000 7efcdeee5640 1 --2- 172.21.15.38:0/2526901817 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efcd8102790 0x7efcd8198a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:38.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.863+0000 7efcdeee5640 1 --2- 172.21.15.38:0/2526901817 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efcd8101de0 0x7efcd81984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:38.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.863+0000 7efcdeee5640 1 -- 172.21.15.38:0/2526901817 >> 172.21.15.38:0/2526901817 conn(0x7efcd80fd8c0 msgr2=0x7efcd80fecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:38.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.863+0000 7efcdeee5640 1 -- 172.21.15.38:0/2526901817 shutdown_connections 2024-09-16T00:32:38.864 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:38.863+0000 7efcdeee5640 1 -- 172.21.15.38:0/2526901817 wait complete. 2024-09-16T00:32:39.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:38 smithi038 ceph-mon[30945]: pgmap v360: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:32:39.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:38 smithi122 ceph-mon[35412]: pgmap v360: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:32:39.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:38 smithi073 ceph-mon[35254]: pgmap v360: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:32:39.535 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-16T00:32:39.535 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-16T00:32:39.535 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi038.front.sepia.ceph.com 2024-09-16T00:32:39.536 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-16T00:32:39.536 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-16T00:32:39.536 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-16T00:32:39.536 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-16T00:32:39.536 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-16T00:32:39.536 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-16T00:32:39.536 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:32:39.536 DEBUG:teuthology.orchestra.run.smithi038:> ip addr 2024-09-16T00:32:39.555 INFO:teuthology.orchestra.run.smithi038.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-16T00:32:39.555 INFO:teuthology.orchestra.run.smithi038.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-16T00:32:39.556 INFO:teuthology.orchestra.run.smithi038.stdout: inet 127.0.0.1/8 scope host lo 2024-09-16T00:32:39.556 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-16T00:32:39.556 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 ::1/128 scope host 2024-09-16T00:32:39.556 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-16T00:32:39.556 INFO:teuthology.orchestra.run.smithi038.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T00:32:39.556 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a2 brd ff:ff:ff:ff:ff:ff 2024-09-16T00:32:39.556 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f0 2024-09-16T00:32:39.556 INFO:teuthology.orchestra.run.smithi038.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T00:32:39.556 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:6b:fd:a3 brd ff:ff:ff:ff:ff:ff 2024-09-16T00:32:39.556 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp7s0f1 2024-09-16T00:32:39.556 INFO:teuthology.orchestra.run.smithi038.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-16T00:32:39.556 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fc brd ff:ff:ff:ff:ff:ff 2024-09-16T00:32:39.556 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f0 2024-09-16T00:32:39.557 INFO:teuthology.orchestra.run.smithi038.stdout: inet 172.21.15.38/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-16T00:32:39.557 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft 42136sec preferred_lft 42136sec 2024-09-16T00:32:39.557 INFO:teuthology.orchestra.run.smithi038.stdout: inet6 fe80::ec4:7aff:febd:15fc/64 scope link 2024-09-16T00:32:39.557 INFO:teuthology.orchestra.run.smithi038.stdout: valid_lft forever preferred_lft forever 2024-09-16T00:32:39.557 INFO:teuthology.orchestra.run.smithi038.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-16T00:32:39.557 INFO:teuthology.orchestra.run.smithi038.stdout: link/ether 0c:c4:7a:bd:15:fd brd ff:ff:ff:ff:ff:ff 2024-09-16T00:32:39.557 INFO:teuthology.orchestra.run.smithi038.stdout: altname enp1s0f1 2024-09-16T00:32:39.557 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-16T00:32:39.557 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:32:39.558 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-16T00:32:39.558 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link add name ceph-brx type bridge 2024-09-16T00:32:39.558 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip addr flush dev ceph-brx 2024-09-16T00:32:39.558 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set ceph-brx up 2024-09-16T00:32:39.558 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-16T00:32:39.558 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-16T00:32:39.649 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:32:39.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:32:39.705 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:32:39.705 DEBUG:teuthology.orchestra.run.smithi038:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-16T00:32:39.729 INFO:teuthology.orchestra.run.smithi038.stdout:1 2024-09-16T00:32:39.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:32:39.731 DEBUG:teuthology.orchestra.run.smithi038:> route 2024-09-16T00:32:39.788 INFO:teuthology.orchestra.run.smithi038.stdout:Kernel IP routing table 2024-09-16T00:32:39.788 INFO:teuthology.orchestra.run.smithi038.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-16T00:32:39.789 INFO:teuthology.orchestra.run.smithi038.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-16T00:32:39.789 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-16T00:32:39.789 INFO:teuthology.orchestra.run.smithi038.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-16T00:32:39.789 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:32:39.789 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-16T00:32:39.789 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-16T00:32:39.789 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-16T00:32:39.789 DEBUG:teuthology.orchestra.run.smithi038:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-16T00:32:39.789 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-16T00:32:39.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:32:39.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:32:39.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:32:39.919 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-09-16T00:32:39.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:32:39.936 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list-id 2024-09-16T00:32:39.992 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:32:39.993 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-16T00:32:39.993 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T00:32:39.993 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-16T00:32:39.993 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-16T00:32:40.081 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:32:40.102 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:39 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/2526901817' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T00:32:40.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:32:40.112 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-16T00:32:40.112 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:32:40.112 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-16T00:32:40.112 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-16T00:32:40.112 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-16T00:32:40.112 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-16T00:32:40.112 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-16T00:32:40.113 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-16T00:32:40.113 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-16T00:32:40.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:32:40.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:39 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/2526901817' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T00:32:40.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:32:40.229 DEBUG:teuthology.orchestra.run.smithi038:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-16T00:32:40.229 DEBUG:teuthology.orchestra.run.smithi038:> set -e 2024-09-16T00:32:40.230 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set brx.0 up 2024-09-16T00:32:40.230 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link set dev brx.0 master ceph-brx 2024-09-16T00:32:40.230 DEBUG:teuthology.orchestra.run.smithi038:> ') 2024-09-16T00:32:40.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:32:40.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:39 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/2526901817' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-09-16T00:32:40.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:32:40.309 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T00:32:40.309 DEBUG:teuthology.orchestra.run.smithi038:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-16T00:32:40.338 INFO:teuthology.orchestra.run.smithi038.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-16T00:32:40.338 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-16T00:32:40.339 DEBUG:teuthology.orchestra.run.smithi038:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-16T00:32:40.395 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-16T00:32:40.395 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-16T00:32:40.395 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/aaa8e7e9-698e-40fc-9502-7256606ca4b9 2024-09-16T00:32:40.395 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:32:40.395 DEBUG:teuthology.orchestra.run.smithi038:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/aaa8e7e9-698e-40fc-9502-7256606ca4b9 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-16T00:32:40.480 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-16T00:32:40.480 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-09-16T00:32:40.480 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-16T00:32:40.636 INFO:teuthology.orchestra.run.smithi038.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync 2024-09-16T00:32:40.636 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync". 2024-09-16T00:32:40.636 INFO:teuthology.orchestra.run.smithi038.stdout:invalid new device string format 2024-09-16T00:32:40.637 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: resolved to: "172.21.15.38:3300,172.21.15.73:3300,172.21.15.122:3300" 2024-09-16T00:32:40.637 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.38:3300,172.21.15.73:3300,172.21.15.122:3300:/volumes/qa/sv_1/aaa8e7e9-698e-40fc-9502-7256606ca4b9 2024-09-16T00:32:40.637 INFO:teuthology.orchestra.run.smithi038.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=crc,wsync,key=0,fsid=983b2072-73c1-11ef-bceb-c7b262605968" will pass to kernel 2024-09-16T00:32:40.638 INFO:teuthology.orchestra.run.smithi038.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-16T00:32:40.638 INFO:teuthology.orchestra.run.smithi038.stdout: You just mounted a file system that supports labels which does not 2024-09-16T00:32:40.638 INFO:teuthology.orchestra.run.smithi038.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-16T00:32:40.638 INFO:teuthology.orchestra.run.smithi038.stdout: applications will generate AVC messages and not be allowed access to 2024-09-16T00:32:40.638 INFO:teuthology.orchestra.run.smithi038.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-16T00:32:40.639 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-16T00:32:40.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:32:40.640 DEBUG:teuthology.orchestra.run.smithi038:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-16T00:32:40.670 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T00:32:40.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T00:32:40.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:32:40.769 INFO:teuthology.orchestra.run.smithi038.stdout:client24914 2024-09-16T00:32:40.769 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-16T00:32:40.769 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T00:32:40.787 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T00:32:40.877 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T00:32:40.879 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:32:40.884 INFO:teuthology.orchestra.run.smithi038.stdout:client24914 2024-09-16T00:32:40.885 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-16T00:32:40.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:40 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-16T00:32:40.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:40 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:32:40.942 INFO:teuthology.orchestra.run.smithi038.stdout:client24914 2024-09-16T00:32:40.942 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/983b2072-73c1-11ef-bceb-c7b262605968.client24914/status') 2024-09-16T00:32:40.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:40 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-16T00:32:41.002 INFO:teuthology.orchestra.run.smithi038.stdout:instance: client.24914 (3)172.21.15.38:0/1439643702 2024-09-16T00:32:41.002 INFO:teuthology.orchestra.run.smithi038.stdout:blocklisted: false 2024-09-16T00:32:41.003 INFO:teuthology.orchestra.run.smithi038.stderr:0+1 records in 2024-09-16T00:32:41.003 INFO:teuthology.orchestra.run.smithi038.stderr:0+1 records out 2024-09-16T00:32:41.003 INFO:teuthology.orchestra.run.smithi038.stderr:71 bytes copied, 2.5144e-05 s, 2.8 MB/s 2024-09-16T00:32:41.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-16T00:32:41.010 INFO:teuthology.run_tasks:Running task exec... 2024-09-16T00:32:41.021 INFO:teuthology.task.exec:Executing custom commands... 2024-09-16T00:32:41.021 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi038.front.sepia.ceph.com 2024-09-16T00:32:41.021 DEBUG:teuthology.orchestra.run.smithi038:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-09-16T00:32:41.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.126+0000 7f89201c4640 1 -- 172.21.15.38:0/3735193046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8918107840 msgr2=0x7f8918109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:41.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.126+0000 7f89201c4640 1 --2- 172.21.15.38:0/3735193046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8918107840 0x7f8918109cb0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f8910009900 tx=0x7f891002a410 comp rx=0 tx=0).stop 2024-09-16T00:32:41.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.126+0000 7f89201c4640 1 -- 172.21.15.38:0/3735193046 shutdown_connections 2024-09-16T00:32:41.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.126+0000 7f89201c4640 1 --2- 172.21.15.38:0/3735193046 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8918107840 0x7f8918109cb0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:41.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.126+0000 7f89201c4640 1 --2- 172.21.15.38:0/3735193046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8918104f10 0x7f8918107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:41.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.126+0000 7f89201c4640 1 --2- 172.21.15.38:0/3735193046 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f89181025e0 0x7f89181049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:41.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.126+0000 7f89201c4640 1 -- 172.21.15.38:0/3735193046 >> 172.21.15.38:0/3735193046 conn(0x7f89180fc1e0 msgr2=0x7f89180fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:41.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.126+0000 7f89201c4640 1 -- 172.21.15.38:0/3735193046 shutdown_connections 2024-09-16T00:32:41.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.126+0000 7f89201c4640 1 -- 172.21.15.38:0/3735193046 wait complete. 2024-09-16T00:32:41.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.127+0000 7f89201c4640 1 Processor -- start 2024-09-16T00:32:41.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.127+0000 7f89201c4640 1 -- start start 2024-09-16T00:32:41.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.127+0000 7f89201c4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89181025e0 0x7f8918101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:41.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.127+0000 7f89201c4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8918104f10 0x7f8918100590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:41.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.127+0000 7f89201c4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8918107840 0x7f8918100ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:41.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.127+0000 7f89201c4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f891810d800 con 0x7f89181025e0 2024-09-16T00:32:41.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.127+0000 7f89201c4640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f891810d680 con 0x7f8918107840 2024-09-16T00:32:41.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.127+0000 7f89201c4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f891810d980 con 0x7f8918104f10 2024-09-16T00:32:41.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.127+0000 7f891df60640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89181025e0 0x7f8918101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:41.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.127+0000 7f891df60640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89181025e0 0x7f8918101f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46954/0 (socket says 172.21.15.38:46954) 2024-09-16T00:32:41.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.127+0000 7f891df60640 1 -- 172.21.15.38:0/4163186939 learned_addr learned my addr 172.21.15.38:0/4163186939 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:41.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.127+0000 7f891e761640 1 --2- 172.21.15.38:0/4163186939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8918107840 0x7f8918100ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:41.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.128+0000 7f891d75f640 1 --2- 172.21.15.38:0/4163186939 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8918104f10 0x7f8918100590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:41.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.128+0000 7f891df60640 1 -- 172.21.15.38:0/4163186939 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8918104f10 msgr2=0x7f8918100590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:41.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.128+0000 7f891df60640 1 --2- 172.21.15.38:0/4163186939 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8918104f10 0x7f8918100590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:41.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.128+0000 7f891df60640 1 -- 172.21.15.38:0/4163186939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8918107840 msgr2=0x7f8918100ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:41.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.128+0000 7f891df60640 1 --2- 172.21.15.38:0/4163186939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8918107840 0x7f8918100ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:41.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.128+0000 7f891df60640 1 -- 172.21.15.38:0/4163186939 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8918101360 con 0x7f89181025e0 2024-09-16T00:32:41.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.128+0000 7f891df60640 1 --2- 172.21.15.38:0/4163186939 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89181025e0 0x7f8918101f50 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f891400cce0 tx=0x7f891400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:41.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.128+0000 7f890effd640 1 -- 172.21.15.38:0/4163186939 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8914004b20 con 0x7f89181025e0 2024-09-16T00:32:41.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.128+0000 7f890effd640 1 -- 172.21.15.38:0/4163186939 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8914004cc0 con 0x7f89181025e0 2024-09-16T00:32:41.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.128+0000 7f89201c4640 1 -- 172.21.15.38:0/4163186939 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89181a1f20 con 0x7f89181025e0 2024-09-16T00:32:41.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.128+0000 7f890effd640 1 -- 172.21.15.38:0/4163186939 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f891400f7d0 con 0x7f89181025e0 2024-09-16T00:32:41.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.128+0000 7f89201c4640 1 -- 172.21.15.38:0/4163186939 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89181a2330 con 0x7f89181025e0 2024-09-16T00:32:41.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.129+0000 7f89201c4640 1 -- 172.21.15.38:0/4163186939 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89180fdcd0 con 0x7f89181025e0 2024-09-16T00:32:41.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.130+0000 7f890effd640 1 -- 172.21.15.38:0/4163186939 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f891400f970 con 0x7f89181025e0 2024-09-16T00:32:41.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.131+0000 7f890effd640 1 --2- 172.21.15.38:0/4163186939 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f89040a32f0 0x7f89040a57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:41.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.131+0000 7f891d75f640 1 --2- 172.21.15.38:0/4163186939 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f89040a32f0 0x7f89040a57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:41.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.131+0000 7f890effd640 1 -- 172.21.15.38:0/4163186939 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f89140d0110 con 0x7f89181025e0 2024-09-16T00:32:41.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.132+0000 7f891d75f640 1 --2- 172.21.15.38:0/4163186939 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f89040a32f0 0x7f89040a57b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8908005fd0 tx=0x7f8908005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:41.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.134+0000 7f890effd640 1 -- 172.21.15.38:0/4163186939 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8914014030 con 0x7f89181025e0 2024-09-16T00:32:41.214 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:40 smithi038 ceph-mon[30945]: pgmap v361: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T00:32:41.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:40 smithi122 ceph-mon[35412]: pgmap v361: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T00:32:41.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:40 smithi073 ceph-mon[35254]: pgmap v361: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T00:32:41.354 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.354+0000 7f89201c4640 1 -- 172.21.15.38:0/4163186939 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7f891802d7c0 con 0x7f89181025e0 2024-09-16T00:32:41.866 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.866+0000 7f890effd640 1 -- 172.21.15.38:0/4163186939 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v26) ==== 118+0+0 (secure 0 0 0) 0x7f891409ca50 con 0x7f89181025e0 2024-09-16T00:32:41.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.869+0000 7f89201c4640 1 -- 172.21.15.38:0/4163186939 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f89040a32f0 msgr2=0x7f89040a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:41.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.869+0000 7f89201c4640 1 --2- 172.21.15.38:0/4163186939 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f89040a32f0 0x7f89040a57b0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8908005fd0 tx=0x7f8908005850 comp rx=0 tx=0).stop 2024-09-16T00:32:41.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.869+0000 7f89201c4640 1 -- 172.21.15.38:0/4163186939 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89181025e0 msgr2=0x7f8918101f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:41.869 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.869+0000 7f89201c4640 1 --2- 172.21.15.38:0/4163186939 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89181025e0 0x7f8918101f50 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f891400cce0 tx=0x7f891400a620 comp rx=0 tx=0).stop 2024-09-16T00:32:41.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.869+0000 7f89201c4640 1 -- 172.21.15.38:0/4163186939 shutdown_connections 2024-09-16T00:32:41.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.869+0000 7f89201c4640 1 --2- 172.21.15.38:0/4163186939 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f89040a32f0 0x7f89040a57b0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:41.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.869+0000 7f89201c4640 1 --2- 172.21.15.38:0/4163186939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8918107840 0x7f8918100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:41.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.869+0000 7f89201c4640 1 --2- 172.21.15.38:0/4163186939 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8918104f10 0x7f8918100590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:41.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.869+0000 7f89201c4640 1 --2- 172.21.15.38:0/4163186939 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f89181025e0 0x7f8918101f50 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:41.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.869+0000 7f89201c4640 1 -- 172.21.15.38:0/4163186939 >> 172.21.15.38:0/4163186939 conn(0x7f89180fc1e0 msgr2=0x7f8918102e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:41.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.869+0000 7f89201c4640 1 -- 172.21.15.38:0/4163186939 shutdown_connections 2024-09-16T00:32:41.870 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.869+0000 7f89201c4640 1 -- 172.21.15.38:0/4163186939 wait complete. 2024-09-16T00:32:41.882 DEBUG:teuthology.orchestra.run.smithi038:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-09-16T00:32:41.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.993+0000 7ffbc2f13640 1 -- 172.21.15.38:0/3743458472 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbbc100a60 msgr2=0x7ffbbc100e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:41.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.993+0000 7ffbc2f13640 1 --2- 172.21.15.38:0/3743458472 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbbc100a60 0x7ffbbc100e60 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ffbac009900 tx=0x7ffbac02a410 comp rx=0 tx=0).stop 2024-09-16T00:32:41.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.994+0000 7ffbc2f13640 1 -- 172.21.15.38:0/3743458472 shutdown_connections 2024-09-16T00:32:41.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.994+0000 7ffbc2f13640 1 --2- 172.21.15.38:0/3743458472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbbc102610 0x7ffbbc108ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:41.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.994+0000 7ffbc2f13640 1 --2- 172.21.15.38:0/3743458472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffbbc101c50 0x7ffbbc1020d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:41.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.994+0000 7ffbc2f13640 1 --2- 172.21.15.38:0/3743458472 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbbc100a60 0x7ffbbc100e60 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:41.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.994+0000 7ffbc2f13640 1 -- 172.21.15.38:0/3743458472 >> 172.21.15.38:0/3743458472 conn(0x7ffbbc0fc1e0 msgr2=0x7ffbbc0fe600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:41.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.994+0000 7ffbc2f13640 1 -- 172.21.15.38:0/3743458472 shutdown_connections 2024-09-16T00:32:41.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.994+0000 7ffbc2f13640 1 -- 172.21.15.38:0/3743458472 wait complete. 2024-09-16T00:32:41.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.994+0000 7ffbc2f13640 1 Processor -- start 2024-09-16T00:32:41.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.994+0000 7ffbc2f13640 1 -- start start 2024-09-16T00:32:41.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.994+0000 7ffbc2f13640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbbc100a60 0x7ffbbc197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:41.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.994+0000 7ffbc2f13640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffbbc101c50 0x7ffbbc198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:41.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.994+0000 7ffbc2f13640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbbc102610 0x7ffbbc19f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:41.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.994+0000 7ffbc2f13640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ffbbc10c730 con 0x7ffbbc102610 2024-09-16T00:32:41.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.994+0000 7ffbc2f13640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ffbbc10c5b0 con 0x7ffbbc101c50 2024-09-16T00:32:41.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.994+0000 7ffbc2f13640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffbbc10c8b0 con 0x7ffbbc100a60 2024-09-16T00:32:41.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.995+0000 7ffbc14b0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbbc102610 0x7ffbbc19f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:41.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.995+0000 7ffbc14b0640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbbc102610 0x7ffbbc19f3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46980/0 (socket says 172.21.15.38:46980) 2024-09-16T00:32:41.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.995+0000 7ffbc0caf640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbbc100a60 0x7ffbbc197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:41.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.995+0000 7ffbc14b0640 1 -- 172.21.15.38:0/1652706898 learned_addr learned my addr 172.21.15.38:0/1652706898 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:41.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.995+0000 7ffbbbfff640 1 --2- 172.21.15.38:0/1652706898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffbbc101c50 0x7ffbbc198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:41.998 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.995+0000 7ffbc14b0640 1 -- 172.21.15.38:0/1652706898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbbc100a60 msgr2=0x7ffbbc197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:41.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.995+0000 7ffbc14b0640 1 --2- 172.21.15.38:0/1652706898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbbc100a60 0x7ffbbc197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:41.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.995+0000 7ffbc14b0640 1 -- 172.21.15.38:0/1652706898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffbbc101c50 msgr2=0x7ffbbc198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:41.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.995+0000 7ffbc14b0640 1 --2- 172.21.15.38:0/1652706898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffbbc101c50 0x7ffbbc198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:41.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.995+0000 7ffbc14b0640 1 -- 172.21.15.38:0/1652706898 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbbc19fa30 con 0x7ffbbc102610 2024-09-16T00:32:41.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.995+0000 7ffbc14b0640 1 --2- 172.21.15.38:0/1652706898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbbc102610 0x7ffbbc19f3c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400eaa0 tx=0x7ffbb400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:41.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.995+0000 7ffbb9ffb640 1 -- 172.21.15.38:0/1652706898 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb400cdb0 con 0x7ffbbc102610 2024-09-16T00:32:41.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.995+0000 7ffbb9ffb640 1 -- 172.21.15.38:0/1652706898 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffbb4005020 con 0x7ffbbc102610 2024-09-16T00:32:41.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.995+0000 7ffbb9ffb640 1 -- 172.21.15.38:0/1652706898 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb40116b0 con 0x7ffbbc102610 2024-09-16T00:32:41.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.995+0000 7ffbc2f13640 1 -- 172.21.15.38:0/1652706898 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbbc19fd20 con 0x7ffbbc102610 2024-09-16T00:32:41.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.996+0000 7ffbc2f13640 1 -- 172.21.15.38:0/1652706898 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbbc1a0260 con 0x7ffbbc102610 2024-09-16T00:32:42.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.997+0000 7ffbc2f13640 1 -- 172.21.15.38:0/1652706898 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbbc1a04f0 con 0x7ffbbc102610 2024-09-16T00:32:42.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.998+0000 7ffbb9ffb640 1 -- 172.21.15.38:0/1652706898 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7ffbb4005560 con 0x7ffbbc102610 2024-09-16T00:32:42.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.998+0000 7ffbb9ffb640 1 --2- 172.21.15.38:0/1652706898 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ffba40a3370 0x7ffba40a5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:42.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.998+0000 7ffbb9ffb640 1 -- 172.21.15.38:0/1652706898 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7ffbb40cfd90 con 0x7ffbbc102610 2024-09-16T00:32:42.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.998+0000 7ffbc0caf640 1 --2- 172.21.15.38:0/1652706898 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ffba40a3370 0x7ffba40a5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:42.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:41.999+0000 7ffbc0caf640 1 --2- 172.21.15.38:0/1652706898 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ffba40a3370 0x7ffba40a5830 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ffbac02aa00 tx=0x7ffbac0315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:42.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.002+0000 7ffbb9ffb640 1 -- 172.21.15.38:0/1652706898 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ffbb4016030 con 0x7ffbbc102610 2024-09-16T00:32:42.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.207+0000 7ffbc2f13640 1 -- 172.21.15.38:0/1652706898 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7ffbbc0fe3c0 con 0x7ffba40a3370 2024-09-16T00:32:42.213 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:41 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/4163186939' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-16T00:32:42.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:41 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/4163186939' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-16T00:32:42.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.224+0000 7ffbb9ffb640 1 -- 172.21.15.38:0/1652706898 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7ffbbc0fe3c0 con 0x7ffba40a3370 2024-09-16T00:32:42.224 INFO:teuthology.orchestra.run.smithi038.stdout:{} 2024-09-16T00:32:42.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.226+0000 7ffbc2f13640 1 -- 172.21.15.38:0/1652706898 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ffba40a3370 msgr2=0x7ffba40a5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:42.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.226+0000 7ffbc2f13640 1 --2- 172.21.15.38:0/1652706898 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ffba40a3370 0x7ffba40a5830 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ffbac02aa00 tx=0x7ffbac0315b0 comp rx=0 tx=0).stop 2024-09-16T00:32:42.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.226+0000 7ffbc2f13640 1 -- 172.21.15.38:0/1652706898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbbc102610 msgr2=0x7ffbbc19f3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:42.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.226+0000 7ffbc2f13640 1 --2- 172.21.15.38:0/1652706898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbbc102610 0x7ffbbc19f3c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400eaa0 tx=0x7ffbb400ef60 comp rx=0 tx=0).stop 2024-09-16T00:32:42.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.226+0000 7ffbc2f13640 1 -- 172.21.15.38:0/1652706898 shutdown_connections 2024-09-16T00:32:42.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.226+0000 7ffbc2f13640 1 --2- 172.21.15.38:0/1652706898 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ffba40a3370 0x7ffba40a5830 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:42.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.226+0000 7ffbc2f13640 1 --2- 172.21.15.38:0/1652706898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffbbc102610 0x7ffbbc19f3c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:42.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.226+0000 7ffbc2f13640 1 --2- 172.21.15.38:0/1652706898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffbbc101c50 0x7ffbbc198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:42.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.226+0000 7ffbc2f13640 1 --2- 172.21.15.38:0/1652706898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffbbc100a60 0x7ffbbc197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:42.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.226+0000 7ffbc2f13640 1 -- 172.21.15.38:0/1652706898 >> 172.21.15.38:0/1652706898 conn(0x7ffbbc0fc1e0 msgr2=0x7ffbbc0fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:42.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.226+0000 7ffbc2f13640 1 -- 172.21.15.38:0/1652706898 shutdown_connections 2024-09-16T00:32:42.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.226+0000 7ffbc2f13640 1 -- 172.21.15.38:0/1652706898 wait complete. 2024-09-16T00:32:42.237 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-16T00:32:42.250 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-16T00:32:42.261 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T00:32:42.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:41 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/4163186939' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-09-16T00:32:42.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.364+0000 7f624ef8d640 1 -- 172.21.15.38:0/2928682032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62481025c0 msgr2=0x7f6248108e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:42.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.364+0000 7f624ef8d640 1 --2- 172.21.15.38:0/2928682032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62481025c0 0x7f6248108e50 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6240009ca0 tx=0x7f624002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:32:42.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.364+0000 7f624ef8d640 1 -- 172.21.15.38:0/2928682032 shutdown_connections 2024-09-16T00:32:42.367 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.364+0000 7f624ef8d640 1 --2- 172.21.15.38:0/2928682032 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f62481025c0 0x7f6248108e50 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:42.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.364+0000 7f624ef8d640 1 --2- 172.21.15.38:0/2928682032 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6248101c00 0x7f6248102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:42.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.364+0000 7f624ef8d640 1 --2- 172.21.15.38:0/2928682032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6248100a10 0x7f6248100e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:42.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.364+0000 7f624ef8d640 1 -- 172.21.15.38:0/2928682032 >> 172.21.15.38:0/2928682032 conn(0x7f62480fc170 msgr2=0x7f62480fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:42.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.364+0000 7f624ef8d640 1 -- 172.21.15.38:0/2928682032 shutdown_connections 2024-09-16T00:32:42.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.364+0000 7f624ef8d640 1 -- 172.21.15.38:0/2928682032 wait complete. 2024-09-16T00:32:42.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.365+0000 7f624ef8d640 1 Processor -- start 2024-09-16T00:32:42.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.365+0000 7f624ef8d640 1 -- start start 2024-09-16T00:32:42.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.365+0000 7f624ef8d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6248100a10 0x7f6248199ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:42.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.365+0000 7f624ef8d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6248101c00 0x7f624819a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:42.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.365+0000 7f624ef8d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62481025c0 0x7f62481a15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:42.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.365+0000 7f624ef8d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f624810c670 con 0x7f6248100a10 2024-09-16T00:32:42.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.365+0000 7f624ef8d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f624810c4f0 con 0x7f62481025c0 2024-09-16T00:32:42.368 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.365+0000 7f624ef8d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f624810c7f0 con 0x7f6248101c00 2024-09-16T00:32:42.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.365+0000 7f624cd29640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6248100a10 0x7f6248199ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:42.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.365+0000 7f624cd29640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6248100a10 0x7f6248199ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47008/0 (socket says 172.21.15.38:47008) 2024-09-16T00:32:42.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.365+0000 7f624cd29640 1 -- 172.21.15.38:0/133326061 learned_addr learned my addr 172.21.15.38:0/133326061 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:42.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.365+0000 7f6247fff640 1 --2- 172.21.15.38:0/133326061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6248101c00 0x7f624819a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:42.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.365+0000 7f624d52a640 1 --2- 172.21.15.38:0/133326061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62481025c0 0x7f62481a15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:42.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.365+0000 7f624cd29640 1 -- 172.21.15.38:0/133326061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6248101c00 msgr2=0x7f624819a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:42.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.365+0000 7f624cd29640 1 --2- 172.21.15.38:0/133326061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6248101c00 0x7f624819a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:42.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.365+0000 7f624cd29640 1 -- 172.21.15.38:0/133326061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62481025c0 msgr2=0x7f62481a15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:42.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.365+0000 7f624cd29640 1 --2- 172.21.15.38:0/133326061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62481025c0 0x7f62481a15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:42.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.365+0000 7f624cd29640 1 -- 172.21.15.38:0/133326061 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62481a1c10 con 0x7f6248100a10 2024-09-16T00:32:42.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.366+0000 7f624cd29640 1 --2- 172.21.15.38:0/133326061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6248100a10 0x7f6248199ff0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f623c00d770 tx=0x7f623c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:42.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.366+0000 7f6245ffb640 1 -- 172.21.15.38:0/133326061 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623c017070 con 0x7f6248100a10 2024-09-16T00:32:42.369 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.366+0000 7f6245ffb640 1 -- 172.21.15.38:0/133326061 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f623c009c30 con 0x7f6248100a10 2024-09-16T00:32:42.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.366+0000 7f624ef8d640 1 -- 172.21.15.38:0/133326061 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62481a1f00 con 0x7f6248100a10 2024-09-16T00:32:42.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.366+0000 7f6245ffb640 1 -- 172.21.15.38:0/133326061 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f623c003dd0 con 0x7f6248100a10 2024-09-16T00:32:42.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.366+0000 7f624ef8d640 1 -- 172.21.15.38:0/133326061 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62481a2360 con 0x7f6248100a10 2024-09-16T00:32:42.370 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.367+0000 7f624ef8d640 1 -- 172.21.15.38:0/133326061 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62481a25c0 con 0x7f6248100a10 2024-09-16T00:32:42.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.368+0000 7f6245ffb640 1 -- 172.21.15.38:0/133326061 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f623c003fb0 con 0x7f6248100a10 2024-09-16T00:32:42.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.369+0000 7f6245ffb640 1 --2- 172.21.15.38:0/133326061 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f62380a3250 0x7f62380a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:42.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.369+0000 7f6247fff640 1 --2- 172.21.15.38:0/133326061 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f62380a3250 0x7f62380a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:42.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.369+0000 7f6245ffb640 1 -- 172.21.15.38:0/133326061 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f623c0cf730 con 0x7f6248100a10 2024-09-16T00:32:42.371 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.369+0000 7f6247fff640 1 --2- 172.21.15.38:0/133326061 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f62380a3250 0x7f62380a5710 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6234005fd0 tx=0x7f6234005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:42.373 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.373+0000 7f6245ffb640 1 -- 172.21.15.38:0/133326061 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f623c09c070 con 0x7f6248100a10 2024-09-16T00:32:42.465 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:32:42 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:32:42.212+0000 7f1b6169f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:32:42.465 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:32:42 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:32:42.212+0000 7f1b6169f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:32:42.465 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:32:42 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:32:42.212+0000 7f1b6169f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:32:42.465 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:32:42 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:32:42.212+0000 7f1b6169f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:32:42.465 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:32:42 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:32:42.212+0000 7f1b6169f640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-16T00:32:42.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.552+0000 7f624ef8d640 1 -- 172.21.15.38:0/133326061 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f62481a2e90 con 0x7f6248100a10 2024-09-16T00:32:42.554 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.553+0000 7f6245ffb640 1 -- 172.21.15.38:0/133326061 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24767 (secure 0 0 0) 0x7f623c009de0 con 0x7f6248100a10 2024-09-16T00:32:42.554 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:32:42.554 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":92,"fsid":"983b2072-73c1-11ef-bceb-c7b262605968","created":"2024-09-16T00:22:34.897053+0000","modified":"2024-09-16T00:32:24.296695+0000","last_up_change":"2024-09-16T00:29:34.503134+0000","last_in_change":"2024-09-16T00:29:10.135656+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:25:41.931977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:30:22.741889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:30:25.042590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:30:31.554350+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3afc20e3-869f-43a9-a911-b0e8d07de86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6803","nonce":1622028921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6805","nonce":1622028921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6809","nonce":1622028921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6807","nonce":1622028921}]},"public_addr":"172.21.15.38:6803/1622028921","cluster_addr":"172.21.15.38:6805/1622028921","heartbeat_back_addr":"172.21.15.38:6809/1622028921","heartbeat_front_addr":"172.21.15.38:6807/1622028921","state":["exists","up"]},{"osd":1,"uuid":"6d9d7acb-45ed-4543-8afc-6433fc7f096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6801","nonce":3811690336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6803","nonce":3811690336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6807","nonce":3811690336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6805","nonce":3811690336}]},"public_addr":"172.21.15.73:6801/3811690336","cluster_addr":"172.21.15.73:6803/3811690336","heartbeat_back_addr":"172.21.15.73:6807/3811690336","heartbeat_front_addr":"172.21.15.73:6805/3811690336","state":["exists","up"]},{"osd":2,"uuid":"48690f35-99e9-4f45-9bee-487857caa8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6801","nonce":1754741655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6803","nonce":1754741655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6807","nonce":1754741655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6805","nonce":1754741655}]},"public_addr":"172.21.15.122:6801/1754741655","cluster_addr":"172.21.15.122:6803/1754741655","heartbeat_back_addr":"172.21.15.122:6807/1754741655","heartbeat_front_addr":"172.21.15.122:6805/1754741655","state":["exists","up"]},{"osd":3,"uuid":"28ebb4a0-92e7-4f57-82a5-b9b761eca269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6811","nonce":3523190079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6813","nonce":3523190079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6817","nonce":3523190079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6815","nonce":3523190079}]},"public_addr":"172.21.15.38:6811/3523190079","cluster_addr":"172.21.15.38:6813/3523190079","heartbeat_back_addr":"172.21.15.38:6817/3523190079","heartbeat_front_addr":"172.21.15.38:6815/3523190079","state":["exists","up"]},{"osd":4,"uuid":"5ea363f7-7ad8-45ed-b7e4-4b010fe7f8cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6809","nonce":1469927628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6811","nonce":1469927628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6815","nonce":1469927628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6813","nonce":1469927628}]},"public_addr":"172.21.15.73:6809/1469927628","cluster_addr":"172.21.15.73:6811/1469927628","heartbeat_back_addr":"172.21.15.73:6815/1469927628","heartbeat_front_addr":"172.21.15.73:6813/1469927628","state":["exists","up"]},{"osd":5,"uuid":"e3c66d43-dd28-4bb1-a886-62a869be82cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6809","nonce":2954413058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6811","nonce":2954413058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6815","nonce":2954413058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6813","nonce":2954413058}]},"public_addr":"172.21.15.122:6809/2954413058","cluster_addr":"172.21.15.122:6811/2954413058","heartbeat_back_addr":"172.21.15.122:6815/2954413058","heartbeat_front_addr":"172.21.15.122:6813/2954413058","state":["exists","up"]},{"osd":6,"uuid":"211497ac-7c48-4095-bfdb-1d4df0149f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6819","nonce":1415719550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6821","nonce":1415719550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6825","nonce":1415719550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6823","nonce":1415719550}]},"public_addr":"172.21.15.38:6819/1415719550","cluster_addr":"172.21.15.38:6821/1415719550","heartbeat_back_addr":"172.21.15.38:6825/1415719550","heartbeat_front_addr":"172.21.15.38:6823/1415719550","state":["exists","up"]},{"osd":7,"uuid":"06512315-3f5e-44f8-976d-d3f5f1ba00e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6817","nonce":2634361473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6819","nonce":2634361473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6823","nonce":2634361473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6821","nonce":2634361473}]},"public_addr":"172.21.15.73:6817/2634361473","cluster_addr":"172.21.15.73:6819/2634361473","heartbeat_back_addr":"172.21.15.73:6823/2634361473","heartbeat_front_addr":"172.21.15.73:6821/2634361473","state":["exists","up"]},{"osd":8,"uuid":"db703c0a-1f87-46f0-be5f-f826f7f9daac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6817","nonce":1800567633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6819","nonce":1800567633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6823","nonce":1800567633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6821","nonce":1800567633}]},"public_addr":"172.21.15.122:6817/1800567633","cluster_addr":"172.21.15.122:6819/1800567633","heartbeat_back_addr":"172.21.15.122:6823/1800567633","heartbeat_front_addr":"172.21.15.122:6821/1800567633","state":["exists","up"]},{"osd":9,"uuid":"f5384200-b3b4-4936-85e1-fc57a9fe2bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6827","nonce":445727833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6829","nonce":445727833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6833","nonce":445727833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6831","nonce":445727833}]},"public_addr":"172.21.15.38:6827/445727833","cluster_addr":"172.21.15.38:6829/445727833","heartbeat_back_addr":"172.21.15.38:6833/445727833","heartbeat_front_addr":"172.21.15.38:6831/445727833","state":["exists","up"]},{"osd":10,"uuid":"c22883d9-6672-4796-a961-963d21eb37fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6825","nonce":43297139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6827","nonce":43297139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6831","nonce":43297139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6829","nonce":43297139}]},"public_addr":"172.21.15.73:6825/43297139","cluster_addr":"172.21.15.73:6827/43297139","heartbeat_back_addr":"172.21.15.73:6831/43297139","heartbeat_front_addr":"172.21.15.73:6829/43297139","state":["exists","up"]},{"osd":11,"uuid":"9f98e8cb-1bff-4b8b-aa19-56b691387383","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6825","nonce":1886446061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6827","nonce":1886446061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6831","nonce":1886446061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6829","nonce":1886446061}]},"public_addr":"172.21.15.122:6825/1886446061","cluster_addr":"172.21.15.122:6827/1886446061","heartbeat_back_addr":"172.21.15.122:6831/1886446061","heartbeat_front_addr":"172.21.15.122:6829/1886446061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:47.384012+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:12.834488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:38.770055+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:04.800466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:30.808076+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:56.646967+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:27:22.721270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:27:48.510011+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:28:13.922944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:28:40.514947+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:29:06.031052+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:29:31.882062+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6801/1779324916":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/924150447":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/141237849":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/2302081750":"2024-09-17T00:22:56.844212+0000","172.21.15.38:0/2245468594":"2024-09-17T00:22:56.844212+0000","172.21.15.38:6801/583006373":"2024-09-17T00:22:56.844212+0000","172.21.15.38:0/2435492667":"2024-09-17T00:23:21.417483+0000","172.21.15.38:6800/583006373":"2024-09-17T00:22:56.844212+0000","172.21.15.38:6800/1779324916":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/1103516837":"2024-09-17T00:22:56.844212+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:32:42.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.556+0000 7f624ef8d640 1 -- 172.21.15.38:0/133326061 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f62380a3250 msgr2=0x7f62380a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:42.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.556+0000 7f624ef8d640 1 --2- 172.21.15.38:0/133326061 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f62380a3250 0x7f62380a5710 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6234005fd0 tx=0x7f6234005850 comp rx=0 tx=0).stop 2024-09-16T00:32:42.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.556+0000 7f624ef8d640 1 -- 172.21.15.38:0/133326061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6248100a10 msgr2=0x7f6248199ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:42.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.556+0000 7f624ef8d640 1 --2- 172.21.15.38:0/133326061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6248100a10 0x7f6248199ff0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f623c00d770 tx=0x7f623c00dc30 comp rx=0 tx=0).stop 2024-09-16T00:32:42.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.556+0000 7f624ef8d640 1 -- 172.21.15.38:0/133326061 shutdown_connections 2024-09-16T00:32:42.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.556+0000 7f624ef8d640 1 --2- 172.21.15.38:0/133326061 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f62380a3250 0x7f62380a5710 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:42.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.556+0000 7f624ef8d640 1 --2- 172.21.15.38:0/133326061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62481025c0 0x7f62481a15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:42.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.556+0000 7f624ef8d640 1 --2- 172.21.15.38:0/133326061 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6248101c00 0x7f624819a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:42.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.556+0000 7f624ef8d640 1 --2- 172.21.15.38:0/133326061 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6248100a10 0x7f6248199ff0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:42.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.556+0000 7f624ef8d640 1 -- 172.21.15.38:0/133326061 >> 172.21.15.38:0/133326061 conn(0x7f62480fc170 msgr2=0x7f62480693b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:42.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.556+0000 7f624ef8d640 1 -- 172.21.15.38:0/133326061 shutdown_connections 2024-09-16T00:32:42.557 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.556+0000 7f624ef8d640 1 -- 172.21.15.38:0/133326061 wait complete. 2024-09-16T00:32:42.567 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T00:25:41.931977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-16T00:30:22.741889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-16T00:30:25.042590+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-16T00:30:31.554350+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-16T00:32:42.568 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T00:32:42.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.675+0000 7fbed20c9640 1 -- 172.21.15.38:0/2383258613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbecc102600 msgr2=0x7fbecc1049f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:42.677 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.675+0000 7fbed20c9640 1 --2- 172.21.15.38:0/2383258613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbecc102600 0x7fbecc1049f0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fbebc009900 tx=0x7fbebc02a410 comp rx=0 tx=0).stop 2024-09-16T00:32:42.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.675+0000 7fbed20c9640 1 -- 172.21.15.38:0/2383258613 shutdown_connections 2024-09-16T00:32:42.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.675+0000 7fbed20c9640 1 --2- 172.21.15.38:0/2383258613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbecc107860 0x7fbecc109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:42.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.675+0000 7fbed20c9640 1 --2- 172.21.15.38:0/2383258613 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbecc104f30 0x7fbecc107320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:42.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.675+0000 7fbed20c9640 1 --2- 172.21.15.38:0/2383258613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbecc102600 0x7fbecc1049f0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:42.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.675+0000 7fbed20c9640 1 -- 172.21.15.38:0/2383258613 >> 172.21.15.38:0/2383258613 conn(0x7fbecc0fc140 msgr2=0x7fbecc0fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:42.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.675+0000 7fbed20c9640 1 -- 172.21.15.38:0/2383258613 shutdown_connections 2024-09-16T00:32:42.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.675+0000 7fbed20c9640 1 -- 172.21.15.38:0/2383258613 wait complete. 2024-09-16T00:32:42.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.675+0000 7fbed20c9640 1 Processor -- start 2024-09-16T00:32:42.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.675+0000 7fbed20c9640 1 -- start start 2024-09-16T00:32:42.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.676+0000 7fbed20c9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbecc102600 0x7fbecc197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:42.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.676+0000 7fbed20c9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbecc104f30 0x7fbecc198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:42.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.676+0000 7fbed20c9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbecc107860 0x7fbecc19f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:42.678 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.676+0000 7fbed20c9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbecc10d7e0 con 0x7fbecc102600 2024-09-16T00:32:42.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.676+0000 7fbed20c9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbecc10d660 con 0x7fbecc104f30 2024-09-16T00:32:42.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.676+0000 7fbed20c9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbecc10d960 con 0x7fbecc107860 2024-09-16T00:32:42.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.676+0000 7fbecb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbecc102600 0x7fbecc197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:42.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.676+0000 7fbecb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbecc102600 0x7fbecc197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47024/0 (socket says 172.21.15.38:47024) 2024-09-16T00:32:42.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.676+0000 7fbecb7fe640 1 -- 172.21.15.38:0/1275390073 learned_addr learned my addr 172.21.15.38:0/1275390073 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:42.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.676+0000 7fbecaffd640 1 --2- 172.21.15.38:0/1275390073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbecc104f30 0x7fbecc198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:42.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.676+0000 7fbecbfff640 1 --2- 172.21.15.38:0/1275390073 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbecc107860 0x7fbecc19f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:42.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.676+0000 7fbecb7fe640 1 -- 172.21.15.38:0/1275390073 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbecc107860 msgr2=0x7fbecc19f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:42.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.676+0000 7fbecb7fe640 1 --2- 172.21.15.38:0/1275390073 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbecc107860 0x7fbecc19f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:42.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.676+0000 7fbecb7fe640 1 -- 172.21.15.38:0/1275390073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbecc104f30 msgr2=0x7fbecc198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:42.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.676+0000 7fbecb7fe640 1 --2- 172.21.15.38:0/1275390073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbecc104f30 0x7fbecc198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:42.679 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.676+0000 7fbecb7fe640 1 -- 172.21.15.38:0/1275390073 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbecc19fb10 con 0x7fbecc102600 2024-09-16T00:32:42.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.676+0000 7fbecb7fe640 1 --2- 172.21.15.38:0/1275390073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbecc102600 0x7fbecc197ef0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbebc009230 tx=0x7fbebc004360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:42.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.677+0000 7fbec8ff9640 1 -- 172.21.15.38:0/1275390073 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbebc03f030 con 0x7fbecc102600 2024-09-16T00:32:42.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.677+0000 7fbec8ff9640 1 -- 172.21.15.38:0/1275390073 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbebc003ce0 con 0x7fbecc102600 2024-09-16T00:32:42.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.677+0000 7fbed20c9640 1 -- 172.21.15.38:0/1275390073 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbecc19fda0 con 0x7fbecc102600 2024-09-16T00:32:42.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.677+0000 7fbec8ff9640 1 -- 172.21.15.38:0/1275390073 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbebc035070 con 0x7fbecc102600 2024-09-16T00:32:42.680 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.677+0000 7fbed20c9640 1 -- 172.21.15.38:0/1275390073 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbecc1a01d0 con 0x7fbecc102600 2024-09-16T00:32:42.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.678+0000 7fbed20c9640 1 -- 172.21.15.38:0/1275390073 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbecc0fdc30 con 0x7fbecc102600 2024-09-16T00:32:42.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.679+0000 7fbec8ff9640 1 -- 172.21.15.38:0/1275390073 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7fbebc042030 con 0x7fbecc102600 2024-09-16T00:32:42.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.680+0000 7fbec8ff9640 1 --2- 172.21.15.38:0/1275390073 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fbeb40a3250 0x7fbeb40a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:42.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.680+0000 7fbecaffd640 1 --2- 172.21.15.38:0/1275390073 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fbeb40a3250 0x7fbeb40a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:42.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.680+0000 7fbec8ff9640 1 -- 172.21.15.38:0/1275390073 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7fbebc0f0680 con 0x7fbecc102600 2024-09-16T00:32:42.681 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.680+0000 7fbecaffd640 1 --2- 172.21.15.38:0/1275390073 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fbeb40a3250 0x7fbeb40a5710 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbecc199400 tx=0x7fbec000b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:42.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.684+0000 7fbec8ff9640 1 -- 172.21.15.38:0/1275390073 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbebc0bd020 con 0x7fbecc102600 2024-09-16T00:32:42.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.859+0000 7fbed20c9640 1 -- 172.21.15.38:0/1275390073 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbecc062e90 con 0x7fbecc102600 2024-09-16T00:32:42.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.859+0000 7fbec8ff9640 1 -- 172.21.15.38:0/1275390073 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7fbebc0bc920 con 0x7fbecc102600 2024-09-16T00:32:42.861 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-09-16T00:32:42.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.861+0000 7fbed20c9640 1 -- 172.21.15.38:0/1275390073 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fbeb40a3250 msgr2=0x7fbeb40a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:42.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.861+0000 7fbed20c9640 1 --2- 172.21.15.38:0/1275390073 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fbeb40a3250 0x7fbeb40a5710 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fbecc199400 tx=0x7fbec000b390 comp rx=0 tx=0).stop 2024-09-16T00:32:42.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.861+0000 7fbed20c9640 1 -- 172.21.15.38:0/1275390073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbecc102600 msgr2=0x7fbecc197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:42.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.861+0000 7fbed20c9640 1 --2- 172.21.15.38:0/1275390073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbecc102600 0x7fbecc197ef0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fbebc009230 tx=0x7fbebc004360 comp rx=0 tx=0).stop 2024-09-16T00:32:42.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.861+0000 7fbed20c9640 1 -- 172.21.15.38:0/1275390073 shutdown_connections 2024-09-16T00:32:42.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.861+0000 7fbed20c9640 1 --2- 172.21.15.38:0/1275390073 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fbeb40a3250 0x7fbeb40a5710 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:42.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.861+0000 7fbed20c9640 1 --2- 172.21.15.38:0/1275390073 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbecc107860 0x7fbecc19f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:42.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.861+0000 7fbed20c9640 1 --2- 172.21.15.38:0/1275390073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbecc104f30 0x7fbecc198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:42.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.861+0000 7fbed20c9640 1 --2- 172.21.15.38:0/1275390073 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbecc102600 0x7fbecc197ef0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:42.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.861+0000 7fbed20c9640 1 -- 172.21.15.38:0/1275390073 >> 172.21.15.38:0/1275390073 conn(0x7fbecc0fc140 msgr2=0x7fbecc102fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:42.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.861+0000 7fbed20c9640 1 -- 172.21.15.38:0/1275390073 shutdown_connections 2024-09-16T00:32:42.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.861+0000 7fbed20c9640 1 -- 172.21.15.38:0/1275390073 wait complete. 2024-09-16T00:32:42.873 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-16T00:32:42.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.980+0000 7f5404913640 1 -- 172.21.15.38:0/924743675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5400101c20 msgr2=0x7f54001020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:42.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.980+0000 7f5404913640 1 --2- 172.21.15.38:0/924743675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5400101c20 0x7f54001020a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f53f4009900 tx=0x7f53f402a410 comp rx=0 tx=0).stop 2024-09-16T00:32:42.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.980+0000 7f5404913640 1 -- 172.21.15.38:0/924743675 shutdown_connections 2024-09-16T00:32:42.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.980+0000 7f5404913640 1 --2- 172.21.15.38:0/924743675 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f54001025e0 0x7f5400108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:42.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.980+0000 7f5404913640 1 --2- 172.21.15.38:0/924743675 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5400101c20 0x7f54001020a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:42.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.980+0000 7f5404913640 1 --2- 172.21.15.38:0/924743675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5400100a30 0x7f5400100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:42.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.980+0000 7f5404913640 1 -- 172.21.15.38:0/924743675 >> 172.21.15.38:0/924743675 conn(0x7f54000fc1b0 msgr2=0x7f54000fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:42.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.980+0000 7f5404913640 1 -- 172.21.15.38:0/924743675 shutdown_connections 2024-09-16T00:32:42.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.980+0000 7f5404913640 1 -- 172.21.15.38:0/924743675 wait complete. 2024-09-16T00:32:42.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.981+0000 7f5404913640 1 Processor -- start 2024-09-16T00:32:42.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.981+0000 7f5404913640 1 -- start start 2024-09-16T00:32:42.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.981+0000 7f5404913640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5400100a30 0x7f5400197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:42.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.981+0000 7f5404913640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5400101c20 0x7f5400198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:42.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.981+0000 7f5404913640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54001025e0 0x7f540019f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:42.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.981+0000 7f5404913640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f540010d7f0 con 0x7f54001025e0 2024-09-16T00:32:42.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.981+0000 7f5404913640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f540010d670 con 0x7f5400101c20 2024-09-16T00:32:42.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.981+0000 7f5404913640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f540010d970 con 0x7f5400100a30 2024-09-16T00:32:42.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.981+0000 7f53fed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54001025e0 0x7f540019f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:42.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.981+0000 7f53fed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54001025e0 0x7f540019f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47046/0 (socket says 172.21.15.38:47046) 2024-09-16T00:32:42.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.981+0000 7f53fed9d640 1 -- 172.21.15.38:0/2591920086 learned_addr learned my addr 172.21.15.38:0/2591920086 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:42.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.981+0000 7f53fe59c640 1 --2- 172.21.15.38:0/2591920086 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5400100a30 0x7f5400197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:42.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.981+0000 7f53fdd9b640 1 --2- 172.21.15.38:0/2591920086 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5400101c20 0x7f5400198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:42.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.981+0000 7f53fed9d640 1 -- 172.21.15.38:0/2591920086 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5400100a30 msgr2=0x7f5400197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:42.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.982+0000 7f53fed9d640 1 --2- 172.21.15.38:0/2591920086 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5400100a30 0x7f5400197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:42.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.982+0000 7f53fed9d640 1 -- 172.21.15.38:0/2591920086 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5400101c20 msgr2=0x7f5400198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:42.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.982+0000 7f53fed9d640 1 --2- 172.21.15.38:0/2591920086 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5400101c20 0x7f5400198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:42.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.982+0000 7f53fed9d640 1 -- 172.21.15.38:0/2591920086 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f540019fb10 con 0x7f54001025e0 2024-09-16T00:32:42.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.982+0000 7f53fed9d640 1 --2- 172.21.15.38:0/2591920086 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54001025e0 0x7f540019f4a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f53f800d750 tx=0x7f53f800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:42.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.982+0000 7f53ef7fe640 1 -- 172.21.15.38:0/2591920086 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f8017070 con 0x7f54001025e0 2024-09-16T00:32:42.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.982+0000 7f53ef7fe640 1 -- 172.21.15.38:0/2591920086 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f53f8005500 con 0x7f54001025e0 2024-09-16T00:32:42.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.982+0000 7f5404913640 1 -- 172.21.15.38:0/2591920086 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f540019fe00 con 0x7f54001025e0 2024-09-16T00:32:42.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.982+0000 7f53ef7fe640 1 -- 172.21.15.38:0/2591920086 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53f8004d90 con 0x7f54001025e0 2024-09-16T00:32:42.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.982+0000 7f5404913640 1 -- 172.21.15.38:0/2591920086 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54001a02c0 con 0x7f54001025e0 2024-09-16T00:32:42.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.983+0000 7f5404913640 1 -- 172.21.15.38:0/2591920086 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5400107e90 con 0x7f54001025e0 2024-09-16T00:32:42.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.984+0000 7f53ef7fe640 1 -- 172.21.15.38:0/2591920086 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f53f8004f30 con 0x7f54001025e0 2024-09-16T00:32:42.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.985+0000 7f53ef7fe640 1 --2- 172.21.15.38:0/2591920086 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f53e80a3250 0x7f53e80a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:42.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.985+0000 7f53fe59c640 1 --2- 172.21.15.38:0/2591920086 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f53e80a3250 0x7f53e80a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:42.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.985+0000 7f53ef7fe640 1 -- 172.21.15.38:0/2591920086 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f53f80cf5b0 con 0x7f54001025e0 2024-09-16T00:32:42.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.985+0000 7f53fe59c640 1 --2- 172.21.15.38:0/2591920086 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f53e80a3250 0x7f53e80a5710 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5400101a80 tx=0x7f53f0009670 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:42.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:42.989+0000 7f53ef7fe640 1 -- 172.21.15.38:0/2591920086 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f53f809b900 con 0x7f54001025e0 2024-09-16T00:32:43.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.167+0000 7f5404913640 1 -- 172.21.15.38:0/2591920086 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5400062e90 con 0x7f54001025e0 2024-09-16T00:32:43.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.167+0000 7f53ef7fe640 1 -- 172.21.15.38:0/2591920086 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f53f809fc00 con 0x7f54001025e0 2024-09-16T00:32:43.167 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-09-16T00:32:43.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.169+0000 7f5404913640 1 -- 172.21.15.38:0/2591920086 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f53e80a3250 msgr2=0x7f53e80a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:43.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.169+0000 7f5404913640 1 --2- 172.21.15.38:0/2591920086 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f53e80a3250 0x7f53e80a5710 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5400101a80 tx=0x7f53f0009670 comp rx=0 tx=0).stop 2024-09-16T00:32:43.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.169+0000 7f5404913640 1 -- 172.21.15.38:0/2591920086 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54001025e0 msgr2=0x7f540019f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:43.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.169+0000 7f5404913640 1 --2- 172.21.15.38:0/2591920086 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54001025e0 0x7f540019f4a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f53f800d750 tx=0x7f53f800dc10 comp rx=0 tx=0).stop 2024-09-16T00:32:43.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.169+0000 7f5404913640 1 -- 172.21.15.38:0/2591920086 shutdown_connections 2024-09-16T00:32:43.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.169+0000 7f5404913640 1 --2- 172.21.15.38:0/2591920086 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f53e80a3250 0x7f53e80a5710 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:43.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.169+0000 7f5404913640 1 --2- 172.21.15.38:0/2591920086 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f54001025e0 0x7f540019f4a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:43.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.169+0000 7f5404913640 1 --2- 172.21.15.38:0/2591920086 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5400101c20 0x7f5400198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:43.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.169+0000 7f5404913640 1 --2- 172.21.15.38:0/2591920086 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5400100a30 0x7f5400197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:43.170 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.169+0000 7f5404913640 1 -- 172.21.15.38:0/2591920086 >> 172.21.15.38:0/2591920086 conn(0x7f54000fc1b0 msgr2=0x7f5400105060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:43.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.170+0000 7f5404913640 1 -- 172.21.15.38:0/2591920086 shutdown_connections 2024-09-16T00:32:43.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.170+0000 7f5404913640 1 -- 172.21.15.38:0/2591920086 wait complete. 2024-09-16T00:32:43.180 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:42 smithi038 ceph-mon[30945]: pgmap v362: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T00:32:43.180 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:42 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/4163186939' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-16T00:32:43.180 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:42 smithi038 ceph-mon[30945]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-16T00:32:43.180 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:42 smithi038 ceph-mon[30945]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:43.180 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:42 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/133326061' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:32:43.180 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:42 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/1275390073' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T00:32:43.181 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-16T00:32:43.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:42 smithi122 ceph-mon[35412]: pgmap v362: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T00:32:43.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:42 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/4163186939' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-16T00:32:43.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:42 smithi122 ceph-mon[35412]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-16T00:32:43.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:42 smithi122 ceph-mon[35412]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:43.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:42 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/133326061' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:32:43.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:42 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/1275390073' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T00:32:43.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.288+0000 7f3f55a42640 1 -- 172.21.15.38:0/1446132742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f50101b30 msgr2=0x7f3f50101f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:43.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.288+0000 7f3f55a42640 1 --2- 172.21.15.38:0/1446132742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f50101b30 0x7f3f50101f30 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f3f40009900 tx=0x7f3f4002a410 comp rx=0 tx=0).stop 2024-09-16T00:32:43.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.288+0000 7f3f55a42640 1 -- 172.21.15.38:0/1446132742 shutdown_connections 2024-09-16T00:32:43.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.288+0000 7f3f55a42640 1 --2- 172.21.15.38:0/1446132742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f501036e0 0x7f3f50109f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:43.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.288+0000 7f3f55a42640 1 --2- 172.21.15.38:0/1446132742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f50102d20 0x7f3f501031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:43.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.288+0000 7f3f55a42640 1 --2- 172.21.15.38:0/1446132742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f50101b30 0x7f3f50101f30 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:43.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.288+0000 7f3f55a42640 1 -- 172.21.15.38:0/1446132742 >> 172.21.15.38:0/1446132742 conn(0x7f3f500fd290 msgr2=0x7f3f500ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:43.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.288+0000 7f3f55a42640 1 -- 172.21.15.38:0/1446132742 shutdown_connections 2024-09-16T00:32:43.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.288+0000 7f3f55a42640 1 -- 172.21.15.38:0/1446132742 wait complete. 2024-09-16T00:32:43.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.288+0000 7f3f55a42640 1 Processor -- start 2024-09-16T00:32:43.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.288+0000 7f3f55a42640 1 -- start start 2024-09-16T00:32:43.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.289+0000 7f3f55a42640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f50101b30 0x7f3f50075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:43.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.289+0000 7f3f55a42640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f50102d20 0x7f3f500745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:43.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.289+0000 7f3f55a42640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f501036e0 0x7f3f50074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:43.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.289+0000 7f3f55a42640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5010d800 con 0x7f3f501036e0 2024-09-16T00:32:43.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.289+0000 7f3f55a42640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5010d680 con 0x7f3f50102d20 2024-09-16T00:32:43.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.289+0000 7f3f55a42640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f3f5010d980 con 0x7f3f50101b30 2024-09-16T00:32:43.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.289+0000 7f3f4f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f501036e0 0x7f3f50074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:43.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.289+0000 7f3f4f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f501036e0 0x7f3f50074b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47066/0 (socket says 172.21.15.38:47066) 2024-09-16T00:32:43.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.289+0000 7f3f4f7fe640 1 -- 172.21.15.38:0/2504952940 learned_addr learned my addr 172.21.15.38:0/2504952940 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:43.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.289+0000 7f3f4effd640 1 --2- 172.21.15.38:0/2504952940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f50101b30 0x7f3f50075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:43.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.289+0000 7f3f4e7fc640 1 --2- 172.21.15.38:0/2504952940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f50102d20 0x7f3f500745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:43.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.289+0000 7f3f4f7fe640 1 -- 172.21.15.38:0/2504952940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f50101b30 msgr2=0x7f3f50075fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:43.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.289+0000 7f3f4f7fe640 1 --2- 172.21.15.38:0/2504952940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f50101b30 0x7f3f50075fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:43.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.289+0000 7f3f4f7fe640 1 -- 172.21.15.38:0/2504952940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f50102d20 msgr2=0x7f3f500745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:43.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.289+0000 7f3f4f7fe640 1 --2- 172.21.15.38:0/2504952940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f50102d20 0x7f3f500745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:43.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.289+0000 7f3f4f7fe640 1 -- 172.21.15.38:0/2504952940 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f500753b0 con 0x7f3f501036e0 2024-09-16T00:32:43.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.289+0000 7f3f4e7fc640 1 --2- 172.21.15.38:0/2504952940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f50102d20 0x7f3f500745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:43.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.289+0000 7f3f4f7fe640 1 --2- 172.21.15.38:0/2504952940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f501036e0 0x7f3f50074b20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3f4800cad0 tx=0x7f3f4800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:43.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.290+0000 7f3f54a40640 1 -- 172.21.15.38:0/2504952940 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4800ae20 con 0x7f3f501036e0 2024-09-16T00:32:43.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.290+0000 7f3f54a40640 1 -- 172.21.15.38:0/2504952940 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3f48005020 con 0x7f3f501036e0 2024-09-16T00:32:43.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.290+0000 7f3f55a42640 1 -- 172.21.15.38:0/2504952940 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f501a1fa0 con 0x7f3f501036e0 2024-09-16T00:32:43.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.290+0000 7f3f54a40640 1 -- 172.21.15.38:0/2504952940 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4800f6f0 con 0x7f3f501036e0 2024-09-16T00:32:43.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.290+0000 7f3f55a42640 1 -- 172.21.15.38:0/2504952940 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f501a2410 con 0x7f3f501036e0 2024-09-16T00:32:43.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.291+0000 7f3f55a42640 1 -- 172.21.15.38:0/2504952940 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f50108f70 con 0x7f3f501036e0 2024-09-16T00:32:43.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.292+0000 7f3f54a40640 1 -- 172.21.15.38:0/2504952940 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f3f480051d0 con 0x7f3f501036e0 2024-09-16T00:32:43.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.292+0000 7f3f54a40640 1 --2- 172.21.15.38:0/2504952940 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f3f380a3250 0x7f3f380a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:43.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.293+0000 7f3f4effd640 1 --2- 172.21.15.38:0/2504952940 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f3f380a3250 0x7f3f380a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:43.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.293+0000 7f3f54a40640 1 -- 172.21.15.38:0/2504952940 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f3f480cfd70 con 0x7f3f501036e0 2024-09-16T00:32:43.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.293+0000 7f3f4effd640 1 --2- 172.21.15.38:0/2504952940 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f3f380a3250 0x7f3f380a5710 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3f4002aa00 tx=0x7f3f400315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:43.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:42 smithi073 ceph-mon[35254]: pgmap v362: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T00:32:43.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:42 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/4163186939' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-09-16T00:32:43.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:42 smithi073 ceph-mon[35254]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2024-09-16T00:32:43.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:42 smithi073 ceph-mon[35254]: from='client.15237 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-09-16T00:32:43.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:42 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/133326061' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:32:43.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:42 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/1275390073' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T00:32:43.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.297+0000 7f3f54a40640 1 -- 172.21.15.38:0/2504952940 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f3f48014030 con 0x7f3f501036e0 2024-09-16T00:32:43.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.471+0000 7f3f55a42640 1 -- 172.21.15.38:0/2504952940 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f3f50062df0 con 0x7f3f501036e0 2024-09-16T00:32:43.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.471+0000 7f3f54a40640 1 -- 172.21.15.38:0/2504952940 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f3f4809c710 con 0x7f3f501036e0 2024-09-16T00:32:43.472 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-09-16T00:32:43.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.473+0000 7f3f55a42640 1 -- 172.21.15.38:0/2504952940 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f3f380a3250 msgr2=0x7f3f380a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:43.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.473+0000 7f3f55a42640 1 --2- 172.21.15.38:0/2504952940 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f3f380a3250 0x7f3f380a5710 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3f4002aa00 tx=0x7f3f400315b0 comp rx=0 tx=0).stop 2024-09-16T00:32:43.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.473+0000 7f3f55a42640 1 -- 172.21.15.38:0/2504952940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f501036e0 msgr2=0x7f3f50074b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:43.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.473+0000 7f3f55a42640 1 --2- 172.21.15.38:0/2504952940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f501036e0 0x7f3f50074b20 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f3f4800cad0 tx=0x7f3f4800cf90 comp rx=0 tx=0).stop 2024-09-16T00:32:43.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.473+0000 7f3f55a42640 1 -- 172.21.15.38:0/2504952940 shutdown_connections 2024-09-16T00:32:43.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.473+0000 7f3f55a42640 1 --2- 172.21.15.38:0/2504952940 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f3f380a3250 0x7f3f380a5710 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:43.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.473+0000 7f3f55a42640 1 --2- 172.21.15.38:0/2504952940 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f3f501036e0 0x7f3f50074b20 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:43.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.473+0000 7f3f55a42640 1 --2- 172.21.15.38:0/2504952940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f50102d20 0x7f3f500745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:43.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.473+0000 7f3f55a42640 1 --2- 172.21.15.38:0/2504952940 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f3f50101b30 0x7f3f50075fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:43.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.473+0000 7f3f55a42640 1 -- 172.21.15.38:0/2504952940 >> 172.21.15.38:0/2504952940 conn(0x7f3f500fd290 msgr2=0x7f3f50106140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:43.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.473+0000 7f3f55a42640 1 -- 172.21.15.38:0/2504952940 shutdown_connections 2024-09-16T00:32:43.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.473+0000 7f3f55a42640 1 -- 172.21.15.38:0/2504952940 wait complete. 2024-09-16T00:32:43.485 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-16T00:32:43.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.591+0000 7f5be7442640 1 -- 172.21.15.38:0/450130505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5be01025b0 msgr2=0x7f5be0108e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:43.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.591+0000 7f5be7442640 1 --2- 172.21.15.38:0/450130505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5be01025b0 0x7f5be0108e40 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8009ca0 tx=0x7f5bd802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:32:43.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.591+0000 7f5be7442640 1 -- 172.21.15.38:0/450130505 shutdown_connections 2024-09-16T00:32:43.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.591+0000 7f5be7442640 1 --2- 172.21.15.38:0/450130505 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5be01025b0 0x7f5be0108e40 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:43.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.591+0000 7f5be7442640 1 --2- 172.21.15.38:0/450130505 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5be0101bf0 0x7f5be0102070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:43.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.591+0000 7f5be7442640 1 --2- 172.21.15.38:0/450130505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5be0100a00 0x7f5be0100e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:43.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.591+0000 7f5be7442640 1 -- 172.21.15.38:0/450130505 >> 172.21.15.38:0/450130505 conn(0x7f5be00fc140 msgr2=0x7f5be00fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:43.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.591+0000 7f5be7442640 1 -- 172.21.15.38:0/450130505 shutdown_connections 2024-09-16T00:32:43.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.591+0000 7f5be7442640 1 -- 172.21.15.38:0/450130505 wait complete. 2024-09-16T00:32:43.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.591+0000 7f5be7442640 1 Processor -- start 2024-09-16T00:32:43.594 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.591+0000 7f5be7442640 1 -- start start 2024-09-16T00:32:43.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.592+0000 7f5be7442640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5be0100a00 0x7f5be0196c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:43.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.592+0000 7f5be7442640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5be0101bf0 0x7f5be01971c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:43.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.592+0000 7f5be7442640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5be01025b0 0x7f5be019e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:43.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.592+0000 7f5be7442640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5be010c660 con 0x7f5be0100a00 2024-09-16T00:32:43.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.592+0000 7f5be7442640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5be010c4e0 con 0x7f5be0101bf0 2024-09-16T00:32:43.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.592+0000 7f5be7442640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5be010c7e0 con 0x7f5be01025b0 2024-09-16T00:32:43.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.592+0000 7f5be51de640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5be0100a00 0x7f5be0196c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:43.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.592+0000 7f5be51de640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5be0100a00 0x7f5be0196c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47078/0 (socket says 172.21.15.38:47078) 2024-09-16T00:32:43.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.592+0000 7f5be51de640 1 -- 172.21.15.38:0/1035755858 learned_addr learned my addr 172.21.15.38:0/1035755858 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:43.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.592+0000 7f5be49dd640 1 --2- 172.21.15.38:0/1035755858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5be0101bf0 0x7f5be01971c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:43.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.592+0000 7f5be59df640 1 --2- 172.21.15.38:0/1035755858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5be01025b0 0x7f5be019e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:43.595 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.592+0000 7f5be51de640 1 -- 172.21.15.38:0/1035755858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5be01025b0 msgr2=0x7f5be019e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:43.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.592+0000 7f5be51de640 1 --2- 172.21.15.38:0/1035755858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5be01025b0 0x7f5be019e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:43.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.592+0000 7f5be51de640 1 -- 172.21.15.38:0/1035755858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5be0101bf0 msgr2=0x7f5be01971c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:43.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.592+0000 7f5be51de640 1 --2- 172.21.15.38:0/1035755858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5be0101bf0 0x7f5be01971c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:43.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.592+0000 7f5be51de640 1 -- 172.21.15.38:0/1035755858 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5be019e890 con 0x7f5be0100a00 2024-09-16T00:32:43.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.592+0000 7f5be49dd640 1 --2- 172.21.15.38:0/1035755858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5be0101bf0 0x7f5be01971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:43.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.592+0000 7f5be51de640 1 --2- 172.21.15.38:0/1035755858 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5be0100a00 0x7f5be0196c80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc00d770 tx=0x7f5bdc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:43.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.592+0000 7f5bd67fc640 1 -- 172.21.15.38:0/1035755858 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc017070 con 0x7f5be0100a00 2024-09-16T00:32:43.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.592+0000 7f5be7442640 1 -- 172.21.15.38:0/1035755858 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5be019eb80 con 0x7f5be0100a00 2024-09-16T00:32:43.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.592+0000 7f5bd67fc640 1 -- 172.21.15.38:0/1035755858 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5bdc009c30 con 0x7f5be0100a00 2024-09-16T00:32:43.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.593+0000 7f5bd67fc640 1 -- 172.21.15.38:0/1035755858 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bdc003dd0 con 0x7f5be0100a00 2024-09-16T00:32:43.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.593+0000 7f5be7442640 1 -- 172.21.15.38:0/1035755858 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5be019f0d0 con 0x7f5be0100a00 2024-09-16T00:32:43.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.593+0000 7f5be7442640 1 -- 172.21.15.38:0/1035755858 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5be0107d20 con 0x7f5be0100a00 2024-09-16T00:32:43.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.595+0000 7f5bd67fc640 1 -- 172.21.15.38:0/1035755858 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 18) ==== 138355+0+0 (secure 0 0 0) 0x7f5bdc003f70 con 0x7f5be0100a00 2024-09-16T00:32:43.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.595+0000 7f5bd67fc640 1 --2- 172.21.15.38:0/1035755858 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f5bcc0a3250 0x7f5bcc0a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:43.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.595+0000 7f5bd67fc640 1 -- 172.21.15.38:0/1035755858 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f5bdc0cfb40 con 0x7f5be0100a00 2024-09-16T00:32:43.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.595+0000 7f5be49dd640 1 --2- 172.21.15.38:0/1035755858 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f5bcc0a3250 0x7f5bcc0a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:43.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.596+0000 7f5be49dd640 1 --2- 172.21.15.38:0/1035755858 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f5bcc0a3250 0x7f5bcc0a5710 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000abe0 tx=0x7f5bd000a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:43.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.599+0000 7f5bd67fc640 1 -- 172.21.15.38:0/1035755858 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f5bdc009de0 con 0x7f5be0100a00 2024-09-16T00:32:43.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.774+0000 7f5be7442640 1 -- 172.21.15.38:0/1035755858 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f5be002d970 con 0x7f5be0100a00 2024-09-16T00:32:43.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.775+0000 7f5bd67fc640 1 -- 172.21.15.38:0/1035755858 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7f5bdc09c4e0 con 0x7f5be0100a00 2024-09-16T00:32:43.775 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-09-16T00:32:43.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.777+0000 7f5be7442640 1 -- 172.21.15.38:0/1035755858 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f5bcc0a3250 msgr2=0x7f5bcc0a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:43.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.777+0000 7f5be7442640 1 --2- 172.21.15.38:0/1035755858 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f5bcc0a3250 0x7f5bcc0a5710 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5bd000abe0 tx=0x7f5bd000a5c0 comp rx=0 tx=0).stop 2024-09-16T00:32:43.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.777+0000 7f5be7442640 1 -- 172.21.15.38:0/1035755858 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5be0100a00 msgr2=0x7f5be0196c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:43.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.777+0000 7f5be7442640 1 --2- 172.21.15.38:0/1035755858 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5be0100a00 0x7f5be0196c80 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5bdc00d770 tx=0x7f5bdc00dc30 comp rx=0 tx=0).stop 2024-09-16T00:32:43.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.777+0000 7f5be7442640 1 -- 172.21.15.38:0/1035755858 shutdown_connections 2024-09-16T00:32:43.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.777+0000 7f5be7442640 1 --2- 172.21.15.38:0/1035755858 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f5bcc0a3250 0x7f5bcc0a5710 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:43.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.777+0000 7f5be7442640 1 --2- 172.21.15.38:0/1035755858 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5be01025b0 0x7f5be019e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:43.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.777+0000 7f5be7442640 1 --2- 172.21.15.38:0/1035755858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5be0101bf0 0x7f5be01971c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:43.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.777+0000 7f5be7442640 1 --2- 172.21.15.38:0/1035755858 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5be0100a00 0x7f5be0196c80 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:43.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.777+0000 7f5be7442640 1 -- 172.21.15.38:0/1035755858 >> 172.21.15.38:0/1035755858 conn(0x7f5be00fc140 msgr2=0x7f5be0104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:43.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.777+0000 7f5be7442640 1 -- 172.21.15.38:0/1035755858 shutdown_connections 2024-09-16T00:32:43.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:43.777+0000 7f5be7442640 1 -- 172.21.15.38:0/1035755858 wait complete. 2024-09-16T00:32:43.789 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-16T00:32:43.789 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:32:44.054 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:44.176 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:43 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/2591920086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T00:32:44.176 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:43 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/2504952940' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T00:32:44.176 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:43 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/1035755858' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-16T00:32:44.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:43 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/2591920086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T00:32:44.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:43 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/2504952940' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T00:32:44.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:43 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/1035755858' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-16T00:32:44.296 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:43 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/2591920086' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T00:32:44.296 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:43 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/2504952940' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T00:32:44.297 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:43 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/1035755858' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-16T00:32:44.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.551+0000 7f531d7c8640 1 -- 172.21.15.38:0/711254361 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f531810c170 msgr2=0x7f531810e610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:44.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.551+0000 7f531d7c8640 1 --2- 172.21.15.38:0/711254361 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f531810c170 0x7f531810e610 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f5310009ca0 tx=0x7f531002a950 comp rx=0 tx=0).stop 2024-09-16T00:32:44.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.551+0000 7f531d7c8640 1 -- 172.21.15.38:0/711254361 shutdown_connections 2024-09-16T00:32:44.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.551+0000 7f531d7c8640 1 --2- 172.21.15.38:0/711254361 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f531810c170 0x7f531810e610 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:44.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.551+0000 7f531d7c8640 1 --2- 172.21.15.38:0/711254361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5318103660 0x7f531810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:44.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.551+0000 7f531d7c8640 1 --2- 172.21.15.38:0/711254361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5318102d40 0x7f5318103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:44.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.551+0000 7f531d7c8640 1 -- 172.21.15.38:0/711254361 >> 172.21.15.38:0/711254361 conn(0x7f53180fc7a0 msgr2=0x7f53180febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:44.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.551+0000 7f531d7c8640 1 -- 172.21.15.38:0/711254361 shutdown_connections 2024-09-16T00:32:44.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.551+0000 7f531d7c8640 1 -- 172.21.15.38:0/711254361 wait complete. 2024-09-16T00:32:44.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.552+0000 7f531d7c8640 1 Processor -- start 2024-09-16T00:32:44.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.552+0000 7f531d7c8640 1 -- start start 2024-09-16T00:32:44.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.552+0000 7f531d7c8640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5318102d40 0x7f53181009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:44.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.552+0000 7f531d7c8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5318103660 0x7f5318100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:44.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.552+0000 7f531d7c8640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f531810c170 0x7f5318101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:44.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.552+0000 7f531d7c8640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5318111f70 con 0x7f5318103660 2024-09-16T00:32:44.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.552+0000 7f531d7c8640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5318111df0 con 0x7f5318102d40 2024-09-16T00:32:44.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.552+0000 7f531d7c8640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f53181120f0 con 0x7f531810c170 2024-09-16T00:32:44.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.552+0000 7f53167fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5318103660 0x7f5318100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:44.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.552+0000 7f53167fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5318103660 0x7f5318100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47096/0 (socket says 172.21.15.38:47096) 2024-09-16T00:32:44.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.552+0000 7f53167fc640 1 -- 172.21.15.38:0/1681819617 learned_addr learned my addr 172.21.15.38:0/1681819617 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:44.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.553+0000 7f53177fe640 1 --2- 172.21.15.38:0/1681819617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f531810c170 0x7f5318101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:44.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.553+0000 7f5316ffd640 1 --2- 172.21.15.38:0/1681819617 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5318102d40 0x7f53181009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:44.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.553+0000 7f53167fc640 1 -- 172.21.15.38:0/1681819617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f531810c170 msgr2=0x7f5318101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:44.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.553+0000 7f53167fc640 1 --2- 172.21.15.38:0/1681819617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f531810c170 0x7f5318101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:44.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.553+0000 7f53167fc640 1 -- 172.21.15.38:0/1681819617 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5318102d40 msgr2=0x7f53181009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:44.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.553+0000 7f53167fc640 1 --2- 172.21.15.38:0/1681819617 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5318102d40 0x7f53181009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:44.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.553+0000 7f53167fc640 1 -- 172.21.15.38:0/1681819617 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53181a6890 con 0x7f5318103660 2024-09-16T00:32:44.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.553+0000 7f5316ffd640 1 --2- 172.21.15.38:0/1681819617 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5318102d40 0x7f53181009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:44.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.553+0000 7f53167fc640 1 --2- 172.21.15.38:0/1681819617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5318103660 0x7f5318100ee0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f530000d740 tx=0x7f530000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:44.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.553+0000 7f530ffff640 1 -- 172.21.15.38:0/1681819617 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5300017070 con 0x7f5318103660 2024-09-16T00:32:44.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.553+0000 7f531d7c8640 1 -- 172.21.15.38:0/1681819617 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53181a6b80 con 0x7f5318103660 2024-09-16T00:32:44.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.553+0000 7f530ffff640 1 -- 172.21.15.38:0/1681819617 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f530000de50 con 0x7f5318103660 2024-09-16T00:32:44.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.553+0000 7f530ffff640 1 -- 172.21.15.38:0/1681819617 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f530000f620 con 0x7f5318103660 2024-09-16T00:32:44.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.553+0000 7f531d7c8640 1 -- 172.21.15.38:0/1681819617 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53181a70c0 con 0x7f5318103660 2024-09-16T00:32:44.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.554+0000 7f531d7c8640 1 -- 172.21.15.38:0/1681819617 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5318103120 con 0x7f5318103660 2024-09-16T00:32:44.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.555+0000 7f530ffff640 1 -- 172.21.15.38:0/1681819617 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f5300009c30 con 0x7f5318103660 2024-09-16T00:32:44.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.556+0000 7f530ffff640 1 --2- 172.21.15.38:0/1681819617 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f53040a32d0 0x7f53040a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:44.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.556+0000 7f530ffff640 1 -- 172.21.15.38:0/1681819617 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f53000cfa00 con 0x7f5318103660 2024-09-16T00:32:44.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.556+0000 7f5316ffd640 1 --2- 172.21.15.38:0/1681819617 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f53040a32d0 0x7f53040a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:44.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.557+0000 7f5316ffd640 1 --2- 172.21.15.38:0/1681819617 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f53040a32d0 0x7f53040a5790 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f53080058c0 tx=0x7f5308005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:44.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.560+0000 7f530ffff640 1 -- 172.21.15.38:0/1681819617 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f530009c1e0 con 0x7f5318103660 2024-09-16T00:32:44.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.775+0000 7f531d7c8640 1 -- 172.21.15.38:0/1681819617 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5318106c80 con 0x7f5318103660 2024-09-16T00:32:44.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.775+0000 7f530ffff640 1 -- 172.21.15.38:0/1681819617 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f530009bae0 con 0x7f5318103660 2024-09-16T00:32:44.776 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:32:44.776 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"btime":"2024-09-16T00:32:41:853514+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/367974527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":367974527},{"type":"v1","addr":"172.21.15.38:6835","nonce":367974527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/335077136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":335077136},{"type":"v1","addr":"172.21.15.38:6837","nonce":335077136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/3501211160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3501211160},{"type":"v1","addr":"172.21.15.38:6839","nonce":3501211160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6841/2623130915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":2623130915},{"type":"v1","addr":"172.21.15.38:6841","nonce":2623130915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/412277519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":412277519},{"type":"v1","addr":"172.21.15.122:6833","nonce":412277519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3987234757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3987234757},{"type":"v1","addr":"172.21.15.122:6835","nonce":3987234757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:30:26.764565+0000","modified":"2024-09-16T00:32:41.355828+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24440},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1210328177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1210328177},{"type":"v1","addr":"172.21.15.73:6833","nonce":1210328177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6837/1858299680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1858299680},{"type":"v1","addr":"172.21.15.122:6837","nonce":1858299680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1983975311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1983975311},{"type":"v1","addr":"172.21.15.122:6839","nonce":1983975311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.73:6835/2866363113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2866363113},{"type":"v1","addr":"172.21.15.73:6835","nonce":2866363113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.73:6837/2681081265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2681081265},{"type":"v1","addr":"172.21.15.73:6837","nonce":2681081265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.73:6839/2396224525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2396224525},{"type":"v1","addr":"172.21.15.73:6839","nonce":2396224525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24464,24440]},"id":1}]} 2024-09-16T00:32:44.776 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 26 2024-09-16T00:32:44.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.778+0000 7f531d7c8640 1 -- 172.21.15.38:0/1681819617 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f53040a32d0 msgr2=0x7f53040a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:44.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.778+0000 7f531d7c8640 1 --2- 172.21.15.38:0/1681819617 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f53040a32d0 0x7f53040a5790 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f53080058c0 tx=0x7f5308005850 comp rx=0 tx=0).stop 2024-09-16T00:32:44.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.778+0000 7f531d7c8640 1 -- 172.21.15.38:0/1681819617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5318103660 msgr2=0x7f5318100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:44.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.778+0000 7f531d7c8640 1 --2- 172.21.15.38:0/1681819617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5318103660 0x7f5318100ee0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f530000d740 tx=0x7f530000dc00 comp rx=0 tx=0).stop 2024-09-16T00:32:44.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.778+0000 7f531d7c8640 1 -- 172.21.15.38:0/1681819617 shutdown_connections 2024-09-16T00:32:44.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.778+0000 7f531d7c8640 1 --2- 172.21.15.38:0/1681819617 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f53040a32d0 0x7f53040a5790 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:44.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.778+0000 7f531d7c8640 1 --2- 172.21.15.38:0/1681819617 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f531810c170 0x7f5318101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:44.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.778+0000 7f531d7c8640 1 --2- 172.21.15.38:0/1681819617 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5318103660 0x7f5318100ee0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:44.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.778+0000 7f531d7c8640 1 --2- 172.21.15.38:0/1681819617 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5318102d40 0x7f53181009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:44.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.778+0000 7f531d7c8640 1 -- 172.21.15.38:0/1681819617 >> 172.21.15.38:0/1681819617 conn(0x7f53180fc7a0 msgr2=0x7f53180fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:44.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.779+0000 7f531d7c8640 1 -- 172.21.15.38:0/1681819617 shutdown_connections 2024-09-16T00:32:44.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:44.779+0000 7f531d7c8640 1 -- 172.21.15.38:0/1681819617 wait complete. 2024-09-16T00:32:45.213 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:44 smithi038 ceph-mon[30945]: pgmap v363: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-16T00:32:45.213 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:44 smithi038 ceph-mon[30945]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-16T00:32:45.213 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:44 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/1681819617' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:32:45.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:44 smithi122 ceph-mon[35412]: pgmap v363: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-16T00:32:45.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:44 smithi122 ceph-mon[35412]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-16T00:32:45.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:44 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/1681819617' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:32:45.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:44 smithi073 ceph-mon[35254]: pgmap v363: 129 pgs: 129 active+clean; 642 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 6 op/s 2024-09-16T00:32:45.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:44 smithi073 ceph-mon[35254]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-09-16T00:32:45.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:44 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/1681819617' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:32:45.509 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-16T00:32:45.509 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-16T00:32:45.509 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:32:45.623 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.620+0000 7f77874b7640 1 -- 172.21.15.38:0/149267704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77801025b0 msgr2=0x7f77801049a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:45.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.620+0000 7f77874b7640 1 --2- 172.21.15.38:0/149267704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77801025b0 0x7f77801049a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f777c009900 tx=0x7f777c02a410 comp rx=0 tx=0).stop 2024-09-16T00:32:45.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.620+0000 7f77874b7640 1 -- 172.21.15.38:0/149267704 shutdown_connections 2024-09-16T00:32:45.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.620+0000 7f77874b7640 1 --2- 172.21.15.38:0/149267704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7780107810 0x7f7780109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:45.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.620+0000 7f77874b7640 1 --2- 172.21.15.38:0/149267704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7780104ee0 0x7f77801072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:45.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.620+0000 7f77874b7640 1 --2- 172.21.15.38:0/149267704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77801025b0 0x7f77801049a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:45.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.620+0000 7f77874b7640 1 -- 172.21.15.38:0/149267704 >> 172.21.15.38:0/149267704 conn(0x7f77800fc1b0 msgr2=0x7f77800fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:45.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.620+0000 7f77874b7640 1 -- 172.21.15.38:0/149267704 shutdown_connections 2024-09-16T00:32:45.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.620+0000 7f77874b7640 1 -- 172.21.15.38:0/149267704 wait complete. 2024-09-16T00:32:45.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.621+0000 7f77874b7640 1 Processor -- start 2024-09-16T00:32:45.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.621+0000 7f77874b7640 1 -- start start 2024-09-16T00:32:45.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.621+0000 7f77874b7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77801025b0 0x7f7780197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:45.624 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.621+0000 7f77874b7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7780104ee0 0x7f7780198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:45.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.621+0000 7f77874b7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7780107810 0x7f778019f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:45.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.621+0000 7f77874b7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f778010d7f0 con 0x7f77801025b0 2024-09-16T00:32:45.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.621+0000 7f77874b7640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f778010d670 con 0x7f7780104ee0 2024-09-16T00:32:45.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.621+0000 7f77874b7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f778010d970 con 0x7f7780107810 2024-09-16T00:32:45.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.621+0000 7f7785253640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77801025b0 0x7f7780197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:45.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.621+0000 7f7785253640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77801025b0 0x7f7780197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47104/0 (socket says 172.21.15.38:47104) 2024-09-16T00:32:45.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.621+0000 7f7785253640 1 -- 172.21.15.38:0/3809174190 learned_addr learned my addr 172.21.15.38:0/3809174190 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:45.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.622+0000 7f7784a52640 1 --2- 172.21.15.38:0/3809174190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7780104ee0 0x7f7780198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:45.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.622+0000 7f7785253640 1 -- 172.21.15.38:0/3809174190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7780107810 msgr2=0x7f778019f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:45.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.622+0000 7f7785a54640 1 --2- 172.21.15.38:0/3809174190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7780107810 0x7f778019f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:45.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.622+0000 7f7785253640 1 --2- 172.21.15.38:0/3809174190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7780107810 0x7f778019f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:45.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.622+0000 7f7785253640 1 -- 172.21.15.38:0/3809174190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7780104ee0 msgr2=0x7f7780198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:45.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.622+0000 7f7785253640 1 --2- 172.21.15.38:0/3809174190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7780104ee0 0x7f7780198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:45.625 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.622+0000 7f7785253640 1 -- 172.21.15.38:0/3809174190 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f778019faf0 con 0x7f77801025b0 2024-09-16T00:32:45.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.622+0000 7f7784a52640 1 --2- 172.21.15.38:0/3809174190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7780104ee0 0x7f7780198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:45.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.622+0000 7f7785a54640 1 --2- 172.21.15.38:0/3809174190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7780107810 0x7f778019f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:32:45.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.622+0000 7f7785253640 1 --2- 172.21.15.38:0/3809174190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77801025b0 0x7f7780197ed0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f777c009230 tx=0x7f777c031b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:45.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.622+0000 7f77767fc640 1 -- 172.21.15.38:0/3809174190 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f777c03f030 con 0x7f77801025b0 2024-09-16T00:32:45.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.622+0000 7f77767fc640 1 -- 172.21.15.38:0/3809174190 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f777c003ad0 con 0x7f77801025b0 2024-09-16T00:32:45.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.622+0000 7f77874b7640 1 -- 172.21.15.38:0/3809174190 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f778019fd80 con 0x7f77801025b0 2024-09-16T00:32:45.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.622+0000 7f77767fc640 1 -- 172.21.15.38:0/3809174190 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f777c035070 con 0x7f77801025b0 2024-09-16T00:32:45.626 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.622+0000 7f77874b7640 1 -- 172.21.15.38:0/3809174190 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77801a0200 con 0x7f77801025b0 2024-09-16T00:32:45.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.623+0000 7f77874b7640 1 -- 172.21.15.38:0/3809174190 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77800fdca0 con 0x7f77801025b0 2024-09-16T00:32:45.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.623+0000 7f77767fc640 1 -- 172.21.15.38:0/3809174190 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f777c042030 con 0x7f77801025b0 2024-09-16T00:32:45.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.624+0000 7f77767fc640 1 --2- 172.21.15.38:0/3809174190 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f776c0a3360 0x7f776c0a5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:45.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.625+0000 7f7784a52640 1 --2- 172.21.15.38:0/3809174190 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f776c0a3360 0x7f776c0a5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:45.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.625+0000 7f77767fc640 1 -- 172.21.15.38:0/3809174190 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f777c0f0040 con 0x7f77801025b0 2024-09-16T00:32:45.627 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.625+0000 7f7784a52640 1 --2- 172.21.15.38:0/3809174190 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f776c0a3360 0x7f776c0a5820 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7770005d00 tx=0x7f7770005c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:45.630 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.629+0000 7f77767fc640 1 -- 172.21.15.38:0/3809174190 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f777c0bc880 con 0x7f77801025b0 2024-09-16T00:32:45.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.802+0000 7f77874b7640 1 -- 172.21.15.38:0/3809174190 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7780062630 con 0x7f776c0a3360 2024-09-16T00:32:45.811 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.810+0000 7f77767fc640 1 -- 172.21.15.38:0/3809174190 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340493 (secure 0 0 0) 0x7f7780062630 con 0x7f776c0a3360 2024-09-16T00:32:45.817 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:32:45.819 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-16T00:32:45.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.814+0000 7f77874b7640 1 -- 172.21.15.38:0/3809174190 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f776c0a3360 msgr2=0x7f776c0a5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:45.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.814+0000 7f77874b7640 1 --2- 172.21.15.38:0/3809174190 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f776c0a3360 0x7f776c0a5820 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7770005d00 tx=0x7f7770005c70 comp rx=0 tx=0).stop 2024-09-16T00:32:45.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.814+0000 7f77874b7640 1 -- 172.21.15.38:0/3809174190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77801025b0 msgr2=0x7f7780197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:45.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.814+0000 7f77874b7640 1 --2- 172.21.15.38:0/3809174190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77801025b0 0x7f7780197ed0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f777c009230 tx=0x7f777c031b60 comp rx=0 tx=0).stop 2024-09-16T00:32:45.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.814+0000 7f77874b7640 1 -- 172.21.15.38:0/3809174190 shutdown_connections 2024-09-16T00:32:45.821 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.814+0000 7f77874b7640 1 --2- 172.21.15.38:0/3809174190 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f776c0a3360 0x7f776c0a5820 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:45.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.814+0000 7f77874b7640 1 --2- 172.21.15.38:0/3809174190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7780107810 0x7f778019f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:45.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.814+0000 7f77874b7640 1 --2- 172.21.15.38:0/3809174190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7780104ee0 0x7f7780198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:45.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.814+0000 7f77874b7640 1 --2- 172.21.15.38:0/3809174190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f77801025b0 0x7f7780197ed0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:45.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.814+0000 7f77874b7640 1 -- 172.21.15.38:0/3809174190 >> 172.21.15.38:0/3809174190 conn(0x7f77800fc1b0 msgr2=0x7f7780103070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:45.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.814+0000 7f77874b7640 1 -- 172.21.15.38:0/3809174190 shutdown_connections 2024-09-16T00:32:45.822 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.814+0000 7f77874b7640 1 -- 172.21.15.38:0/3809174190 wait complete. 2024-09-16T00:32:45.826 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2024-09-16T00:32:45.543344+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663538,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":763,"num_read_kb":508,"num_write":366,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1180792,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":795,"ondisk_log_size":795,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380376,"kb_used_data":14440,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561448,"statfs":{"total":1151940427776,"available":1151550922752,"internally_reserved":0,"allocated":14786560,"data_stored":8238233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6467,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":83,"num_read_kb":53,"num_write":10,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003434"},"pg_stats":[{"pgid":"2.1f","version":"88'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.337120+0000","last_change":"2024-09-16T00:30:23.933355+0000","last_active":"2024-09-16T00:32:24.337120+0000","last_peered":"2024-09-16T00:32:24.337120+0000","last_clean":"2024-09-16T00:32:24.337120+0000","last_became_active":"2024-09-16T00:30:23.933057+0000","last_became_peered":"2024-09-16T00:30:23.933057+0000","last_unstale":"2024-09-16T00:32:24.337120+0000","last_undegraded":"2024-09-16T00:32:24.337120+0000","last_fullsized":"2024-09-16T00:32:24.337120+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:23:01.713440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.596165+0000","last_change":"2024-09-16T00:30:26.084537+0000","last_active":"2024-09-16T00:32:24.596165+0000","last_peered":"2024-09-16T00:32:24.596165+0000","last_clean":"2024-09-16T00:32:24.596165+0000","last_became_active":"2024-09-16T00:30:26.084226+0000","last_became_peered":"2024-09-16T00:30:26.084226+0000","last_unstale":"2024-09-16T00:32:24.596165+0000","last_undegraded":"2024-09-16T00:32:24.596165+0000","last_fullsized":"2024-09-16T00:32:24.596165+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:43:41.628877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594728+0000","last_change":"2024-09-16T00:30:33.154873+0000","last_active":"2024-09-16T00:32:24.594728+0000","last_peered":"2024-09-16T00:32:24.594728+0000","last_clean":"2024-09-16T00:32:24.594728+0000","last_became_active":"2024-09-16T00:30:33.154433+0000","last_became_peered":"2024-09-16T00:30:33.154433+0000","last_unstale":"2024-09-16T00:32:24.594728+0000","last_undegraded":"2024-09-16T00:32:24.594728+0000","last_fullsized":"2024-09-16T00:32:24.594728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:50.805806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.596255+0000","last_change":"2024-09-16T00:30:24.422274+0000","last_active":"2024-09-16T00:32:24.596255+0000","last_peered":"2024-09-16T00:32:24.596255+0000","last_clean":"2024-09-16T00:32:24.596255+0000","last_became_active":"2024-09-16T00:30:24.421902+0000","last_became_peered":"2024-09-16T00:30:24.421902+0000","last_unstale":"2024-09-16T00:32:24.596255+0000","last_undegraded":"2024-09-16T00:32:24.596255+0000","last_fullsized":"2024-09-16T00:32:24.596255+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:01.910187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595568+0000","last_change":"2024-09-16T00:30:26.088165+0000","last_active":"2024-09-16T00:32:24.595568+0000","last_peered":"2024-09-16T00:32:24.595568+0000","last_clean":"2024-09-16T00:32:24.595568+0000","last_became_active":"2024-09-16T00:30:26.087593+0000","last_became_peered":"2024-09-16T00:30:26.087593+0000","last_unstale":"2024-09-16T00:32:24.595568+0000","last_undegraded":"2024-09-16T00:32:24.595568+0000","last_fullsized":"2024-09-16T00:32:24.595568+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:54.149930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454083+0000","last_change":"2024-09-16T00:30:33.154586+0000","last_active":"2024-09-16T00:32:24.454083+0000","last_peered":"2024-09-16T00:32:24.454083+0000","last_clean":"2024-09-16T00:32:24.454083+0000","last_became_active":"2024-09-16T00:30:33.154209+0000","last_became_peered":"2024-09-16T00:30:33.154209+0000","last_unstale":"2024-09-16T00:32:24.454083+0000","last_undegraded":"2024-09-16T00:32:24.454083+0000","last_fullsized":"2024-09-16T00:32:24.454083+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:45.776895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339220+0000","last_change":"2024-09-16T00:30:23.932442+0000","last_active":"2024-09-16T00:32:24.339220+0000","last_peered":"2024-09-16T00:32:24.339220+0000","last_clean":"2024-09-16T00:32:24.339220+0000","last_became_active":"2024-09-16T00:30:23.932009+0000","last_became_peered":"2024-09-16T00:30:23.932009+0000","last_unstale":"2024-09-16T00:32:24.339220+0000","last_undegraded":"2024-09-16T00:32:24.339220+0000","last_fullsized":"2024-09-16T00:32:24.339220+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:09:27.078187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561572+0000","last_change":"2024-09-16T00:30:26.087715+0000","last_active":"2024-09-16T00:32:24.561572+0000","last_peered":"2024-09-16T00:32:24.561572+0000","last_clean":"2024-09-16T00:32:24.561572+0000","last_became_active":"2024-09-16T00:30:26.087415+0000","last_became_peered":"2024-09-16T00:30:26.087415+0000","last_unstale":"2024-09-16T00:32:24.561572+0000","last_undegraded":"2024-09-16T00:32:24.561572+0000","last_fullsized":"2024-09-16T00:32:24.561572+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:52.373992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454447+0000","last_change":"2024-09-16T00:30:33.152328+0000","last_active":"2024-09-16T00:32:24.454447+0000","last_peered":"2024-09-16T00:32:24.454447+0000","last_clean":"2024-09-16T00:32:24.454447+0000","last_became_active":"2024-09-16T00:30:33.151954+0000","last_became_peered":"2024-09-16T00:30:33.151954+0000","last_unstale":"2024-09-16T00:32:24.454447+0000","last_undegraded":"2024-09-16T00:32:24.454447+0000","last_fullsized":"2024-09-16T00:32:24.454447+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:36.546729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339346+0000","last_change":"2024-09-16T00:30:23.928988+0000","last_active":"2024-09-16T00:32:24.339346+0000","last_peered":"2024-09-16T00:32:24.339346+0000","last_clean":"2024-09-16T00:32:24.339346+0000","last_became_active":"2024-09-16T00:30:23.928638+0000","last_became_peered":"2024-09-16T00:30:23.928638+0000","last_unstale":"2024-09-16T00:32:24.339346+0000","last_undegraded":"2024-09-16T00:32:24.339346+0000","last_fullsized":"2024-09-16T00:32:24.339346+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:45:58.203455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.336619+0000","last_change":"2024-09-16T00:31:23.446550+0000","last_active":"2024-09-16T00:32:24.336619+0000","last_peered":"2024-09-16T00:32:24.336619+0000","last_clean":"2024-09-16T00:32:24.336619+0000","last_became_active":"2024-09-16T00:31:23.446179+0000","last_became_peered":"2024-09-16T00:31:23.446179+0000","last_unstale":"2024-09-16T00:32:24.336619+0000","last_undegraded":"2024-09-16T00:32:24.336619+0000","last_fullsized":"2024-09-16T00:32:24.336619+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:04:21.280174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454185+0000","last_change":"2024-09-16T00:30:33.147564+0000","last_active":"2024-09-16T00:32:24.454185+0000","last_peered":"2024-09-16T00:32:24.454185+0000","last_clean":"2024-09-16T00:32:24.454185+0000","last_became_active":"2024-09-16T00:30:33.147119+0000","last_became_peered":"2024-09-16T00:30:33.147119+0000","last_unstale":"2024-09-16T00:32:24.454185+0000","last_undegraded":"2024-09-16T00:32:24.454185+0000","last_fullsized":"2024-09-16T00:32:24.454185+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:00.509122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595076+0000","last_change":"2024-09-16T00:30:26.080954+0000","last_active":"2024-09-16T00:32:24.595076+0000","last_peered":"2024-09-16T00:32:24.595076+0000","last_clean":"2024-09-16T00:32:24.595076+0000","last_became_active":"2024-09-16T00:30:26.080624+0000","last_became_peered":"2024-09-16T00:30:26.080624+0000","last_unstale":"2024-09-16T00:32:24.595076+0000","last_undegraded":"2024-09-16T00:32:24.595076+0000","last_fullsized":"2024-09-16T00:32:24.595076+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:05:10.440980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339169+0000","last_change":"2024-09-16T00:30:23.933715+0000","last_active":"2024-09-16T00:32:24.339169+0000","last_peered":"2024-09-16T00:32:24.339169+0000","last_clean":"2024-09-16T00:32:24.339169+0000","last_became_active":"2024-09-16T00:30:23.933275+0000","last_became_peered":"2024-09-16T00:30:23.933275+0000","last_unstale":"2024-09-16T00:32:24.339169+0000","last_undegraded":"2024-09-16T00:32:24.339169+0000","last_fullsized":"2024-09-16T00:32:24.339169+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:40:44.846053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454555+0000","last_change":"2024-09-16T00:30:33.604602+0000","last_active":"2024-09-16T00:32:24.454555+0000","last_peered":"2024-09-16T00:32:24.454555+0000","last_clean":"2024-09-16T00:32:24.454555+0000","last_became_active":"2024-09-16T00:30:33.604107+0000","last_became_peered":"2024-09-16T00:30:33.604107+0000","last_unstale":"2024-09-16T00:32:24.454555+0000","last_undegraded":"2024-09-16T00:32:24.454555+0000","last_fullsized":"2024-09-16T00:32:24.454555+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:41.254445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595459+0000","last_change":"2024-09-16T00:30:26.080612+0000","last_active":"2024-09-16T00:32:24.595459+0000","last_peered":"2024-09-16T00:32:24.595459+0000","last_clean":"2024-09-16T00:32:24.595459+0000","last_became_active":"2024-09-16T00:30:26.080317+0000","last_became_peered":"2024-09-16T00:30:26.080317+0000","last_unstale":"2024-09-16T00:32:24.595459+0000","last_undegraded":"2024-09-16T00:32:24.595459+0000","last_fullsized":"2024-09-16T00:32:24.595459+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:23.814241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594893+0000","last_change":"2024-09-16T00:30:23.936778+0000","last_active":"2024-09-16T00:32:24.594893+0000","last_peered":"2024-09-16T00:32:24.594893+0000","last_clean":"2024-09-16T00:32:24.594893+0000","last_became_active":"2024-09-16T00:30:23.936265+0000","last_became_peered":"2024-09-16T00:30:23.936265+0000","last_unstale":"2024-09-16T00:32:24.594893+0000","last_undegraded":"2024-09-16T00:32:24.594893+0000","last_fullsized":"2024-09-16T00:32:24.594893+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:35:28.288476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454520+0000","last_change":"2024-09-16T00:30:33.148340+0000","last_active":"2024-09-16T00:32:24.454520+0000","last_peered":"2024-09-16T00:32:24.454520+0000","last_clean":"2024-09-16T00:32:24.454520+0000","last_became_active":"2024-09-16T00:30:33.147289+0000","last_became_peered":"2024-09-16T00:30:33.147289+0000","last_unstale":"2024-09-16T00:32:24.454520+0000","last_undegraded":"2024-09-16T00:32:24.454520+0000","last_fullsized":"2024-09-16T00:32:24.454520+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:21.259254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.338936+0000","last_change":"2024-09-16T00:30:26.077325+0000","last_active":"2024-09-16T00:32:24.338936+0000","last_peered":"2024-09-16T00:32:24.338936+0000","last_clean":"2024-09-16T00:32:24.338936+0000","last_became_active":"2024-09-16T00:30:26.077002+0000","last_became_peered":"2024-09-16T00:30:26.077002+0000","last_unstale":"2024-09-16T00:32:24.338936+0000","last_undegraded":"2024-09-16T00:32:24.338936+0000","last_fullsized":"2024-09-16T00:32:24.338936+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:12:38.569247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595646+0000","last_change":"2024-09-16T00:30:23.933025+0000","last_active":"2024-09-16T00:32:24.595646+0000","last_peered":"2024-09-16T00:32:24.595646+0000","last_clean":"2024-09-16T00:32:24.595646+0000","last_became_active":"2024-09-16T00:30:23.932669+0000","last_became_peered":"2024-09-16T00:30:23.932669+0000","last_unstale":"2024-09-16T00:32:24.595646+0000","last_undegraded":"2024-09-16T00:32:24.595646+0000","last_fullsized":"2024-09-16T00:32:24.595646+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:20.593665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.597170+0000","last_change":"2024-09-16T00:32:24.597170+0000","last_active":"2024-09-16T00:32:24.597170+0000","last_peered":"2024-09-16T00:32:24.597170+0000","last_clean":"2024-09-16T00:32:24.597170+0000","last_became_active":"2024-09-16T00:32:24.596810+0000","last_became_peered":"2024-09-16T00:32:24.596810+0000","last_unstale":"2024-09-16T00:32:24.597170+0000","last_undegraded":"2024-09-16T00:32:24.597170+0000","last_fullsized":"2024-09-16T00:32:24.597170+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:26:18.737332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.338711+0000","last_change":"2024-09-16T00:30:26.076104+0000","last_active":"2024-09-16T00:32:24.338711+0000","last_peered":"2024-09-16T00:32:24.338711+0000","last_clean":"2024-09-16T00:32:24.338711+0000","last_became_active":"2024-09-16T00:30:26.075764+0000","last_became_peered":"2024-09-16T00:30:26.075764+0000","last_unstale":"2024-09-16T00:32:24.338711+0000","last_undegraded":"2024-09-16T00:32:24.338711+0000","last_fullsized":"2024-09-16T00:32:24.338711+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:31.228533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561411+0000","last_change":"2024-09-16T00:30:24.422411+0000","last_active":"2024-09-16T00:32:24.561411+0000","last_peered":"2024-09-16T00:32:24.561411+0000","last_clean":"2024-09-16T00:32:24.561411+0000","last_became_active":"2024-09-16T00:30:24.422102+0000","last_became_peered":"2024-09-16T00:30:24.422102+0000","last_unstale":"2024-09-16T00:32:24.561411+0000","last_undegraded":"2024-09-16T00:32:24.561411+0000","last_fullsized":"2024-09-16T00:32:24.561411+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:06:00.572070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454559+0000","last_change":"2024-09-16T00:30:33.605397+0000","last_active":"2024-09-16T00:32:24.454559+0000","last_peered":"2024-09-16T00:32:24.454559+0000","last_clean":"2024-09-16T00:32:24.454559+0000","last_became_active":"2024-09-16T00:30:33.604978+0000","last_became_peered":"2024-09-16T00:30:33.604978+0000","last_unstale":"2024-09-16T00:32:24.454559+0000","last_undegraded":"2024-09-16T00:32:24.454559+0000","last_fullsized":"2024-09-16T00:32:24.454559+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:35:16.766056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595380+0000","last_change":"2024-09-16T00:30:26.082684+0000","last_active":"2024-09-16T00:32:24.595380+0000","last_peered":"2024-09-16T00:32:24.595380+0000","last_clean":"2024-09-16T00:32:24.595380+0000","last_became_active":"2024-09-16T00:30:26.082335+0000","last_became_peered":"2024-09-16T00:30:26.082335+0000","last_unstale":"2024-09-16T00:32:24.595380+0000","last_undegraded":"2024-09-16T00:32:24.595380+0000","last_fullsized":"2024-09-16T00:32:24.595380+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:51:21.389627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454909+0000","last_change":"2024-09-16T00:30:24.587840+0000","last_active":"2024-09-16T00:32:24.454909+0000","last_peered":"2024-09-16T00:32:24.454909+0000","last_clean":"2024-09-16T00:32:24.454909+0000","last_became_active":"2024-09-16T00:30:24.587436+0000","last_became_peered":"2024-09-16T00:30:24.587436+0000","last_unstale":"2024-09-16T00:32:24.454909+0000","last_undegraded":"2024-09-16T00:32:24.454909+0000","last_fullsized":"2024-09-16T00:32:24.454909+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:06.758585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594628+0000","last_change":"2024-09-16T00:30:33.145051+0000","last_active":"2024-09-16T00:32:24.594628+0000","last_peered":"2024-09-16T00:32:24.594628+0000","last_clean":"2024-09-16T00:32:24.594628+0000","last_became_active":"2024-09-16T00:30:33.144585+0000","last_became_peered":"2024-09-16T00:30:33.144585+0000","last_unstale":"2024-09-16T00:32:24.594628+0000","last_undegraded":"2024-09-16T00:32:24.594628+0000","last_fullsized":"2024-09-16T00:32:24.594628+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:48:23.588495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.336844+0000","last_change":"2024-09-16T00:31:23.444802+0000","last_active":"2024-09-16T00:32:24.336844+0000","last_peered":"2024-09-16T00:32:24.336844+0000","last_clean":"2024-09-16T00:32:24.336844+0000","last_became_active":"2024-09-16T00:31:23.444447+0000","last_became_peered":"2024-09-16T00:31:23.444447+0000","last_unstale":"2024-09-16T00:32:24.336844+0000","last_undegraded":"2024-09-16T00:32:24.336844+0000","last_fullsized":"2024-09-16T00:32:24.336844+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:46:37.285250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.6","version":"92'14","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:41.088268+0000","last_change":"2024-09-16T00:30:23.933284+0000","last_active":"2024-09-16T00:32:41.088268+0000","last_peered":"2024-09-16T00:32:41.088268+0000","last_clean":"2024-09-16T00:32:41.088268+0000","last_became_active":"2024-09-16T00:30:23.932920+0000","last_became_peered":"2024-09-16T00:30:23.932920+0000","last_unstale":"2024-09-16T00:32:41.088268+0000","last_undegraded":"2024-09-16T00:32:41.088268+0000","last_fullsized":"2024-09-16T00:32:41.088268+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:38.199391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594942+0000","last_change":"2024-09-16T00:30:33.604308+0000","last_active":"2024-09-16T00:32:24.594942+0000","last_peered":"2024-09-16T00:32:24.594942+0000","last_clean":"2024-09-16T00:32:24.594942+0000","last_became_active":"2024-09-16T00:30:33.603889+0000","last_became_peered":"2024-09-16T00:30:33.603889+0000","last_unstale":"2024-09-16T00:32:24.594942+0000","last_undegraded":"2024-09-16T00:32:24.594942+0000","last_fullsized":"2024-09-16T00:32:24.594942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:32:55.173063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452512+0000","last_change":"2024-09-16T00:30:26.078515+0000","last_active":"2024-09-16T00:32:24.452512+0000","last_peered":"2024-09-16T00:32:24.452512+0000","last_clean":"2024-09-16T00:32:24.452512+0000","last_became_active":"2024-09-16T00:30:26.078173+0000","last_became_peered":"2024-09-16T00:30:26.078173+0000","last_unstale":"2024-09-16T00:32:24.452512+0000","last_undegraded":"2024-09-16T00:32:24.452512+0000","last_fullsized":"2024-09-16T00:32:24.452512+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:40:31.471154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339304+0000","last_change":"2024-09-16T00:30:24.423209+0000","last_active":"2024-09-16T00:32:24.339304+0000","last_peered":"2024-09-16T00:32:24.339304+0000","last_clean":"2024-09-16T00:32:24.339304+0000","last_became_active":"2024-09-16T00:30:24.422157+0000","last_became_peered":"2024-09-16T00:30:24.422157+0000","last_unstale":"2024-09-16T00:32:24.339304+0000","last_undegraded":"2024-09-16T00:32:24.339304+0000","last_fullsized":"2024-09-16T00:32:24.339304+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:28.262392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339327+0000","last_change":"2024-09-16T00:30:33.146204+0000","last_active":"2024-09-16T00:32:24.339327+0000","last_peered":"2024-09-16T00:32:24.339327+0000","last_clean":"2024-09-16T00:32:24.339327+0000","last_became_active":"2024-09-16T00:30:33.145553+0000","last_became_peered":"2024-09-16T00:30:33.145553+0000","last_unstale":"2024-09-16T00:32:24.339327+0000","last_undegraded":"2024-09-16T00:32:24.339327+0000","last_fullsized":"2024-09-16T00:32:24.339327+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:22.096050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.337437+0000","last_change":"2024-09-16T00:30:26.070153+0000","last_active":"2024-09-16T00:32:24.337437+0000","last_peered":"2024-09-16T00:32:24.337437+0000","last_clean":"2024-09-16T00:32:24.337437+0000","last_became_active":"2024-09-16T00:30:26.069792+0000","last_became_peered":"2024-09-16T00:30:26.069792+0000","last_unstale":"2024-09-16T00:32:24.337437+0000","last_undegraded":"2024-09-16T00:32:24.337437+0000","last_fullsized":"2024-09-16T00:32:24.337437+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:28.164242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.336815+0000","last_change":"2024-09-16T00:30:24.423261+0000","last_active":"2024-09-16T00:32:24.336815+0000","last_peered":"2024-09-16T00:32:24.336815+0000","last_clean":"2024-09-16T00:32:24.336815+0000","last_became_active":"2024-09-16T00:30:24.422650+0000","last_became_peered":"2024-09-16T00:30:24.422650+0000","last_unstale":"2024-09-16T00:32:24.336815+0000","last_undegraded":"2024-09-16T00:32:24.336815+0000","last_fullsized":"2024-09-16T00:32:24.336815+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:00.211744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339762+0000","last_change":"2024-09-16T00:30:33.604492+0000","last_active":"2024-09-16T00:32:24.339762+0000","last_peered":"2024-09-16T00:32:24.339762+0000","last_clean":"2024-09-16T00:32:24.339762+0000","last_became_active":"2024-09-16T00:30:33.604015+0000","last_became_peered":"2024-09-16T00:30:33.604015+0000","last_unstale":"2024-09-16T00:32:24.339762+0000","last_undegraded":"2024-09-16T00:32:24.339762+0000","last_fullsized":"2024-09-16T00:32:24.339762+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:40:22.516583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.337389+0000","last_change":"2024-09-16T00:30:26.075517+0000","last_active":"2024-09-16T00:32:24.337389+0000","last_peered":"2024-09-16T00:32:24.337389+0000","last_clean":"2024-09-16T00:32:24.337389+0000","last_became_active":"2024-09-16T00:30:26.075113+0000","last_became_peered":"2024-09-16T00:30:26.075113+0000","last_unstale":"2024-09-16T00:32:24.337389+0000","last_undegraded":"2024-09-16T00:32:24.337389+0000","last_fullsized":"2024-09-16T00:32:24.337389+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:11.427010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595204+0000","last_change":"2024-09-16T00:30:23.933639+0000","last_active":"2024-09-16T00:32:24.595204+0000","last_peered":"2024-09-16T00:32:24.595204+0000","last_clean":"2024-09-16T00:32:24.595204+0000","last_became_active":"2024-09-16T00:30:23.933322+0000","last_became_peered":"2024-09-16T00:30:23.933322+0000","last_unstale":"2024-09-16T00:32:24.595204+0000","last_undegraded":"2024-09-16T00:32:24.595204+0000","last_fullsized":"2024-09-16T00:32:24.595204+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:59:59.082702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452139+0000","last_change":"2024-09-16T00:30:33.148508+0000","last_active":"2024-09-16T00:32:24.452139+0000","last_peered":"2024-09-16T00:32:24.452139+0000","last_clean":"2024-09-16T00:32:24.452139+0000","last_became_active":"2024-09-16T00:30:33.148091+0000","last_became_peered":"2024-09-16T00:30:33.148091+0000","last_unstale":"2024-09-16T00:32:24.452139+0000","last_undegraded":"2024-09-16T00:32:24.452139+0000","last_fullsized":"2024-09-16T00:32:24.452139+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:50.052526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454239+0000","last_change":"2024-09-16T00:30:26.080374+0000","last_active":"2024-09-16T00:32:24.454239+0000","last_peered":"2024-09-16T00:32:24.454239+0000","last_clean":"2024-09-16T00:32:24.454239+0000","last_became_active":"2024-09-16T00:30:26.080044+0000","last_became_peered":"2024-09-16T00:30:26.080044+0000","last_unstale":"2024-09-16T00:32:24.454239+0000","last_undegraded":"2024-09-16T00:32:24.454239+0000","last_fullsized":"2024-09-16T00:32:24.454239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:38:36.034903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":145,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:43.480306+0000","last_change":"2024-09-16T00:30:24.422137+0000","last_active":"2024-09-16T00:32:43.480306+0000","last_peered":"2024-09-16T00:32:43.480306+0000","last_clean":"2024-09-16T00:32:43.480306+0000","last_became_active":"2024-09-16T00:30:24.421778+0000","last_became_peered":"2024-09-16T00:30:24.421778+0000","last_unstale":"2024-09-16T00:32:43.480306+0000","last_undegraded":"2024-09-16T00:32:43.480306+0000","last_fullsized":"2024-09-16T00:32:43.480306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:54:21.964099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452309+0000","last_change":"2024-09-16T00:30:33.604263+0000","last_active":"2024-09-16T00:32:24.452309+0000","last_peered":"2024-09-16T00:32:24.452309+0000","last_clean":"2024-09-16T00:32:24.452309+0000","last_became_active":"2024-09-16T00:30:33.603804+0000","last_became_peered":"2024-09-16T00:30:33.603804+0000","last_unstale":"2024-09-16T00:32:24.452309+0000","last_undegraded":"2024-09-16T00:32:24.452309+0000","last_fullsized":"2024-09-16T00:32:24.452309+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:08:37.590827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.451792+0000","last_change":"2024-09-16T00:30:26.082916+0000","last_active":"2024-09-16T00:32:24.451792+0000","last_peered":"2024-09-16T00:32:24.451792+0000","last_clean":"2024-09-16T00:32:24.451792+0000","last_became_active":"2024-09-16T00:30:26.082606+0000","last_became_peered":"2024-09-16T00:30:26.082606+0000","last_unstale":"2024-09-16T00:32:24.451792+0000","last_undegraded":"2024-09-16T00:32:24.451792+0000","last_fullsized":"2024-09-16T00:32:24.451792+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:56.777640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594991+0000","last_change":"2024-09-16T00:30:23.937602+0000","last_active":"2024-09-16T00:32:24.594991+0000","last_peered":"2024-09-16T00:32:24.594991+0000","last_clean":"2024-09-16T00:32:24.594991+0000","last_became_active":"2024-09-16T00:30:23.937261+0000","last_became_peered":"2024-09-16T00:30:23.937261+0000","last_unstale":"2024-09-16T00:32:24.594991+0000","last_undegraded":"2024-09-16T00:32:24.594991+0000","last_fullsized":"2024-09-16T00:32:24.594991+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:24.698758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454567+0000","last_change":"2024-09-16T00:30:33.605536+0000","last_active":"2024-09-16T00:32:24.454567+0000","last_peered":"2024-09-16T00:32:24.454567+0000","last_clean":"2024-09-16T00:32:24.454567+0000","last_became_active":"2024-09-16T00:30:33.605073+0000","last_became_peered":"2024-09-16T00:30:33.605073+0000","last_unstale":"2024-09-16T00:32:24.454567+0000","last_undegraded":"2024-09-16T00:32:24.454567+0000","last_fullsized":"2024-09-16T00:32:24.454567+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:11.143052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595152+0000","last_change":"2024-09-16T00:30:26.084507+0000","last_active":"2024-09-16T00:32:24.595152+0000","last_peered":"2024-09-16T00:32:24.595152+0000","last_clean":"2024-09-16T00:32:24.595152+0000","last_became_active":"2024-09-16T00:30:26.084152+0000","last_became_peered":"2024-09-16T00:30:26.084152+0000","last_unstale":"2024-09-16T00:32:24.595152+0000","last_undegraded":"2024-09-16T00:32:24.595152+0000","last_fullsized":"2024-09-16T00:32:24.595152+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:27:21.557409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454701+0000","last_change":"2024-09-16T00:30:23.932686+0000","last_active":"2024-09-16T00:32:24.454701+0000","last_peered":"2024-09-16T00:32:24.454701+0000","last_clean":"2024-09-16T00:32:24.454701+0000","last_became_active":"2024-09-16T00:30:23.932343+0000","last_became_peered":"2024-09-16T00:30:23.932343+0000","last_unstale":"2024-09-16T00:32:24.454701+0000","last_undegraded":"2024-09-16T00:32:24.454701+0000","last_fullsized":"2024-09-16T00:32:24.454701+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:13:44.185712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":351,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595072+0000","last_change":"2024-09-16T00:29:11.143642+0000","last_active":"2024-09-16T00:32:24.595072+0000","last_peered":"2024-09-16T00:32:24.595072+0000","last_clean":"2024-09-16T00:32:24.595072+0000","last_became_active":"2024-09-16T00:29:11.143227+0000","last_became_peered":"2024-09-16T00:29:11.143227+0000","last_unstale":"2024-09-16T00:32:24.595072+0000","last_undegraded":"2024-09-16T00:32:24.595072+0000","last_fullsized":"2024-09-16T00:32:24.595072+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:25:42.361316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:25:42.361316+0000","last_clean_scrub_stamp":"2024-09-16T00:25:42.361316+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:29:12.740474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595397+0000","last_change":"2024-09-16T00:30:33.155555+0000","last_active":"2024-09-16T00:32:24.595397+0000","last_peered":"2024-09-16T00:32:24.595397+0000","last_clean":"2024-09-16T00:32:24.595397+0000","last_became_active":"2024-09-16T00:30:33.155170+0000","last_became_peered":"2024-09-16T00:30:33.155170+0000","last_unstale":"2024-09-16T00:32:24.595397+0000","last_undegraded":"2024-09-16T00:32:24.595397+0000","last_fullsized":"2024-09-16T00:32:24.595397+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:34:46.214422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595573+0000","last_change":"2024-09-16T00:30:26.076003+0000","last_active":"2024-09-16T00:32:24.595573+0000","last_peered":"2024-09-16T00:32:24.595573+0000","last_clean":"2024-09-16T00:32:24.595573+0000","last_became_active":"2024-09-16T00:30:26.075666+0000","last_became_peered":"2024-09-16T00:30:26.075666+0000","last_unstale":"2024-09-16T00:32:24.595573+0000","last_undegraded":"2024-09-16T00:32:24.595573+0000","last_fullsized":"2024-09-16T00:32:24.595573+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:30.113380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594420+0000","last_change":"2024-09-16T00:30:24.587469+0000","last_active":"2024-09-16T00:32:24.594420+0000","last_peered":"2024-09-16T00:32:24.594420+0000","last_clean":"2024-09-16T00:32:24.594420+0000","last_became_active":"2024-09-16T00:30:24.587049+0000","last_became_peered":"2024-09-16T00:30:24.587049+0000","last_unstale":"2024-09-16T00:32:24.594420+0000","last_undegraded":"2024-09-16T00:32:24.594420+0000","last_fullsized":"2024-09-16T00:32:24.594420+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:17:45.934505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561433+0000","last_change":"2024-09-16T00:30:33.145921+0000","last_active":"2024-09-16T00:32:24.561433+0000","last_peered":"2024-09-16T00:32:24.561433+0000","last_clean":"2024-09-16T00:32:24.561433+0000","last_became_active":"2024-09-16T00:30:33.145450+0000","last_became_peered":"2024-09-16T00:30:33.145450+0000","last_unstale":"2024-09-16T00:32:24.561433+0000","last_undegraded":"2024-09-16T00:32:24.561433+0000","last_fullsized":"2024-09-16T00:32:24.561433+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:53:51.698686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595330+0000","last_change":"2024-09-16T00:30:26.087940+0000","last_active":"2024-09-16T00:32:24.595330+0000","last_peered":"2024-09-16T00:32:24.595330+0000","last_clean":"2024-09-16T00:32:24.595330+0000","last_became_active":"2024-09-16T00:30:26.087599+0000","last_became_peered":"2024-09-16T00:30:26.087599+0000","last_unstale":"2024-09-16T00:32:24.595330+0000","last_undegraded":"2024-09-16T00:32:24.595330+0000","last_fullsized":"2024-09-16T00:32:24.595330+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:43.934839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454847+0000","last_change":"2024-09-16T00:30:23.930986+0000","last_active":"2024-09-16T00:32:24.454847+0000","last_peered":"2024-09-16T00:32:24.454847+0000","last_clean":"2024-09-16T00:32:24.454847+0000","last_became_active":"2024-09-16T00:30:23.930588+0000","last_became_peered":"2024-09-16T00:30:23.930588+0000","last_unstale":"2024-09-16T00:32:24.454847+0000","last_undegraded":"2024-09-16T00:32:24.454847+0000","last_fullsized":"2024-09-16T00:32:24.454847+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:04:41.902358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594672+0000","last_change":"2024-09-16T00:30:33.154894+0000","last_active":"2024-09-16T00:32:24.594672+0000","last_peered":"2024-09-16T00:32:24.594672+0000","last_clean":"2024-09-16T00:32:24.594672+0000","last_became_active":"2024-09-16T00:30:33.154490+0000","last_became_peered":"2024-09-16T00:30:33.154490+0000","last_unstale":"2024-09-16T00:32:24.594672+0000","last_undegraded":"2024-09-16T00:32:24.594672+0000","last_fullsized":"2024-09-16T00:32:24.594672+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:59:53.854212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595549+0000","last_change":"2024-09-16T00:30:26.077069+0000","last_active":"2024-09-16T00:32:24.595549+0000","last_peered":"2024-09-16T00:32:24.595549+0000","last_clean":"2024-09-16T00:32:24.595549+0000","last_became_active":"2024-09-16T00:30:26.076718+0000","last_became_peered":"2024-09-16T00:30:26.076718+0000","last_unstale":"2024-09-16T00:32:24.595549+0000","last_undegraded":"2024-09-16T00:32:24.595549+0000","last_fullsized":"2024-09-16T00:32:24.595549+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:25:37.541704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452322+0000","last_change":"2024-09-16T00:30:23.932081+0000","last_active":"2024-09-16T00:32:24.452322+0000","last_peered":"2024-09-16T00:32:24.452322+0000","last_clean":"2024-09-16T00:32:24.452322+0000","last_became_active":"2024-09-16T00:30:23.931729+0000","last_became_peered":"2024-09-16T00:30:23.931729+0000","last_unstale":"2024-09-16T00:32:24.452322+0000","last_undegraded":"2024-09-16T00:32:24.452322+0000","last_fullsized":"2024-09-16T00:32:24.452322+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:20:05.178580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454073+0000","last_change":"2024-09-16T00:30:33.147738+0000","last_active":"2024-09-16T00:32:24.454073+0000","last_peered":"2024-09-16T00:32:24.454073+0000","last_clean":"2024-09-16T00:32:24.454073+0000","last_became_active":"2024-09-16T00:30:33.147326+0000","last_became_peered":"2024-09-16T00:30:33.147326+0000","last_unstale":"2024-09-16T00:32:24.454073+0000","last_undegraded":"2024-09-16T00:32:24.454073+0000","last_fullsized":"2024-09-16T00:32:24.454073+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:44:55.123980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339213+0000","last_change":"2024-09-16T00:30:26.079312+0000","last_active":"2024-09-16T00:32:24.339213+0000","last_peered":"2024-09-16T00:32:24.339213+0000","last_clean":"2024-09-16T00:32:24.339213+0000","last_became_active":"2024-09-16T00:30:26.078986+0000","last_became_peered":"2024-09-16T00:30:26.078986+0000","last_unstale":"2024-09-16T00:32:24.339213+0000","last_undegraded":"2024-09-16T00:32:24.339213+0000","last_fullsized":"2024-09-16T00:32:24.339213+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:02:06.602850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339614+0000","last_change":"2024-09-16T00:31:23.452102+0000","last_active":"2024-09-16T00:32:24.339614+0000","last_peered":"2024-09-16T00:32:24.339614+0000","last_clean":"2024-09-16T00:32:24.339614+0000","last_became_active":"2024-09-16T00:31:23.445974+0000","last_became_peered":"2024-09-16T00:31:23.445974+0000","last_unstale":"2024-09-16T00:32:24.339614+0000","last_undegraded":"2024-09-16T00:32:24.339614+0000","last_fullsized":"2024-09-16T00:32:24.339614+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:52.314494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454166+0000","last_change":"2024-09-16T00:30:33.149154+0000","last_active":"2024-09-16T00:32:24.454166+0000","last_peered":"2024-09-16T00:32:24.454166+0000","last_clean":"2024-09-16T00:32:24.454166+0000","last_became_active":"2024-09-16T00:30:33.148763+0000","last_became_peered":"2024-09-16T00:30:33.148763+0000","last_unstale":"2024-09-16T00:32:24.454166+0000","last_undegraded":"2024-09-16T00:32:24.454166+0000","last_fullsized":"2024-09-16T00:32:24.454166+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:58:05.884919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454241+0000","last_change":"2024-09-16T00:30:26.086905+0000","last_active":"2024-09-16T00:32:24.454241+0000","last_peered":"2024-09-16T00:32:24.454241+0000","last_clean":"2024-09-16T00:32:24.454241+0000","last_became_active":"2024-09-16T00:30:26.086519+0000","last_became_peered":"2024-09-16T00:30:26.086519+0000","last_unstale":"2024-09-16T00:32:24.454241+0000","last_undegraded":"2024-09-16T00:32:24.454241+0000","last_fullsized":"2024-09-16T00:32:24.454241+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:51:08.787679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.338768+0000","last_change":"2024-09-16T00:30:24.422893+0000","last_active":"2024-09-16T00:32:24.338768+0000","last_peered":"2024-09-16T00:32:24.338768+0000","last_clean":"2024-09-16T00:32:24.338768+0000","last_became_active":"2024-09-16T00:30:24.422073+0000","last_became_peered":"2024-09-16T00:30:24.422073+0000","last_unstale":"2024-09-16T00:32:24.338768+0000","last_undegraded":"2024-09-16T00:32:24.338768+0000","last_fullsized":"2024-09-16T00:32:24.338768+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:15:11.375957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452155+0000","last_change":"2024-09-16T00:30:33.152582+0000","last_active":"2024-09-16T00:32:24.452155+0000","last_peered":"2024-09-16T00:32:24.452155+0000","last_clean":"2024-09-16T00:32:24.452155+0000","last_became_active":"2024-09-16T00:30:33.152180+0000","last_became_peered":"2024-09-16T00:30:33.152180+0000","last_unstale":"2024-09-16T00:32:24.452155+0000","last_undegraded":"2024-09-16T00:32:24.452155+0000","last_fullsized":"2024-09-16T00:32:24.452155+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:41:10.499391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594802+0000","last_change":"2024-09-16T00:30:26.086052+0000","last_active":"2024-09-16T00:32:24.594802+0000","last_peered":"2024-09-16T00:32:24.594802+0000","last_clean":"2024-09-16T00:32:24.594802+0000","last_became_active":"2024-09-16T00:30:26.085756+0000","last_became_peered":"2024-09-16T00:30:26.085756+0000","last_unstale":"2024-09-16T00:32:24.594802+0000","last_undegraded":"2024-09-16T00:32:24.594802+0000","last_fullsized":"2024-09-16T00:32:24.594802+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:52.582166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":144,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:42.479580+0000","last_change":"2024-09-16T00:30:23.933528+0000","last_active":"2024-09-16T00:32:42.479580+0000","last_peered":"2024-09-16T00:32:42.479580+0000","last_clean":"2024-09-16T00:32:42.479580+0000","last_became_active":"2024-09-16T00:30:23.933186+0000","last_became_peered":"2024-09-16T00:30:23.933186+0000","last_unstale":"2024-09-16T00:32:42.479580+0000","last_undegraded":"2024-09-16T00:32:42.479580+0000","last_fullsized":"2024-09-16T00:32:42.479580+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:11:00.489676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.340727+0000","last_change":"2024-09-16T00:31:23.578056+0000","last_active":"2024-09-16T00:32:24.340727+0000","last_peered":"2024-09-16T00:32:24.340727+0000","last_clean":"2024-09-16T00:32:24.340727+0000","last_became_active":"2024-09-16T00:31:23.577721+0000","last_became_peered":"2024-09-16T00:31:23.577721+0000","last_unstale":"2024-09-16T00:32:24.340727+0000","last_undegraded":"2024-09-16T00:32:24.340727+0000","last_fullsized":"2024-09-16T00:32:24.340727+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:53.632440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595248+0000","last_change":"2024-09-16T00:30:26.084509+0000","last_active":"2024-09-16T00:32:24.595248+0000","last_peered":"2024-09-16T00:32:24.595248+0000","last_clean":"2024-09-16T00:32:24.595248+0000","last_became_active":"2024-09-16T00:30:26.084188+0000","last_became_peered":"2024-09-16T00:30:26.084188+0000","last_unstale":"2024-09-16T00:32:24.595248+0000","last_undegraded":"2024-09-16T00:32:24.595248+0000","last_fullsized":"2024-09-16T00:32:24.595248+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:21:48.981277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454529+0000","last_change":"2024-09-16T00:30:23.933499+0000","last_active":"2024-09-16T00:32:24.454529+0000","last_peered":"2024-09-16T00:32:24.454529+0000","last_clean":"2024-09-16T00:32:24.454529+0000","last_became_active":"2024-09-16T00:30:23.933082+0000","last_became_peered":"2024-09-16T00:30:23.933082+0000","last_unstale":"2024-09-16T00:32:24.454529+0000","last_undegraded":"2024-09-16T00:32:24.454529+0000","last_fullsized":"2024-09-16T00:32:24.454529+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:42:18.998433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.596318+0000","last_change":"2024-09-16T00:32:24.596318+0000","last_active":"2024-09-16T00:32:24.596318+0000","last_peered":"2024-09-16T00:32:24.596318+0000","last_clean":"2024-09-16T00:32:24.596318+0000","last_became_active":"2024-09-16T00:32:24.595906+0000","last_became_peered":"2024-09-16T00:32:24.595906+0000","last_unstale":"2024-09-16T00:32:24.596318+0000","last_undegraded":"2024-09-16T00:32:24.596318+0000","last_fullsized":"2024-09-16T00:32:24.596318+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:13:56.601745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452018+0000","last_change":"2024-09-16T00:30:26.080517+0000","last_active":"2024-09-16T00:32:24.452018+0000","last_peered":"2024-09-16T00:32:24.452018+0000","last_clean":"2024-09-16T00:32:24.452018+0000","last_became_active":"2024-09-16T00:30:26.080197+0000","last_became_peered":"2024-09-16T00:30:26.080197+0000","last_unstale":"2024-09-16T00:32:24.452018+0000","last_undegraded":"2024-09-16T00:32:24.452018+0000","last_fullsized":"2024-09-16T00:32:24.452018+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:15.508562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339267+0000","last_change":"2024-09-16T00:31:23.457047+0000","last_active":"2024-09-16T00:32:24.339267+0000","last_peered":"2024-09-16T00:32:24.339267+0000","last_clean":"2024-09-16T00:32:24.339267+0000","last_became_active":"2024-09-16T00:31:23.447014+0000","last_became_peered":"2024-09-16T00:31:23.447014+0000","last_unstale":"2024-09-16T00:32:24.339267+0000","last_undegraded":"2024-09-16T00:32:24.339267+0000","last_fullsized":"2024-09-16T00:32:24.339267+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:34:42.040228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454641+0000","last_change":"2024-09-16T00:30:33.145803+0000","last_active":"2024-09-16T00:32:24.454641+0000","last_peered":"2024-09-16T00:32:24.454641+0000","last_clean":"2024-09-16T00:32:24.454641+0000","last_became_active":"2024-09-16T00:30:33.145366+0000","last_became_peered":"2024-09-16T00:30:33.145366+0000","last_unstale":"2024-09-16T00:32:24.454641+0000","last_undegraded":"2024-09-16T00:32:24.454641+0000","last_fullsized":"2024-09-16T00:32:24.454641+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:47:28.689305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339085+0000","last_change":"2024-09-16T00:30:26.079415+0000","last_active":"2024-09-16T00:32:24.339085+0000","last_peered":"2024-09-16T00:32:24.339085+0000","last_clean":"2024-09-16T00:32:24.339085+0000","last_became_active":"2024-09-16T00:30:26.079075+0000","last_became_peered":"2024-09-16T00:30:26.079075+0000","last_unstale":"2024-09-16T00:32:24.339085+0000","last_undegraded":"2024-09-16T00:32:24.339085+0000","last_fullsized":"2024-09-16T00:32:24.339085+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:18:18.236342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"92'39","reported_seq":291,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:43.484104+0000","last_change":"2024-09-16T00:30:23.933172+0000","last_active":"2024-09-16T00:32:43.484104+0000","last_peered":"2024-09-16T00:32:43.484104+0000","last_clean":"2024-09-16T00:32:43.484104+0000","last_became_active":"2024-09-16T00:30:23.932492+0000","last_became_peered":"2024-09-16T00:30:23.932492+0000","last_unstale":"2024-09-16T00:32:43.484104+0000","last_undegraded":"2024-09-16T00:32:43.484104+0000","last_fullsized":"2024-09-16T00:32:43.484104+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:29.169448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68399,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":213,"num_read_kb":175,"num_write":40,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.337053+0000","last_change":"2024-09-16T00:31:23.445673+0000","last_active":"2024-09-16T00:32:24.337053+0000","last_peered":"2024-09-16T00:32:24.337053+0000","last_clean":"2024-09-16T00:32:24.337053+0000","last_became_active":"2024-09-16T00:31:23.445299+0000","last_became_peered":"2024-09-16T00:31:23.445299+0000","last_unstale":"2024-09-16T00:32:24.337053+0000","last_undegraded":"2024-09-16T00:32:24.337053+0000","last_fullsized":"2024-09-16T00:32:24.337053+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:07:44.235340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561729+0000","last_change":"2024-09-16T00:30:26.085419+0000","last_active":"2024-09-16T00:32:24.561729+0000","last_peered":"2024-09-16T00:32:24.561729+0000","last_clean":"2024-09-16T00:32:24.561729+0000","last_became_active":"2024-09-16T00:30:26.085103+0000","last_became_peered":"2024-09-16T00:30:26.085103+0000","last_unstale":"2024-09-16T00:32:24.561729+0000","last_undegraded":"2024-09-16T00:32:24.561729+0000","last_fullsized":"2024-09-16T00:32:24.561729+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:56:26.352779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.453906+0000","last_change":"2024-09-16T00:30:24.422646+0000","last_active":"2024-09-16T00:32:24.453906+0000","last_peered":"2024-09-16T00:32:24.453906+0000","last_clean":"2024-09-16T00:32:24.453906+0000","last_became_active":"2024-09-16T00:30:24.422065+0000","last_became_peered":"2024-09-16T00:30:24.422065+0000","last_unstale":"2024-09-16T00:32:24.453906+0000","last_undegraded":"2024-09-16T00:32:24.453906+0000","last_fullsized":"2024-09-16T00:32:24.453906+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:53:12.401613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454317+0000","last_change":"2024-09-16T00:30:33.147692+0000","last_active":"2024-09-16T00:32:24.454317+0000","last_peered":"2024-09-16T00:32:24.454317+0000","last_clean":"2024-09-16T00:32:24.454317+0000","last_became_active":"2024-09-16T00:30:33.147115+0000","last_became_peered":"2024-09-16T00:30:33.147115+0000","last_unstale":"2024-09-16T00:32:24.454317+0000","last_undegraded":"2024-09-16T00:32:24.454317+0000","last_fullsized":"2024-09-16T00:32:24.454317+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:44:04.536894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595596+0000","last_change":"2024-09-16T00:30:26.088427+0000","last_active":"2024-09-16T00:32:24.595596+0000","last_peered":"2024-09-16T00:32:24.595596+0000","last_clean":"2024-09-16T00:32:24.595596+0000","last_became_active":"2024-09-16T00:30:26.086454+0000","last_became_peered":"2024-09-16T00:30:26.086454+0000","last_unstale":"2024-09-16T00:32:24.595596+0000","last_undegraded":"2024-09-16T00:32:24.595596+0000","last_fullsized":"2024-09-16T00:32:24.595596+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:54:29.679959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595460+0000","last_change":"2024-09-16T00:31:23.453484+0000","last_active":"2024-09-16T00:32:24.595460+0000","last_peered":"2024-09-16T00:32:24.595460+0000","last_clean":"2024-09-16T00:32:24.595460+0000","last_became_active":"2024-09-16T00:31:23.446700+0000","last_became_peered":"2024-09-16T00:31:23.446700+0000","last_unstale":"2024-09-16T00:32:24.595460+0000","last_undegraded":"2024-09-16T00:32:24.595460+0000","last_fullsized":"2024-09-16T00:32:24.595460+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:43:26.465657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.336273+0000","last_change":"2024-09-16T00:30:33.156510+0000","last_active":"2024-09-16T00:32:24.336273+0000","last_peered":"2024-09-16T00:32:24.336273+0000","last_clean":"2024-09-16T00:32:24.336273+0000","last_became_active":"2024-09-16T00:30:33.156088+0000","last_became_peered":"2024-09-16T00:30:33.156088+0000","last_unstale":"2024-09-16T00:32:24.336273+0000","last_undegraded":"2024-09-16T00:32:24.336273+0000","last_fullsized":"2024-09-16T00:32:24.336273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:34.230901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561872+0000","last_change":"2024-09-16T00:30:26.082598+0000","last_active":"2024-09-16T00:32:24.561872+0000","last_peered":"2024-09-16T00:32:24.561872+0000","last_clean":"2024-09-16T00:32:24.561872+0000","last_became_active":"2024-09-16T00:30:26.082286+0000","last_became_peered":"2024-09-16T00:30:26.082286+0000","last_unstale":"2024-09-16T00:32:24.561872+0000","last_undegraded":"2024-09-16T00:32:24.561872+0000","last_fullsized":"2024-09-16T00:32:24.561872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:22:18.579149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"92'5","reported_seq":144,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:42.481119+0000","last_change":"2024-09-16T00:30:23.931097+0000","last_active":"2024-09-16T00:32:42.481119+0000","last_peered":"2024-09-16T00:32:42.481119+0000","last_clean":"2024-09-16T00:32:42.481119+0000","last_became_active":"2024-09-16T00:30:23.930719+0000","last_became_peered":"2024-09-16T00:30:23.930719+0000","last_unstale":"2024-09-16T00:32:42.481119+0000","last_undegraded":"2024-09-16T00:32:42.481119+0000","last_fullsized":"2024-09-16T00:32:42.481119+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:13:47.887754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595241+0000","last_change":"2024-09-16T00:30:33.142581+0000","last_active":"2024-09-16T00:32:24.595241+0000","last_peered":"2024-09-16T00:32:24.595241+0000","last_clean":"2024-09-16T00:32:24.595241+0000","last_became_active":"2024-09-16T00:30:33.142263+0000","last_became_peered":"2024-09-16T00:30:33.142263+0000","last_unstale":"2024-09-16T00:32:24.595241+0000","last_undegraded":"2024-09-16T00:32:24.595241+0000","last_fullsized":"2024-09-16T00:32:24.595241+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:10:06.456011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595181+0000","last_change":"2024-09-16T00:30:33.152084+0000","last_active":"2024-09-16T00:32:24.595181+0000","last_peered":"2024-09-16T00:32:24.595181+0000","last_clean":"2024-09-16T00:32:24.595181+0000","last_became_active":"2024-09-16T00:30:33.151681+0000","last_became_peered":"2024-09-16T00:30:33.151681+0000","last_unstale":"2024-09-16T00:32:24.595181+0000","last_undegraded":"2024-09-16T00:32:24.595181+0000","last_fullsized":"2024-09-16T00:32:24.595181+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:36.966498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.336370+0000","last_change":"2024-09-16T00:30:26.084200+0000","last_active":"2024-09-16T00:32:24.336370+0000","last_peered":"2024-09-16T00:32:24.336370+0000","last_clean":"2024-09-16T00:32:24.336370+0000","last_became_active":"2024-09-16T00:30:26.083858+0000","last_became_peered":"2024-09-16T00:30:26.083858+0000","last_unstale":"2024-09-16T00:32:24.336370+0000","last_undegraded":"2024-09-16T00:32:24.336370+0000","last_fullsized":"2024-09-16T00:32:24.336370+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:42.712624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454319+0000","last_change":"2024-09-16T00:30:23.932399+0000","last_active":"2024-09-16T00:32:24.454319+0000","last_peered":"2024-09-16T00:32:24.454319+0000","last_clean":"2024-09-16T00:32:24.454319+0000","last_became_active":"2024-09-16T00:30:23.931996+0000","last_became_peered":"2024-09-16T00:30:23.931996+0000","last_unstale":"2024-09-16T00:32:24.454319+0000","last_undegraded":"2024-09-16T00:32:24.454319+0000","last_fullsized":"2024-09-16T00:32:24.454319+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:47:09.634507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339553+0000","last_change":"2024-09-16T00:30:33.154563+0000","last_active":"2024-09-16T00:32:24.339553+0000","last_peered":"2024-09-16T00:32:24.339553+0000","last_clean":"2024-09-16T00:32:24.339553+0000","last_became_active":"2024-09-16T00:30:33.154105+0000","last_became_peered":"2024-09-16T00:30:33.154105+0000","last_unstale":"2024-09-16T00:32:24.339553+0000","last_undegraded":"2024-09-16T00:32:24.339553+0000","last_fullsized":"2024-09-16T00:32:24.339553+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:59.966255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595005+0000","last_change":"2024-09-16T00:30:26.075126+0000","last_active":"2024-09-16T00:32:24.595005+0000","last_peered":"2024-09-16T00:32:24.595005+0000","last_clean":"2024-09-16T00:32:24.595005+0000","last_became_active":"2024-09-16T00:30:26.074819+0000","last_became_peered":"2024-09-16T00:30:26.074819+0000","last_unstale":"2024-09-16T00:32:24.595005+0000","last_undegraded":"2024-09-16T00:32:24.595005+0000","last_fullsized":"2024-09-16T00:32:24.595005+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:36.996346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.451802+0000","last_change":"2024-09-16T00:30:23.929943+0000","last_active":"2024-09-16T00:32:24.451802+0000","last_peered":"2024-09-16T00:32:24.451802+0000","last_clean":"2024-09-16T00:32:24.451802+0000","last_became_active":"2024-09-16T00:30:23.929578+0000","last_became_peered":"2024-09-16T00:30:23.929578+0000","last_unstale":"2024-09-16T00:32:24.451802+0000","last_undegraded":"2024-09-16T00:32:24.451802+0000","last_fullsized":"2024-09-16T00:32:24.451802+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:05:12.505543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.337224+0000","last_change":"2024-09-16T00:30:23.935758+0000","last_active":"2024-09-16T00:32:24.337224+0000","last_peered":"2024-09-16T00:32:24.337224+0000","last_clean":"2024-09-16T00:32:24.337224+0000","last_became_active":"2024-09-16T00:30:23.935321+0000","last_became_peered":"2024-09-16T00:30:23.935321+0000","last_unstale":"2024-09-16T00:32:24.337224+0000","last_undegraded":"2024-09-16T00:32:24.337224+0000","last_fullsized":"2024-09-16T00:32:24.337224+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:35:40.513435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454389+0000","last_change":"2024-09-16T00:30:26.086316+0000","last_active":"2024-09-16T00:32:24.454389+0000","last_peered":"2024-09-16T00:32:24.454389+0000","last_clean":"2024-09-16T00:32:24.454389+0000","last_became_active":"2024-09-16T00:30:26.085800+0000","last_became_peered":"2024-09-16T00:30:26.085800+0000","last_unstale":"2024-09-16T00:32:24.454389+0000","last_undegraded":"2024-09-16T00:32:24.454389+0000","last_fullsized":"2024-09-16T00:32:24.454389+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:05:36.277127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595014+0000","last_change":"2024-09-16T00:30:33.155665+0000","last_active":"2024-09-16T00:32:24.595014+0000","last_peered":"2024-09-16T00:32:24.595014+0000","last_clean":"2024-09-16T00:32:24.595014+0000","last_became_active":"2024-09-16T00:30:33.155253+0000","last_became_peered":"2024-09-16T00:30:33.155253+0000","last_unstale":"2024-09-16T00:32:24.595014+0000","last_undegraded":"2024-09-16T00:32:24.595014+0000","last_fullsized":"2024-09-16T00:32:24.595014+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:37.545232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'8","reported_seq":151,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:42.481651+0000","last_change":"2024-09-16T00:30:23.934487+0000","last_active":"2024-09-16T00:32:42.481651+0000","last_peered":"2024-09-16T00:32:42.481651+0000","last_clean":"2024-09-16T00:32:42.481651+0000","last_became_active":"2024-09-16T00:30:23.934137+0000","last_became_peered":"2024-09-16T00:30:23.934137+0000","last_unstale":"2024-09-16T00:32:42.481651+0000","last_undegraded":"2024-09-16T00:32:42.481651+0000","last_fullsized":"2024-09-16T00:32:42.481651+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:14:33.088776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1549,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.338839+0000","last_change":"2024-09-16T00:30:26.077533+0000","last_active":"2024-09-16T00:32:24.338839+0000","last_peered":"2024-09-16T00:32:24.338839+0000","last_clean":"2024-09-16T00:32:24.338839+0000","last_became_active":"2024-09-16T00:30:26.076443+0000","last_became_peered":"2024-09-16T00:30:26.076443+0000","last_unstale":"2024-09-16T00:32:24.338839+0000","last_undegraded":"2024-09-16T00:32:24.338839+0000","last_fullsized":"2024-09-16T00:32:24.338839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:39:17.495521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595151+0000","last_change":"2024-09-16T00:30:33.146123+0000","last_active":"2024-09-16T00:32:24.595151+0000","last_peered":"2024-09-16T00:32:24.595151+0000","last_clean":"2024-09-16T00:32:24.595151+0000","last_became_active":"2024-09-16T00:30:33.145716+0000","last_became_peered":"2024-09-16T00:30:33.145716+0000","last_unstale":"2024-09-16T00:32:24.595151+0000","last_undegraded":"2024-09-16T00:32:24.595151+0000","last_fullsized":"2024-09-16T00:32:24.595151+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:14:27.281367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.338818+0000","last_change":"2024-09-16T00:30:26.083155+0000","last_active":"2024-09-16T00:32:24.338818+0000","last_peered":"2024-09-16T00:32:24.338818+0000","last_clean":"2024-09-16T00:32:24.338818+0000","last_became_active":"2024-09-16T00:30:26.082782+0000","last_became_peered":"2024-09-16T00:30:26.082782+0000","last_unstale":"2024-09-16T00:32:24.338818+0000","last_undegraded":"2024-09-16T00:32:24.338818+0000","last_fullsized":"2024-09-16T00:32:24.338818+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:15.929343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594891+0000","last_change":"2024-09-16T00:31:23.447453+0000","last_active":"2024-09-16T00:32:24.594891+0000","last_peered":"2024-09-16T00:32:24.594891+0000","last_clean":"2024-09-16T00:32:24.594891+0000","last_became_active":"2024-09-16T00:31:23.445507+0000","last_became_peered":"2024-09-16T00:31:23.445507+0000","last_unstale":"2024-09-16T00:32:24.594891+0000","last_undegraded":"2024-09-16T00:32:24.594891+0000","last_fullsized":"2024-09-16T00:32:24.594891+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:18:50.885809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594798+0000","last_change":"2024-09-16T00:30:26.076252+0000","last_active":"2024-09-16T00:32:24.594798+0000","last_peered":"2024-09-16T00:32:24.594798+0000","last_clean":"2024-09-16T00:32:24.594798+0000","last_became_active":"2024-09-16T00:30:26.075922+0000","last_became_peered":"2024-09-16T00:30:26.075922+0000","last_unstale":"2024-09-16T00:32:24.594798+0000","last_undegraded":"2024-09-16T00:32:24.594798+0000","last_fullsized":"2024-09-16T00:32:24.594798+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:28:04.822817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594821+0000","last_change":"2024-09-16T00:30:26.085123+0000","last_active":"2024-09-16T00:32:24.594821+0000","last_peered":"2024-09-16T00:32:24.594821+0000","last_clean":"2024-09-16T00:32:24.594821+0000","last_became_active":"2024-09-16T00:30:26.084782+0000","last_became_peered":"2024-09-16T00:30:26.084782+0000","last_unstale":"2024-09-16T00:32:24.594821+0000","last_undegraded":"2024-09-16T00:32:24.594821+0000","last_fullsized":"2024-09-16T00:32:24.594821+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:09:33.766340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452404+0000","last_change":"2024-09-16T00:30:26.077048+0000","last_active":"2024-09-16T00:32:24.452404+0000","last_peered":"2024-09-16T00:32:24.452404+0000","last_clean":"2024-09-16T00:32:24.452404+0000","last_became_active":"2024-09-16T00:30:26.076719+0000","last_became_peered":"2024-09-16T00:30:26.076719+0000","last_unstale":"2024-09-16T00:32:24.452404+0000","last_undegraded":"2024-09-16T00:32:24.452404+0000","last_fullsized":"2024-09-16T00:32:24.452404+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:58.584246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339277+0000","last_change":"2024-09-16T00:30:26.078446+0000","last_active":"2024-09-16T00:32:24.339277+0000","last_peered":"2024-09-16T00:32:24.339277+0000","last_clean":"2024-09-16T00:32:24.339277+0000","last_became_active":"2024-09-16T00:30:26.078145+0000","last_became_peered":"2024-09-16T00:30:26.078145+0000","last_unstale":"2024-09-16T00:32:24.339277+0000","last_undegraded":"2024-09-16T00:32:24.339277+0000","last_fullsized":"2024-09-16T00:32:24.339277+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:52:28.739834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595347+0000","last_change":"2024-09-16T00:30:26.075875+0000","last_active":"2024-09-16T00:32:24.595347+0000","last_peered":"2024-09-16T00:32:24.595347+0000","last_clean":"2024-09-16T00:32:24.595347+0000","last_became_active":"2024-09-16T00:30:26.075424+0000","last_became_peered":"2024-09-16T00:30:26.075424+0000","last_unstale":"2024-09-16T00:32:24.595347+0000","last_undegraded":"2024-09-16T00:32:24.595347+0000","last_fullsized":"2024-09-16T00:32:24.595347+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:21:07.958507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595253+0000","last_change":"2024-09-16T00:30:26.077694+0000","last_active":"2024-09-16T00:32:24.595253+0000","last_peered":"2024-09-16T00:32:24.595253+0000","last_clean":"2024-09-16T00:32:24.595253+0000","last_became_active":"2024-09-16T00:30:26.077374+0000","last_became_peered":"2024-09-16T00:30:26.077374+0000","last_unstale":"2024-09-16T00:32:24.595253+0000","last_undegraded":"2024-09-16T00:32:24.595253+0000","last_fullsized":"2024-09-16T00:32:24.595253+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:16:30.822050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454143+0000","last_change":"2024-09-16T00:30:26.085931+0000","last_active":"2024-09-16T00:32:24.454143+0000","last_peered":"2024-09-16T00:32:24.454143+0000","last_clean":"2024-09-16T00:32:24.454143+0000","last_became_active":"2024-09-16T00:30:26.085420+0000","last_became_peered":"2024-09-16T00:30:26.085420+0000","last_unstale":"2024-09-16T00:32:24.454143+0000","last_undegraded":"2024-09-16T00:32:24.454143+0000","last_fullsized":"2024-09-16T00:32:24.454143+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:20:36.019203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595470+0000","last_change":"2024-09-16T00:30:26.074217+0000","last_active":"2024-09-16T00:32:24.595470+0000","last_peered":"2024-09-16T00:32:24.595470+0000","last_clean":"2024-09-16T00:32:24.595470+0000","last_became_active":"2024-09-16T00:30:26.073856+0000","last_became_peered":"2024-09-16T00:30:26.073856+0000","last_unstale":"2024-09-16T00:32:24.595470+0000","last_undegraded":"2024-09-16T00:32:24.595470+0000","last_fullsized":"2024-09-16T00:32:24.595470+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:53:57.078465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561639+0000","last_change":"2024-09-16T00:30:26.084248+0000","last_active":"2024-09-16T00:32:24.561639+0000","last_peered":"2024-09-16T00:32:24.561639+0000","last_clean":"2024-09-16T00:32:24.561639+0000","last_became_active":"2024-09-16T00:30:26.083530+0000","last_became_peered":"2024-09-16T00:30:26.083530+0000","last_unstale":"2024-09-16T00:32:24.561639+0000","last_undegraded":"2024-09-16T00:32:24.561639+0000","last_fullsized":"2024-09-16T00:32:24.561639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:16:49.483881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452162+0000","last_change":"2024-09-16T00:30:26.080803+0000","last_active":"2024-09-16T00:32:24.452162+0000","last_peered":"2024-09-16T00:32:24.452162+0000","last_clean":"2024-09-16T00:32:24.452162+0000","last_became_active":"2024-09-16T00:30:26.080482+0000","last_became_peered":"2024-09-16T00:30:26.080482+0000","last_unstale":"2024-09-16T00:32:24.452162+0000","last_undegraded":"2024-09-16T00:32:24.452162+0000","last_fullsized":"2024-09-16T00:32:24.452162+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:35:41.741025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339770+0000","last_change":"2024-09-16T00:30:26.077680+0000","last_active":"2024-09-16T00:32:24.339770+0000","last_peered":"2024-09-16T00:32:24.339770+0000","last_clean":"2024-09-16T00:32:24.339770+0000","last_became_active":"2024-09-16T00:30:26.077371+0000","last_became_peered":"2024-09-16T00:30:26.077371+0000","last_unstale":"2024-09-16T00:32:24.339770+0000","last_undegraded":"2024-09-16T00:32:24.339770+0000","last_fullsized":"2024-09-16T00:32:24.339770+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:31:34.338617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595364+0000","last_change":"2024-09-16T00:30:26.071531+0000","last_active":"2024-09-16T00:32:24.595364+0000","last_peered":"2024-09-16T00:32:24.595364+0000","last_clean":"2024-09-16T00:32:24.595364+0000","last_became_active":"2024-09-16T00:30:26.071189+0000","last_became_peered":"2024-09-16T00:30:26.071189+0000","last_unstale":"2024-09-16T00:32:24.595364+0000","last_undegraded":"2024-09-16T00:32:24.595364+0000","last_fullsized":"2024-09-16T00:32:24.595364+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:29:18.406272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.336865+0000","last_change":"2024-09-16T00:30:26.085398+0000","last_active":"2024-09-16T00:32:24.336865+0000","last_peered":"2024-09-16T00:32:24.336865+0000","last_clean":"2024-09-16T00:32:24.336865+0000","last_became_active":"2024-09-16T00:30:26.085076+0000","last_became_peered":"2024-09-16T00:30:26.085076+0000","last_unstale":"2024-09-16T00:32:24.336865+0000","last_undegraded":"2024-09-16T00:32:24.336865+0000","last_fullsized":"2024-09-16T00:32:24.336865+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:20:20.864660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.340070+0000","last_change":"2024-09-16T00:30:26.080953+0000","last_active":"2024-09-16T00:32:24.340070+0000","last_peered":"2024-09-16T00:32:24.340070+0000","last_clean":"2024-09-16T00:32:24.340070+0000","last_became_active":"2024-09-16T00:30:26.080185+0000","last_became_peered":"2024-09-16T00:30:26.080185+0000","last_unstale":"2024-09-16T00:32:24.340070+0000","last_undegraded":"2024-09-16T00:32:24.340070+0000","last_fullsized":"2024-09-16T00:32:24.340070+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:56.769182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561297+0000","last_change":"2024-09-16T00:30:26.079320+0000","last_active":"2024-09-16T00:32:24.561297+0000","last_peered":"2024-09-16T00:32:24.561297+0000","last_clean":"2024-09-16T00:32:24.561297+0000","last_became_active":"2024-09-16T00:30:26.078980+0000","last_became_peered":"2024-09-16T00:30:26.078980+0000","last_unstale":"2024-09-16T00:32:24.561297+0000","last_undegraded":"2024-09-16T00:32:24.561297+0000","last_fullsized":"2024-09-16T00:32:24.561297+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:03:15.386657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594661+0000","last_change":"2024-09-16T00:30:26.084948+0000","last_active":"2024-09-16T00:32:24.594661+0000","last_peered":"2024-09-16T00:32:24.594661+0000","last_clean":"2024-09-16T00:32:24.594661+0000","last_became_active":"2024-09-16T00:30:26.084602+0000","last_became_peered":"2024-09-16T00:30:26.084602+0000","last_unstale":"2024-09-16T00:32:24.594661+0000","last_undegraded":"2024-09-16T00:32:24.594661+0000","last_fullsized":"2024-09-16T00:32:24.594661+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:01.709332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594560+0000","last_change":"2024-09-16T00:30:26.080491+0000","last_active":"2024-09-16T00:32:24.594560+0000","last_peered":"2024-09-16T00:32:24.594560+0000","last_clean":"2024-09-16T00:32:24.594560+0000","last_became_active":"2024-09-16T00:30:26.080189+0000","last_became_peered":"2024-09-16T00:30:26.080189+0000","last_unstale":"2024-09-16T00:32:24.594560+0000","last_undegraded":"2024-09-16T00:32:24.594560+0000","last_fullsized":"2024-09-16T00:32:24.594560+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:38.358919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452035+0000","last_change":"2024-09-16T00:30:26.077410+0000","last_active":"2024-09-16T00:32:24.452035+0000","last_peered":"2024-09-16T00:32:24.452035+0000","last_clean":"2024-09-16T00:32:24.452035+0000","last_became_active":"2024-09-16T00:30:26.076498+0000","last_became_peered":"2024-09-16T00:30:26.076498+0000","last_unstale":"2024-09-16T00:32:24.452035+0000","last_undegraded":"2024-09-16T00:32:24.452035+0000","last_fullsized":"2024-09-16T00:32:24.452035+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:54:17.712774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454822+0000","last_change":"2024-09-16T00:30:26.074500+0000","last_active":"2024-09-16T00:32:24.454822+0000","last_peered":"2024-09-16T00:32:24.454822+0000","last_clean":"2024-09-16T00:32:24.454822+0000","last_became_active":"2024-09-16T00:30:26.074177+0000","last_became_peered":"2024-09-16T00:30:26.074177+0000","last_unstale":"2024-09-16T00:32:24.454822+0000","last_undegraded":"2024-09-16T00:32:24.454822+0000","last_fullsized":"2024-09-16T00:32:24.454822+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:43:13.792231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595503+0000","last_change":"2024-09-16T00:30:26.078358+0000","last_active":"2024-09-16T00:32:24.595503+0000","last_peered":"2024-09-16T00:32:24.595503+0000","last_clean":"2024-09-16T00:32:24.595503+0000","last_became_active":"2024-09-16T00:30:26.077993+0000","last_became_peered":"2024-09-16T00:30:26.077993+0000","last_unstale":"2024-09-16T00:32:24.595503+0000","last_undegraded":"2024-09-16T00:32:24.595503+0000","last_fullsized":"2024-09-16T00:32:24.595503+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:25:10.711394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.337671+0000","last_change":"2024-09-16T00:31:23.446836+0000","last_active":"2024-09-16T00:32:24.337671+0000","last_peered":"2024-09-16T00:32:24.337671+0000","last_clean":"2024-09-16T00:32:24.337671+0000","last_became_active":"2024-09-16T00:31:23.446471+0000","last_became_peered":"2024-09-16T00:31:23.446471+0000","last_unstale":"2024-09-16T00:32:24.337671+0000","last_undegraded":"2024-09-16T00:32:24.337671+0000","last_fullsized":"2024-09-16T00:32:24.337671+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:08:34.887618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561867+0000","last_change":"2024-09-16T00:30:26.086313+0000","last_active":"2024-09-16T00:32:24.561867+0000","last_peered":"2024-09-16T00:32:24.561867+0000","last_clean":"2024-09-16T00:32:24.561867+0000","last_became_active":"2024-09-16T00:30:26.085760+0000","last_became_peered":"2024-09-16T00:30:26.085760+0000","last_unstale":"2024-09-16T00:32:24.561867+0000","last_undegraded":"2024-09-16T00:32:24.561867+0000","last_fullsized":"2024-09-16T00:32:24.561867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:40:38.236789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594968+0000","last_change":"2024-09-16T00:30:26.085289+0000","last_active":"2024-09-16T00:32:24.594968+0000","last_peered":"2024-09-16T00:32:24.594968+0000","last_clean":"2024-09-16T00:32:24.594968+0000","last_became_active":"2024-09-16T00:30:26.084995+0000","last_became_peered":"2024-09-16T00:30:26.084995+0000","last_unstale":"2024-09-16T00:32:24.594968+0000","last_undegraded":"2024-09-16T00:32:24.594968+0000","last_fullsized":"2024-09-16T00:32:24.594968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:45.667892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594850+0000","last_change":"2024-09-16T00:30:26.088690+0000","last_active":"2024-09-16T00:32:24.594850+0000","last_peered":"2024-09-16T00:32:24.594850+0000","last_clean":"2024-09-16T00:32:24.594850+0000","last_became_active":"2024-09-16T00:30:26.087700+0000","last_became_peered":"2024-09-16T00:30:26.087700+0000","last_unstale":"2024-09-16T00:32:24.594850+0000","last_undegraded":"2024-09-16T00:32:24.594850+0000","last_fullsized":"2024-09-16T00:32:24.594850+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:39:51.445324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.337214+0000","last_change":"2024-09-16T00:30:26.080215+0000","last_active":"2024-09-16T00:32:24.337214+0000","last_peered":"2024-09-16T00:32:24.337214+0000","last_clean":"2024-09-16T00:32:24.337214+0000","last_became_active":"2024-09-16T00:30:26.079038+0000","last_became_peered":"2024-09-16T00:30:26.079038+0000","last_unstale":"2024-09-16T00:32:24.337214+0000","last_undegraded":"2024-09-16T00:32:24.337214+0000","last_fullsized":"2024-09-16T00:32:24.337214+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:01.573960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339169+0000","last_change":"2024-09-16T00:30:26.080966+0000","last_active":"2024-09-16T00:32:24.339169+0000","last_peered":"2024-09-16T00:32:24.339169+0000","last_clean":"2024-09-16T00:32:24.339169+0000","last_became_active":"2024-09-16T00:30:26.080641+0000","last_became_peered":"2024-09-16T00:30:26.080641+0000","last_unstale":"2024-09-16T00:32:24.339169+0000","last_undegraded":"2024-09-16T00:32:24.339169+0000","last_fullsized":"2024-09-16T00:32:24.339169+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:33:01.538169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454762+0000","last_change":"2024-09-16T00:30:26.072644+0000","last_active":"2024-09-16T00:32:24.454762+0000","last_peered":"2024-09-16T00:32:24.454762+0000","last_clean":"2024-09-16T00:32:24.454762+0000","last_became_active":"2024-09-16T00:30:26.072336+0000","last_became_peered":"2024-09-16T00:30:26.072336+0000","last_unstale":"2024-09-16T00:32:24.454762+0000","last_undegraded":"2024-09-16T00:32:24.454762+0000","last_fullsized":"2024-09-16T00:32:24.454762+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:44:45.760057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595215+0000","last_change":"2024-09-16T00:30:26.079155+0000","last_active":"2024-09-16T00:32:24.595215+0000","last_peered":"2024-09-16T00:32:24.595215+0000","last_clean":"2024-09-16T00:32:24.595215+0000","last_became_active":"2024-09-16T00:30:26.078820+0000","last_became_peered":"2024-09-16T00:30:26.078820+0000","last_unstale":"2024-09-16T00:32:24.595215+0000","last_undegraded":"2024-09-16T00:32:24.595215+0000","last_fullsized":"2024-09-16T00:32:24.595215+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:35.071143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595361+0000","last_change":"2024-09-16T00:30:26.081537+0000","last_active":"2024-09-16T00:32:24.595361+0000","last_peered":"2024-09-16T00:32:24.595361+0000","last_clean":"2024-09-16T00:32:24.595361+0000","last_became_active":"2024-09-16T00:30:26.080744+0000","last_became_peered":"2024-09-16T00:30:26.080744+0000","last_unstale":"2024-09-16T00:32:24.595361+0000","last_undegraded":"2024-09-16T00:32:24.595361+0000","last_fullsized":"2024-09-16T00:32:24.595361+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:46:38.179888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.340007+0000","last_change":"2024-09-16T00:30:26.081066+0000","last_active":"2024-09-16T00:32:24.340007+0000","last_peered":"2024-09-16T00:32:24.340007+0000","last_clean":"2024-09-16T00:32:24.340007+0000","last_became_active":"2024-09-16T00:30:26.080630+0000","last_became_peered":"2024-09-16T00:30:26.080630+0000","last_unstale":"2024-09-16T00:32:24.340007+0000","last_undegraded":"2024-09-16T00:32:24.340007+0000","last_fullsized":"2024-09-16T00:32:24.340007+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:44:42.644792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72936,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":637,"num_read_kb":399,"num_write":183,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":438272,"data_stored":212649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514538,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1060864,"data_stored":523210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743471,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32892,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712260,"statfs":{"total":95995035648,"available":95961354240,"internally_reserved":0,"allocated":1748992,"data_stored":1178821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906996,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":527486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135929,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1064960,"data_stored":523288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397566,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1662976,"data_stored":1119755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561092,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1048576,"data_stored":518566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790024,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28656,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716496,"statfs":{"total":95995035648,"available":95965691904,"internally_reserved":0,"allocated":1671168,"data_stored":1115343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953550,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1060864,"data_stored":523216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182481,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1060864,"data_stored":518760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1138688,"data_stored":582959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":586875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1073152,"data_stored":519954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":64483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":69977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:32:45.832 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:32:45.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.968+0000 7f794a166640 1 -- 172.21.15.38:0/221069880 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7944101c20 msgr2=0x7f79441020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:45.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.968+0000 7f794a166640 1 --2- 172.21.15.38:0/221069880 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7944101c20 0x7f79441020a0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7938009900 tx=0x7f793802a410 comp rx=0 tx=0).stop 2024-09-16T00:32:45.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.968+0000 7f794a166640 1 -- 172.21.15.38:0/221069880 shutdown_connections 2024-09-16T00:32:45.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.968+0000 7f794a166640 1 --2- 172.21.15.38:0/221069880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79441025e0 0x7f7944108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:45.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.968+0000 7f794a166640 1 --2- 172.21.15.38:0/221069880 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7944101c20 0x7f79441020a0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:45.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.968+0000 7f794a166640 1 --2- 172.21.15.38:0/221069880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7944100a30 0x7f7944100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:45.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.968+0000 7f794a166640 1 -- 172.21.15.38:0/221069880 >> 172.21.15.38:0/221069880 conn(0x7f79440fc1b0 msgr2=0x7f79440fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:45.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.968+0000 7f794a166640 1 -- 172.21.15.38:0/221069880 shutdown_connections 2024-09-16T00:32:45.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.968+0000 7f794a166640 1 -- 172.21.15.38:0/221069880 wait complete. 2024-09-16T00:32:45.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.969+0000 7f794a166640 1 Processor -- start 2024-09-16T00:32:45.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.969+0000 7f794a166640 1 -- start start 2024-09-16T00:32:45.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.969+0000 7f794a166640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7944100a30 0x7f794406af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:45.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.969+0000 7f794a166640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7944101c20 0x7f794419cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:45.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.969+0000 7f794a166640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79441025e0 0x7f794419f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:45.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.969+0000 7f794a166640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f794410c6b0 con 0x7f7944100a30 2024-09-16T00:32:45.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.969+0000 7f794a166640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f794410c530 con 0x7f7944101c20 2024-09-16T00:32:45.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.969+0000 7f794a166640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f794410c830 con 0x7f79441025e0 2024-09-16T00:32:45.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.969+0000 7f79437fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7944100a30 0x7f794406af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:45.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.969+0000 7f79437fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7944100a30 0x7f794406af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:47128/0 (socket says 172.21.15.38:47128) 2024-09-16T00:32:45.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.969+0000 7f79437fe640 1 -- 172.21.15.38:0/272710511 learned_addr learned my addr 172.21.15.38:0/272710511 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:45.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.970+0000 7f7942ffd640 1 --2- 172.21.15.38:0/272710511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7944101c20 0x7f794419cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:45.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.970+0000 7f7943fff640 1 --2- 172.21.15.38:0/272710511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79441025e0 0x7f794419f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:45.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.970+0000 7f79437fe640 1 -- 172.21.15.38:0/272710511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79441025e0 msgr2=0x7f794419f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:45.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.970+0000 7f79437fe640 1 --2- 172.21.15.38:0/272710511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79441025e0 0x7f794419f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:45.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.970+0000 7f79437fe640 1 -- 172.21.15.38:0/272710511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7944101c20 msgr2=0x7f794419cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:45.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.970+0000 7f79437fe640 1 --2- 172.21.15.38:0/272710511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7944101c20 0x7f794419cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:45.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.970+0000 7f79437fe640 1 -- 172.21.15.38:0/272710511 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f794419fab0 con 0x7f7944100a30 2024-09-16T00:32:45.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.970+0000 7f79437fe640 1 --2- 172.21.15.38:0/272710511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7944100a30 0x7f794406af70 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f793400d950 tx=0x7f793400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:45.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.970+0000 7f7940ff9640 1 -- 172.21.15.38:0/272710511 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7934017070 con 0x7f7944100a30 2024-09-16T00:32:45.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.970+0000 7f7940ff9640 1 -- 172.21.15.38:0/272710511 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7934009c30 con 0x7f7944100a30 2024-09-16T00:32:45.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.970+0000 7f7940ff9640 1 -- 172.21.15.38:0/272710511 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793400f600 con 0x7f7944100a30 2024-09-16T00:32:45.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.970+0000 7f794a166640 1 -- 172.21.15.38:0/272710511 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f794419fda0 con 0x7f7944100a30 2024-09-16T00:32:45.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.970+0000 7f794a166640 1 -- 172.21.15.38:0/272710511 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79441a01b0 con 0x7f7944100a30 2024-09-16T00:32:45.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.971+0000 7f794a166640 1 -- 172.21.15.38:0/272710511 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7944107e70 con 0x7f7944100a30 2024-09-16T00:32:45.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.972+0000 7f7940ff9640 1 -- 172.21.15.38:0/272710511 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f793401a070 con 0x7f7944100a30 2024-09-16T00:32:45.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.973+0000 7f7940ff9640 1 --2- 172.21.15.38:0/272710511 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f792c0a32e0 0x7f792c0a57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:45.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.973+0000 7f7942ffd640 1 --2- 172.21.15.38:0/272710511 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f792c0a32e0 0x7f792c0a57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:45.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.974+0000 7f7940ff9640 1 -- 172.21.15.38:0/272710511 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f79340cf8c0 con 0x7f7944100a30 2024-09-16T00:32:45.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.974+0000 7f7942ffd640 1 --2- 172.21.15.38:0/272710511 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f792c0a32e0 0x7f792c0a57a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f79380314e0 tx=0x7f7938000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:45.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:45.978+0000 7f7940ff9640 1 -- 172.21.15.38:0/272710511 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7934009de0 con 0x7f7944100a30 2024-09-16T00:32:46.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.152+0000 7f794a166640 1 -- 172.21.15.38:0/272710511 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7944062630 con 0x7f792c0a32e0 2024-09-16T00:32:46.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.160+0000 7f7940ff9640 1 -- 172.21.15.38:0/272710511 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340493 (secure 0 0 0) 0x7f7944062630 con 0x7f792c0a32e0 2024-09-16T00:32:46.165 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:32:46.167 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-16T00:32:46.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.163+0000 7f794a166640 1 -- 172.21.15.38:0/272710511 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f792c0a32e0 msgr2=0x7f792c0a57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:46.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.163+0000 7f794a166640 1 --2- 172.21.15.38:0/272710511 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f792c0a32e0 0x7f792c0a57a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f79380314e0 tx=0x7f7938000bc0 comp rx=0 tx=0).stop 2024-09-16T00:32:46.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.163+0000 7f794a166640 1 -- 172.21.15.38:0/272710511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7944100a30 msgr2=0x7f794406af70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:46.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.163+0000 7f794a166640 1 --2- 172.21.15.38:0/272710511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7944100a30 0x7f794406af70 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f793400d950 tx=0x7f793400de10 comp rx=0 tx=0).stop 2024-09-16T00:32:46.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.163+0000 7f794a166640 1 -- 172.21.15.38:0/272710511 shutdown_connections 2024-09-16T00:32:46.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.163+0000 7f794a166640 1 --2- 172.21.15.38:0/272710511 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f792c0a32e0 0x7f792c0a57a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:46.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.163+0000 7f794a166640 1 --2- 172.21.15.38:0/272710511 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f79441025e0 0x7f794419f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:46.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.163+0000 7f794a166640 1 --2- 172.21.15.38:0/272710511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7944101c20 0x7f794419cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:46.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.163+0000 7f794a166640 1 --2- 172.21.15.38:0/272710511 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7944100a30 0x7f794406af70 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:46.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.163+0000 7f794a166640 1 -- 172.21.15.38:0/272710511 >> 172.21.15.38:0/272710511 conn(0x7f79440fc1b0 msgr2=0x7f7944105040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:46.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.164+0000 7f794a166640 1 -- 172.21.15.38:0/272710511 shutdown_connections 2024-09-16T00:32:46.169 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.164+0000 7f794a166640 1 -- 172.21.15.38:0/272710511 wait complete. 2024-09-16T00:32:46.176 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2024-09-16T00:32:45.543344+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663538,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":763,"num_read_kb":508,"num_write":366,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1180792,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":795,"ondisk_log_size":795,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380376,"kb_used_data":14440,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561448,"statfs":{"total":1151940427776,"available":1151550922752,"internally_reserved":0,"allocated":14786560,"data_stored":8238233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6467,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":83,"num_read_kb":53,"num_write":10,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003434"},"pg_stats":[{"pgid":"2.1f","version":"88'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.337120+0000","last_change":"2024-09-16T00:30:23.933355+0000","last_active":"2024-09-16T00:32:24.337120+0000","last_peered":"2024-09-16T00:32:24.337120+0000","last_clean":"2024-09-16T00:32:24.337120+0000","last_became_active":"2024-09-16T00:30:23.933057+0000","last_became_peered":"2024-09-16T00:30:23.933057+0000","last_unstale":"2024-09-16T00:32:24.337120+0000","last_undegraded":"2024-09-16T00:32:24.337120+0000","last_fullsized":"2024-09-16T00:32:24.337120+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:23:01.713440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.596165+0000","last_change":"2024-09-16T00:30:26.084537+0000","last_active":"2024-09-16T00:32:24.596165+0000","last_peered":"2024-09-16T00:32:24.596165+0000","last_clean":"2024-09-16T00:32:24.596165+0000","last_became_active":"2024-09-16T00:30:26.084226+0000","last_became_peered":"2024-09-16T00:30:26.084226+0000","last_unstale":"2024-09-16T00:32:24.596165+0000","last_undegraded":"2024-09-16T00:32:24.596165+0000","last_fullsized":"2024-09-16T00:32:24.596165+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:43:41.628877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594728+0000","last_change":"2024-09-16T00:30:33.154873+0000","last_active":"2024-09-16T00:32:24.594728+0000","last_peered":"2024-09-16T00:32:24.594728+0000","last_clean":"2024-09-16T00:32:24.594728+0000","last_became_active":"2024-09-16T00:30:33.154433+0000","last_became_peered":"2024-09-16T00:30:33.154433+0000","last_unstale":"2024-09-16T00:32:24.594728+0000","last_undegraded":"2024-09-16T00:32:24.594728+0000","last_fullsized":"2024-09-16T00:32:24.594728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:50.805806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.596255+0000","last_change":"2024-09-16T00:30:24.422274+0000","last_active":"2024-09-16T00:32:24.596255+0000","last_peered":"2024-09-16T00:32:24.596255+0000","last_clean":"2024-09-16T00:32:24.596255+0000","last_became_active":"2024-09-16T00:30:24.421902+0000","last_became_peered":"2024-09-16T00:30:24.421902+0000","last_unstale":"2024-09-16T00:32:24.596255+0000","last_undegraded":"2024-09-16T00:32:24.596255+0000","last_fullsized":"2024-09-16T00:32:24.596255+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:01.910187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595568+0000","last_change":"2024-09-16T00:30:26.088165+0000","last_active":"2024-09-16T00:32:24.595568+0000","last_peered":"2024-09-16T00:32:24.595568+0000","last_clean":"2024-09-16T00:32:24.595568+0000","last_became_active":"2024-09-16T00:30:26.087593+0000","last_became_peered":"2024-09-16T00:30:26.087593+0000","last_unstale":"2024-09-16T00:32:24.595568+0000","last_undegraded":"2024-09-16T00:32:24.595568+0000","last_fullsized":"2024-09-16T00:32:24.595568+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:54.149930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454083+0000","last_change":"2024-09-16T00:30:33.154586+0000","last_active":"2024-09-16T00:32:24.454083+0000","last_peered":"2024-09-16T00:32:24.454083+0000","last_clean":"2024-09-16T00:32:24.454083+0000","last_became_active":"2024-09-16T00:30:33.154209+0000","last_became_peered":"2024-09-16T00:30:33.154209+0000","last_unstale":"2024-09-16T00:32:24.454083+0000","last_undegraded":"2024-09-16T00:32:24.454083+0000","last_fullsized":"2024-09-16T00:32:24.454083+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:45.776895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339220+0000","last_change":"2024-09-16T00:30:23.932442+0000","last_active":"2024-09-16T00:32:24.339220+0000","last_peered":"2024-09-16T00:32:24.339220+0000","last_clean":"2024-09-16T00:32:24.339220+0000","last_became_active":"2024-09-16T00:30:23.932009+0000","last_became_peered":"2024-09-16T00:30:23.932009+0000","last_unstale":"2024-09-16T00:32:24.339220+0000","last_undegraded":"2024-09-16T00:32:24.339220+0000","last_fullsized":"2024-09-16T00:32:24.339220+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:09:27.078187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561572+0000","last_change":"2024-09-16T00:30:26.087715+0000","last_active":"2024-09-16T00:32:24.561572+0000","last_peered":"2024-09-16T00:32:24.561572+0000","last_clean":"2024-09-16T00:32:24.561572+0000","last_became_active":"2024-09-16T00:30:26.087415+0000","last_became_peered":"2024-09-16T00:30:26.087415+0000","last_unstale":"2024-09-16T00:32:24.561572+0000","last_undegraded":"2024-09-16T00:32:24.561572+0000","last_fullsized":"2024-09-16T00:32:24.561572+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:52.373992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454447+0000","last_change":"2024-09-16T00:30:33.152328+0000","last_active":"2024-09-16T00:32:24.454447+0000","last_peered":"2024-09-16T00:32:24.454447+0000","last_clean":"2024-09-16T00:32:24.454447+0000","last_became_active":"2024-09-16T00:30:33.151954+0000","last_became_peered":"2024-09-16T00:30:33.151954+0000","last_unstale":"2024-09-16T00:32:24.454447+0000","last_undegraded":"2024-09-16T00:32:24.454447+0000","last_fullsized":"2024-09-16T00:32:24.454447+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:36.546729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339346+0000","last_change":"2024-09-16T00:30:23.928988+0000","last_active":"2024-09-16T00:32:24.339346+0000","last_peered":"2024-09-16T00:32:24.339346+0000","last_clean":"2024-09-16T00:32:24.339346+0000","last_became_active":"2024-09-16T00:30:23.928638+0000","last_became_peered":"2024-09-16T00:30:23.928638+0000","last_unstale":"2024-09-16T00:32:24.339346+0000","last_undegraded":"2024-09-16T00:32:24.339346+0000","last_fullsized":"2024-09-16T00:32:24.339346+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:45:58.203455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.336619+0000","last_change":"2024-09-16T00:31:23.446550+0000","last_active":"2024-09-16T00:32:24.336619+0000","last_peered":"2024-09-16T00:32:24.336619+0000","last_clean":"2024-09-16T00:32:24.336619+0000","last_became_active":"2024-09-16T00:31:23.446179+0000","last_became_peered":"2024-09-16T00:31:23.446179+0000","last_unstale":"2024-09-16T00:32:24.336619+0000","last_undegraded":"2024-09-16T00:32:24.336619+0000","last_fullsized":"2024-09-16T00:32:24.336619+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:04:21.280174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454185+0000","last_change":"2024-09-16T00:30:33.147564+0000","last_active":"2024-09-16T00:32:24.454185+0000","last_peered":"2024-09-16T00:32:24.454185+0000","last_clean":"2024-09-16T00:32:24.454185+0000","last_became_active":"2024-09-16T00:30:33.147119+0000","last_became_peered":"2024-09-16T00:30:33.147119+0000","last_unstale":"2024-09-16T00:32:24.454185+0000","last_undegraded":"2024-09-16T00:32:24.454185+0000","last_fullsized":"2024-09-16T00:32:24.454185+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:00.509122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595076+0000","last_change":"2024-09-16T00:30:26.080954+0000","last_active":"2024-09-16T00:32:24.595076+0000","last_peered":"2024-09-16T00:32:24.595076+0000","last_clean":"2024-09-16T00:32:24.595076+0000","last_became_active":"2024-09-16T00:30:26.080624+0000","last_became_peered":"2024-09-16T00:30:26.080624+0000","last_unstale":"2024-09-16T00:32:24.595076+0000","last_undegraded":"2024-09-16T00:32:24.595076+0000","last_fullsized":"2024-09-16T00:32:24.595076+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:05:10.440980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339169+0000","last_change":"2024-09-16T00:30:23.933715+0000","last_active":"2024-09-16T00:32:24.339169+0000","last_peered":"2024-09-16T00:32:24.339169+0000","last_clean":"2024-09-16T00:32:24.339169+0000","last_became_active":"2024-09-16T00:30:23.933275+0000","last_became_peered":"2024-09-16T00:30:23.933275+0000","last_unstale":"2024-09-16T00:32:24.339169+0000","last_undegraded":"2024-09-16T00:32:24.339169+0000","last_fullsized":"2024-09-16T00:32:24.339169+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:40:44.846053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454555+0000","last_change":"2024-09-16T00:30:33.604602+0000","last_active":"2024-09-16T00:32:24.454555+0000","last_peered":"2024-09-16T00:32:24.454555+0000","last_clean":"2024-09-16T00:32:24.454555+0000","last_became_active":"2024-09-16T00:30:33.604107+0000","last_became_peered":"2024-09-16T00:30:33.604107+0000","last_unstale":"2024-09-16T00:32:24.454555+0000","last_undegraded":"2024-09-16T00:32:24.454555+0000","last_fullsized":"2024-09-16T00:32:24.454555+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:41.254445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595459+0000","last_change":"2024-09-16T00:30:26.080612+0000","last_active":"2024-09-16T00:32:24.595459+0000","last_peered":"2024-09-16T00:32:24.595459+0000","last_clean":"2024-09-16T00:32:24.595459+0000","last_became_active":"2024-09-16T00:30:26.080317+0000","last_became_peered":"2024-09-16T00:30:26.080317+0000","last_unstale":"2024-09-16T00:32:24.595459+0000","last_undegraded":"2024-09-16T00:32:24.595459+0000","last_fullsized":"2024-09-16T00:32:24.595459+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:23.814241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594893+0000","last_change":"2024-09-16T00:30:23.936778+0000","last_active":"2024-09-16T00:32:24.594893+0000","last_peered":"2024-09-16T00:32:24.594893+0000","last_clean":"2024-09-16T00:32:24.594893+0000","last_became_active":"2024-09-16T00:30:23.936265+0000","last_became_peered":"2024-09-16T00:30:23.936265+0000","last_unstale":"2024-09-16T00:32:24.594893+0000","last_undegraded":"2024-09-16T00:32:24.594893+0000","last_fullsized":"2024-09-16T00:32:24.594893+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:35:28.288476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454520+0000","last_change":"2024-09-16T00:30:33.148340+0000","last_active":"2024-09-16T00:32:24.454520+0000","last_peered":"2024-09-16T00:32:24.454520+0000","last_clean":"2024-09-16T00:32:24.454520+0000","last_became_active":"2024-09-16T00:30:33.147289+0000","last_became_peered":"2024-09-16T00:30:33.147289+0000","last_unstale":"2024-09-16T00:32:24.454520+0000","last_undegraded":"2024-09-16T00:32:24.454520+0000","last_fullsized":"2024-09-16T00:32:24.454520+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:21.259254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.338936+0000","last_change":"2024-09-16T00:30:26.077325+0000","last_active":"2024-09-16T00:32:24.338936+0000","last_peered":"2024-09-16T00:32:24.338936+0000","last_clean":"2024-09-16T00:32:24.338936+0000","last_became_active":"2024-09-16T00:30:26.077002+0000","last_became_peered":"2024-09-16T00:30:26.077002+0000","last_unstale":"2024-09-16T00:32:24.338936+0000","last_undegraded":"2024-09-16T00:32:24.338936+0000","last_fullsized":"2024-09-16T00:32:24.338936+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:12:38.569247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595646+0000","last_change":"2024-09-16T00:30:23.933025+0000","last_active":"2024-09-16T00:32:24.595646+0000","last_peered":"2024-09-16T00:32:24.595646+0000","last_clean":"2024-09-16T00:32:24.595646+0000","last_became_active":"2024-09-16T00:30:23.932669+0000","last_became_peered":"2024-09-16T00:30:23.932669+0000","last_unstale":"2024-09-16T00:32:24.595646+0000","last_undegraded":"2024-09-16T00:32:24.595646+0000","last_fullsized":"2024-09-16T00:32:24.595646+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:20.593665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.597170+0000","last_change":"2024-09-16T00:32:24.597170+0000","last_active":"2024-09-16T00:32:24.597170+0000","last_peered":"2024-09-16T00:32:24.597170+0000","last_clean":"2024-09-16T00:32:24.597170+0000","last_became_active":"2024-09-16T00:32:24.596810+0000","last_became_peered":"2024-09-16T00:32:24.596810+0000","last_unstale":"2024-09-16T00:32:24.597170+0000","last_undegraded":"2024-09-16T00:32:24.597170+0000","last_fullsized":"2024-09-16T00:32:24.597170+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:26:18.737332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.338711+0000","last_change":"2024-09-16T00:30:26.076104+0000","last_active":"2024-09-16T00:32:24.338711+0000","last_peered":"2024-09-16T00:32:24.338711+0000","last_clean":"2024-09-16T00:32:24.338711+0000","last_became_active":"2024-09-16T00:30:26.075764+0000","last_became_peered":"2024-09-16T00:30:26.075764+0000","last_unstale":"2024-09-16T00:32:24.338711+0000","last_undegraded":"2024-09-16T00:32:24.338711+0000","last_fullsized":"2024-09-16T00:32:24.338711+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:31.228533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561411+0000","last_change":"2024-09-16T00:30:24.422411+0000","last_active":"2024-09-16T00:32:24.561411+0000","last_peered":"2024-09-16T00:32:24.561411+0000","last_clean":"2024-09-16T00:32:24.561411+0000","last_became_active":"2024-09-16T00:30:24.422102+0000","last_became_peered":"2024-09-16T00:30:24.422102+0000","last_unstale":"2024-09-16T00:32:24.561411+0000","last_undegraded":"2024-09-16T00:32:24.561411+0000","last_fullsized":"2024-09-16T00:32:24.561411+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:06:00.572070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454559+0000","last_change":"2024-09-16T00:30:33.605397+0000","last_active":"2024-09-16T00:32:24.454559+0000","last_peered":"2024-09-16T00:32:24.454559+0000","last_clean":"2024-09-16T00:32:24.454559+0000","last_became_active":"2024-09-16T00:30:33.604978+0000","last_became_peered":"2024-09-16T00:30:33.604978+0000","last_unstale":"2024-09-16T00:32:24.454559+0000","last_undegraded":"2024-09-16T00:32:24.454559+0000","last_fullsized":"2024-09-16T00:32:24.454559+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:35:16.766056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595380+0000","last_change":"2024-09-16T00:30:26.082684+0000","last_active":"2024-09-16T00:32:24.595380+0000","last_peered":"2024-09-16T00:32:24.595380+0000","last_clean":"2024-09-16T00:32:24.595380+0000","last_became_active":"2024-09-16T00:30:26.082335+0000","last_became_peered":"2024-09-16T00:30:26.082335+0000","last_unstale":"2024-09-16T00:32:24.595380+0000","last_undegraded":"2024-09-16T00:32:24.595380+0000","last_fullsized":"2024-09-16T00:32:24.595380+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:51:21.389627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454909+0000","last_change":"2024-09-16T00:30:24.587840+0000","last_active":"2024-09-16T00:32:24.454909+0000","last_peered":"2024-09-16T00:32:24.454909+0000","last_clean":"2024-09-16T00:32:24.454909+0000","last_became_active":"2024-09-16T00:30:24.587436+0000","last_became_peered":"2024-09-16T00:30:24.587436+0000","last_unstale":"2024-09-16T00:32:24.454909+0000","last_undegraded":"2024-09-16T00:32:24.454909+0000","last_fullsized":"2024-09-16T00:32:24.454909+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:06.758585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594628+0000","last_change":"2024-09-16T00:30:33.145051+0000","last_active":"2024-09-16T00:32:24.594628+0000","last_peered":"2024-09-16T00:32:24.594628+0000","last_clean":"2024-09-16T00:32:24.594628+0000","last_became_active":"2024-09-16T00:30:33.144585+0000","last_became_peered":"2024-09-16T00:30:33.144585+0000","last_unstale":"2024-09-16T00:32:24.594628+0000","last_undegraded":"2024-09-16T00:32:24.594628+0000","last_fullsized":"2024-09-16T00:32:24.594628+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:48:23.588495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.336844+0000","last_change":"2024-09-16T00:31:23.444802+0000","last_active":"2024-09-16T00:32:24.336844+0000","last_peered":"2024-09-16T00:32:24.336844+0000","last_clean":"2024-09-16T00:32:24.336844+0000","last_became_active":"2024-09-16T00:31:23.444447+0000","last_became_peered":"2024-09-16T00:31:23.444447+0000","last_unstale":"2024-09-16T00:32:24.336844+0000","last_undegraded":"2024-09-16T00:32:24.336844+0000","last_fullsized":"2024-09-16T00:32:24.336844+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:46:37.285250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.6","version":"92'14","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:41.088268+0000","last_change":"2024-09-16T00:30:23.933284+0000","last_active":"2024-09-16T00:32:41.088268+0000","last_peered":"2024-09-16T00:32:41.088268+0000","last_clean":"2024-09-16T00:32:41.088268+0000","last_became_active":"2024-09-16T00:30:23.932920+0000","last_became_peered":"2024-09-16T00:30:23.932920+0000","last_unstale":"2024-09-16T00:32:41.088268+0000","last_undegraded":"2024-09-16T00:32:41.088268+0000","last_fullsized":"2024-09-16T00:32:41.088268+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:38.199391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594942+0000","last_change":"2024-09-16T00:30:33.604308+0000","last_active":"2024-09-16T00:32:24.594942+0000","last_peered":"2024-09-16T00:32:24.594942+0000","last_clean":"2024-09-16T00:32:24.594942+0000","last_became_active":"2024-09-16T00:30:33.603889+0000","last_became_peered":"2024-09-16T00:30:33.603889+0000","last_unstale":"2024-09-16T00:32:24.594942+0000","last_undegraded":"2024-09-16T00:32:24.594942+0000","last_fullsized":"2024-09-16T00:32:24.594942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:32:55.173063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452512+0000","last_change":"2024-09-16T00:30:26.078515+0000","last_active":"2024-09-16T00:32:24.452512+0000","last_peered":"2024-09-16T00:32:24.452512+0000","last_clean":"2024-09-16T00:32:24.452512+0000","last_became_active":"2024-09-16T00:30:26.078173+0000","last_became_peered":"2024-09-16T00:30:26.078173+0000","last_unstale":"2024-09-16T00:32:24.452512+0000","last_undegraded":"2024-09-16T00:32:24.452512+0000","last_fullsized":"2024-09-16T00:32:24.452512+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:40:31.471154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339304+0000","last_change":"2024-09-16T00:30:24.423209+0000","last_active":"2024-09-16T00:32:24.339304+0000","last_peered":"2024-09-16T00:32:24.339304+0000","last_clean":"2024-09-16T00:32:24.339304+0000","last_became_active":"2024-09-16T00:30:24.422157+0000","last_became_peered":"2024-09-16T00:30:24.422157+0000","last_unstale":"2024-09-16T00:32:24.339304+0000","last_undegraded":"2024-09-16T00:32:24.339304+0000","last_fullsized":"2024-09-16T00:32:24.339304+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:28.262392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339327+0000","last_change":"2024-09-16T00:30:33.146204+0000","last_active":"2024-09-16T00:32:24.339327+0000","last_peered":"2024-09-16T00:32:24.339327+0000","last_clean":"2024-09-16T00:32:24.339327+0000","last_became_active":"2024-09-16T00:30:33.145553+0000","last_became_peered":"2024-09-16T00:30:33.145553+0000","last_unstale":"2024-09-16T00:32:24.339327+0000","last_undegraded":"2024-09-16T00:32:24.339327+0000","last_fullsized":"2024-09-16T00:32:24.339327+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:22.096050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.337437+0000","last_change":"2024-09-16T00:30:26.070153+0000","last_active":"2024-09-16T00:32:24.337437+0000","last_peered":"2024-09-16T00:32:24.337437+0000","last_clean":"2024-09-16T00:32:24.337437+0000","last_became_active":"2024-09-16T00:30:26.069792+0000","last_became_peered":"2024-09-16T00:30:26.069792+0000","last_unstale":"2024-09-16T00:32:24.337437+0000","last_undegraded":"2024-09-16T00:32:24.337437+0000","last_fullsized":"2024-09-16T00:32:24.337437+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:28.164242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.336815+0000","last_change":"2024-09-16T00:30:24.423261+0000","last_active":"2024-09-16T00:32:24.336815+0000","last_peered":"2024-09-16T00:32:24.336815+0000","last_clean":"2024-09-16T00:32:24.336815+0000","last_became_active":"2024-09-16T00:30:24.422650+0000","last_became_peered":"2024-09-16T00:30:24.422650+0000","last_unstale":"2024-09-16T00:32:24.336815+0000","last_undegraded":"2024-09-16T00:32:24.336815+0000","last_fullsized":"2024-09-16T00:32:24.336815+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:00.211744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339762+0000","last_change":"2024-09-16T00:30:33.604492+0000","last_active":"2024-09-16T00:32:24.339762+0000","last_peered":"2024-09-16T00:32:24.339762+0000","last_clean":"2024-09-16T00:32:24.339762+0000","last_became_active":"2024-09-16T00:30:33.604015+0000","last_became_peered":"2024-09-16T00:30:33.604015+0000","last_unstale":"2024-09-16T00:32:24.339762+0000","last_undegraded":"2024-09-16T00:32:24.339762+0000","last_fullsized":"2024-09-16T00:32:24.339762+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:40:22.516583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.337389+0000","last_change":"2024-09-16T00:30:26.075517+0000","last_active":"2024-09-16T00:32:24.337389+0000","last_peered":"2024-09-16T00:32:24.337389+0000","last_clean":"2024-09-16T00:32:24.337389+0000","last_became_active":"2024-09-16T00:30:26.075113+0000","last_became_peered":"2024-09-16T00:30:26.075113+0000","last_unstale":"2024-09-16T00:32:24.337389+0000","last_undegraded":"2024-09-16T00:32:24.337389+0000","last_fullsized":"2024-09-16T00:32:24.337389+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:11.427010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595204+0000","last_change":"2024-09-16T00:30:23.933639+0000","last_active":"2024-09-16T00:32:24.595204+0000","last_peered":"2024-09-16T00:32:24.595204+0000","last_clean":"2024-09-16T00:32:24.595204+0000","last_became_active":"2024-09-16T00:30:23.933322+0000","last_became_peered":"2024-09-16T00:30:23.933322+0000","last_unstale":"2024-09-16T00:32:24.595204+0000","last_undegraded":"2024-09-16T00:32:24.595204+0000","last_fullsized":"2024-09-16T00:32:24.595204+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:59:59.082702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452139+0000","last_change":"2024-09-16T00:30:33.148508+0000","last_active":"2024-09-16T00:32:24.452139+0000","last_peered":"2024-09-16T00:32:24.452139+0000","last_clean":"2024-09-16T00:32:24.452139+0000","last_became_active":"2024-09-16T00:30:33.148091+0000","last_became_peered":"2024-09-16T00:30:33.148091+0000","last_unstale":"2024-09-16T00:32:24.452139+0000","last_undegraded":"2024-09-16T00:32:24.452139+0000","last_fullsized":"2024-09-16T00:32:24.452139+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:50.052526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454239+0000","last_change":"2024-09-16T00:30:26.080374+0000","last_active":"2024-09-16T00:32:24.454239+0000","last_peered":"2024-09-16T00:32:24.454239+0000","last_clean":"2024-09-16T00:32:24.454239+0000","last_became_active":"2024-09-16T00:30:26.080044+0000","last_became_peered":"2024-09-16T00:30:26.080044+0000","last_unstale":"2024-09-16T00:32:24.454239+0000","last_undegraded":"2024-09-16T00:32:24.454239+0000","last_fullsized":"2024-09-16T00:32:24.454239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:38:36.034903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":145,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:43.480306+0000","last_change":"2024-09-16T00:30:24.422137+0000","last_active":"2024-09-16T00:32:43.480306+0000","last_peered":"2024-09-16T00:32:43.480306+0000","last_clean":"2024-09-16T00:32:43.480306+0000","last_became_active":"2024-09-16T00:30:24.421778+0000","last_became_peered":"2024-09-16T00:30:24.421778+0000","last_unstale":"2024-09-16T00:32:43.480306+0000","last_undegraded":"2024-09-16T00:32:43.480306+0000","last_fullsized":"2024-09-16T00:32:43.480306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:54:21.964099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452309+0000","last_change":"2024-09-16T00:30:33.604263+0000","last_active":"2024-09-16T00:32:24.452309+0000","last_peered":"2024-09-16T00:32:24.452309+0000","last_clean":"2024-09-16T00:32:24.452309+0000","last_became_active":"2024-09-16T00:30:33.603804+0000","last_became_peered":"2024-09-16T00:30:33.603804+0000","last_unstale":"2024-09-16T00:32:24.452309+0000","last_undegraded":"2024-09-16T00:32:24.452309+0000","last_fullsized":"2024-09-16T00:32:24.452309+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:08:37.590827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.451792+0000","last_change":"2024-09-16T00:30:26.082916+0000","last_active":"2024-09-16T00:32:24.451792+0000","last_peered":"2024-09-16T00:32:24.451792+0000","last_clean":"2024-09-16T00:32:24.451792+0000","last_became_active":"2024-09-16T00:30:26.082606+0000","last_became_peered":"2024-09-16T00:30:26.082606+0000","last_unstale":"2024-09-16T00:32:24.451792+0000","last_undegraded":"2024-09-16T00:32:24.451792+0000","last_fullsized":"2024-09-16T00:32:24.451792+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:56.777640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594991+0000","last_change":"2024-09-16T00:30:23.937602+0000","last_active":"2024-09-16T00:32:24.594991+0000","last_peered":"2024-09-16T00:32:24.594991+0000","last_clean":"2024-09-16T00:32:24.594991+0000","last_became_active":"2024-09-16T00:30:23.937261+0000","last_became_peered":"2024-09-16T00:30:23.937261+0000","last_unstale":"2024-09-16T00:32:24.594991+0000","last_undegraded":"2024-09-16T00:32:24.594991+0000","last_fullsized":"2024-09-16T00:32:24.594991+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:24.698758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454567+0000","last_change":"2024-09-16T00:30:33.605536+0000","last_active":"2024-09-16T00:32:24.454567+0000","last_peered":"2024-09-16T00:32:24.454567+0000","last_clean":"2024-09-16T00:32:24.454567+0000","last_became_active":"2024-09-16T00:30:33.605073+0000","last_became_peered":"2024-09-16T00:30:33.605073+0000","last_unstale":"2024-09-16T00:32:24.454567+0000","last_undegraded":"2024-09-16T00:32:24.454567+0000","last_fullsized":"2024-09-16T00:32:24.454567+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:11.143052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595152+0000","last_change":"2024-09-16T00:30:26.084507+0000","last_active":"2024-09-16T00:32:24.595152+0000","last_peered":"2024-09-16T00:32:24.595152+0000","last_clean":"2024-09-16T00:32:24.595152+0000","last_became_active":"2024-09-16T00:30:26.084152+0000","last_became_peered":"2024-09-16T00:30:26.084152+0000","last_unstale":"2024-09-16T00:32:24.595152+0000","last_undegraded":"2024-09-16T00:32:24.595152+0000","last_fullsized":"2024-09-16T00:32:24.595152+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:27:21.557409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454701+0000","last_change":"2024-09-16T00:30:23.932686+0000","last_active":"2024-09-16T00:32:24.454701+0000","last_peered":"2024-09-16T00:32:24.454701+0000","last_clean":"2024-09-16T00:32:24.454701+0000","last_became_active":"2024-09-16T00:30:23.932343+0000","last_became_peered":"2024-09-16T00:30:23.932343+0000","last_unstale":"2024-09-16T00:32:24.454701+0000","last_undegraded":"2024-09-16T00:32:24.454701+0000","last_fullsized":"2024-09-16T00:32:24.454701+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:13:44.185712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":351,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595072+0000","last_change":"2024-09-16T00:29:11.143642+0000","last_active":"2024-09-16T00:32:24.595072+0000","last_peered":"2024-09-16T00:32:24.595072+0000","last_clean":"2024-09-16T00:32:24.595072+0000","last_became_active":"2024-09-16T00:29:11.143227+0000","last_became_peered":"2024-09-16T00:29:11.143227+0000","last_unstale":"2024-09-16T00:32:24.595072+0000","last_undegraded":"2024-09-16T00:32:24.595072+0000","last_fullsized":"2024-09-16T00:32:24.595072+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:25:42.361316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:25:42.361316+0000","last_clean_scrub_stamp":"2024-09-16T00:25:42.361316+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:29:12.740474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595397+0000","last_change":"2024-09-16T00:30:33.155555+0000","last_active":"2024-09-16T00:32:24.595397+0000","last_peered":"2024-09-16T00:32:24.595397+0000","last_clean":"2024-09-16T00:32:24.595397+0000","last_became_active":"2024-09-16T00:30:33.155170+0000","last_became_peered":"2024-09-16T00:30:33.155170+0000","last_unstale":"2024-09-16T00:32:24.595397+0000","last_undegraded":"2024-09-16T00:32:24.595397+0000","last_fullsized":"2024-09-16T00:32:24.595397+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:34:46.214422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595573+0000","last_change":"2024-09-16T00:30:26.076003+0000","last_active":"2024-09-16T00:32:24.595573+0000","last_peered":"2024-09-16T00:32:24.595573+0000","last_clean":"2024-09-16T00:32:24.595573+0000","last_became_active":"2024-09-16T00:30:26.075666+0000","last_became_peered":"2024-09-16T00:30:26.075666+0000","last_unstale":"2024-09-16T00:32:24.595573+0000","last_undegraded":"2024-09-16T00:32:24.595573+0000","last_fullsized":"2024-09-16T00:32:24.595573+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:30.113380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594420+0000","last_change":"2024-09-16T00:30:24.587469+0000","last_active":"2024-09-16T00:32:24.594420+0000","last_peered":"2024-09-16T00:32:24.594420+0000","last_clean":"2024-09-16T00:32:24.594420+0000","last_became_active":"2024-09-16T00:30:24.587049+0000","last_became_peered":"2024-09-16T00:30:24.587049+0000","last_unstale":"2024-09-16T00:32:24.594420+0000","last_undegraded":"2024-09-16T00:32:24.594420+0000","last_fullsized":"2024-09-16T00:32:24.594420+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:17:45.934505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561433+0000","last_change":"2024-09-16T00:30:33.145921+0000","last_active":"2024-09-16T00:32:24.561433+0000","last_peered":"2024-09-16T00:32:24.561433+0000","last_clean":"2024-09-16T00:32:24.561433+0000","last_became_active":"2024-09-16T00:30:33.145450+0000","last_became_peered":"2024-09-16T00:30:33.145450+0000","last_unstale":"2024-09-16T00:32:24.561433+0000","last_undegraded":"2024-09-16T00:32:24.561433+0000","last_fullsized":"2024-09-16T00:32:24.561433+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:53:51.698686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595330+0000","last_change":"2024-09-16T00:30:26.087940+0000","last_active":"2024-09-16T00:32:24.595330+0000","last_peered":"2024-09-16T00:32:24.595330+0000","last_clean":"2024-09-16T00:32:24.595330+0000","last_became_active":"2024-09-16T00:30:26.087599+0000","last_became_peered":"2024-09-16T00:30:26.087599+0000","last_unstale":"2024-09-16T00:32:24.595330+0000","last_undegraded":"2024-09-16T00:32:24.595330+0000","last_fullsized":"2024-09-16T00:32:24.595330+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:43.934839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454847+0000","last_change":"2024-09-16T00:30:23.930986+0000","last_active":"2024-09-16T00:32:24.454847+0000","last_peered":"2024-09-16T00:32:24.454847+0000","last_clean":"2024-09-16T00:32:24.454847+0000","last_became_active":"2024-09-16T00:30:23.930588+0000","last_became_peered":"2024-09-16T00:30:23.930588+0000","last_unstale":"2024-09-16T00:32:24.454847+0000","last_undegraded":"2024-09-16T00:32:24.454847+0000","last_fullsized":"2024-09-16T00:32:24.454847+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:04:41.902358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594672+0000","last_change":"2024-09-16T00:30:33.154894+0000","last_active":"2024-09-16T00:32:24.594672+0000","last_peered":"2024-09-16T00:32:24.594672+0000","last_clean":"2024-09-16T00:32:24.594672+0000","last_became_active":"2024-09-16T00:30:33.154490+0000","last_became_peered":"2024-09-16T00:30:33.154490+0000","last_unstale":"2024-09-16T00:32:24.594672+0000","last_undegraded":"2024-09-16T00:32:24.594672+0000","last_fullsized":"2024-09-16T00:32:24.594672+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:59:53.854212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595549+0000","last_change":"2024-09-16T00:30:26.077069+0000","last_active":"2024-09-16T00:32:24.595549+0000","last_peered":"2024-09-16T00:32:24.595549+0000","last_clean":"2024-09-16T00:32:24.595549+0000","last_became_active":"2024-09-16T00:30:26.076718+0000","last_became_peered":"2024-09-16T00:30:26.076718+0000","last_unstale":"2024-09-16T00:32:24.595549+0000","last_undegraded":"2024-09-16T00:32:24.595549+0000","last_fullsized":"2024-09-16T00:32:24.595549+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:25:37.541704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452322+0000","last_change":"2024-09-16T00:30:23.932081+0000","last_active":"2024-09-16T00:32:24.452322+0000","last_peered":"2024-09-16T00:32:24.452322+0000","last_clean":"2024-09-16T00:32:24.452322+0000","last_became_active":"2024-09-16T00:30:23.931729+0000","last_became_peered":"2024-09-16T00:30:23.931729+0000","last_unstale":"2024-09-16T00:32:24.452322+0000","last_undegraded":"2024-09-16T00:32:24.452322+0000","last_fullsized":"2024-09-16T00:32:24.452322+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:20:05.178580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454073+0000","last_change":"2024-09-16T00:30:33.147738+0000","last_active":"2024-09-16T00:32:24.454073+0000","last_peered":"2024-09-16T00:32:24.454073+0000","last_clean":"2024-09-16T00:32:24.454073+0000","last_became_active":"2024-09-16T00:30:33.147326+0000","last_became_peered":"2024-09-16T00:30:33.147326+0000","last_unstale":"2024-09-16T00:32:24.454073+0000","last_undegraded":"2024-09-16T00:32:24.454073+0000","last_fullsized":"2024-09-16T00:32:24.454073+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:44:55.123980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339213+0000","last_change":"2024-09-16T00:30:26.079312+0000","last_active":"2024-09-16T00:32:24.339213+0000","last_peered":"2024-09-16T00:32:24.339213+0000","last_clean":"2024-09-16T00:32:24.339213+0000","last_became_active":"2024-09-16T00:30:26.078986+0000","last_became_peered":"2024-09-16T00:30:26.078986+0000","last_unstale":"2024-09-16T00:32:24.339213+0000","last_undegraded":"2024-09-16T00:32:24.339213+0000","last_fullsized":"2024-09-16T00:32:24.339213+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:02:06.602850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339614+0000","last_change":"2024-09-16T00:31:23.452102+0000","last_active":"2024-09-16T00:32:24.339614+0000","last_peered":"2024-09-16T00:32:24.339614+0000","last_clean":"2024-09-16T00:32:24.339614+0000","last_became_active":"2024-09-16T00:31:23.445974+0000","last_became_peered":"2024-09-16T00:31:23.445974+0000","last_unstale":"2024-09-16T00:32:24.339614+0000","last_undegraded":"2024-09-16T00:32:24.339614+0000","last_fullsized":"2024-09-16T00:32:24.339614+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:52.314494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454166+0000","last_change":"2024-09-16T00:30:33.149154+0000","last_active":"2024-09-16T00:32:24.454166+0000","last_peered":"2024-09-16T00:32:24.454166+0000","last_clean":"2024-09-16T00:32:24.454166+0000","last_became_active":"2024-09-16T00:30:33.148763+0000","last_became_peered":"2024-09-16T00:30:33.148763+0000","last_unstale":"2024-09-16T00:32:24.454166+0000","last_undegraded":"2024-09-16T00:32:24.454166+0000","last_fullsized":"2024-09-16T00:32:24.454166+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:58:05.884919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454241+0000","last_change":"2024-09-16T00:30:26.086905+0000","last_active":"2024-09-16T00:32:24.454241+0000","last_peered":"2024-09-16T00:32:24.454241+0000","last_clean":"2024-09-16T00:32:24.454241+0000","last_became_active":"2024-09-16T00:30:26.086519+0000","last_became_peered":"2024-09-16T00:30:26.086519+0000","last_unstale":"2024-09-16T00:32:24.454241+0000","last_undegraded":"2024-09-16T00:32:24.454241+0000","last_fullsized":"2024-09-16T00:32:24.454241+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:51:08.787679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.338768+0000","last_change":"2024-09-16T00:30:24.422893+0000","last_active":"2024-09-16T00:32:24.338768+0000","last_peered":"2024-09-16T00:32:24.338768+0000","last_clean":"2024-09-16T00:32:24.338768+0000","last_became_active":"2024-09-16T00:30:24.422073+0000","last_became_peered":"2024-09-16T00:30:24.422073+0000","last_unstale":"2024-09-16T00:32:24.338768+0000","last_undegraded":"2024-09-16T00:32:24.338768+0000","last_fullsized":"2024-09-16T00:32:24.338768+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:15:11.375957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452155+0000","last_change":"2024-09-16T00:30:33.152582+0000","last_active":"2024-09-16T00:32:24.452155+0000","last_peered":"2024-09-16T00:32:24.452155+0000","last_clean":"2024-09-16T00:32:24.452155+0000","last_became_active":"2024-09-16T00:30:33.152180+0000","last_became_peered":"2024-09-16T00:30:33.152180+0000","last_unstale":"2024-09-16T00:32:24.452155+0000","last_undegraded":"2024-09-16T00:32:24.452155+0000","last_fullsized":"2024-09-16T00:32:24.452155+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:41:10.499391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594802+0000","last_change":"2024-09-16T00:30:26.086052+0000","last_active":"2024-09-16T00:32:24.594802+0000","last_peered":"2024-09-16T00:32:24.594802+0000","last_clean":"2024-09-16T00:32:24.594802+0000","last_became_active":"2024-09-16T00:30:26.085756+0000","last_became_peered":"2024-09-16T00:30:26.085756+0000","last_unstale":"2024-09-16T00:32:24.594802+0000","last_undegraded":"2024-09-16T00:32:24.594802+0000","last_fullsized":"2024-09-16T00:32:24.594802+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:52.582166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":144,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:42.479580+0000","last_change":"2024-09-16T00:30:23.933528+0000","last_active":"2024-09-16T00:32:42.479580+0000","last_peered":"2024-09-16T00:32:42.479580+0000","last_clean":"2024-09-16T00:32:42.479580+0000","last_became_active":"2024-09-16T00:30:23.933186+0000","last_became_peered":"2024-09-16T00:30:23.933186+0000","last_unstale":"2024-09-16T00:32:42.479580+0000","last_undegraded":"2024-09-16T00:32:42.479580+0000","last_fullsized":"2024-09-16T00:32:42.479580+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:11:00.489676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.340727+0000","last_change":"2024-09-16T00:31:23.578056+0000","last_active":"2024-09-16T00:32:24.340727+0000","last_peered":"2024-09-16T00:32:24.340727+0000","last_clean":"2024-09-16T00:32:24.340727+0000","last_became_active":"2024-09-16T00:31:23.577721+0000","last_became_peered":"2024-09-16T00:31:23.577721+0000","last_unstale":"2024-09-16T00:32:24.340727+0000","last_undegraded":"2024-09-16T00:32:24.340727+0000","last_fullsized":"2024-09-16T00:32:24.340727+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:53.632440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595248+0000","last_change":"2024-09-16T00:30:26.084509+0000","last_active":"2024-09-16T00:32:24.595248+0000","last_peered":"2024-09-16T00:32:24.595248+0000","last_clean":"2024-09-16T00:32:24.595248+0000","last_became_active":"2024-09-16T00:30:26.084188+0000","last_became_peered":"2024-09-16T00:30:26.084188+0000","last_unstale":"2024-09-16T00:32:24.595248+0000","last_undegraded":"2024-09-16T00:32:24.595248+0000","last_fullsized":"2024-09-16T00:32:24.595248+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:21:48.981277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454529+0000","last_change":"2024-09-16T00:30:23.933499+0000","last_active":"2024-09-16T00:32:24.454529+0000","last_peered":"2024-09-16T00:32:24.454529+0000","last_clean":"2024-09-16T00:32:24.454529+0000","last_became_active":"2024-09-16T00:30:23.933082+0000","last_became_peered":"2024-09-16T00:30:23.933082+0000","last_unstale":"2024-09-16T00:32:24.454529+0000","last_undegraded":"2024-09-16T00:32:24.454529+0000","last_fullsized":"2024-09-16T00:32:24.454529+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:42:18.998433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.596318+0000","last_change":"2024-09-16T00:32:24.596318+0000","last_active":"2024-09-16T00:32:24.596318+0000","last_peered":"2024-09-16T00:32:24.596318+0000","last_clean":"2024-09-16T00:32:24.596318+0000","last_became_active":"2024-09-16T00:32:24.595906+0000","last_became_peered":"2024-09-16T00:32:24.595906+0000","last_unstale":"2024-09-16T00:32:24.596318+0000","last_undegraded":"2024-09-16T00:32:24.596318+0000","last_fullsized":"2024-09-16T00:32:24.596318+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:13:56.601745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452018+0000","last_change":"2024-09-16T00:30:26.080517+0000","last_active":"2024-09-16T00:32:24.452018+0000","last_peered":"2024-09-16T00:32:24.452018+0000","last_clean":"2024-09-16T00:32:24.452018+0000","last_became_active":"2024-09-16T00:30:26.080197+0000","last_became_peered":"2024-09-16T00:30:26.080197+0000","last_unstale":"2024-09-16T00:32:24.452018+0000","last_undegraded":"2024-09-16T00:32:24.452018+0000","last_fullsized":"2024-09-16T00:32:24.452018+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:15.508562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339267+0000","last_change":"2024-09-16T00:31:23.457047+0000","last_active":"2024-09-16T00:32:24.339267+0000","last_peered":"2024-09-16T00:32:24.339267+0000","last_clean":"2024-09-16T00:32:24.339267+0000","last_became_active":"2024-09-16T00:31:23.447014+0000","last_became_peered":"2024-09-16T00:31:23.447014+0000","last_unstale":"2024-09-16T00:32:24.339267+0000","last_undegraded":"2024-09-16T00:32:24.339267+0000","last_fullsized":"2024-09-16T00:32:24.339267+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:34:42.040228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454641+0000","last_change":"2024-09-16T00:30:33.145803+0000","last_active":"2024-09-16T00:32:24.454641+0000","last_peered":"2024-09-16T00:32:24.454641+0000","last_clean":"2024-09-16T00:32:24.454641+0000","last_became_active":"2024-09-16T00:30:33.145366+0000","last_became_peered":"2024-09-16T00:30:33.145366+0000","last_unstale":"2024-09-16T00:32:24.454641+0000","last_undegraded":"2024-09-16T00:32:24.454641+0000","last_fullsized":"2024-09-16T00:32:24.454641+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:47:28.689305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339085+0000","last_change":"2024-09-16T00:30:26.079415+0000","last_active":"2024-09-16T00:32:24.339085+0000","last_peered":"2024-09-16T00:32:24.339085+0000","last_clean":"2024-09-16T00:32:24.339085+0000","last_became_active":"2024-09-16T00:30:26.079075+0000","last_became_peered":"2024-09-16T00:30:26.079075+0000","last_unstale":"2024-09-16T00:32:24.339085+0000","last_undegraded":"2024-09-16T00:32:24.339085+0000","last_fullsized":"2024-09-16T00:32:24.339085+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:18:18.236342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"92'39","reported_seq":291,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:43.484104+0000","last_change":"2024-09-16T00:30:23.933172+0000","last_active":"2024-09-16T00:32:43.484104+0000","last_peered":"2024-09-16T00:32:43.484104+0000","last_clean":"2024-09-16T00:32:43.484104+0000","last_became_active":"2024-09-16T00:30:23.932492+0000","last_became_peered":"2024-09-16T00:30:23.932492+0000","last_unstale":"2024-09-16T00:32:43.484104+0000","last_undegraded":"2024-09-16T00:32:43.484104+0000","last_fullsized":"2024-09-16T00:32:43.484104+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:29.169448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68399,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":213,"num_read_kb":175,"num_write":40,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.337053+0000","last_change":"2024-09-16T00:31:23.445673+0000","last_active":"2024-09-16T00:32:24.337053+0000","last_peered":"2024-09-16T00:32:24.337053+0000","last_clean":"2024-09-16T00:32:24.337053+0000","last_became_active":"2024-09-16T00:31:23.445299+0000","last_became_peered":"2024-09-16T00:31:23.445299+0000","last_unstale":"2024-09-16T00:32:24.337053+0000","last_undegraded":"2024-09-16T00:32:24.337053+0000","last_fullsized":"2024-09-16T00:32:24.337053+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:07:44.235340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561729+0000","last_change":"2024-09-16T00:30:26.085419+0000","last_active":"2024-09-16T00:32:24.561729+0000","last_peered":"2024-09-16T00:32:24.561729+0000","last_clean":"2024-09-16T00:32:24.561729+0000","last_became_active":"2024-09-16T00:30:26.085103+0000","last_became_peered":"2024-09-16T00:30:26.085103+0000","last_unstale":"2024-09-16T00:32:24.561729+0000","last_undegraded":"2024-09-16T00:32:24.561729+0000","last_fullsized":"2024-09-16T00:32:24.561729+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:56:26.352779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.453906+0000","last_change":"2024-09-16T00:30:24.422646+0000","last_active":"2024-09-16T00:32:24.453906+0000","last_peered":"2024-09-16T00:32:24.453906+0000","last_clean":"2024-09-16T00:32:24.453906+0000","last_became_active":"2024-09-16T00:30:24.422065+0000","last_became_peered":"2024-09-16T00:30:24.422065+0000","last_unstale":"2024-09-16T00:32:24.453906+0000","last_undegraded":"2024-09-16T00:32:24.453906+0000","last_fullsized":"2024-09-16T00:32:24.453906+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:53:12.401613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454317+0000","last_change":"2024-09-16T00:30:33.147692+0000","last_active":"2024-09-16T00:32:24.454317+0000","last_peered":"2024-09-16T00:32:24.454317+0000","last_clean":"2024-09-16T00:32:24.454317+0000","last_became_active":"2024-09-16T00:30:33.147115+0000","last_became_peered":"2024-09-16T00:30:33.147115+0000","last_unstale":"2024-09-16T00:32:24.454317+0000","last_undegraded":"2024-09-16T00:32:24.454317+0000","last_fullsized":"2024-09-16T00:32:24.454317+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:44:04.536894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595596+0000","last_change":"2024-09-16T00:30:26.088427+0000","last_active":"2024-09-16T00:32:24.595596+0000","last_peered":"2024-09-16T00:32:24.595596+0000","last_clean":"2024-09-16T00:32:24.595596+0000","last_became_active":"2024-09-16T00:30:26.086454+0000","last_became_peered":"2024-09-16T00:30:26.086454+0000","last_unstale":"2024-09-16T00:32:24.595596+0000","last_undegraded":"2024-09-16T00:32:24.595596+0000","last_fullsized":"2024-09-16T00:32:24.595596+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:54:29.679959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595460+0000","last_change":"2024-09-16T00:31:23.453484+0000","last_active":"2024-09-16T00:32:24.595460+0000","last_peered":"2024-09-16T00:32:24.595460+0000","last_clean":"2024-09-16T00:32:24.595460+0000","last_became_active":"2024-09-16T00:31:23.446700+0000","last_became_peered":"2024-09-16T00:31:23.446700+0000","last_unstale":"2024-09-16T00:32:24.595460+0000","last_undegraded":"2024-09-16T00:32:24.595460+0000","last_fullsized":"2024-09-16T00:32:24.595460+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:43:26.465657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.336273+0000","last_change":"2024-09-16T00:30:33.156510+0000","last_active":"2024-09-16T00:32:24.336273+0000","last_peered":"2024-09-16T00:32:24.336273+0000","last_clean":"2024-09-16T00:32:24.336273+0000","last_became_active":"2024-09-16T00:30:33.156088+0000","last_became_peered":"2024-09-16T00:30:33.156088+0000","last_unstale":"2024-09-16T00:32:24.336273+0000","last_undegraded":"2024-09-16T00:32:24.336273+0000","last_fullsized":"2024-09-16T00:32:24.336273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:34.230901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561872+0000","last_change":"2024-09-16T00:30:26.082598+0000","last_active":"2024-09-16T00:32:24.561872+0000","last_peered":"2024-09-16T00:32:24.561872+0000","last_clean":"2024-09-16T00:32:24.561872+0000","last_became_active":"2024-09-16T00:30:26.082286+0000","last_became_peered":"2024-09-16T00:30:26.082286+0000","last_unstale":"2024-09-16T00:32:24.561872+0000","last_undegraded":"2024-09-16T00:32:24.561872+0000","last_fullsized":"2024-09-16T00:32:24.561872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:22:18.579149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"92'5","reported_seq":144,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:42.481119+0000","last_change":"2024-09-16T00:30:23.931097+0000","last_active":"2024-09-16T00:32:42.481119+0000","last_peered":"2024-09-16T00:32:42.481119+0000","last_clean":"2024-09-16T00:32:42.481119+0000","last_became_active":"2024-09-16T00:30:23.930719+0000","last_became_peered":"2024-09-16T00:30:23.930719+0000","last_unstale":"2024-09-16T00:32:42.481119+0000","last_undegraded":"2024-09-16T00:32:42.481119+0000","last_fullsized":"2024-09-16T00:32:42.481119+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:13:47.887754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595241+0000","last_change":"2024-09-16T00:30:33.142581+0000","last_active":"2024-09-16T00:32:24.595241+0000","last_peered":"2024-09-16T00:32:24.595241+0000","last_clean":"2024-09-16T00:32:24.595241+0000","last_became_active":"2024-09-16T00:30:33.142263+0000","last_became_peered":"2024-09-16T00:30:33.142263+0000","last_unstale":"2024-09-16T00:32:24.595241+0000","last_undegraded":"2024-09-16T00:32:24.595241+0000","last_fullsized":"2024-09-16T00:32:24.595241+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:10:06.456011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595181+0000","last_change":"2024-09-16T00:30:33.152084+0000","last_active":"2024-09-16T00:32:24.595181+0000","last_peered":"2024-09-16T00:32:24.595181+0000","last_clean":"2024-09-16T00:32:24.595181+0000","last_became_active":"2024-09-16T00:30:33.151681+0000","last_became_peered":"2024-09-16T00:30:33.151681+0000","last_unstale":"2024-09-16T00:32:24.595181+0000","last_undegraded":"2024-09-16T00:32:24.595181+0000","last_fullsized":"2024-09-16T00:32:24.595181+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:36.966498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.336370+0000","last_change":"2024-09-16T00:30:26.084200+0000","last_active":"2024-09-16T00:32:24.336370+0000","last_peered":"2024-09-16T00:32:24.336370+0000","last_clean":"2024-09-16T00:32:24.336370+0000","last_became_active":"2024-09-16T00:30:26.083858+0000","last_became_peered":"2024-09-16T00:30:26.083858+0000","last_unstale":"2024-09-16T00:32:24.336370+0000","last_undegraded":"2024-09-16T00:32:24.336370+0000","last_fullsized":"2024-09-16T00:32:24.336370+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:42.712624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454319+0000","last_change":"2024-09-16T00:30:23.932399+0000","last_active":"2024-09-16T00:32:24.454319+0000","last_peered":"2024-09-16T00:32:24.454319+0000","last_clean":"2024-09-16T00:32:24.454319+0000","last_became_active":"2024-09-16T00:30:23.931996+0000","last_became_peered":"2024-09-16T00:30:23.931996+0000","last_unstale":"2024-09-16T00:32:24.454319+0000","last_undegraded":"2024-09-16T00:32:24.454319+0000","last_fullsized":"2024-09-16T00:32:24.454319+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:47:09.634507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339553+0000","last_change":"2024-09-16T00:30:33.154563+0000","last_active":"2024-09-16T00:32:24.339553+0000","last_peered":"2024-09-16T00:32:24.339553+0000","last_clean":"2024-09-16T00:32:24.339553+0000","last_became_active":"2024-09-16T00:30:33.154105+0000","last_became_peered":"2024-09-16T00:30:33.154105+0000","last_unstale":"2024-09-16T00:32:24.339553+0000","last_undegraded":"2024-09-16T00:32:24.339553+0000","last_fullsized":"2024-09-16T00:32:24.339553+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:59.966255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595005+0000","last_change":"2024-09-16T00:30:26.075126+0000","last_active":"2024-09-16T00:32:24.595005+0000","last_peered":"2024-09-16T00:32:24.595005+0000","last_clean":"2024-09-16T00:32:24.595005+0000","last_became_active":"2024-09-16T00:30:26.074819+0000","last_became_peered":"2024-09-16T00:30:26.074819+0000","last_unstale":"2024-09-16T00:32:24.595005+0000","last_undegraded":"2024-09-16T00:32:24.595005+0000","last_fullsized":"2024-09-16T00:32:24.595005+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:36.996346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.451802+0000","last_change":"2024-09-16T00:30:23.929943+0000","last_active":"2024-09-16T00:32:24.451802+0000","last_peered":"2024-09-16T00:32:24.451802+0000","last_clean":"2024-09-16T00:32:24.451802+0000","last_became_active":"2024-09-16T00:30:23.929578+0000","last_became_peered":"2024-09-16T00:30:23.929578+0000","last_unstale":"2024-09-16T00:32:24.451802+0000","last_undegraded":"2024-09-16T00:32:24.451802+0000","last_fullsized":"2024-09-16T00:32:24.451802+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:05:12.505543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.337224+0000","last_change":"2024-09-16T00:30:23.935758+0000","last_active":"2024-09-16T00:32:24.337224+0000","last_peered":"2024-09-16T00:32:24.337224+0000","last_clean":"2024-09-16T00:32:24.337224+0000","last_became_active":"2024-09-16T00:30:23.935321+0000","last_became_peered":"2024-09-16T00:30:23.935321+0000","last_unstale":"2024-09-16T00:32:24.337224+0000","last_undegraded":"2024-09-16T00:32:24.337224+0000","last_fullsized":"2024-09-16T00:32:24.337224+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:35:40.513435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454389+0000","last_change":"2024-09-16T00:30:26.086316+0000","last_active":"2024-09-16T00:32:24.454389+0000","last_peered":"2024-09-16T00:32:24.454389+0000","last_clean":"2024-09-16T00:32:24.454389+0000","last_became_active":"2024-09-16T00:30:26.085800+0000","last_became_peered":"2024-09-16T00:30:26.085800+0000","last_unstale":"2024-09-16T00:32:24.454389+0000","last_undegraded":"2024-09-16T00:32:24.454389+0000","last_fullsized":"2024-09-16T00:32:24.454389+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:05:36.277127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595014+0000","last_change":"2024-09-16T00:30:33.155665+0000","last_active":"2024-09-16T00:32:24.595014+0000","last_peered":"2024-09-16T00:32:24.595014+0000","last_clean":"2024-09-16T00:32:24.595014+0000","last_became_active":"2024-09-16T00:30:33.155253+0000","last_became_peered":"2024-09-16T00:30:33.155253+0000","last_unstale":"2024-09-16T00:32:24.595014+0000","last_undegraded":"2024-09-16T00:32:24.595014+0000","last_fullsized":"2024-09-16T00:32:24.595014+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:37.545232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'8","reported_seq":151,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:42.481651+0000","last_change":"2024-09-16T00:30:23.934487+0000","last_active":"2024-09-16T00:32:42.481651+0000","last_peered":"2024-09-16T00:32:42.481651+0000","last_clean":"2024-09-16T00:32:42.481651+0000","last_became_active":"2024-09-16T00:30:23.934137+0000","last_became_peered":"2024-09-16T00:30:23.934137+0000","last_unstale":"2024-09-16T00:32:42.481651+0000","last_undegraded":"2024-09-16T00:32:42.481651+0000","last_fullsized":"2024-09-16T00:32:42.481651+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:14:33.088776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1549,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.338839+0000","last_change":"2024-09-16T00:30:26.077533+0000","last_active":"2024-09-16T00:32:24.338839+0000","last_peered":"2024-09-16T00:32:24.338839+0000","last_clean":"2024-09-16T00:32:24.338839+0000","last_became_active":"2024-09-16T00:30:26.076443+0000","last_became_peered":"2024-09-16T00:30:26.076443+0000","last_unstale":"2024-09-16T00:32:24.338839+0000","last_undegraded":"2024-09-16T00:32:24.338839+0000","last_fullsized":"2024-09-16T00:32:24.338839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:39:17.495521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595151+0000","last_change":"2024-09-16T00:30:33.146123+0000","last_active":"2024-09-16T00:32:24.595151+0000","last_peered":"2024-09-16T00:32:24.595151+0000","last_clean":"2024-09-16T00:32:24.595151+0000","last_became_active":"2024-09-16T00:30:33.145716+0000","last_became_peered":"2024-09-16T00:30:33.145716+0000","last_unstale":"2024-09-16T00:32:24.595151+0000","last_undegraded":"2024-09-16T00:32:24.595151+0000","last_fullsized":"2024-09-16T00:32:24.595151+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:14:27.281367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.338818+0000","last_change":"2024-09-16T00:30:26.083155+0000","last_active":"2024-09-16T00:32:24.338818+0000","last_peered":"2024-09-16T00:32:24.338818+0000","last_clean":"2024-09-16T00:32:24.338818+0000","last_became_active":"2024-09-16T00:30:26.082782+0000","last_became_peered":"2024-09-16T00:30:26.082782+0000","last_unstale":"2024-09-16T00:32:24.338818+0000","last_undegraded":"2024-09-16T00:32:24.338818+0000","last_fullsized":"2024-09-16T00:32:24.338818+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:15.929343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594891+0000","last_change":"2024-09-16T00:31:23.447453+0000","last_active":"2024-09-16T00:32:24.594891+0000","last_peered":"2024-09-16T00:32:24.594891+0000","last_clean":"2024-09-16T00:32:24.594891+0000","last_became_active":"2024-09-16T00:31:23.445507+0000","last_became_peered":"2024-09-16T00:31:23.445507+0000","last_unstale":"2024-09-16T00:32:24.594891+0000","last_undegraded":"2024-09-16T00:32:24.594891+0000","last_fullsized":"2024-09-16T00:32:24.594891+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:18:50.885809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594798+0000","last_change":"2024-09-16T00:30:26.076252+0000","last_active":"2024-09-16T00:32:24.594798+0000","last_peered":"2024-09-16T00:32:24.594798+0000","last_clean":"2024-09-16T00:32:24.594798+0000","last_became_active":"2024-09-16T00:30:26.075922+0000","last_became_peered":"2024-09-16T00:30:26.075922+0000","last_unstale":"2024-09-16T00:32:24.594798+0000","last_undegraded":"2024-09-16T00:32:24.594798+0000","last_fullsized":"2024-09-16T00:32:24.594798+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:28:04.822817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594821+0000","last_change":"2024-09-16T00:30:26.085123+0000","last_active":"2024-09-16T00:32:24.594821+0000","last_peered":"2024-09-16T00:32:24.594821+0000","last_clean":"2024-09-16T00:32:24.594821+0000","last_became_active":"2024-09-16T00:30:26.084782+0000","last_became_peered":"2024-09-16T00:30:26.084782+0000","last_unstale":"2024-09-16T00:32:24.594821+0000","last_undegraded":"2024-09-16T00:32:24.594821+0000","last_fullsized":"2024-09-16T00:32:24.594821+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:09:33.766340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452404+0000","last_change":"2024-09-16T00:30:26.077048+0000","last_active":"2024-09-16T00:32:24.452404+0000","last_peered":"2024-09-16T00:32:24.452404+0000","last_clean":"2024-09-16T00:32:24.452404+0000","last_became_active":"2024-09-16T00:30:26.076719+0000","last_became_peered":"2024-09-16T00:30:26.076719+0000","last_unstale":"2024-09-16T00:32:24.452404+0000","last_undegraded":"2024-09-16T00:32:24.452404+0000","last_fullsized":"2024-09-16T00:32:24.452404+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:58.584246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339277+0000","last_change":"2024-09-16T00:30:26.078446+0000","last_active":"2024-09-16T00:32:24.339277+0000","last_peered":"2024-09-16T00:32:24.339277+0000","last_clean":"2024-09-16T00:32:24.339277+0000","last_became_active":"2024-09-16T00:30:26.078145+0000","last_became_peered":"2024-09-16T00:30:26.078145+0000","last_unstale":"2024-09-16T00:32:24.339277+0000","last_undegraded":"2024-09-16T00:32:24.339277+0000","last_fullsized":"2024-09-16T00:32:24.339277+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:52:28.739834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595347+0000","last_change":"2024-09-16T00:30:26.075875+0000","last_active":"2024-09-16T00:32:24.595347+0000","last_peered":"2024-09-16T00:32:24.595347+0000","last_clean":"2024-09-16T00:32:24.595347+0000","last_became_active":"2024-09-16T00:30:26.075424+0000","last_became_peered":"2024-09-16T00:30:26.075424+0000","last_unstale":"2024-09-16T00:32:24.595347+0000","last_undegraded":"2024-09-16T00:32:24.595347+0000","last_fullsized":"2024-09-16T00:32:24.595347+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:21:07.958507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595253+0000","last_change":"2024-09-16T00:30:26.077694+0000","last_active":"2024-09-16T00:32:24.595253+0000","last_peered":"2024-09-16T00:32:24.595253+0000","last_clean":"2024-09-16T00:32:24.595253+0000","last_became_active":"2024-09-16T00:30:26.077374+0000","last_became_peered":"2024-09-16T00:30:26.077374+0000","last_unstale":"2024-09-16T00:32:24.595253+0000","last_undegraded":"2024-09-16T00:32:24.595253+0000","last_fullsized":"2024-09-16T00:32:24.595253+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:16:30.822050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454143+0000","last_change":"2024-09-16T00:30:26.085931+0000","last_active":"2024-09-16T00:32:24.454143+0000","last_peered":"2024-09-16T00:32:24.454143+0000","last_clean":"2024-09-16T00:32:24.454143+0000","last_became_active":"2024-09-16T00:30:26.085420+0000","last_became_peered":"2024-09-16T00:30:26.085420+0000","last_unstale":"2024-09-16T00:32:24.454143+0000","last_undegraded":"2024-09-16T00:32:24.454143+0000","last_fullsized":"2024-09-16T00:32:24.454143+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:20:36.019203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595470+0000","last_change":"2024-09-16T00:30:26.074217+0000","last_active":"2024-09-16T00:32:24.595470+0000","last_peered":"2024-09-16T00:32:24.595470+0000","last_clean":"2024-09-16T00:32:24.595470+0000","last_became_active":"2024-09-16T00:30:26.073856+0000","last_became_peered":"2024-09-16T00:30:26.073856+0000","last_unstale":"2024-09-16T00:32:24.595470+0000","last_undegraded":"2024-09-16T00:32:24.595470+0000","last_fullsized":"2024-09-16T00:32:24.595470+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:53:57.078465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561639+0000","last_change":"2024-09-16T00:30:26.084248+0000","last_active":"2024-09-16T00:32:24.561639+0000","last_peered":"2024-09-16T00:32:24.561639+0000","last_clean":"2024-09-16T00:32:24.561639+0000","last_became_active":"2024-09-16T00:30:26.083530+0000","last_became_peered":"2024-09-16T00:30:26.083530+0000","last_unstale":"2024-09-16T00:32:24.561639+0000","last_undegraded":"2024-09-16T00:32:24.561639+0000","last_fullsized":"2024-09-16T00:32:24.561639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:16:49.483881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452162+0000","last_change":"2024-09-16T00:30:26.080803+0000","last_active":"2024-09-16T00:32:24.452162+0000","last_peered":"2024-09-16T00:32:24.452162+0000","last_clean":"2024-09-16T00:32:24.452162+0000","last_became_active":"2024-09-16T00:30:26.080482+0000","last_became_peered":"2024-09-16T00:30:26.080482+0000","last_unstale":"2024-09-16T00:32:24.452162+0000","last_undegraded":"2024-09-16T00:32:24.452162+0000","last_fullsized":"2024-09-16T00:32:24.452162+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:35:41.741025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339770+0000","last_change":"2024-09-16T00:30:26.077680+0000","last_active":"2024-09-16T00:32:24.339770+0000","last_peered":"2024-09-16T00:32:24.339770+0000","last_clean":"2024-09-16T00:32:24.339770+0000","last_became_active":"2024-09-16T00:30:26.077371+0000","last_became_peered":"2024-09-16T00:30:26.077371+0000","last_unstale":"2024-09-16T00:32:24.339770+0000","last_undegraded":"2024-09-16T00:32:24.339770+0000","last_fullsized":"2024-09-16T00:32:24.339770+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:31:34.338617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595364+0000","last_change":"2024-09-16T00:30:26.071531+0000","last_active":"2024-09-16T00:32:24.595364+0000","last_peered":"2024-09-16T00:32:24.595364+0000","last_clean":"2024-09-16T00:32:24.595364+0000","last_became_active":"2024-09-16T00:30:26.071189+0000","last_became_peered":"2024-09-16T00:30:26.071189+0000","last_unstale":"2024-09-16T00:32:24.595364+0000","last_undegraded":"2024-09-16T00:32:24.595364+0000","last_fullsized":"2024-09-16T00:32:24.595364+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:29:18.406272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.336865+0000","last_change":"2024-09-16T00:30:26.085398+0000","last_active":"2024-09-16T00:32:24.336865+0000","last_peered":"2024-09-16T00:32:24.336865+0000","last_clean":"2024-09-16T00:32:24.336865+0000","last_became_active":"2024-09-16T00:30:26.085076+0000","last_became_peered":"2024-09-16T00:30:26.085076+0000","last_unstale":"2024-09-16T00:32:24.336865+0000","last_undegraded":"2024-09-16T00:32:24.336865+0000","last_fullsized":"2024-09-16T00:32:24.336865+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:20:20.864660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.340070+0000","last_change":"2024-09-16T00:30:26.080953+0000","last_active":"2024-09-16T00:32:24.340070+0000","last_peered":"2024-09-16T00:32:24.340070+0000","last_clean":"2024-09-16T00:32:24.340070+0000","last_became_active":"2024-09-16T00:30:26.080185+0000","last_became_peered":"2024-09-16T00:30:26.080185+0000","last_unstale":"2024-09-16T00:32:24.340070+0000","last_undegraded":"2024-09-16T00:32:24.340070+0000","last_fullsized":"2024-09-16T00:32:24.340070+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:56.769182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561297+0000","last_change":"2024-09-16T00:30:26.079320+0000","last_active":"2024-09-16T00:32:24.561297+0000","last_peered":"2024-09-16T00:32:24.561297+0000","last_clean":"2024-09-16T00:32:24.561297+0000","last_became_active":"2024-09-16T00:30:26.078980+0000","last_became_peered":"2024-09-16T00:30:26.078980+0000","last_unstale":"2024-09-16T00:32:24.561297+0000","last_undegraded":"2024-09-16T00:32:24.561297+0000","last_fullsized":"2024-09-16T00:32:24.561297+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:03:15.386657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594661+0000","last_change":"2024-09-16T00:30:26.084948+0000","last_active":"2024-09-16T00:32:24.594661+0000","last_peered":"2024-09-16T00:32:24.594661+0000","last_clean":"2024-09-16T00:32:24.594661+0000","last_became_active":"2024-09-16T00:30:26.084602+0000","last_became_peered":"2024-09-16T00:30:26.084602+0000","last_unstale":"2024-09-16T00:32:24.594661+0000","last_undegraded":"2024-09-16T00:32:24.594661+0000","last_fullsized":"2024-09-16T00:32:24.594661+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:01.709332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594560+0000","last_change":"2024-09-16T00:30:26.080491+0000","last_active":"2024-09-16T00:32:24.594560+0000","last_peered":"2024-09-16T00:32:24.594560+0000","last_clean":"2024-09-16T00:32:24.594560+0000","last_became_active":"2024-09-16T00:30:26.080189+0000","last_became_peered":"2024-09-16T00:30:26.080189+0000","last_unstale":"2024-09-16T00:32:24.594560+0000","last_undegraded":"2024-09-16T00:32:24.594560+0000","last_fullsized":"2024-09-16T00:32:24.594560+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:38.358919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452035+0000","last_change":"2024-09-16T00:30:26.077410+0000","last_active":"2024-09-16T00:32:24.452035+0000","last_peered":"2024-09-16T00:32:24.452035+0000","last_clean":"2024-09-16T00:32:24.452035+0000","last_became_active":"2024-09-16T00:30:26.076498+0000","last_became_peered":"2024-09-16T00:30:26.076498+0000","last_unstale":"2024-09-16T00:32:24.452035+0000","last_undegraded":"2024-09-16T00:32:24.452035+0000","last_fullsized":"2024-09-16T00:32:24.452035+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:54:17.712774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454822+0000","last_change":"2024-09-16T00:30:26.074500+0000","last_active":"2024-09-16T00:32:24.454822+0000","last_peered":"2024-09-16T00:32:24.454822+0000","last_clean":"2024-09-16T00:32:24.454822+0000","last_became_active":"2024-09-16T00:30:26.074177+0000","last_became_peered":"2024-09-16T00:30:26.074177+0000","last_unstale":"2024-09-16T00:32:24.454822+0000","last_undegraded":"2024-09-16T00:32:24.454822+0000","last_fullsized":"2024-09-16T00:32:24.454822+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:43:13.792231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595503+0000","last_change":"2024-09-16T00:30:26.078358+0000","last_active":"2024-09-16T00:32:24.595503+0000","last_peered":"2024-09-16T00:32:24.595503+0000","last_clean":"2024-09-16T00:32:24.595503+0000","last_became_active":"2024-09-16T00:30:26.077993+0000","last_became_peered":"2024-09-16T00:30:26.077993+0000","last_unstale":"2024-09-16T00:32:24.595503+0000","last_undegraded":"2024-09-16T00:32:24.595503+0000","last_fullsized":"2024-09-16T00:32:24.595503+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:25:10.711394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.337671+0000","last_change":"2024-09-16T00:31:23.446836+0000","last_active":"2024-09-16T00:32:24.337671+0000","last_peered":"2024-09-16T00:32:24.337671+0000","last_clean":"2024-09-16T00:32:24.337671+0000","last_became_active":"2024-09-16T00:31:23.446471+0000","last_became_peered":"2024-09-16T00:31:23.446471+0000","last_unstale":"2024-09-16T00:32:24.337671+0000","last_undegraded":"2024-09-16T00:32:24.337671+0000","last_fullsized":"2024-09-16T00:32:24.337671+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:08:34.887618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561867+0000","last_change":"2024-09-16T00:30:26.086313+0000","last_active":"2024-09-16T00:32:24.561867+0000","last_peered":"2024-09-16T00:32:24.561867+0000","last_clean":"2024-09-16T00:32:24.561867+0000","last_became_active":"2024-09-16T00:30:26.085760+0000","last_became_peered":"2024-09-16T00:30:26.085760+0000","last_unstale":"2024-09-16T00:32:24.561867+0000","last_undegraded":"2024-09-16T00:32:24.561867+0000","last_fullsized":"2024-09-16T00:32:24.561867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:40:38.236789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594968+0000","last_change":"2024-09-16T00:30:26.085289+0000","last_active":"2024-09-16T00:32:24.594968+0000","last_peered":"2024-09-16T00:32:24.594968+0000","last_clean":"2024-09-16T00:32:24.594968+0000","last_became_active":"2024-09-16T00:30:26.084995+0000","last_became_peered":"2024-09-16T00:30:26.084995+0000","last_unstale":"2024-09-16T00:32:24.594968+0000","last_undegraded":"2024-09-16T00:32:24.594968+0000","last_fullsized":"2024-09-16T00:32:24.594968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:45.667892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594850+0000","last_change":"2024-09-16T00:30:26.088690+0000","last_active":"2024-09-16T00:32:24.594850+0000","last_peered":"2024-09-16T00:32:24.594850+0000","last_clean":"2024-09-16T00:32:24.594850+0000","last_became_active":"2024-09-16T00:30:26.087700+0000","last_became_peered":"2024-09-16T00:30:26.087700+0000","last_unstale":"2024-09-16T00:32:24.594850+0000","last_undegraded":"2024-09-16T00:32:24.594850+0000","last_fullsized":"2024-09-16T00:32:24.594850+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:39:51.445324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.337214+0000","last_change":"2024-09-16T00:30:26.080215+0000","last_active":"2024-09-16T00:32:24.337214+0000","last_peered":"2024-09-16T00:32:24.337214+0000","last_clean":"2024-09-16T00:32:24.337214+0000","last_became_active":"2024-09-16T00:30:26.079038+0000","last_became_peered":"2024-09-16T00:30:26.079038+0000","last_unstale":"2024-09-16T00:32:24.337214+0000","last_undegraded":"2024-09-16T00:32:24.337214+0000","last_fullsized":"2024-09-16T00:32:24.337214+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:01.573960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339169+0000","last_change":"2024-09-16T00:30:26.080966+0000","last_active":"2024-09-16T00:32:24.339169+0000","last_peered":"2024-09-16T00:32:24.339169+0000","last_clean":"2024-09-16T00:32:24.339169+0000","last_became_active":"2024-09-16T00:30:26.080641+0000","last_became_peered":"2024-09-16T00:30:26.080641+0000","last_unstale":"2024-09-16T00:32:24.339169+0000","last_undegraded":"2024-09-16T00:32:24.339169+0000","last_fullsized":"2024-09-16T00:32:24.339169+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:33:01.538169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454762+0000","last_change":"2024-09-16T00:30:26.072644+0000","last_active":"2024-09-16T00:32:24.454762+0000","last_peered":"2024-09-16T00:32:24.454762+0000","last_clean":"2024-09-16T00:32:24.454762+0000","last_became_active":"2024-09-16T00:30:26.072336+0000","last_became_peered":"2024-09-16T00:30:26.072336+0000","last_unstale":"2024-09-16T00:32:24.454762+0000","last_undegraded":"2024-09-16T00:32:24.454762+0000","last_fullsized":"2024-09-16T00:32:24.454762+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:44:45.760057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595215+0000","last_change":"2024-09-16T00:30:26.079155+0000","last_active":"2024-09-16T00:32:24.595215+0000","last_peered":"2024-09-16T00:32:24.595215+0000","last_clean":"2024-09-16T00:32:24.595215+0000","last_became_active":"2024-09-16T00:30:26.078820+0000","last_became_peered":"2024-09-16T00:30:26.078820+0000","last_unstale":"2024-09-16T00:32:24.595215+0000","last_undegraded":"2024-09-16T00:32:24.595215+0000","last_fullsized":"2024-09-16T00:32:24.595215+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:35.071143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595361+0000","last_change":"2024-09-16T00:30:26.081537+0000","last_active":"2024-09-16T00:32:24.595361+0000","last_peered":"2024-09-16T00:32:24.595361+0000","last_clean":"2024-09-16T00:32:24.595361+0000","last_became_active":"2024-09-16T00:30:26.080744+0000","last_became_peered":"2024-09-16T00:30:26.080744+0000","last_unstale":"2024-09-16T00:32:24.595361+0000","last_undegraded":"2024-09-16T00:32:24.595361+0000","last_fullsized":"2024-09-16T00:32:24.595361+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:46:38.179888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.340007+0000","last_change":"2024-09-16T00:30:26.081066+0000","last_active":"2024-09-16T00:32:24.340007+0000","last_peered":"2024-09-16T00:32:24.340007+0000","last_clean":"2024-09-16T00:32:24.340007+0000","last_became_active":"2024-09-16T00:30:26.080630+0000","last_became_peered":"2024-09-16T00:30:26.080630+0000","last_unstale":"2024-09-16T00:32:24.340007+0000","last_undegraded":"2024-09-16T00:32:24.340007+0000","last_fullsized":"2024-09-16T00:32:24.340007+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:44:42.644792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72936,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":637,"num_read_kb":399,"num_write":183,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":438272,"data_stored":212649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514538,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1060864,"data_stored":523210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743471,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32892,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712260,"statfs":{"total":95995035648,"available":95961354240,"internally_reserved":0,"allocated":1748992,"data_stored":1178821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906996,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":527486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135929,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1064960,"data_stored":523288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397566,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1662976,"data_stored":1119755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561092,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1048576,"data_stored":518566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790024,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28656,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716496,"statfs":{"total":95995035648,"available":95965691904,"internally_reserved":0,"allocated":1671168,"data_stored":1115343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953550,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1060864,"data_stored":523216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182481,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1060864,"data_stored":518760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1138688,"data_stored":582959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":586875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1073152,"data_stored":519954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":64483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":69977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:32:46.181 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-16T00:32:46.181 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-16T00:32:46.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.318+0000 7f64c1aa5640 1 -- 172.21.15.38:0/339511121 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64bc104ee0 msgr2=0x7f64bc1072d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:46.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.318+0000 7f64c1aa5640 1 --2- 172.21.15.38:0/339511121 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64bc104ee0 0x7f64bc1072d0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f64b0009900 tx=0x7f64b002a410 comp rx=0 tx=0).stop 2024-09-16T00:32:46.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.318+0000 7f64c1aa5640 1 -- 172.21.15.38:0/339511121 shutdown_connections 2024-09-16T00:32:46.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.318+0000 7f64c1aa5640 1 --2- 172.21.15.38:0/339511121 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64bc107810 0x7f64bc109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:46.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.318+0000 7f64c1aa5640 1 --2- 172.21.15.38:0/339511121 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64bc104ee0 0x7f64bc1072d0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:46.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.318+0000 7f64c1aa5640 1 --2- 172.21.15.38:0/339511121 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64bc1025b0 0x7f64bc1049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:46.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.318+0000 7f64c1aa5640 1 -- 172.21.15.38:0/339511121 >> 172.21.15.38:0/339511121 conn(0x7f64bc0fc1b0 msgr2=0x7f64bc0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:46.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.318+0000 7f64c1aa5640 1 -- 172.21.15.38:0/339511121 shutdown_connections 2024-09-16T00:32:46.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.318+0000 7f64c1aa5640 1 -- 172.21.15.38:0/339511121 wait complete. 2024-09-16T00:32:46.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.319+0000 7f64c1aa5640 1 Processor -- start 2024-09-16T00:32:46.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.319+0000 7f64c1aa5640 1 -- start start 2024-09-16T00:32:46.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.319+0000 7f64c1aa5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64bc1025b0 0x7f64bc101f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:46.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.319+0000 7f64c1aa5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64bc104ee0 0x7f64bc100560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:46.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.319+0000 7f64c1aa5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64bc107810 0x7f64bc100aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:46.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.319+0000 7f64c1aa5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc10d7e0 con 0x7f64bc1025b0 2024-09-16T00:32:46.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.319+0000 7f64c1aa5640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc10d660 con 0x7f64bc104ee0 2024-09-16T00:32:46.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.319+0000 7f64c1aa5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f64bc10d960 con 0x7f64bc107810 2024-09-16T00:32:46.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.319+0000 7f64bb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64bc1025b0 0x7f64bc101f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:46.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.319+0000 7f64bb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64bc1025b0 0x7f64bc101f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57186/0 (socket says 172.21.15.38:57186) 2024-09-16T00:32:46.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.319+0000 7f64bb7fe640 1 -- 172.21.15.38:0/941392366 learned_addr learned my addr 172.21.15.38:0/941392366 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:46.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.319+0000 7f64baffd640 1 --2- 172.21.15.38:0/941392366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64bc104ee0 0x7f64bc100560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:46.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.319+0000 7f64bb7fe640 1 -- 172.21.15.38:0/941392366 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64bc107810 msgr2=0x7f64bc100aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:46.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.319+0000 7f64bb7fe640 1 --2- 172.21.15.38:0/941392366 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64bc107810 0x7f64bc100aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:46.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.319+0000 7f64bb7fe640 1 -- 172.21.15.38:0/941392366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64bc104ee0 msgr2=0x7f64bc100560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:46.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.319+0000 7f64bb7fe640 1 --2- 172.21.15.38:0/941392366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64bc104ee0 0x7f64bc100560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:46.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.319+0000 7f64bb7fe640 1 -- 172.21.15.38:0/941392366 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64bc101330 con 0x7f64bc1025b0 2024-09-16T00:32:46.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.319+0000 7f64baffd640 1 --2- 172.21.15.38:0/941392366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64bc104ee0 0x7f64bc100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:32:46.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.320+0000 7f64bb7fe640 1 --2- 172.21.15.38:0/941392366 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64bc1025b0 0x7f64bc101f20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f64ac00d770 tx=0x7f64ac00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:46.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.320+0000 7f64b8ff9640 1 -- 172.21.15.38:0/941392366 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac017070 con 0x7f64bc1025b0 2024-09-16T00:32:46.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.320+0000 7f64c1aa5640 1 -- 172.21.15.38:0/941392366 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64bc1a1fc0 con 0x7f64bc1025b0 2024-09-16T00:32:46.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.320+0000 7f64b8ff9640 1 -- 172.21.15.38:0/941392366 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64ac005500 con 0x7f64bc1025b0 2024-09-16T00:32:46.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.320+0000 7f64b8ff9640 1 -- 172.21.15.38:0/941392366 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64ac004e00 con 0x7f64bc1025b0 2024-09-16T00:32:46.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.320+0000 7f64c1aa5640 1 -- 172.21.15.38:0/941392366 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64bc1a2420 con 0x7f64bc1025b0 2024-09-16T00:32:46.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.321+0000 7f64c1aa5640 1 -- 172.21.15.38:0/941392366 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64bc0fdca0 con 0x7f64bc1025b0 2024-09-16T00:32:46.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.322+0000 7f64b8ff9640 1 -- 172.21.15.38:0/941392366 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f64ac004fa0 con 0x7f64bc1025b0 2024-09-16T00:32:46.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.323+0000 7f64b8ff9640 1 --2- 172.21.15.38:0/941392366 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f64a40a32e0 0x7f64a40a57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:46.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.323+0000 7f64baffd640 1 --2- 172.21.15.38:0/941392366 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f64a40a32e0 0x7f64a40a57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:46.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.323+0000 7f64b8ff9640 1 -- 172.21.15.38:0/941392366 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f64ac0cfd00 con 0x7f64bc1025b0 2024-09-16T00:32:46.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.324+0000 7f64baffd640 1 --2- 172.21.15.38:0/941392366 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f64a40a32e0 0x7f64a40a57a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f64b0000c00 tx=0x7f64b0031400 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:46.327 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.326+0000 7f64b8ff9640 1 -- 172.21.15.38:0/941392366 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f64ac09c540 con 0x7f64bc1025b0 2024-09-16T00:32:46.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.499+0000 7f64c1aa5640 1 -- 172.21.15.38:0/941392366 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f64bc062630 con 0x7f64a40a32e0 2024-09-16T00:32:46.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.507+0000 7f64b8ff9640 1 -- 172.21.15.38:0/941392366 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340493 (secure 0 0 0) 0x7f64bc062630 con 0x7f64a40a32e0 2024-09-16T00:32:46.512 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:32:46.514 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-16T00:32:46.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.510+0000 7f64c1aa5640 1 -- 172.21.15.38:0/941392366 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f64a40a32e0 msgr2=0x7f64a40a57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:46.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.510+0000 7f64c1aa5640 1 --2- 172.21.15.38:0/941392366 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f64a40a32e0 0x7f64a40a57a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f64b0000c00 tx=0x7f64b0031400 comp rx=0 tx=0).stop 2024-09-16T00:32:46.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.510+0000 7f64c1aa5640 1 -- 172.21.15.38:0/941392366 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64bc1025b0 msgr2=0x7f64bc101f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:46.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.510+0000 7f64c1aa5640 1 --2- 172.21.15.38:0/941392366 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64bc1025b0 0x7f64bc101f20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f64ac00d770 tx=0x7f64ac00dc30 comp rx=0 tx=0).stop 2024-09-16T00:32:46.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.510+0000 7f64c1aa5640 1 -- 172.21.15.38:0/941392366 shutdown_connections 2024-09-16T00:32:46.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.510+0000 7f64c1aa5640 1 --2- 172.21.15.38:0/941392366 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f64a40a32e0 0x7f64a40a57a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:46.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.510+0000 7f64c1aa5640 1 --2- 172.21.15.38:0/941392366 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64bc107810 0x7f64bc100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:46.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.510+0000 7f64c1aa5640 1 --2- 172.21.15.38:0/941392366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64bc104ee0 0x7f64bc100560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:46.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.510+0000 7f64c1aa5640 1 --2- 172.21.15.38:0/941392366 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f64bc1025b0 0x7f64bc101f20 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:46.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.510+0000 7f64c1aa5640 1 -- 172.21.15.38:0/941392366 >> 172.21.15.38:0/941392366 conn(0x7f64bc0fc1b0 msgr2=0x7f64bc1030b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:46.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.510+0000 7f64c1aa5640 1 -- 172.21.15.38:0/941392366 shutdown_connections 2024-09-16T00:32:46.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:46.511+0000 7f64c1aa5640 1 -- 172.21.15.38:0/941392366 wait complete. 2024-09-16T00:32:46.523 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":364,"stamp":"2024-09-16T00:32:45.543344+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663538,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":763,"num_read_kb":508,"num_write":366,"num_write_kb":3073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1180792,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":795,"ondisk_log_size":795,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380376,"kb_used_data":14440,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561448,"statfs":{"total":1151940427776,"available":1151550922752,"internally_reserved":0,"allocated":14786560,"data_stored":8238233,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6467,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":83,"num_read_kb":53,"num_write":10,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003434"},"pg_stats":[{"pgid":"2.1f","version":"88'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.337120+0000","last_change":"2024-09-16T00:30:23.933355+0000","last_active":"2024-09-16T00:32:24.337120+0000","last_peered":"2024-09-16T00:32:24.337120+0000","last_clean":"2024-09-16T00:32:24.337120+0000","last_became_active":"2024-09-16T00:30:23.933057+0000","last_became_peered":"2024-09-16T00:30:23.933057+0000","last_unstale":"2024-09-16T00:32:24.337120+0000","last_undegraded":"2024-09-16T00:32:24.337120+0000","last_fullsized":"2024-09-16T00:32:24.337120+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:23:01.713440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.596165+0000","last_change":"2024-09-16T00:30:26.084537+0000","last_active":"2024-09-16T00:32:24.596165+0000","last_peered":"2024-09-16T00:32:24.596165+0000","last_clean":"2024-09-16T00:32:24.596165+0000","last_became_active":"2024-09-16T00:30:26.084226+0000","last_became_peered":"2024-09-16T00:30:26.084226+0000","last_unstale":"2024-09-16T00:32:24.596165+0000","last_undegraded":"2024-09-16T00:32:24.596165+0000","last_fullsized":"2024-09-16T00:32:24.596165+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:43:41.628877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594728+0000","last_change":"2024-09-16T00:30:33.154873+0000","last_active":"2024-09-16T00:32:24.594728+0000","last_peered":"2024-09-16T00:32:24.594728+0000","last_clean":"2024-09-16T00:32:24.594728+0000","last_became_active":"2024-09-16T00:30:33.154433+0000","last_became_peered":"2024-09-16T00:30:33.154433+0000","last_unstale":"2024-09-16T00:32:24.594728+0000","last_undegraded":"2024-09-16T00:32:24.594728+0000","last_fullsized":"2024-09-16T00:32:24.594728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:50.805806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.596255+0000","last_change":"2024-09-16T00:30:24.422274+0000","last_active":"2024-09-16T00:32:24.596255+0000","last_peered":"2024-09-16T00:32:24.596255+0000","last_clean":"2024-09-16T00:32:24.596255+0000","last_became_active":"2024-09-16T00:30:24.421902+0000","last_became_peered":"2024-09-16T00:30:24.421902+0000","last_unstale":"2024-09-16T00:32:24.596255+0000","last_undegraded":"2024-09-16T00:32:24.596255+0000","last_fullsized":"2024-09-16T00:32:24.596255+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:01.910187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595568+0000","last_change":"2024-09-16T00:30:26.088165+0000","last_active":"2024-09-16T00:32:24.595568+0000","last_peered":"2024-09-16T00:32:24.595568+0000","last_clean":"2024-09-16T00:32:24.595568+0000","last_became_active":"2024-09-16T00:30:26.087593+0000","last_became_peered":"2024-09-16T00:30:26.087593+0000","last_unstale":"2024-09-16T00:32:24.595568+0000","last_undegraded":"2024-09-16T00:32:24.595568+0000","last_fullsized":"2024-09-16T00:32:24.595568+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:54.149930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454083+0000","last_change":"2024-09-16T00:30:33.154586+0000","last_active":"2024-09-16T00:32:24.454083+0000","last_peered":"2024-09-16T00:32:24.454083+0000","last_clean":"2024-09-16T00:32:24.454083+0000","last_became_active":"2024-09-16T00:30:33.154209+0000","last_became_peered":"2024-09-16T00:30:33.154209+0000","last_unstale":"2024-09-16T00:32:24.454083+0000","last_undegraded":"2024-09-16T00:32:24.454083+0000","last_fullsized":"2024-09-16T00:32:24.454083+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:45.776895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339220+0000","last_change":"2024-09-16T00:30:23.932442+0000","last_active":"2024-09-16T00:32:24.339220+0000","last_peered":"2024-09-16T00:32:24.339220+0000","last_clean":"2024-09-16T00:32:24.339220+0000","last_became_active":"2024-09-16T00:30:23.932009+0000","last_became_peered":"2024-09-16T00:30:23.932009+0000","last_unstale":"2024-09-16T00:32:24.339220+0000","last_undegraded":"2024-09-16T00:32:24.339220+0000","last_fullsized":"2024-09-16T00:32:24.339220+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:09:27.078187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561572+0000","last_change":"2024-09-16T00:30:26.087715+0000","last_active":"2024-09-16T00:32:24.561572+0000","last_peered":"2024-09-16T00:32:24.561572+0000","last_clean":"2024-09-16T00:32:24.561572+0000","last_became_active":"2024-09-16T00:30:26.087415+0000","last_became_peered":"2024-09-16T00:30:26.087415+0000","last_unstale":"2024-09-16T00:32:24.561572+0000","last_undegraded":"2024-09-16T00:32:24.561572+0000","last_fullsized":"2024-09-16T00:32:24.561572+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:52.373992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454447+0000","last_change":"2024-09-16T00:30:33.152328+0000","last_active":"2024-09-16T00:32:24.454447+0000","last_peered":"2024-09-16T00:32:24.454447+0000","last_clean":"2024-09-16T00:32:24.454447+0000","last_became_active":"2024-09-16T00:30:33.151954+0000","last_became_peered":"2024-09-16T00:30:33.151954+0000","last_unstale":"2024-09-16T00:32:24.454447+0000","last_undegraded":"2024-09-16T00:32:24.454447+0000","last_fullsized":"2024-09-16T00:32:24.454447+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:36.546729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339346+0000","last_change":"2024-09-16T00:30:23.928988+0000","last_active":"2024-09-16T00:32:24.339346+0000","last_peered":"2024-09-16T00:32:24.339346+0000","last_clean":"2024-09-16T00:32:24.339346+0000","last_became_active":"2024-09-16T00:30:23.928638+0000","last_became_peered":"2024-09-16T00:30:23.928638+0000","last_unstale":"2024-09-16T00:32:24.339346+0000","last_undegraded":"2024-09-16T00:32:24.339346+0000","last_fullsized":"2024-09-16T00:32:24.339346+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:45:58.203455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.336619+0000","last_change":"2024-09-16T00:31:23.446550+0000","last_active":"2024-09-16T00:32:24.336619+0000","last_peered":"2024-09-16T00:32:24.336619+0000","last_clean":"2024-09-16T00:32:24.336619+0000","last_became_active":"2024-09-16T00:31:23.446179+0000","last_became_peered":"2024-09-16T00:31:23.446179+0000","last_unstale":"2024-09-16T00:32:24.336619+0000","last_undegraded":"2024-09-16T00:32:24.336619+0000","last_fullsized":"2024-09-16T00:32:24.336619+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:04:21.280174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454185+0000","last_change":"2024-09-16T00:30:33.147564+0000","last_active":"2024-09-16T00:32:24.454185+0000","last_peered":"2024-09-16T00:32:24.454185+0000","last_clean":"2024-09-16T00:32:24.454185+0000","last_became_active":"2024-09-16T00:30:33.147119+0000","last_became_peered":"2024-09-16T00:30:33.147119+0000","last_unstale":"2024-09-16T00:32:24.454185+0000","last_undegraded":"2024-09-16T00:32:24.454185+0000","last_fullsized":"2024-09-16T00:32:24.454185+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:00.509122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595076+0000","last_change":"2024-09-16T00:30:26.080954+0000","last_active":"2024-09-16T00:32:24.595076+0000","last_peered":"2024-09-16T00:32:24.595076+0000","last_clean":"2024-09-16T00:32:24.595076+0000","last_became_active":"2024-09-16T00:30:26.080624+0000","last_became_peered":"2024-09-16T00:30:26.080624+0000","last_unstale":"2024-09-16T00:32:24.595076+0000","last_undegraded":"2024-09-16T00:32:24.595076+0000","last_fullsized":"2024-09-16T00:32:24.595076+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:05:10.440980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339169+0000","last_change":"2024-09-16T00:30:23.933715+0000","last_active":"2024-09-16T00:32:24.339169+0000","last_peered":"2024-09-16T00:32:24.339169+0000","last_clean":"2024-09-16T00:32:24.339169+0000","last_became_active":"2024-09-16T00:30:23.933275+0000","last_became_peered":"2024-09-16T00:30:23.933275+0000","last_unstale":"2024-09-16T00:32:24.339169+0000","last_undegraded":"2024-09-16T00:32:24.339169+0000","last_fullsized":"2024-09-16T00:32:24.339169+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:40:44.846053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454555+0000","last_change":"2024-09-16T00:30:33.604602+0000","last_active":"2024-09-16T00:32:24.454555+0000","last_peered":"2024-09-16T00:32:24.454555+0000","last_clean":"2024-09-16T00:32:24.454555+0000","last_became_active":"2024-09-16T00:30:33.604107+0000","last_became_peered":"2024-09-16T00:30:33.604107+0000","last_unstale":"2024-09-16T00:32:24.454555+0000","last_undegraded":"2024-09-16T00:32:24.454555+0000","last_fullsized":"2024-09-16T00:32:24.454555+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:41.254445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595459+0000","last_change":"2024-09-16T00:30:26.080612+0000","last_active":"2024-09-16T00:32:24.595459+0000","last_peered":"2024-09-16T00:32:24.595459+0000","last_clean":"2024-09-16T00:32:24.595459+0000","last_became_active":"2024-09-16T00:30:26.080317+0000","last_became_peered":"2024-09-16T00:30:26.080317+0000","last_unstale":"2024-09-16T00:32:24.595459+0000","last_undegraded":"2024-09-16T00:32:24.595459+0000","last_fullsized":"2024-09-16T00:32:24.595459+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:23.814241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594893+0000","last_change":"2024-09-16T00:30:23.936778+0000","last_active":"2024-09-16T00:32:24.594893+0000","last_peered":"2024-09-16T00:32:24.594893+0000","last_clean":"2024-09-16T00:32:24.594893+0000","last_became_active":"2024-09-16T00:30:23.936265+0000","last_became_peered":"2024-09-16T00:30:23.936265+0000","last_unstale":"2024-09-16T00:32:24.594893+0000","last_undegraded":"2024-09-16T00:32:24.594893+0000","last_fullsized":"2024-09-16T00:32:24.594893+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:35:28.288476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454520+0000","last_change":"2024-09-16T00:30:33.148340+0000","last_active":"2024-09-16T00:32:24.454520+0000","last_peered":"2024-09-16T00:32:24.454520+0000","last_clean":"2024-09-16T00:32:24.454520+0000","last_became_active":"2024-09-16T00:30:33.147289+0000","last_became_peered":"2024-09-16T00:30:33.147289+0000","last_unstale":"2024-09-16T00:32:24.454520+0000","last_undegraded":"2024-09-16T00:32:24.454520+0000","last_fullsized":"2024-09-16T00:32:24.454520+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:21.259254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.338936+0000","last_change":"2024-09-16T00:30:26.077325+0000","last_active":"2024-09-16T00:32:24.338936+0000","last_peered":"2024-09-16T00:32:24.338936+0000","last_clean":"2024-09-16T00:32:24.338936+0000","last_became_active":"2024-09-16T00:30:26.077002+0000","last_became_peered":"2024-09-16T00:30:26.077002+0000","last_unstale":"2024-09-16T00:32:24.338936+0000","last_undegraded":"2024-09-16T00:32:24.338936+0000","last_fullsized":"2024-09-16T00:32:24.338936+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:12:38.569247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595646+0000","last_change":"2024-09-16T00:30:23.933025+0000","last_active":"2024-09-16T00:32:24.595646+0000","last_peered":"2024-09-16T00:32:24.595646+0000","last_clean":"2024-09-16T00:32:24.595646+0000","last_became_active":"2024-09-16T00:30:23.932669+0000","last_became_peered":"2024-09-16T00:30:23.932669+0000","last_unstale":"2024-09-16T00:32:24.595646+0000","last_undegraded":"2024-09-16T00:32:24.595646+0000","last_fullsized":"2024-09-16T00:32:24.595646+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:20.593665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.597170+0000","last_change":"2024-09-16T00:32:24.597170+0000","last_active":"2024-09-16T00:32:24.597170+0000","last_peered":"2024-09-16T00:32:24.597170+0000","last_clean":"2024-09-16T00:32:24.597170+0000","last_became_active":"2024-09-16T00:32:24.596810+0000","last_became_peered":"2024-09-16T00:32:24.596810+0000","last_unstale":"2024-09-16T00:32:24.597170+0000","last_undegraded":"2024-09-16T00:32:24.597170+0000","last_fullsized":"2024-09-16T00:32:24.597170+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:26:18.737332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.338711+0000","last_change":"2024-09-16T00:30:26.076104+0000","last_active":"2024-09-16T00:32:24.338711+0000","last_peered":"2024-09-16T00:32:24.338711+0000","last_clean":"2024-09-16T00:32:24.338711+0000","last_became_active":"2024-09-16T00:30:26.075764+0000","last_became_peered":"2024-09-16T00:30:26.075764+0000","last_unstale":"2024-09-16T00:32:24.338711+0000","last_undegraded":"2024-09-16T00:32:24.338711+0000","last_fullsized":"2024-09-16T00:32:24.338711+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:31.228533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561411+0000","last_change":"2024-09-16T00:30:24.422411+0000","last_active":"2024-09-16T00:32:24.561411+0000","last_peered":"2024-09-16T00:32:24.561411+0000","last_clean":"2024-09-16T00:32:24.561411+0000","last_became_active":"2024-09-16T00:30:24.422102+0000","last_became_peered":"2024-09-16T00:30:24.422102+0000","last_unstale":"2024-09-16T00:32:24.561411+0000","last_undegraded":"2024-09-16T00:32:24.561411+0000","last_fullsized":"2024-09-16T00:32:24.561411+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:06:00.572070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454559+0000","last_change":"2024-09-16T00:30:33.605397+0000","last_active":"2024-09-16T00:32:24.454559+0000","last_peered":"2024-09-16T00:32:24.454559+0000","last_clean":"2024-09-16T00:32:24.454559+0000","last_became_active":"2024-09-16T00:30:33.604978+0000","last_became_peered":"2024-09-16T00:30:33.604978+0000","last_unstale":"2024-09-16T00:32:24.454559+0000","last_undegraded":"2024-09-16T00:32:24.454559+0000","last_fullsized":"2024-09-16T00:32:24.454559+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:35:16.766056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595380+0000","last_change":"2024-09-16T00:30:26.082684+0000","last_active":"2024-09-16T00:32:24.595380+0000","last_peered":"2024-09-16T00:32:24.595380+0000","last_clean":"2024-09-16T00:32:24.595380+0000","last_became_active":"2024-09-16T00:30:26.082335+0000","last_became_peered":"2024-09-16T00:30:26.082335+0000","last_unstale":"2024-09-16T00:32:24.595380+0000","last_undegraded":"2024-09-16T00:32:24.595380+0000","last_fullsized":"2024-09-16T00:32:24.595380+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:51:21.389627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454909+0000","last_change":"2024-09-16T00:30:24.587840+0000","last_active":"2024-09-16T00:32:24.454909+0000","last_peered":"2024-09-16T00:32:24.454909+0000","last_clean":"2024-09-16T00:32:24.454909+0000","last_became_active":"2024-09-16T00:30:24.587436+0000","last_became_peered":"2024-09-16T00:30:24.587436+0000","last_unstale":"2024-09-16T00:32:24.454909+0000","last_undegraded":"2024-09-16T00:32:24.454909+0000","last_fullsized":"2024-09-16T00:32:24.454909+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:06.758585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594628+0000","last_change":"2024-09-16T00:30:33.145051+0000","last_active":"2024-09-16T00:32:24.594628+0000","last_peered":"2024-09-16T00:32:24.594628+0000","last_clean":"2024-09-16T00:32:24.594628+0000","last_became_active":"2024-09-16T00:30:33.144585+0000","last_became_peered":"2024-09-16T00:30:33.144585+0000","last_unstale":"2024-09-16T00:32:24.594628+0000","last_undegraded":"2024-09-16T00:32:24.594628+0000","last_fullsized":"2024-09-16T00:32:24.594628+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:48:23.588495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.336844+0000","last_change":"2024-09-16T00:31:23.444802+0000","last_active":"2024-09-16T00:32:24.336844+0000","last_peered":"2024-09-16T00:32:24.336844+0000","last_clean":"2024-09-16T00:32:24.336844+0000","last_became_active":"2024-09-16T00:31:23.444447+0000","last_became_peered":"2024-09-16T00:31:23.444447+0000","last_unstale":"2024-09-16T00:32:24.336844+0000","last_undegraded":"2024-09-16T00:32:24.336844+0000","last_fullsized":"2024-09-16T00:32:24.336844+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:46:37.285250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.6","version":"92'14","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:41.088268+0000","last_change":"2024-09-16T00:30:23.933284+0000","last_active":"2024-09-16T00:32:41.088268+0000","last_peered":"2024-09-16T00:32:41.088268+0000","last_clean":"2024-09-16T00:32:41.088268+0000","last_became_active":"2024-09-16T00:30:23.932920+0000","last_became_peered":"2024-09-16T00:30:23.932920+0000","last_unstale":"2024-09-16T00:32:41.088268+0000","last_undegraded":"2024-09-16T00:32:41.088268+0000","last_fullsized":"2024-09-16T00:32:41.088268+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:38.199391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594942+0000","last_change":"2024-09-16T00:30:33.604308+0000","last_active":"2024-09-16T00:32:24.594942+0000","last_peered":"2024-09-16T00:32:24.594942+0000","last_clean":"2024-09-16T00:32:24.594942+0000","last_became_active":"2024-09-16T00:30:33.603889+0000","last_became_peered":"2024-09-16T00:30:33.603889+0000","last_unstale":"2024-09-16T00:32:24.594942+0000","last_undegraded":"2024-09-16T00:32:24.594942+0000","last_fullsized":"2024-09-16T00:32:24.594942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:32:55.173063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452512+0000","last_change":"2024-09-16T00:30:26.078515+0000","last_active":"2024-09-16T00:32:24.452512+0000","last_peered":"2024-09-16T00:32:24.452512+0000","last_clean":"2024-09-16T00:32:24.452512+0000","last_became_active":"2024-09-16T00:30:26.078173+0000","last_became_peered":"2024-09-16T00:30:26.078173+0000","last_unstale":"2024-09-16T00:32:24.452512+0000","last_undegraded":"2024-09-16T00:32:24.452512+0000","last_fullsized":"2024-09-16T00:32:24.452512+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:40:31.471154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339304+0000","last_change":"2024-09-16T00:30:24.423209+0000","last_active":"2024-09-16T00:32:24.339304+0000","last_peered":"2024-09-16T00:32:24.339304+0000","last_clean":"2024-09-16T00:32:24.339304+0000","last_became_active":"2024-09-16T00:30:24.422157+0000","last_became_peered":"2024-09-16T00:30:24.422157+0000","last_unstale":"2024-09-16T00:32:24.339304+0000","last_undegraded":"2024-09-16T00:32:24.339304+0000","last_fullsized":"2024-09-16T00:32:24.339304+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:28.262392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339327+0000","last_change":"2024-09-16T00:30:33.146204+0000","last_active":"2024-09-16T00:32:24.339327+0000","last_peered":"2024-09-16T00:32:24.339327+0000","last_clean":"2024-09-16T00:32:24.339327+0000","last_became_active":"2024-09-16T00:30:33.145553+0000","last_became_peered":"2024-09-16T00:30:33.145553+0000","last_unstale":"2024-09-16T00:32:24.339327+0000","last_undegraded":"2024-09-16T00:32:24.339327+0000","last_fullsized":"2024-09-16T00:32:24.339327+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:22.096050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.337437+0000","last_change":"2024-09-16T00:30:26.070153+0000","last_active":"2024-09-16T00:32:24.337437+0000","last_peered":"2024-09-16T00:32:24.337437+0000","last_clean":"2024-09-16T00:32:24.337437+0000","last_became_active":"2024-09-16T00:30:26.069792+0000","last_became_peered":"2024-09-16T00:30:26.069792+0000","last_unstale":"2024-09-16T00:32:24.337437+0000","last_undegraded":"2024-09-16T00:32:24.337437+0000","last_fullsized":"2024-09-16T00:32:24.337437+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:28.164242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.336815+0000","last_change":"2024-09-16T00:30:24.423261+0000","last_active":"2024-09-16T00:32:24.336815+0000","last_peered":"2024-09-16T00:32:24.336815+0000","last_clean":"2024-09-16T00:32:24.336815+0000","last_became_active":"2024-09-16T00:30:24.422650+0000","last_became_peered":"2024-09-16T00:30:24.422650+0000","last_unstale":"2024-09-16T00:32:24.336815+0000","last_undegraded":"2024-09-16T00:32:24.336815+0000","last_fullsized":"2024-09-16T00:32:24.336815+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:00.211744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339762+0000","last_change":"2024-09-16T00:30:33.604492+0000","last_active":"2024-09-16T00:32:24.339762+0000","last_peered":"2024-09-16T00:32:24.339762+0000","last_clean":"2024-09-16T00:32:24.339762+0000","last_became_active":"2024-09-16T00:30:33.604015+0000","last_became_peered":"2024-09-16T00:30:33.604015+0000","last_unstale":"2024-09-16T00:32:24.339762+0000","last_undegraded":"2024-09-16T00:32:24.339762+0000","last_fullsized":"2024-09-16T00:32:24.339762+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:40:22.516583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.337389+0000","last_change":"2024-09-16T00:30:26.075517+0000","last_active":"2024-09-16T00:32:24.337389+0000","last_peered":"2024-09-16T00:32:24.337389+0000","last_clean":"2024-09-16T00:32:24.337389+0000","last_became_active":"2024-09-16T00:30:26.075113+0000","last_became_peered":"2024-09-16T00:30:26.075113+0000","last_unstale":"2024-09-16T00:32:24.337389+0000","last_undegraded":"2024-09-16T00:32:24.337389+0000","last_fullsized":"2024-09-16T00:32:24.337389+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:11.427010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595204+0000","last_change":"2024-09-16T00:30:23.933639+0000","last_active":"2024-09-16T00:32:24.595204+0000","last_peered":"2024-09-16T00:32:24.595204+0000","last_clean":"2024-09-16T00:32:24.595204+0000","last_became_active":"2024-09-16T00:30:23.933322+0000","last_became_peered":"2024-09-16T00:30:23.933322+0000","last_unstale":"2024-09-16T00:32:24.595204+0000","last_undegraded":"2024-09-16T00:32:24.595204+0000","last_fullsized":"2024-09-16T00:32:24.595204+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:59:59.082702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452139+0000","last_change":"2024-09-16T00:30:33.148508+0000","last_active":"2024-09-16T00:32:24.452139+0000","last_peered":"2024-09-16T00:32:24.452139+0000","last_clean":"2024-09-16T00:32:24.452139+0000","last_became_active":"2024-09-16T00:30:33.148091+0000","last_became_peered":"2024-09-16T00:30:33.148091+0000","last_unstale":"2024-09-16T00:32:24.452139+0000","last_undegraded":"2024-09-16T00:32:24.452139+0000","last_fullsized":"2024-09-16T00:32:24.452139+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:50.052526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454239+0000","last_change":"2024-09-16T00:30:26.080374+0000","last_active":"2024-09-16T00:32:24.454239+0000","last_peered":"2024-09-16T00:32:24.454239+0000","last_clean":"2024-09-16T00:32:24.454239+0000","last_became_active":"2024-09-16T00:30:26.080044+0000","last_became_peered":"2024-09-16T00:30:26.080044+0000","last_unstale":"2024-09-16T00:32:24.454239+0000","last_undegraded":"2024-09-16T00:32:24.454239+0000","last_fullsized":"2024-09-16T00:32:24.454239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:38:36.034903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":145,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:43.480306+0000","last_change":"2024-09-16T00:30:24.422137+0000","last_active":"2024-09-16T00:32:43.480306+0000","last_peered":"2024-09-16T00:32:43.480306+0000","last_clean":"2024-09-16T00:32:43.480306+0000","last_became_active":"2024-09-16T00:30:24.421778+0000","last_became_peered":"2024-09-16T00:30:24.421778+0000","last_unstale":"2024-09-16T00:32:43.480306+0000","last_undegraded":"2024-09-16T00:32:43.480306+0000","last_fullsized":"2024-09-16T00:32:43.480306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:54:21.964099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452309+0000","last_change":"2024-09-16T00:30:33.604263+0000","last_active":"2024-09-16T00:32:24.452309+0000","last_peered":"2024-09-16T00:32:24.452309+0000","last_clean":"2024-09-16T00:32:24.452309+0000","last_became_active":"2024-09-16T00:30:33.603804+0000","last_became_peered":"2024-09-16T00:30:33.603804+0000","last_unstale":"2024-09-16T00:32:24.452309+0000","last_undegraded":"2024-09-16T00:32:24.452309+0000","last_fullsized":"2024-09-16T00:32:24.452309+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:08:37.590827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.451792+0000","last_change":"2024-09-16T00:30:26.082916+0000","last_active":"2024-09-16T00:32:24.451792+0000","last_peered":"2024-09-16T00:32:24.451792+0000","last_clean":"2024-09-16T00:32:24.451792+0000","last_became_active":"2024-09-16T00:30:26.082606+0000","last_became_peered":"2024-09-16T00:30:26.082606+0000","last_unstale":"2024-09-16T00:32:24.451792+0000","last_undegraded":"2024-09-16T00:32:24.451792+0000","last_fullsized":"2024-09-16T00:32:24.451792+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:56.777640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594991+0000","last_change":"2024-09-16T00:30:23.937602+0000","last_active":"2024-09-16T00:32:24.594991+0000","last_peered":"2024-09-16T00:32:24.594991+0000","last_clean":"2024-09-16T00:32:24.594991+0000","last_became_active":"2024-09-16T00:30:23.937261+0000","last_became_peered":"2024-09-16T00:30:23.937261+0000","last_unstale":"2024-09-16T00:32:24.594991+0000","last_undegraded":"2024-09-16T00:32:24.594991+0000","last_fullsized":"2024-09-16T00:32:24.594991+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:24.698758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454567+0000","last_change":"2024-09-16T00:30:33.605536+0000","last_active":"2024-09-16T00:32:24.454567+0000","last_peered":"2024-09-16T00:32:24.454567+0000","last_clean":"2024-09-16T00:32:24.454567+0000","last_became_active":"2024-09-16T00:30:33.605073+0000","last_became_peered":"2024-09-16T00:30:33.605073+0000","last_unstale":"2024-09-16T00:32:24.454567+0000","last_undegraded":"2024-09-16T00:32:24.454567+0000","last_fullsized":"2024-09-16T00:32:24.454567+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:11.143052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595152+0000","last_change":"2024-09-16T00:30:26.084507+0000","last_active":"2024-09-16T00:32:24.595152+0000","last_peered":"2024-09-16T00:32:24.595152+0000","last_clean":"2024-09-16T00:32:24.595152+0000","last_became_active":"2024-09-16T00:30:26.084152+0000","last_became_peered":"2024-09-16T00:30:26.084152+0000","last_unstale":"2024-09-16T00:32:24.595152+0000","last_undegraded":"2024-09-16T00:32:24.595152+0000","last_fullsized":"2024-09-16T00:32:24.595152+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:27:21.557409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454701+0000","last_change":"2024-09-16T00:30:23.932686+0000","last_active":"2024-09-16T00:32:24.454701+0000","last_peered":"2024-09-16T00:32:24.454701+0000","last_clean":"2024-09-16T00:32:24.454701+0000","last_became_active":"2024-09-16T00:30:23.932343+0000","last_became_peered":"2024-09-16T00:30:23.932343+0000","last_unstale":"2024-09-16T00:32:24.454701+0000","last_undegraded":"2024-09-16T00:32:24.454701+0000","last_fullsized":"2024-09-16T00:32:24.454701+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:13:44.185712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":351,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595072+0000","last_change":"2024-09-16T00:29:11.143642+0000","last_active":"2024-09-16T00:32:24.595072+0000","last_peered":"2024-09-16T00:32:24.595072+0000","last_clean":"2024-09-16T00:32:24.595072+0000","last_became_active":"2024-09-16T00:29:11.143227+0000","last_became_peered":"2024-09-16T00:29:11.143227+0000","last_unstale":"2024-09-16T00:32:24.595072+0000","last_undegraded":"2024-09-16T00:32:24.595072+0000","last_fullsized":"2024-09-16T00:32:24.595072+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:25:42.361316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:25:42.361316+0000","last_clean_scrub_stamp":"2024-09-16T00:25:42.361316+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:29:12.740474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595397+0000","last_change":"2024-09-16T00:30:33.155555+0000","last_active":"2024-09-16T00:32:24.595397+0000","last_peered":"2024-09-16T00:32:24.595397+0000","last_clean":"2024-09-16T00:32:24.595397+0000","last_became_active":"2024-09-16T00:30:33.155170+0000","last_became_peered":"2024-09-16T00:30:33.155170+0000","last_unstale":"2024-09-16T00:32:24.595397+0000","last_undegraded":"2024-09-16T00:32:24.595397+0000","last_fullsized":"2024-09-16T00:32:24.595397+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:34:46.214422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595573+0000","last_change":"2024-09-16T00:30:26.076003+0000","last_active":"2024-09-16T00:32:24.595573+0000","last_peered":"2024-09-16T00:32:24.595573+0000","last_clean":"2024-09-16T00:32:24.595573+0000","last_became_active":"2024-09-16T00:30:26.075666+0000","last_became_peered":"2024-09-16T00:30:26.075666+0000","last_unstale":"2024-09-16T00:32:24.595573+0000","last_undegraded":"2024-09-16T00:32:24.595573+0000","last_fullsized":"2024-09-16T00:32:24.595573+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:30.113380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594420+0000","last_change":"2024-09-16T00:30:24.587469+0000","last_active":"2024-09-16T00:32:24.594420+0000","last_peered":"2024-09-16T00:32:24.594420+0000","last_clean":"2024-09-16T00:32:24.594420+0000","last_became_active":"2024-09-16T00:30:24.587049+0000","last_became_peered":"2024-09-16T00:30:24.587049+0000","last_unstale":"2024-09-16T00:32:24.594420+0000","last_undegraded":"2024-09-16T00:32:24.594420+0000","last_fullsized":"2024-09-16T00:32:24.594420+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:17:45.934505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561433+0000","last_change":"2024-09-16T00:30:33.145921+0000","last_active":"2024-09-16T00:32:24.561433+0000","last_peered":"2024-09-16T00:32:24.561433+0000","last_clean":"2024-09-16T00:32:24.561433+0000","last_became_active":"2024-09-16T00:30:33.145450+0000","last_became_peered":"2024-09-16T00:30:33.145450+0000","last_unstale":"2024-09-16T00:32:24.561433+0000","last_undegraded":"2024-09-16T00:32:24.561433+0000","last_fullsized":"2024-09-16T00:32:24.561433+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:53:51.698686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595330+0000","last_change":"2024-09-16T00:30:26.087940+0000","last_active":"2024-09-16T00:32:24.595330+0000","last_peered":"2024-09-16T00:32:24.595330+0000","last_clean":"2024-09-16T00:32:24.595330+0000","last_became_active":"2024-09-16T00:30:26.087599+0000","last_became_peered":"2024-09-16T00:30:26.087599+0000","last_unstale":"2024-09-16T00:32:24.595330+0000","last_undegraded":"2024-09-16T00:32:24.595330+0000","last_fullsized":"2024-09-16T00:32:24.595330+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:43.934839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454847+0000","last_change":"2024-09-16T00:30:23.930986+0000","last_active":"2024-09-16T00:32:24.454847+0000","last_peered":"2024-09-16T00:32:24.454847+0000","last_clean":"2024-09-16T00:32:24.454847+0000","last_became_active":"2024-09-16T00:30:23.930588+0000","last_became_peered":"2024-09-16T00:30:23.930588+0000","last_unstale":"2024-09-16T00:32:24.454847+0000","last_undegraded":"2024-09-16T00:32:24.454847+0000","last_fullsized":"2024-09-16T00:32:24.454847+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:04:41.902358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594672+0000","last_change":"2024-09-16T00:30:33.154894+0000","last_active":"2024-09-16T00:32:24.594672+0000","last_peered":"2024-09-16T00:32:24.594672+0000","last_clean":"2024-09-16T00:32:24.594672+0000","last_became_active":"2024-09-16T00:30:33.154490+0000","last_became_peered":"2024-09-16T00:30:33.154490+0000","last_unstale":"2024-09-16T00:32:24.594672+0000","last_undegraded":"2024-09-16T00:32:24.594672+0000","last_fullsized":"2024-09-16T00:32:24.594672+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:59:53.854212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595549+0000","last_change":"2024-09-16T00:30:26.077069+0000","last_active":"2024-09-16T00:32:24.595549+0000","last_peered":"2024-09-16T00:32:24.595549+0000","last_clean":"2024-09-16T00:32:24.595549+0000","last_became_active":"2024-09-16T00:30:26.076718+0000","last_became_peered":"2024-09-16T00:30:26.076718+0000","last_unstale":"2024-09-16T00:32:24.595549+0000","last_undegraded":"2024-09-16T00:32:24.595549+0000","last_fullsized":"2024-09-16T00:32:24.595549+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:25:37.541704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452322+0000","last_change":"2024-09-16T00:30:23.932081+0000","last_active":"2024-09-16T00:32:24.452322+0000","last_peered":"2024-09-16T00:32:24.452322+0000","last_clean":"2024-09-16T00:32:24.452322+0000","last_became_active":"2024-09-16T00:30:23.931729+0000","last_became_peered":"2024-09-16T00:30:23.931729+0000","last_unstale":"2024-09-16T00:32:24.452322+0000","last_undegraded":"2024-09-16T00:32:24.452322+0000","last_fullsized":"2024-09-16T00:32:24.452322+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:20:05.178580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454073+0000","last_change":"2024-09-16T00:30:33.147738+0000","last_active":"2024-09-16T00:32:24.454073+0000","last_peered":"2024-09-16T00:32:24.454073+0000","last_clean":"2024-09-16T00:32:24.454073+0000","last_became_active":"2024-09-16T00:30:33.147326+0000","last_became_peered":"2024-09-16T00:30:33.147326+0000","last_unstale":"2024-09-16T00:32:24.454073+0000","last_undegraded":"2024-09-16T00:32:24.454073+0000","last_fullsized":"2024-09-16T00:32:24.454073+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:44:55.123980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339213+0000","last_change":"2024-09-16T00:30:26.079312+0000","last_active":"2024-09-16T00:32:24.339213+0000","last_peered":"2024-09-16T00:32:24.339213+0000","last_clean":"2024-09-16T00:32:24.339213+0000","last_became_active":"2024-09-16T00:30:26.078986+0000","last_became_peered":"2024-09-16T00:30:26.078986+0000","last_unstale":"2024-09-16T00:32:24.339213+0000","last_undegraded":"2024-09-16T00:32:24.339213+0000","last_fullsized":"2024-09-16T00:32:24.339213+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:02:06.602850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339614+0000","last_change":"2024-09-16T00:31:23.452102+0000","last_active":"2024-09-16T00:32:24.339614+0000","last_peered":"2024-09-16T00:32:24.339614+0000","last_clean":"2024-09-16T00:32:24.339614+0000","last_became_active":"2024-09-16T00:31:23.445974+0000","last_became_peered":"2024-09-16T00:31:23.445974+0000","last_unstale":"2024-09-16T00:32:24.339614+0000","last_undegraded":"2024-09-16T00:32:24.339614+0000","last_fullsized":"2024-09-16T00:32:24.339614+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:52.314494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454166+0000","last_change":"2024-09-16T00:30:33.149154+0000","last_active":"2024-09-16T00:32:24.454166+0000","last_peered":"2024-09-16T00:32:24.454166+0000","last_clean":"2024-09-16T00:32:24.454166+0000","last_became_active":"2024-09-16T00:30:33.148763+0000","last_became_peered":"2024-09-16T00:30:33.148763+0000","last_unstale":"2024-09-16T00:32:24.454166+0000","last_undegraded":"2024-09-16T00:32:24.454166+0000","last_fullsized":"2024-09-16T00:32:24.454166+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:58:05.884919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454241+0000","last_change":"2024-09-16T00:30:26.086905+0000","last_active":"2024-09-16T00:32:24.454241+0000","last_peered":"2024-09-16T00:32:24.454241+0000","last_clean":"2024-09-16T00:32:24.454241+0000","last_became_active":"2024-09-16T00:30:26.086519+0000","last_became_peered":"2024-09-16T00:30:26.086519+0000","last_unstale":"2024-09-16T00:32:24.454241+0000","last_undegraded":"2024-09-16T00:32:24.454241+0000","last_fullsized":"2024-09-16T00:32:24.454241+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:51:08.787679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":54,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.338768+0000","last_change":"2024-09-16T00:30:24.422893+0000","last_active":"2024-09-16T00:32:24.338768+0000","last_peered":"2024-09-16T00:32:24.338768+0000","last_clean":"2024-09-16T00:32:24.338768+0000","last_became_active":"2024-09-16T00:30:24.422073+0000","last_became_peered":"2024-09-16T00:30:24.422073+0000","last_unstale":"2024-09-16T00:32:24.338768+0000","last_undegraded":"2024-09-16T00:32:24.338768+0000","last_fullsized":"2024-09-16T00:32:24.338768+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:15:11.375957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452155+0000","last_change":"2024-09-16T00:30:33.152582+0000","last_active":"2024-09-16T00:32:24.452155+0000","last_peered":"2024-09-16T00:32:24.452155+0000","last_clean":"2024-09-16T00:32:24.452155+0000","last_became_active":"2024-09-16T00:30:33.152180+0000","last_became_peered":"2024-09-16T00:30:33.152180+0000","last_unstale":"2024-09-16T00:32:24.452155+0000","last_undegraded":"2024-09-16T00:32:24.452155+0000","last_fullsized":"2024-09-16T00:32:24.452155+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:41:10.499391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594802+0000","last_change":"2024-09-16T00:30:26.086052+0000","last_active":"2024-09-16T00:32:24.594802+0000","last_peered":"2024-09-16T00:32:24.594802+0000","last_clean":"2024-09-16T00:32:24.594802+0000","last_became_active":"2024-09-16T00:30:26.085756+0000","last_became_peered":"2024-09-16T00:30:26.085756+0000","last_unstale":"2024-09-16T00:32:24.594802+0000","last_undegraded":"2024-09-16T00:32:24.594802+0000","last_fullsized":"2024-09-16T00:32:24.594802+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:52.582166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":144,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:42.479580+0000","last_change":"2024-09-16T00:30:23.933528+0000","last_active":"2024-09-16T00:32:42.479580+0000","last_peered":"2024-09-16T00:32:42.479580+0000","last_clean":"2024-09-16T00:32:42.479580+0000","last_became_active":"2024-09-16T00:30:23.933186+0000","last_became_peered":"2024-09-16T00:30:23.933186+0000","last_unstale":"2024-09-16T00:32:42.479580+0000","last_undegraded":"2024-09-16T00:32:42.479580+0000","last_fullsized":"2024-09-16T00:32:42.479580+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:11:00.489676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":103,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.340727+0000","last_change":"2024-09-16T00:31:23.578056+0000","last_active":"2024-09-16T00:32:24.340727+0000","last_peered":"2024-09-16T00:32:24.340727+0000","last_clean":"2024-09-16T00:32:24.340727+0000","last_became_active":"2024-09-16T00:31:23.577721+0000","last_became_peered":"2024-09-16T00:31:23.577721+0000","last_unstale":"2024-09-16T00:32:24.340727+0000","last_undegraded":"2024-09-16T00:32:24.340727+0000","last_fullsized":"2024-09-16T00:32:24.340727+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:53.632440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595248+0000","last_change":"2024-09-16T00:30:26.084509+0000","last_active":"2024-09-16T00:32:24.595248+0000","last_peered":"2024-09-16T00:32:24.595248+0000","last_clean":"2024-09-16T00:32:24.595248+0000","last_became_active":"2024-09-16T00:30:26.084188+0000","last_became_peered":"2024-09-16T00:30:26.084188+0000","last_unstale":"2024-09-16T00:32:24.595248+0000","last_undegraded":"2024-09-16T00:32:24.595248+0000","last_fullsized":"2024-09-16T00:32:24.595248+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:21:48.981277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454529+0000","last_change":"2024-09-16T00:30:23.933499+0000","last_active":"2024-09-16T00:32:24.454529+0000","last_peered":"2024-09-16T00:32:24.454529+0000","last_clean":"2024-09-16T00:32:24.454529+0000","last_became_active":"2024-09-16T00:30:23.933082+0000","last_became_peered":"2024-09-16T00:30:23.933082+0000","last_unstale":"2024-09-16T00:32:24.454529+0000","last_undegraded":"2024-09-16T00:32:24.454529+0000","last_fullsized":"2024-09-16T00:32:24.454529+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:42:18.998433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.596318+0000","last_change":"2024-09-16T00:32:24.596318+0000","last_active":"2024-09-16T00:32:24.596318+0000","last_peered":"2024-09-16T00:32:24.596318+0000","last_clean":"2024-09-16T00:32:24.596318+0000","last_became_active":"2024-09-16T00:32:24.595906+0000","last_became_peered":"2024-09-16T00:32:24.595906+0000","last_unstale":"2024-09-16T00:32:24.596318+0000","last_undegraded":"2024-09-16T00:32:24.596318+0000","last_fullsized":"2024-09-16T00:32:24.596318+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:13:56.601745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452018+0000","last_change":"2024-09-16T00:30:26.080517+0000","last_active":"2024-09-16T00:32:24.452018+0000","last_peered":"2024-09-16T00:32:24.452018+0000","last_clean":"2024-09-16T00:32:24.452018+0000","last_became_active":"2024-09-16T00:30:26.080197+0000","last_became_peered":"2024-09-16T00:30:26.080197+0000","last_unstale":"2024-09-16T00:32:24.452018+0000","last_undegraded":"2024-09-16T00:32:24.452018+0000","last_fullsized":"2024-09-16T00:32:24.452018+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:15.508562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339267+0000","last_change":"2024-09-16T00:31:23.457047+0000","last_active":"2024-09-16T00:32:24.339267+0000","last_peered":"2024-09-16T00:32:24.339267+0000","last_clean":"2024-09-16T00:32:24.339267+0000","last_became_active":"2024-09-16T00:31:23.447014+0000","last_became_peered":"2024-09-16T00:31:23.447014+0000","last_unstale":"2024-09-16T00:32:24.339267+0000","last_undegraded":"2024-09-16T00:32:24.339267+0000","last_fullsized":"2024-09-16T00:32:24.339267+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:34:42.040228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454641+0000","last_change":"2024-09-16T00:30:33.145803+0000","last_active":"2024-09-16T00:32:24.454641+0000","last_peered":"2024-09-16T00:32:24.454641+0000","last_clean":"2024-09-16T00:32:24.454641+0000","last_became_active":"2024-09-16T00:30:33.145366+0000","last_became_peered":"2024-09-16T00:30:33.145366+0000","last_unstale":"2024-09-16T00:32:24.454641+0000","last_undegraded":"2024-09-16T00:32:24.454641+0000","last_fullsized":"2024-09-16T00:32:24.454641+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:47:28.689305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339085+0000","last_change":"2024-09-16T00:30:26.079415+0000","last_active":"2024-09-16T00:32:24.339085+0000","last_peered":"2024-09-16T00:32:24.339085+0000","last_clean":"2024-09-16T00:32:24.339085+0000","last_became_active":"2024-09-16T00:30:26.079075+0000","last_became_peered":"2024-09-16T00:30:26.079075+0000","last_unstale":"2024-09-16T00:32:24.339085+0000","last_undegraded":"2024-09-16T00:32:24.339085+0000","last_fullsized":"2024-09-16T00:32:24.339085+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:18:18.236342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"92'39","reported_seq":291,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:43.484104+0000","last_change":"2024-09-16T00:30:23.933172+0000","last_active":"2024-09-16T00:32:43.484104+0000","last_peered":"2024-09-16T00:32:43.484104+0000","last_clean":"2024-09-16T00:32:43.484104+0000","last_became_active":"2024-09-16T00:30:23.932492+0000","last_became_peered":"2024-09-16T00:30:23.932492+0000","last_unstale":"2024-09-16T00:32:43.484104+0000","last_undegraded":"2024-09-16T00:32:43.484104+0000","last_fullsized":"2024-09-16T00:32:43.484104+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:29.169448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68399,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":213,"num_read_kb":175,"num_write":40,"num_write_kb":87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.337053+0000","last_change":"2024-09-16T00:31:23.445673+0000","last_active":"2024-09-16T00:32:24.337053+0000","last_peered":"2024-09-16T00:32:24.337053+0000","last_clean":"2024-09-16T00:32:24.337053+0000","last_became_active":"2024-09-16T00:31:23.445299+0000","last_became_peered":"2024-09-16T00:31:23.445299+0000","last_unstale":"2024-09-16T00:32:24.337053+0000","last_undegraded":"2024-09-16T00:32:24.337053+0000","last_fullsized":"2024-09-16T00:32:24.337053+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:07:44.235340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561729+0000","last_change":"2024-09-16T00:30:26.085419+0000","last_active":"2024-09-16T00:32:24.561729+0000","last_peered":"2024-09-16T00:32:24.561729+0000","last_clean":"2024-09-16T00:32:24.561729+0000","last_became_active":"2024-09-16T00:30:26.085103+0000","last_became_peered":"2024-09-16T00:30:26.085103+0000","last_unstale":"2024-09-16T00:32:24.561729+0000","last_undegraded":"2024-09-16T00:32:24.561729+0000","last_fullsized":"2024-09-16T00:32:24.561729+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:56:26.352779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.453906+0000","last_change":"2024-09-16T00:30:24.422646+0000","last_active":"2024-09-16T00:32:24.453906+0000","last_peered":"2024-09-16T00:32:24.453906+0000","last_clean":"2024-09-16T00:32:24.453906+0000","last_became_active":"2024-09-16T00:30:24.422065+0000","last_became_peered":"2024-09-16T00:30:24.422065+0000","last_unstale":"2024-09-16T00:32:24.453906+0000","last_undegraded":"2024-09-16T00:32:24.453906+0000","last_fullsized":"2024-09-16T00:32:24.453906+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:53:12.401613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454317+0000","last_change":"2024-09-16T00:30:33.147692+0000","last_active":"2024-09-16T00:32:24.454317+0000","last_peered":"2024-09-16T00:32:24.454317+0000","last_clean":"2024-09-16T00:32:24.454317+0000","last_became_active":"2024-09-16T00:30:33.147115+0000","last_became_peered":"2024-09-16T00:30:33.147115+0000","last_unstale":"2024-09-16T00:32:24.454317+0000","last_undegraded":"2024-09-16T00:32:24.454317+0000","last_fullsized":"2024-09-16T00:32:24.454317+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:44:04.536894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595596+0000","last_change":"2024-09-16T00:30:26.088427+0000","last_active":"2024-09-16T00:32:24.595596+0000","last_peered":"2024-09-16T00:32:24.595596+0000","last_clean":"2024-09-16T00:32:24.595596+0000","last_became_active":"2024-09-16T00:30:26.086454+0000","last_became_peered":"2024-09-16T00:30:26.086454+0000","last_unstale":"2024-09-16T00:32:24.595596+0000","last_undegraded":"2024-09-16T00:32:24.595596+0000","last_fullsized":"2024-09-16T00:32:24.595596+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:54:29.679959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595460+0000","last_change":"2024-09-16T00:31:23.453484+0000","last_active":"2024-09-16T00:32:24.595460+0000","last_peered":"2024-09-16T00:32:24.595460+0000","last_clean":"2024-09-16T00:32:24.595460+0000","last_became_active":"2024-09-16T00:31:23.446700+0000","last_became_peered":"2024-09-16T00:31:23.446700+0000","last_unstale":"2024-09-16T00:32:24.595460+0000","last_undegraded":"2024-09-16T00:32:24.595460+0000","last_fullsized":"2024-09-16T00:32:24.595460+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:43:26.465657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.336273+0000","last_change":"2024-09-16T00:30:33.156510+0000","last_active":"2024-09-16T00:32:24.336273+0000","last_peered":"2024-09-16T00:32:24.336273+0000","last_clean":"2024-09-16T00:32:24.336273+0000","last_became_active":"2024-09-16T00:30:33.156088+0000","last_became_peered":"2024-09-16T00:30:33.156088+0000","last_unstale":"2024-09-16T00:32:24.336273+0000","last_undegraded":"2024-09-16T00:32:24.336273+0000","last_fullsized":"2024-09-16T00:32:24.336273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:34.230901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561872+0000","last_change":"2024-09-16T00:30:26.082598+0000","last_active":"2024-09-16T00:32:24.561872+0000","last_peered":"2024-09-16T00:32:24.561872+0000","last_clean":"2024-09-16T00:32:24.561872+0000","last_became_active":"2024-09-16T00:30:26.082286+0000","last_became_peered":"2024-09-16T00:30:26.082286+0000","last_unstale":"2024-09-16T00:32:24.561872+0000","last_undegraded":"2024-09-16T00:32:24.561872+0000","last_fullsized":"2024-09-16T00:32:24.561872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:22:18.579149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"92'5","reported_seq":144,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:42.481119+0000","last_change":"2024-09-16T00:30:23.931097+0000","last_active":"2024-09-16T00:32:42.481119+0000","last_peered":"2024-09-16T00:32:42.481119+0000","last_clean":"2024-09-16T00:32:42.481119+0000","last_became_active":"2024-09-16T00:30:23.930719+0000","last_became_peered":"2024-09-16T00:30:23.930719+0000","last_unstale":"2024-09-16T00:32:42.481119+0000","last_undegraded":"2024-09-16T00:32:42.481119+0000","last_fullsized":"2024-09-16T00:32:42.481119+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:13:47.887754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":101,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595241+0000","last_change":"2024-09-16T00:30:33.142581+0000","last_active":"2024-09-16T00:32:24.595241+0000","last_peered":"2024-09-16T00:32:24.595241+0000","last_clean":"2024-09-16T00:32:24.595241+0000","last_became_active":"2024-09-16T00:30:33.142263+0000","last_became_peered":"2024-09-16T00:30:33.142263+0000","last_unstale":"2024-09-16T00:32:24.595241+0000","last_undegraded":"2024-09-16T00:32:24.595241+0000","last_fullsized":"2024-09-16T00:32:24.595241+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:10:06.456011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595181+0000","last_change":"2024-09-16T00:30:33.152084+0000","last_active":"2024-09-16T00:32:24.595181+0000","last_peered":"2024-09-16T00:32:24.595181+0000","last_clean":"2024-09-16T00:32:24.595181+0000","last_became_active":"2024-09-16T00:30:33.151681+0000","last_became_peered":"2024-09-16T00:30:33.151681+0000","last_unstale":"2024-09-16T00:32:24.595181+0000","last_undegraded":"2024-09-16T00:32:24.595181+0000","last_fullsized":"2024-09-16T00:32:24.595181+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:36.966498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.336370+0000","last_change":"2024-09-16T00:30:26.084200+0000","last_active":"2024-09-16T00:32:24.336370+0000","last_peered":"2024-09-16T00:32:24.336370+0000","last_clean":"2024-09-16T00:32:24.336370+0000","last_became_active":"2024-09-16T00:30:26.083858+0000","last_became_peered":"2024-09-16T00:30:26.083858+0000","last_unstale":"2024-09-16T00:32:24.336370+0000","last_undegraded":"2024-09-16T00:32:24.336370+0000","last_fullsized":"2024-09-16T00:32:24.336370+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:42.712624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454319+0000","last_change":"2024-09-16T00:30:23.932399+0000","last_active":"2024-09-16T00:32:24.454319+0000","last_peered":"2024-09-16T00:32:24.454319+0000","last_clean":"2024-09-16T00:32:24.454319+0000","last_became_active":"2024-09-16T00:30:23.931996+0000","last_became_peered":"2024-09-16T00:30:23.931996+0000","last_unstale":"2024-09-16T00:32:24.454319+0000","last_undegraded":"2024-09-16T00:32:24.454319+0000","last_fullsized":"2024-09-16T00:32:24.454319+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:47:09.634507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339553+0000","last_change":"2024-09-16T00:30:33.154563+0000","last_active":"2024-09-16T00:32:24.339553+0000","last_peered":"2024-09-16T00:32:24.339553+0000","last_clean":"2024-09-16T00:32:24.339553+0000","last_became_active":"2024-09-16T00:30:33.154105+0000","last_became_peered":"2024-09-16T00:30:33.154105+0000","last_unstale":"2024-09-16T00:32:24.339553+0000","last_undegraded":"2024-09-16T00:32:24.339553+0000","last_fullsized":"2024-09-16T00:32:24.339553+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:59.966255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595005+0000","last_change":"2024-09-16T00:30:26.075126+0000","last_active":"2024-09-16T00:32:24.595005+0000","last_peered":"2024-09-16T00:32:24.595005+0000","last_clean":"2024-09-16T00:32:24.595005+0000","last_became_active":"2024-09-16T00:30:26.074819+0000","last_became_peered":"2024-09-16T00:30:26.074819+0000","last_unstale":"2024-09-16T00:32:24.595005+0000","last_undegraded":"2024-09-16T00:32:24.595005+0000","last_fullsized":"2024-09-16T00:32:24.595005+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:36.996346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.451802+0000","last_change":"2024-09-16T00:30:23.929943+0000","last_active":"2024-09-16T00:32:24.451802+0000","last_peered":"2024-09-16T00:32:24.451802+0000","last_clean":"2024-09-16T00:32:24.451802+0000","last_became_active":"2024-09-16T00:30:23.929578+0000","last_became_peered":"2024-09-16T00:30:23.929578+0000","last_unstale":"2024-09-16T00:32:24.451802+0000","last_undegraded":"2024-09-16T00:32:24.451802+0000","last_fullsized":"2024-09-16T00:32:24.451802+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:05:12.505543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.337224+0000","last_change":"2024-09-16T00:30:23.935758+0000","last_active":"2024-09-16T00:32:24.337224+0000","last_peered":"2024-09-16T00:32:24.337224+0000","last_clean":"2024-09-16T00:32:24.337224+0000","last_became_active":"2024-09-16T00:30:23.935321+0000","last_became_peered":"2024-09-16T00:30:23.935321+0000","last_unstale":"2024-09-16T00:32:24.337224+0000","last_undegraded":"2024-09-16T00:32:24.337224+0000","last_fullsized":"2024-09-16T00:32:24.337224+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:35:40.513435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454389+0000","last_change":"2024-09-16T00:30:26.086316+0000","last_active":"2024-09-16T00:32:24.454389+0000","last_peered":"2024-09-16T00:32:24.454389+0000","last_clean":"2024-09-16T00:32:24.454389+0000","last_became_active":"2024-09-16T00:30:26.085800+0000","last_became_peered":"2024-09-16T00:30:26.085800+0000","last_unstale":"2024-09-16T00:32:24.454389+0000","last_undegraded":"2024-09-16T00:32:24.454389+0000","last_fullsized":"2024-09-16T00:32:24.454389+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:05:36.277127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595014+0000","last_change":"2024-09-16T00:30:33.155665+0000","last_active":"2024-09-16T00:32:24.595014+0000","last_peered":"2024-09-16T00:32:24.595014+0000","last_clean":"2024-09-16T00:32:24.595014+0000","last_became_active":"2024-09-16T00:30:33.155253+0000","last_became_peered":"2024-09-16T00:30:33.155253+0000","last_unstale":"2024-09-16T00:32:24.595014+0000","last_undegraded":"2024-09-16T00:32:24.595014+0000","last_fullsized":"2024-09-16T00:32:24.595014+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:37.545232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'8","reported_seq":151,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:42.481651+0000","last_change":"2024-09-16T00:30:23.934487+0000","last_active":"2024-09-16T00:32:42.481651+0000","last_peered":"2024-09-16T00:32:42.481651+0000","last_clean":"2024-09-16T00:32:42.481651+0000","last_became_active":"2024-09-16T00:30:23.934137+0000","last_became_peered":"2024-09-16T00:30:23.934137+0000","last_unstale":"2024-09-16T00:32:42.481651+0000","last_undegraded":"2024-09-16T00:32:42.481651+0000","last_fullsized":"2024-09-16T00:32:42.481651+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:14:33.088776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1549,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":105,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.338839+0000","last_change":"2024-09-16T00:30:26.077533+0000","last_active":"2024-09-16T00:32:24.338839+0000","last_peered":"2024-09-16T00:32:24.338839+0000","last_clean":"2024-09-16T00:32:24.338839+0000","last_became_active":"2024-09-16T00:30:26.076443+0000","last_became_peered":"2024-09-16T00:30:26.076443+0000","last_unstale":"2024-09-16T00:32:24.338839+0000","last_undegraded":"2024-09-16T00:32:24.338839+0000","last_fullsized":"2024-09-16T00:32:24.338839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:39:17.495521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595151+0000","last_change":"2024-09-16T00:30:33.146123+0000","last_active":"2024-09-16T00:32:24.595151+0000","last_peered":"2024-09-16T00:32:24.595151+0000","last_clean":"2024-09-16T00:32:24.595151+0000","last_became_active":"2024-09-16T00:30:33.145716+0000","last_became_peered":"2024-09-16T00:30:33.145716+0000","last_unstale":"2024-09-16T00:32:24.595151+0000","last_undegraded":"2024-09-16T00:32:24.595151+0000","last_fullsized":"2024-09-16T00:32:24.595151+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:14:27.281367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.338818+0000","last_change":"2024-09-16T00:30:26.083155+0000","last_active":"2024-09-16T00:32:24.338818+0000","last_peered":"2024-09-16T00:32:24.338818+0000","last_clean":"2024-09-16T00:32:24.338818+0000","last_became_active":"2024-09-16T00:30:26.082782+0000","last_became_peered":"2024-09-16T00:30:26.082782+0000","last_unstale":"2024-09-16T00:32:24.338818+0000","last_undegraded":"2024-09-16T00:32:24.338818+0000","last_fullsized":"2024-09-16T00:32:24.338818+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:15.929343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594891+0000","last_change":"2024-09-16T00:31:23.447453+0000","last_active":"2024-09-16T00:32:24.594891+0000","last_peered":"2024-09-16T00:32:24.594891+0000","last_clean":"2024-09-16T00:32:24.594891+0000","last_became_active":"2024-09-16T00:31:23.445507+0000","last_became_peered":"2024-09-16T00:31:23.445507+0000","last_unstale":"2024-09-16T00:32:24.594891+0000","last_undegraded":"2024-09-16T00:32:24.594891+0000","last_fullsized":"2024-09-16T00:32:24.594891+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:18:50.885809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594798+0000","last_change":"2024-09-16T00:30:26.076252+0000","last_active":"2024-09-16T00:32:24.594798+0000","last_peered":"2024-09-16T00:32:24.594798+0000","last_clean":"2024-09-16T00:32:24.594798+0000","last_became_active":"2024-09-16T00:30:26.075922+0000","last_became_peered":"2024-09-16T00:30:26.075922+0000","last_unstale":"2024-09-16T00:32:24.594798+0000","last_undegraded":"2024-09-16T00:32:24.594798+0000","last_fullsized":"2024-09-16T00:32:24.594798+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:28:04.822817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594821+0000","last_change":"2024-09-16T00:30:26.085123+0000","last_active":"2024-09-16T00:32:24.594821+0000","last_peered":"2024-09-16T00:32:24.594821+0000","last_clean":"2024-09-16T00:32:24.594821+0000","last_became_active":"2024-09-16T00:30:26.084782+0000","last_became_peered":"2024-09-16T00:30:26.084782+0000","last_unstale":"2024-09-16T00:32:24.594821+0000","last_undegraded":"2024-09-16T00:32:24.594821+0000","last_fullsized":"2024-09-16T00:32:24.594821+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:09:33.766340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452404+0000","last_change":"2024-09-16T00:30:26.077048+0000","last_active":"2024-09-16T00:32:24.452404+0000","last_peered":"2024-09-16T00:32:24.452404+0000","last_clean":"2024-09-16T00:32:24.452404+0000","last_became_active":"2024-09-16T00:30:26.076719+0000","last_became_peered":"2024-09-16T00:30:26.076719+0000","last_unstale":"2024-09-16T00:32:24.452404+0000","last_undegraded":"2024-09-16T00:32:24.452404+0000","last_fullsized":"2024-09-16T00:32:24.452404+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:58.584246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339277+0000","last_change":"2024-09-16T00:30:26.078446+0000","last_active":"2024-09-16T00:32:24.339277+0000","last_peered":"2024-09-16T00:32:24.339277+0000","last_clean":"2024-09-16T00:32:24.339277+0000","last_became_active":"2024-09-16T00:30:26.078145+0000","last_became_peered":"2024-09-16T00:30:26.078145+0000","last_unstale":"2024-09-16T00:32:24.339277+0000","last_undegraded":"2024-09-16T00:32:24.339277+0000","last_fullsized":"2024-09-16T00:32:24.339277+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:52:28.739834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595347+0000","last_change":"2024-09-16T00:30:26.075875+0000","last_active":"2024-09-16T00:32:24.595347+0000","last_peered":"2024-09-16T00:32:24.595347+0000","last_clean":"2024-09-16T00:32:24.595347+0000","last_became_active":"2024-09-16T00:30:26.075424+0000","last_became_peered":"2024-09-16T00:30:26.075424+0000","last_unstale":"2024-09-16T00:32:24.595347+0000","last_undegraded":"2024-09-16T00:32:24.595347+0000","last_fullsized":"2024-09-16T00:32:24.595347+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:21:07.958507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595253+0000","last_change":"2024-09-16T00:30:26.077694+0000","last_active":"2024-09-16T00:32:24.595253+0000","last_peered":"2024-09-16T00:32:24.595253+0000","last_clean":"2024-09-16T00:32:24.595253+0000","last_became_active":"2024-09-16T00:30:26.077374+0000","last_became_peered":"2024-09-16T00:30:26.077374+0000","last_unstale":"2024-09-16T00:32:24.595253+0000","last_undegraded":"2024-09-16T00:32:24.595253+0000","last_fullsized":"2024-09-16T00:32:24.595253+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:16:30.822050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454143+0000","last_change":"2024-09-16T00:30:26.085931+0000","last_active":"2024-09-16T00:32:24.454143+0000","last_peered":"2024-09-16T00:32:24.454143+0000","last_clean":"2024-09-16T00:32:24.454143+0000","last_became_active":"2024-09-16T00:30:26.085420+0000","last_became_peered":"2024-09-16T00:30:26.085420+0000","last_unstale":"2024-09-16T00:32:24.454143+0000","last_undegraded":"2024-09-16T00:32:24.454143+0000","last_fullsized":"2024-09-16T00:32:24.454143+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:20:36.019203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595470+0000","last_change":"2024-09-16T00:30:26.074217+0000","last_active":"2024-09-16T00:32:24.595470+0000","last_peered":"2024-09-16T00:32:24.595470+0000","last_clean":"2024-09-16T00:32:24.595470+0000","last_became_active":"2024-09-16T00:30:26.073856+0000","last_became_peered":"2024-09-16T00:30:26.073856+0000","last_unstale":"2024-09-16T00:32:24.595470+0000","last_undegraded":"2024-09-16T00:32:24.595470+0000","last_fullsized":"2024-09-16T00:32:24.595470+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:53:57.078465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561639+0000","last_change":"2024-09-16T00:30:26.084248+0000","last_active":"2024-09-16T00:32:24.561639+0000","last_peered":"2024-09-16T00:32:24.561639+0000","last_clean":"2024-09-16T00:32:24.561639+0000","last_became_active":"2024-09-16T00:30:26.083530+0000","last_became_peered":"2024-09-16T00:30:26.083530+0000","last_unstale":"2024-09-16T00:32:24.561639+0000","last_undegraded":"2024-09-16T00:32:24.561639+0000","last_fullsized":"2024-09-16T00:32:24.561639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:16:49.483881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452162+0000","last_change":"2024-09-16T00:30:26.080803+0000","last_active":"2024-09-16T00:32:24.452162+0000","last_peered":"2024-09-16T00:32:24.452162+0000","last_clean":"2024-09-16T00:32:24.452162+0000","last_became_active":"2024-09-16T00:30:26.080482+0000","last_became_peered":"2024-09-16T00:30:26.080482+0000","last_unstale":"2024-09-16T00:32:24.452162+0000","last_undegraded":"2024-09-16T00:32:24.452162+0000","last_fullsized":"2024-09-16T00:32:24.452162+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:35:41.741025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339770+0000","last_change":"2024-09-16T00:30:26.077680+0000","last_active":"2024-09-16T00:32:24.339770+0000","last_peered":"2024-09-16T00:32:24.339770+0000","last_clean":"2024-09-16T00:32:24.339770+0000","last_became_active":"2024-09-16T00:30:26.077371+0000","last_became_peered":"2024-09-16T00:30:26.077371+0000","last_unstale":"2024-09-16T00:32:24.339770+0000","last_undegraded":"2024-09-16T00:32:24.339770+0000","last_fullsized":"2024-09-16T00:32:24.339770+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:31:34.338617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595364+0000","last_change":"2024-09-16T00:30:26.071531+0000","last_active":"2024-09-16T00:32:24.595364+0000","last_peered":"2024-09-16T00:32:24.595364+0000","last_clean":"2024-09-16T00:32:24.595364+0000","last_became_active":"2024-09-16T00:30:26.071189+0000","last_became_peered":"2024-09-16T00:30:26.071189+0000","last_unstale":"2024-09-16T00:32:24.595364+0000","last_undegraded":"2024-09-16T00:32:24.595364+0000","last_fullsized":"2024-09-16T00:32:24.595364+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:29:18.406272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.336865+0000","last_change":"2024-09-16T00:30:26.085398+0000","last_active":"2024-09-16T00:32:24.336865+0000","last_peered":"2024-09-16T00:32:24.336865+0000","last_clean":"2024-09-16T00:32:24.336865+0000","last_became_active":"2024-09-16T00:30:26.085076+0000","last_became_peered":"2024-09-16T00:30:26.085076+0000","last_unstale":"2024-09-16T00:32:24.336865+0000","last_undegraded":"2024-09-16T00:32:24.336865+0000","last_fullsized":"2024-09-16T00:32:24.336865+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:20:20.864660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.340070+0000","last_change":"2024-09-16T00:30:26.080953+0000","last_active":"2024-09-16T00:32:24.340070+0000","last_peered":"2024-09-16T00:32:24.340070+0000","last_clean":"2024-09-16T00:32:24.340070+0000","last_became_active":"2024-09-16T00:30:26.080185+0000","last_became_peered":"2024-09-16T00:30:26.080185+0000","last_unstale":"2024-09-16T00:32:24.340070+0000","last_undegraded":"2024-09-16T00:32:24.340070+0000","last_fullsized":"2024-09-16T00:32:24.340070+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:56.769182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561297+0000","last_change":"2024-09-16T00:30:26.079320+0000","last_active":"2024-09-16T00:32:24.561297+0000","last_peered":"2024-09-16T00:32:24.561297+0000","last_clean":"2024-09-16T00:32:24.561297+0000","last_became_active":"2024-09-16T00:30:26.078980+0000","last_became_peered":"2024-09-16T00:30:26.078980+0000","last_unstale":"2024-09-16T00:32:24.561297+0000","last_undegraded":"2024-09-16T00:32:24.561297+0000","last_fullsized":"2024-09-16T00:32:24.561297+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:03:15.386657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594661+0000","last_change":"2024-09-16T00:30:26.084948+0000","last_active":"2024-09-16T00:32:24.594661+0000","last_peered":"2024-09-16T00:32:24.594661+0000","last_clean":"2024-09-16T00:32:24.594661+0000","last_became_active":"2024-09-16T00:30:26.084602+0000","last_became_peered":"2024-09-16T00:30:26.084602+0000","last_unstale":"2024-09-16T00:32:24.594661+0000","last_undegraded":"2024-09-16T00:32:24.594661+0000","last_fullsized":"2024-09-16T00:32:24.594661+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:01.709332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594560+0000","last_change":"2024-09-16T00:30:26.080491+0000","last_active":"2024-09-16T00:32:24.594560+0000","last_peered":"2024-09-16T00:32:24.594560+0000","last_clean":"2024-09-16T00:32:24.594560+0000","last_became_active":"2024-09-16T00:30:26.080189+0000","last_became_peered":"2024-09-16T00:30:26.080189+0000","last_unstale":"2024-09-16T00:32:24.594560+0000","last_undegraded":"2024-09-16T00:32:24.594560+0000","last_fullsized":"2024-09-16T00:32:24.594560+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:38.358919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452035+0000","last_change":"2024-09-16T00:30:26.077410+0000","last_active":"2024-09-16T00:32:24.452035+0000","last_peered":"2024-09-16T00:32:24.452035+0000","last_clean":"2024-09-16T00:32:24.452035+0000","last_became_active":"2024-09-16T00:30:26.076498+0000","last_became_peered":"2024-09-16T00:30:26.076498+0000","last_unstale":"2024-09-16T00:32:24.452035+0000","last_undegraded":"2024-09-16T00:32:24.452035+0000","last_fullsized":"2024-09-16T00:32:24.452035+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:54:17.712774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454822+0000","last_change":"2024-09-16T00:30:26.074500+0000","last_active":"2024-09-16T00:32:24.454822+0000","last_peered":"2024-09-16T00:32:24.454822+0000","last_clean":"2024-09-16T00:32:24.454822+0000","last_became_active":"2024-09-16T00:30:26.074177+0000","last_became_peered":"2024-09-16T00:30:26.074177+0000","last_unstale":"2024-09-16T00:32:24.454822+0000","last_undegraded":"2024-09-16T00:32:24.454822+0000","last_fullsized":"2024-09-16T00:32:24.454822+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:43:13.792231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595503+0000","last_change":"2024-09-16T00:30:26.078358+0000","last_active":"2024-09-16T00:32:24.595503+0000","last_peered":"2024-09-16T00:32:24.595503+0000","last_clean":"2024-09-16T00:32:24.595503+0000","last_became_active":"2024-09-16T00:30:26.077993+0000","last_became_peered":"2024-09-16T00:30:26.077993+0000","last_unstale":"2024-09-16T00:32:24.595503+0000","last_undegraded":"2024-09-16T00:32:24.595503+0000","last_fullsized":"2024-09-16T00:32:24.595503+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:25:10.711394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.337671+0000","last_change":"2024-09-16T00:31:23.446836+0000","last_active":"2024-09-16T00:32:24.337671+0000","last_peered":"2024-09-16T00:32:24.337671+0000","last_clean":"2024-09-16T00:32:24.337671+0000","last_became_active":"2024-09-16T00:31:23.446471+0000","last_became_peered":"2024-09-16T00:31:23.446471+0000","last_unstale":"2024-09-16T00:32:24.337671+0000","last_undegraded":"2024-09-16T00:32:24.337671+0000","last_fullsized":"2024-09-16T00:32:24.337671+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:08:34.887618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561867+0000","last_change":"2024-09-16T00:30:26.086313+0000","last_active":"2024-09-16T00:32:24.561867+0000","last_peered":"2024-09-16T00:32:24.561867+0000","last_clean":"2024-09-16T00:32:24.561867+0000","last_became_active":"2024-09-16T00:30:26.085760+0000","last_became_peered":"2024-09-16T00:30:26.085760+0000","last_unstale":"2024-09-16T00:32:24.561867+0000","last_undegraded":"2024-09-16T00:32:24.561867+0000","last_fullsized":"2024-09-16T00:32:24.561867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:40:38.236789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594968+0000","last_change":"2024-09-16T00:30:26.085289+0000","last_active":"2024-09-16T00:32:24.594968+0000","last_peered":"2024-09-16T00:32:24.594968+0000","last_clean":"2024-09-16T00:32:24.594968+0000","last_became_active":"2024-09-16T00:30:26.084995+0000","last_became_peered":"2024-09-16T00:30:26.084995+0000","last_unstale":"2024-09-16T00:32:24.594968+0000","last_undegraded":"2024-09-16T00:32:24.594968+0000","last_fullsized":"2024-09-16T00:32:24.594968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:45.667892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594850+0000","last_change":"2024-09-16T00:30:26.088690+0000","last_active":"2024-09-16T00:32:24.594850+0000","last_peered":"2024-09-16T00:32:24.594850+0000","last_clean":"2024-09-16T00:32:24.594850+0000","last_became_active":"2024-09-16T00:30:26.087700+0000","last_became_peered":"2024-09-16T00:30:26.087700+0000","last_unstale":"2024-09-16T00:32:24.594850+0000","last_undegraded":"2024-09-16T00:32:24.594850+0000","last_fullsized":"2024-09-16T00:32:24.594850+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:39:51.445324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.337214+0000","last_change":"2024-09-16T00:30:26.080215+0000","last_active":"2024-09-16T00:32:24.337214+0000","last_peered":"2024-09-16T00:32:24.337214+0000","last_clean":"2024-09-16T00:32:24.337214+0000","last_became_active":"2024-09-16T00:30:26.079038+0000","last_became_peered":"2024-09-16T00:30:26.079038+0000","last_unstale":"2024-09-16T00:32:24.337214+0000","last_undegraded":"2024-09-16T00:32:24.337214+0000","last_fullsized":"2024-09-16T00:32:24.337214+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:01.573960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339169+0000","last_change":"2024-09-16T00:30:26.080966+0000","last_active":"2024-09-16T00:32:24.339169+0000","last_peered":"2024-09-16T00:32:24.339169+0000","last_clean":"2024-09-16T00:32:24.339169+0000","last_became_active":"2024-09-16T00:30:26.080641+0000","last_became_peered":"2024-09-16T00:30:26.080641+0000","last_unstale":"2024-09-16T00:32:24.339169+0000","last_undegraded":"2024-09-16T00:32:24.339169+0000","last_fullsized":"2024-09-16T00:32:24.339169+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:33:01.538169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454762+0000","last_change":"2024-09-16T00:30:26.072644+0000","last_active":"2024-09-16T00:32:24.454762+0000","last_peered":"2024-09-16T00:32:24.454762+0000","last_clean":"2024-09-16T00:32:24.454762+0000","last_became_active":"2024-09-16T00:30:26.072336+0000","last_became_peered":"2024-09-16T00:30:26.072336+0000","last_unstale":"2024-09-16T00:32:24.454762+0000","last_undegraded":"2024-09-16T00:32:24.454762+0000","last_fullsized":"2024-09-16T00:32:24.454762+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:44:45.760057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595215+0000","last_change":"2024-09-16T00:30:26.079155+0000","last_active":"2024-09-16T00:32:24.595215+0000","last_peered":"2024-09-16T00:32:24.595215+0000","last_clean":"2024-09-16T00:32:24.595215+0000","last_became_active":"2024-09-16T00:30:26.078820+0000","last_became_peered":"2024-09-16T00:30:26.078820+0000","last_unstale":"2024-09-16T00:32:24.595215+0000","last_undegraded":"2024-09-16T00:32:24.595215+0000","last_fullsized":"2024-09-16T00:32:24.595215+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:35.071143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595361+0000","last_change":"2024-09-16T00:30:26.081537+0000","last_active":"2024-09-16T00:32:24.595361+0000","last_peered":"2024-09-16T00:32:24.595361+0000","last_clean":"2024-09-16T00:32:24.595361+0000","last_became_active":"2024-09-16T00:30:26.080744+0000","last_became_peered":"2024-09-16T00:30:26.080744+0000","last_unstale":"2024-09-16T00:32:24.595361+0000","last_undegraded":"2024-09-16T00:32:24.595361+0000","last_fullsized":"2024-09-16T00:32:24.595361+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:46:38.179888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.340007+0000","last_change":"2024-09-16T00:30:26.081066+0000","last_active":"2024-09-16T00:32:24.340007+0000","last_peered":"2024-09-16T00:32:24.340007+0000","last_clean":"2024-09-16T00:32:24.340007+0000","last_became_active":"2024-09-16T00:30:26.080630+0000","last_became_peered":"2024-09-16T00:30:26.080630+0000","last_unstale":"2024-09-16T00:32:24.340007+0000","last_undegraded":"2024-09-16T00:32:24.340007+0000","last_fullsized":"2024-09-16T00:32:24.340007+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:44:42.644792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":72936,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":637,"num_read_kb":399,"num_write":183,"num_write_kb":133,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":438272,"data_stored":212649,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514538,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1060864,"data_stored":523210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743471,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32892,"kb_used_data":1708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712260,"statfs":{"total":95995035648,"available":95961354240,"internally_reserved":0,"allocated":1748992,"data_stored":1178821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906996,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":527486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135929,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1064960,"data_stored":523288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397566,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1662976,"data_stored":1119755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561092,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1048576,"data_stored":518566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790024,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28656,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716496,"statfs":{"total":95995035648,"available":95965691904,"internally_reserved":0,"allocated":1671168,"data_stored":1115343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953550,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1060864,"data_stored":523216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182481,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32228,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712924,"statfs":{"total":95995035648,"available":95962034176,"internally_reserved":0,"allocated":1060864,"data_stored":518760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28136,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717016,"statfs":{"total":95995035648,"available":95966224384,"internally_reserved":0,"allocated":1138688,"data_stored":582959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":586875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1073152,"data_stored":519954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":64483,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":110592,"data_stored":69977,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:32:46.528 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:32:46.818 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:47.141 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:46 smithi038 ceph-mon[30945]: pgmap v364: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-16T00:32:47.141 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:46 smithi038 ceph-mon[30945]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:32:47.141 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:46 smithi038 ceph-mon[30945]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:32:47.141 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:46 smithi038 ceph-mon[30945]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:32:47.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:46 smithi122 ceph-mon[35412]: pgmap v364: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-16T00:32:47.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:46 smithi122 ceph-mon[35412]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:32:47.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:46 smithi122 ceph-mon[35412]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:32:47.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:46 smithi122 ceph-mon[35412]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:32:47.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:46 smithi073 ceph-mon[35254]: pgmap v364: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-16T00:32:47.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:46 smithi073 ceph-mon[35254]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:32:47.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:46 smithi073 ceph-mon[35254]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:32:47.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:46 smithi073 ceph-mon[35254]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:32:47.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.337+0000 7f6408cc2640 1 -- 172.21.15.38:0/786619240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6404102790 msgr2=0x7f64041067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:47.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.337+0000 7f6408cc2640 1 --2- 172.21.15.38:0/786619240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6404102790 0x7f64041067e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f63f8009900 tx=0x7f63f802a410 comp rx=0 tx=0).stop 2024-09-16T00:32:47.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.337+0000 7f6408cc2640 1 -- 172.21.15.38:0/786619240 shutdown_connections 2024-09-16T00:32:47.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.337+0000 7f6408cc2640 1 --2- 172.21.15.38:0/786619240 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6404106f10 0x7f640410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.337+0000 7f6408cc2640 1 --2- 172.21.15.38:0/786619240 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6404102790 0x7f64041067e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.337+0000 7f6408cc2640 1 --2- 172.21.15.38:0/786619240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6404101de0 0x7f64041021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.363 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.337+0000 7f6408cc2640 1 -- 172.21.15.38:0/786619240 >> 172.21.15.38:0/786619240 conn(0x7f64040fd8c0 msgr2=0x7f64040ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:47.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.337+0000 7f6408cc2640 1 -- 172.21.15.38:0/786619240 shutdown_connections 2024-09-16T00:32:47.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.337+0000 7f6408cc2640 1 -- 172.21.15.38:0/786619240 wait complete. 2024-09-16T00:32:47.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.337+0000 7f6408cc2640 1 Processor -- start 2024-09-16T00:32:47.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.338+0000 7f6408cc2640 1 -- start start 2024-09-16T00:32:47.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.338+0000 7f6408cc2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6404101de0 0x7f6404198470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:47.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.338+0000 7f6408cc2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6404102790 0x7f64041989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:47.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.338+0000 7f6408cc2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6404106f10 0x7f640419cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:47.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.338+0000 7f6408cc2640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6404111f90 con 0x7f6404102790 2024-09-16T00:32:47.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.338+0000 7f6408cc2640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6404111e10 con 0x7f6404106f10 2024-09-16T00:32:47.364 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.338+0000 7f6408cc2640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6404112110 con 0x7f6404101de0 2024-09-16T00:32:47.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.338+0000 7f6401d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6404102790 0x7f64041989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:47.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.338+0000 7f6401d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6404102790 0x7f64041989b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57218/0 (socket says 172.21.15.38:57218) 2024-09-16T00:32:47.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.338+0000 7f6401d9b640 1 -- 172.21.15.38:0/3139928151 learned_addr learned my addr 172.21.15.38:0/3139928151 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:47.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.338+0000 7f6402d9d640 1 --2- 172.21.15.38:0/3139928151 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6404106f10 0x7f640419cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:47.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.338+0000 7f640259c640 1 --2- 172.21.15.38:0/3139928151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6404101de0 0x7f6404198470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:47.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.338+0000 7f6401d9b640 1 -- 172.21.15.38:0/3139928151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6404101de0 msgr2=0x7f6404198470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:47.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.338+0000 7f6401d9b640 1 --2- 172.21.15.38:0/3139928151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6404101de0 0x7f6404198470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.338+0000 7f6401d9b640 1 -- 172.21.15.38:0/3139928151 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6404106f10 msgr2=0x7f640419cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:47.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.338+0000 7f6401d9b640 1 --2- 172.21.15.38:0/3139928151 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6404106f10 0x7f640419cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.338+0000 7f6401d9b640 1 -- 172.21.15.38:0/3139928151 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f640419d420 con 0x7f6404102790 2024-09-16T00:32:47.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.338+0000 7f6401d9b640 1 --2- 172.21.15.38:0/3139928151 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6404102790 0x7f64041989b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f63f8006010 tx=0x7f63f80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:47.365 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.339+0000 7f63f37fe640 1 -- 172.21.15.38:0/3139928151 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f803f030 con 0x7f6404102790 2024-09-16T00:32:47.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.339+0000 7f63f37fe640 1 -- 172.21.15.38:0/3139928151 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f63f8003ce0 con 0x7f6404102790 2024-09-16T00:32:47.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.339+0000 7f6408cc2640 1 -- 172.21.15.38:0/3139928151 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f640419d6b0 con 0x7f6404102790 2024-09-16T00:32:47.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.339+0000 7f63f37fe640 1 -- 172.21.15.38:0/3139928151 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f8035070 con 0x7f6404102790 2024-09-16T00:32:47.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.339+0000 7f6408cc2640 1 -- 172.21.15.38:0/3139928151 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64041a4ff0 con 0x7f6404102790 2024-09-16T00:32:47.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.340+0000 7f6408cc2640 1 -- 172.21.15.38:0/3139928151 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f64041021c0 con 0x7f6404102790 2024-09-16T00:32:47.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.341+0000 7f63f37fe640 1 -- 172.21.15.38:0/3139928151 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f63f8042030 con 0x7f6404102790 2024-09-16T00:32:47.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.342+0000 7f63f37fe640 1 --2- 172.21.15.38:0/3139928151 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f63ec0a3350 0x7f63ec0a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:47.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.342+0000 7f640259c640 1 --2- 172.21.15.38:0/3139928151 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f63ec0a3350 0x7f63ec0a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:47.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.342+0000 7f63f37fe640 1 -- 172.21.15.38:0/3139928151 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f63f80f06f0 con 0x7f6404102790 2024-09-16T00:32:47.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.342+0000 7f640259c640 1 --2- 172.21.15.38:0/3139928151 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f63ec0a3350 0x7f63ec0a5810 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f63f4005fd0 tx=0x7f63f4009870 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:47.366 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.345+0000 7f63f37fe640 1 -- 172.21.15.38:0/3139928151 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f63f80bcff0 con 0x7f6404102790 2024-09-16T00:32:47.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.554+0000 7f6408cc2640 1 -- 172.21.15.38:0/3139928151 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f64041997d0 con 0x7f6404102790 2024-09-16T00:32:47.556 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.555+0000 7f63f37fe640 1 -- 172.21.15.38:0/3139928151 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f64041997d0 con 0x7f6404102790 2024-09-16T00:32:47.557 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:32:47.557 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"btime":"2024-09-16T00:32:41:853514+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/367974527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":367974527},{"type":"v1","addr":"172.21.15.38:6835","nonce":367974527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/335077136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":335077136},{"type":"v1","addr":"172.21.15.38:6837","nonce":335077136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/3501211160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3501211160},{"type":"v1","addr":"172.21.15.38:6839","nonce":3501211160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6841/2623130915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":2623130915},{"type":"v1","addr":"172.21.15.38:6841","nonce":2623130915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/412277519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":412277519},{"type":"v1","addr":"172.21.15.122:6833","nonce":412277519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3987234757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3987234757},{"type":"v1","addr":"172.21.15.122:6835","nonce":3987234757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:30:26.764565+0000","modified":"2024-09-16T00:32:41.355828+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24440},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1210328177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1210328177},{"type":"v1","addr":"172.21.15.73:6833","nonce":1210328177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6837/1858299680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1858299680},{"type":"v1","addr":"172.21.15.122:6837","nonce":1858299680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1983975311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1983975311},{"type":"v1","addr":"172.21.15.122:6839","nonce":1983975311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.73:6835/2866363113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2866363113},{"type":"v1","addr":"172.21.15.73:6835","nonce":2866363113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.73:6837/2681081265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2681081265},{"type":"v1","addr":"172.21.15.73:6837","nonce":2681081265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.73:6839/2396224525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2396224525},{"type":"v1","addr":"172.21.15.73:6839","nonce":2396224525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24464,24440]},"id":1}]} 2024-09-16T00:32:47.557 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 26 2024-09-16T00:32:47.558 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.557+0000 7f6408cc2640 1 -- 172.21.15.38:0/3139928151 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f63ec0a3350 msgr2=0x7f63ec0a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:47.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.557+0000 7f6408cc2640 1 --2- 172.21.15.38:0/3139928151 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f63ec0a3350 0x7f63ec0a5810 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f63f4005fd0 tx=0x7f63f4009870 comp rx=0 tx=0).stop 2024-09-16T00:32:47.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.557+0000 7f6408cc2640 1 -- 172.21.15.38:0/3139928151 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6404102790 msgr2=0x7f64041989b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:47.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.557+0000 7f6408cc2640 1 --2- 172.21.15.38:0/3139928151 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6404102790 0x7f64041989b0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f63f8006010 tx=0x7f63f80042f0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.557+0000 7f6408cc2640 1 -- 172.21.15.38:0/3139928151 shutdown_connections 2024-09-16T00:32:47.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.557+0000 7f6408cc2640 1 --2- 172.21.15.38:0/3139928151 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f63ec0a3350 0x7f63ec0a5810 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.557+0000 7f6408cc2640 1 --2- 172.21.15.38:0/3139928151 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6404106f10 0x7f640419cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.557+0000 7f6408cc2640 1 --2- 172.21.15.38:0/3139928151 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6404102790 0x7f64041989b0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.557+0000 7f6408cc2640 1 --2- 172.21.15.38:0/3139928151 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6404101de0 0x7f6404198470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:47.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.557+0000 7f6408cc2640 1 -- 172.21.15.38:0/3139928151 >> 172.21.15.38:0/3139928151 conn(0x7f64040fd8c0 msgr2=0x7f64040fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:47.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.557+0000 7f6408cc2640 1 -- 172.21.15.38:0/3139928151 shutdown_connections 2024-09-16T00:32:47.559 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:47.557+0000 7f6408cc2640 1 -- 172.21.15.38:0/3139928151 wait complete. 2024-09-16T00:32:48.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:47 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/3139928151' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:32:48.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:47 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/3139928151' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:32:48.267 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:32:48.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:47 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/3139928151' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:32:48.526 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:48.939 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:48 smithi038 ceph-mon[30945]: pgmap v365: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-16T00:32:49.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.019+0000 7f939d648640 1 -- 172.21.15.38:0/2570100303 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93981002f0 msgr2=0x7f93981006d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:49.039 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.019+0000 7f939d648640 1 --2- 172.21.15.38:0/2570100303 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93981002f0 0x7f93981006d0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9388009930 tx=0x7f938802a420 comp rx=0 tx=0).stop 2024-09-16T00:32:49.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.019+0000 7f939d648640 1 -- 172.21.15.38:0/2570100303 shutdown_connections 2024-09-16T00:32:49.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.020+0000 7f939d648640 1 --2- 172.21.15.38:0/2570100303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9398106f50 0x7f939810eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:49.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.020+0000 7f939d648640 1 --2- 172.21.15.38:0/2570100303 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9398100ca0 0x7f9398106840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:49.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.020+0000 7f939d648640 1 --2- 172.21.15.38:0/2570100303 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f93981002f0 0x7f93981006d0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:49.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.020+0000 7f939d648640 1 -- 172.21.15.38:0/2570100303 >> 172.21.15.38:0/2570100303 conn(0x7f9398077210 msgr2=0x7f93980fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:49.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.020+0000 7f939d648640 1 -- 172.21.15.38:0/2570100303 shutdown_connections 2024-09-16T00:32:49.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.020+0000 7f939d648640 1 -- 172.21.15.38:0/2570100303 wait complete. 2024-09-16T00:32:49.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.020+0000 7f939d648640 1 Processor -- start 2024-09-16T00:32:49.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.020+0000 7f939d648640 1 -- start start 2024-09-16T00:32:49.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.020+0000 7f939d648640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93981002f0 0x7f93981984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:49.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.021+0000 7f939d648640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9398100ca0 0x7f9398198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:49.040 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.021+0000 7f939d648640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9398106f50 0x7f939819cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:49.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.021+0000 7f939d648640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9398111fa0 con 0x7f9398100ca0 2024-09-16T00:32:49.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.021+0000 7f939d648640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9398111e20 con 0x7f9398106f50 2024-09-16T00:32:49.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.021+0000 7f939d648640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9398112120 con 0x7f93981002f0 2024-09-16T00:32:49.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.021+0000 7f93967fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9398100ca0 0x7f9398198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:49.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.021+0000 7f93967fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9398100ca0 0x7f9398198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57242/0 (socket says 172.21.15.38:57242) 2024-09-16T00:32:49.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.021+0000 7f93967fc640 1 -- 172.21.15.38:0/22885484 learned_addr learned my addr 172.21.15.38:0/22885484 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:49.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.021+0000 7f93977fe640 1 --2- 172.21.15.38:0/22885484 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9398106f50 0x7f939819cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:49.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.021+0000 7f9396ffd640 1 --2- 172.21.15.38:0/22885484 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93981002f0 0x7f93981984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:49.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.021+0000 7f93967fc640 1 -- 172.21.15.38:0/22885484 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93981002f0 msgr2=0x7f93981984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:49.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.021+0000 7f93967fc640 1 --2- 172.21.15.38:0/22885484 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93981002f0 0x7f93981984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:49.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.021+0000 7f93967fc640 1 -- 172.21.15.38:0/22885484 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9398106f50 msgr2=0x7f939819cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:49.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.021+0000 7f93967fc640 1 --2- 172.21.15.38:0/22885484 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9398106f50 0x7f939819cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.021+0000 7f93967fc640 1 -- 172.21.15.38:0/22885484 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f939819d480 con 0x7f9398100ca0 2024-09-16T00:32:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.021+0000 7f9396ffd640 1 --2- 172.21.15.38:0/22885484 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93981002f0 0x7f93981984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.021+0000 7f93967fc640 1 --2- 172.21.15.38:0/22885484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9398100ca0 0x7f9398198a10 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f938c005fd0 tx=0x7f938c00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.021+0000 7f937ffff640 1 -- 172.21.15.38:0/22885484 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c017070 con 0x7f9398100ca0 2024-09-16T00:32:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.021+0000 7f937ffff640 1 -- 172.21.15.38:0/22885484 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f938c009c30 con 0x7f9398100ca0 2024-09-16T00:32:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.021+0000 7f939d648640 1 -- 172.21.15.38:0/22885484 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f939819d770 con 0x7f9398100ca0 2024-09-16T00:32:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.021+0000 7f937ffff640 1 -- 172.21.15.38:0/22885484 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f938c010460 con 0x7f9398100ca0 2024-09-16T00:32:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.022+0000 7f939d648640 1 -- 172.21.15.38:0/22885484 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93981a5050 con 0x7f9398100ca0 2024-09-16T00:32:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.022+0000 7f939d648640 1 -- 172.21.15.38:0/22885484 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93981006d0 con 0x7f9398100ca0 2024-09-16T00:32:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.024+0000 7f937ffff640 1 -- 172.21.15.38:0/22885484 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f938c003e70 con 0x7f9398100ca0 2024-09-16T00:32:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.024+0000 7f937ffff640 1 --2- 172.21.15.38:0/22885484 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f93800a32d0 0x7f93800a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.024+0000 7f9396ffd640 1 --2- 172.21.15.38:0/22885484 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f93800a32d0 0x7f93800a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.024+0000 7f937ffff640 1 -- 172.21.15.38:0/22885484 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f938c0cfac0 con 0x7f9398100ca0 2024-09-16T00:32:49.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.025+0000 7f9396ffd640 1 --2- 172.21.15.38:0/22885484 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f93800a32d0 0x7f93800a5790 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9388000c00 tx=0x7f9388005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:49.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.028+0000 7f937ffff640 1 -- 172.21.15.38:0/22885484 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f938c020030 con 0x7f9398100ca0 2024-09-16T00:32:49.207 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.206+0000 7f939d648640 1 -- 172.21.15.38:0/22885484 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f939802d7c0 con 0x7f9398100ca0 2024-09-16T00:32:49.208 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.207+0000 7f937ffff640 1 -- 172.21.15.38:0/22885484 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24767 (secure 0 0 0) 0x7f938c09c300 con 0x7f9398100ca0 2024-09-16T00:32:49.209 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:32:49.209 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":92,"fsid":"983b2072-73c1-11ef-bceb-c7b262605968","created":"2024-09-16T00:22:34.897053+0000","modified":"2024-09-16T00:32:24.296695+0000","last_up_change":"2024-09-16T00:29:34.503134+0000","last_in_change":"2024-09-16T00:29:10.135656+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:25:41.931977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:30:22.741889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:30:25.042590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:30:31.554350+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3afc20e3-869f-43a9-a911-b0e8d07de86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6803","nonce":1622028921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6805","nonce":1622028921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6809","nonce":1622028921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6807","nonce":1622028921}]},"public_addr":"172.21.15.38:6803/1622028921","cluster_addr":"172.21.15.38:6805/1622028921","heartbeat_back_addr":"172.21.15.38:6809/1622028921","heartbeat_front_addr":"172.21.15.38:6807/1622028921","state":["exists","up"]},{"osd":1,"uuid":"6d9d7acb-45ed-4543-8afc-6433fc7f096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6801","nonce":3811690336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6803","nonce":3811690336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6807","nonce":3811690336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6805","nonce":3811690336}]},"public_addr":"172.21.15.73:6801/3811690336","cluster_addr":"172.21.15.73:6803/3811690336","heartbeat_back_addr":"172.21.15.73:6807/3811690336","heartbeat_front_addr":"172.21.15.73:6805/3811690336","state":["exists","up"]},{"osd":2,"uuid":"48690f35-99e9-4f45-9bee-487857caa8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6801","nonce":1754741655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6803","nonce":1754741655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6807","nonce":1754741655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6805","nonce":1754741655}]},"public_addr":"172.21.15.122:6801/1754741655","cluster_addr":"172.21.15.122:6803/1754741655","heartbeat_back_addr":"172.21.15.122:6807/1754741655","heartbeat_front_addr":"172.21.15.122:6805/1754741655","state":["exists","up"]},{"osd":3,"uuid":"28ebb4a0-92e7-4f57-82a5-b9b761eca269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6811","nonce":3523190079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6813","nonce":3523190079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6817","nonce":3523190079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6815","nonce":3523190079}]},"public_addr":"172.21.15.38:6811/3523190079","cluster_addr":"172.21.15.38:6813/3523190079","heartbeat_back_addr":"172.21.15.38:6817/3523190079","heartbeat_front_addr":"172.21.15.38:6815/3523190079","state":["exists","up"]},{"osd":4,"uuid":"5ea363f7-7ad8-45ed-b7e4-4b010fe7f8cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6809","nonce":1469927628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6811","nonce":1469927628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6815","nonce":1469927628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6813","nonce":1469927628}]},"public_addr":"172.21.15.73:6809/1469927628","cluster_addr":"172.21.15.73:6811/1469927628","heartbeat_back_addr":"172.21.15.73:6815/1469927628","heartbeat_front_addr":"172.21.15.73:6813/1469927628","state":["exists","up"]},{"osd":5,"uuid":"e3c66d43-dd28-4bb1-a886-62a869be82cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6809","nonce":2954413058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6811","nonce":2954413058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6815","nonce":2954413058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6813","nonce":2954413058}]},"public_addr":"172.21.15.122:6809/2954413058","cluster_addr":"172.21.15.122:6811/2954413058","heartbeat_back_addr":"172.21.15.122:6815/2954413058","heartbeat_front_addr":"172.21.15.122:6813/2954413058","state":["exists","up"]},{"osd":6,"uuid":"211497ac-7c48-4095-bfdb-1d4df0149f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6819","nonce":1415719550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6821","nonce":1415719550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6825","nonce":1415719550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6823","nonce":1415719550}]},"public_addr":"172.21.15.38:6819/1415719550","cluster_addr":"172.21.15.38:6821/1415719550","heartbeat_back_addr":"172.21.15.38:6825/1415719550","heartbeat_front_addr":"172.21.15.38:6823/1415719550","state":["exists","up"]},{"osd":7,"uuid":"06512315-3f5e-44f8-976d-d3f5f1ba00e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6817","nonce":2634361473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6819","nonce":2634361473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6823","nonce":2634361473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6821","nonce":2634361473}]},"public_addr":"172.21.15.73:6817/2634361473","cluster_addr":"172.21.15.73:6819/2634361473","heartbeat_back_addr":"172.21.15.73:6823/2634361473","heartbeat_front_addr":"172.21.15.73:6821/2634361473","state":["exists","up"]},{"osd":8,"uuid":"db703c0a-1f87-46f0-be5f-f826f7f9daac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6817","nonce":1800567633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6819","nonce":1800567633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6823","nonce":1800567633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6821","nonce":1800567633}]},"public_addr":"172.21.15.122:6817/1800567633","cluster_addr":"172.21.15.122:6819/1800567633","heartbeat_back_addr":"172.21.15.122:6823/1800567633","heartbeat_front_addr":"172.21.15.122:6821/1800567633","state":["exists","up"]},{"osd":9,"uuid":"f5384200-b3b4-4936-85e1-fc57a9fe2bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6827","nonce":445727833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6829","nonce":445727833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6833","nonce":445727833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6831","nonce":445727833}]},"public_addr":"172.21.15.38:6827/445727833","cluster_addr":"172.21.15.38:6829/445727833","heartbeat_back_addr":"172.21.15.38:6833/445727833","heartbeat_front_addr":"172.21.15.38:6831/445727833","state":["exists","up"]},{"osd":10,"uuid":"c22883d9-6672-4796-a961-963d21eb37fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6825","nonce":43297139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6827","nonce":43297139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6831","nonce":43297139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6829","nonce":43297139}]},"public_addr":"172.21.15.73:6825/43297139","cluster_addr":"172.21.15.73:6827/43297139","heartbeat_back_addr":"172.21.15.73:6831/43297139","heartbeat_front_addr":"172.21.15.73:6829/43297139","state":["exists","up"]},{"osd":11,"uuid":"9f98e8cb-1bff-4b8b-aa19-56b691387383","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6825","nonce":1886446061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6827","nonce":1886446061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6831","nonce":1886446061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6829","nonce":1886446061}]},"public_addr":"172.21.15.122:6825/1886446061","cluster_addr":"172.21.15.122:6827/1886446061","heartbeat_back_addr":"172.21.15.122:6831/1886446061","heartbeat_front_addr":"172.21.15.122:6829/1886446061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:47.384012+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:12.834488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:38.770055+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:04.800466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:30.808076+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:56.646967+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:27:22.721270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:27:48.510011+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:28:13.922944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:28:40.514947+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:29:06.031052+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:29:31.882062+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6801/1779324916":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/924150447":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/141237849":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/2302081750":"2024-09-17T00:22:56.844212+0000","172.21.15.38:0/2245468594":"2024-09-17T00:22:56.844212+0000","172.21.15.38:6801/583006373":"2024-09-17T00:22:56.844212+0000","172.21.15.38:0/2435492667":"2024-09-17T00:23:21.417483+0000","172.21.15.38:6800/583006373":"2024-09-17T00:22:56.844212+0000","172.21.15.38:6800/1779324916":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/1103516837":"2024-09-17T00:22:56.844212+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:32:49.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.209+0000 7f939d648640 1 -- 172.21.15.38:0/22885484 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f93800a32d0 msgr2=0x7f93800a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:49.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.209+0000 7f939d648640 1 --2- 172.21.15.38:0/22885484 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f93800a32d0 0x7f93800a5790 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f9388000c00 tx=0x7f9388005a20 comp rx=0 tx=0).stop 2024-09-16T00:32:49.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.209+0000 7f939d648640 1 -- 172.21.15.38:0/22885484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9398100ca0 msgr2=0x7f9398198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:49.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.209+0000 7f939d648640 1 --2- 172.21.15.38:0/22885484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9398100ca0 0x7f9398198a10 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f938c005fd0 tx=0x7f938c00dde0 comp rx=0 tx=0).stop 2024-09-16T00:32:49.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.210+0000 7f939d648640 1 -- 172.21.15.38:0/22885484 shutdown_connections 2024-09-16T00:32:49.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.210+0000 7f939d648640 1 --2- 172.21.15.38:0/22885484 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f93800a32d0 0x7f93800a5790 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:49.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.210+0000 7f939d648640 1 --2- 172.21.15.38:0/22885484 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9398106f50 0x7f939819cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:49.211 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.210+0000 7f939d648640 1 --2- 172.21.15.38:0/22885484 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9398100ca0 0x7f9398198a10 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:49.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.210+0000 7f939d648640 1 --2- 172.21.15.38:0/22885484 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f93981002f0 0x7f93981984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:49.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.210+0000 7f939d648640 1 -- 172.21.15.38:0/22885484 >> 172.21.15.38:0/22885484 conn(0x7f9398077210 msgr2=0x7f93981052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:49.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.210+0000 7f939d648640 1 -- 172.21.15.38:0/22885484 shutdown_connections 2024-09-16T00:32:49.212 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:49.210+0000 7f939d648640 1 -- 172.21.15.38:0/22885484 wait complete. 2024-09-16T00:32:49.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:48 smithi122 ceph-mon[35412]: pgmap v365: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-16T00:32:49.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:48 smithi073 ceph-mon[35254]: pgmap v365: 129 pgs: 129 active+clean; 648 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 1.1 KiB/s wr, 6 op/s 2024-09-16T00:32:49.932 DEBUG:tasks.fwd_scrub:Yielding 2024-09-16T00:32:49.932 INFO:teuthology.run_tasks:Running task quiescer... 2024-09-16T00:32:49.947 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-16T00:32:49.948 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T00:32:49.948 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-16T00:32:49.948 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:32:49.948 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:32:50.214 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:49 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/22885484' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:32:50.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:49 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/22885484' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:32:50.217 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:50.217 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:50.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:49 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/22885484' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:32:50.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.929+0000 7faabd94d640 1 -- 172.21.15.38:0/3609232677 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faab81036e0 msgr2=0x7faab8107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:50.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.929+0000 7faabd94d640 1 --2- 172.21.15.38:0/3609232677 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faab81036e0 0x7faab8107b90 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7faaac009930 tx=0x7faaac02a420 comp rx=0 tx=0).stop 2024-09-16T00:32:50.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.929+0000 7faabd94d640 1 -- 172.21.15.38:0/3609232677 shutdown_connections 2024-09-16T00:32:50.931 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.929+0000 7faabd94d640 1 --2- 172.21.15.38:0/3609232677 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faab81082a0 0x7faab810ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.929+0000 7faabd94d640 1 --2- 172.21.15.38:0/3609232677 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faab81036e0 0x7faab8107b90 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.929+0000 7faabd94d640 1 --2- 172.21.15.38:0/3609232677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faab8102d30 0x7faab8103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.929+0000 7faabd94d640 1 -- 172.21.15.38:0/3609232677 >> 172.21.15.38:0/3609232677 conn(0x7faab80fc7d0 msgr2=0x7faab80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:50.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.929+0000 7faabd94d640 1 -- 172.21.15.38:0/3609232677 shutdown_connections 2024-09-16T00:32:50.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.929+0000 7faabd94d640 1 -- 172.21.15.38:0/3609232677 wait complete. 2024-09-16T00:32:50.932 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.930+0000 7faabd94d640 1 Processor -- start 2024-09-16T00:32:50.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.930+0000 7faabd94d640 1 -- start start 2024-09-16T00:32:50.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.930+0000 7faabd94d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faab8102d30 0x7faab8198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:50.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.930+0000 7faabd94d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faab81036e0 0x7faab8198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:50.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.931+0000 7faabd94d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faab81082a0 0x7faab819ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:50.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.931+0000 7faabd94d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7faab81120c0 con 0x7faab81082a0 2024-09-16T00:32:50.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.931+0000 7faabd94d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7faab8111f40 con 0x7faab8102d30 2024-09-16T00:32:50.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.931+0000 7faabd94d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faab8112240 con 0x7faab81036e0 2024-09-16T00:32:50.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.931+0000 7faab77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faab81082a0 0x7faab819ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:50.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.931+0000 7faab77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faab81082a0 0x7faab819ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57260/0 (socket says 172.21.15.38:57260) 2024-09-16T00:32:50.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.931+0000 7faab77fe640 1 -- 172.21.15.38:0/1629952980 learned_addr learned my addr 172.21.15.38:0/1629952980 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:50.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.931+0000 7faab6ffd640 1 --2- 172.21.15.38:0/1629952980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faab8102d30 0x7faab8198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:50.934 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.931+0000 7faab67fc640 1 --2- 172.21.15.38:0/1629952980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faab81036e0 0x7faab8198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:50.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.931+0000 7faab77fe640 1 -- 172.21.15.38:0/1629952980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faab81036e0 msgr2=0x7faab8198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:50.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.931+0000 7faab77fe640 1 --2- 172.21.15.38:0/1629952980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faab81036e0 0x7faab8198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.931+0000 7faab77fe640 1 -- 172.21.15.38:0/1629952980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faab8102d30 msgr2=0x7faab8198600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:50.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.931+0000 7faab77fe640 1 --2- 172.21.15.38:0/1629952980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faab8102d30 0x7faab8198600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.931+0000 7faab77fe640 1 -- 172.21.15.38:0/1629952980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faab819d5b0 con 0x7faab81082a0 2024-09-16T00:32:50.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.931+0000 7faab77fe640 1 --2- 172.21.15.38:0/1629952980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faab81082a0 0x7faab819ced0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7faab0009fd0 tx=0x7faab000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:50.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.931+0000 7faabc94b640 1 -- 172.21.15.38:0/1629952980 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab0011830 con 0x7faab81082a0 2024-09-16T00:32:50.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.931+0000 7faabc94b640 1 -- 172.21.15.38:0/1629952980 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faab0016070 con 0x7faab81082a0 2024-09-16T00:32:50.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.931+0000 7faabd94d640 1 -- 172.21.15.38:0/1629952980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faab819d8a0 con 0x7faab81082a0 2024-09-16T00:32:50.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.932+0000 7faabc94b640 1 -- 172.21.15.38:0/1629952980 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab000f2e0 con 0x7faab81082a0 2024-09-16T00:32:50.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.932+0000 7faabd94d640 1 -- 172.21.15.38:0/1629952980 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faab81a5180 con 0x7faab81082a0 2024-09-16T00:32:50.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.934+0000 7faabc94b640 1 -- 172.21.15.38:0/1629952980 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7faab00119d0 con 0x7faab81082a0 2024-09-16T00:32:50.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.934+0000 7faabfd26740 1 Processor -- start 2024-09-16T00:32:50.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.934+0000 7faabfd26740 1 -- start start 2024-09-16T00:32:50.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.934+0000 7faabfd26740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffe0e76f70 0x55ffe0e77350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:50.941 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.935+0000 7faabfd26740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffe0e77890 0x55ffe0e81c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:50.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.935+0000 7faabfd26740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffe0e82150 0x55ffe0e84540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:50.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.935+0000 7faabfd26740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ffe0dd8280 con 0x55ffe0e76f70 2024-09-16T00:32:50.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.935+0000 7faabfd26740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ffe0d7fb10 con 0x55ffe0e77890 2024-09-16T00:32:50.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.935+0000 7faabfd26740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ffe0dd4240 con 0x55ffe0e82150 2024-09-16T00:32:50.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.935+0000 7faabc94b640 1 --2- 172.21.15.38:0/1629952980 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7faaa00a32d0 0x7faaa00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:50.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.935+0000 7faab67fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffe0e76f70 0x55ffe0e77350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:50.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.935+0000 7faab67fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffe0e76f70 0x55ffe0e77350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57274/0 (socket says 172.21.15.38:57274) 2024-09-16T00:32:50.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.935+0000 7faab67fc640 1 -- 172.21.15.38:0/2990074136 learned_addr learned my addr 172.21.15.38:0/2990074136 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:50.942 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.935+0000 7faab6ffd640 1 --2- 172.21.15.38:0/2990074136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffe0e77890 0x55ffe0e81c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:50.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.935+0000 7faab77fe640 1 --2- 172.21.15.38:0/1629952980 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7faaa00a32d0 0x7faaa00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:50.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.935+0000 7faab67fc640 1 --2- 172.21.15.38:0/2990074136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffe0e82150 0x55ffe0e84540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:50.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.935+0000 7faabc94b640 1 -- 172.21.15.38:0/1629952980 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7faab00cf370 con 0x7faab81082a0 2024-09-16T00:32:50.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.935+0000 7faab67fc640 1 -- 172.21.15.38:0/2990074136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffe0e82150 msgr2=0x55ffe0e84540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:50.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.935+0000 7faab67fc640 1 --2- 172.21.15.38:0/2990074136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffe0e82150 0x55ffe0e84540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.935+0000 7faab67fc640 1 -- 172.21.15.38:0/2990074136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffe0e77890 msgr2=0x55ffe0e81c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:50.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.935+0000 7faab67fc640 1 --2- 172.21.15.38:0/2990074136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffe0e77890 0x55ffe0e81c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.943 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.935+0000 7faab67fc640 1 -- 172.21.15.38:0/2990074136 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffe0d8c1f0 con 0x55ffe0e76f70 2024-09-16T00:32:50.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.935+0000 7faab6ffd640 1 --2- 172.21.15.38:0/2990074136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffe0e77890 0x55ffe0e81c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:32:50.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.935+0000 7faab67fc640 1 --2- 172.21.15.38:0/2990074136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffe0e76f70 0x55ffe0e77350 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7faaac009900 tx=0x7faaac000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dfa350eb37c70ed server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:50.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.935+0000 7faab77fe640 1 --2- 172.21.15.38:0/1629952980 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7faaa00a32d0 0x7faaa00a5790 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7faab00979c0 tx=0x7faab006e3e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:50.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.936+0000 7faa9dffb640 1 -- 172.21.15.38:0/2990074136 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac044030 con 0x55ffe0e76f70 2024-09-16T00:32:50.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.936+0000 7faa9dffb640 1 -- 172.21.15.38:0/2990074136 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaac005020 con 0x55ffe0e76f70 2024-09-16T00:32:50.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.936+0000 7faa9dffb640 1 -- 172.21.15.38:0/2990074136 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac035040 con 0x55ffe0e76f70 2024-09-16T00:32:50.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.936+0000 7faabfd26740 1 -- 172.21.15.38:0/2990074136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffe0e76f70 msgr2=0x55ffe0e77350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:50.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.936+0000 7faabfd26740 1 --2- 172.21.15.38:0/2990074136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffe0e76f70 0x55ffe0e77350 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7faaac009900 tx=0x7faaac000bc0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.936+0000 7faabfd26740 1 -- 172.21.15.38:0/2990074136 shutdown_connections 2024-09-16T00:32:50.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.936+0000 7faabfd26740 1 --2- 172.21.15.38:0/2990074136 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffe0e82150 0x55ffe0e84540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.936+0000 7faabfd26740 1 --2- 172.21.15.38:0/2990074136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffe0e77890 0x55ffe0e81c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.936+0000 7faabfd26740 1 --2- 172.21.15.38:0/2990074136 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffe0e76f70 0x55ffe0e77350 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.944 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.936+0000 7faabfd26740 1 -- 172.21.15.38:0/2990074136 >> 172.21.15.38:0/2990074136 conn(0x55ffe0e73500 msgr2=0x55ffe0e738e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:50.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.936+0000 7faabfd26740 1 -- 172.21.15.38:0/2990074136 shutdown_connections 2024-09-16T00:32:50.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.936+0000 7faabfd26740 1 -- 172.21.15.38:0/2990074136 wait complete. 2024-09-16T00:32:50.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.937+0000 7faabfd26740 1 -- start start 2024-09-16T00:32:50.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.937+0000 7faabfd26740 1 Processor -- start 2024-09-16T00:32:50.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.937+0000 7faabfd26740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffe0e76f70 0x55ffe0e7eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:50.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.937+0000 7faabfd26740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffe0e77890 0x55ffe0e7f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:50.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.937+0000 7faabfd26740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffe0e82150 0x55ffe0e7fa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:50.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.937+0000 7faabfd26740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ffe0e84d20 con 0x55ffe0e77890 2024-09-16T00:32:50.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.937+0000 7faabfd26740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ffe0d92f10 con 0x55ffe0e82150 2024-09-16T00:32:50.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.937+0000 7faabfd26740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ffe0e84ea0 con 0x55ffe0e76f70 2024-09-16T00:32:50.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.937+0000 7faabfd26740 10 client.0 _ll_register_callbacks cb 0x55ffe0d5ba40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:50.945 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.937+0000 7faab6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffe0e77890 0x55ffe0e7f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:50.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.938+0000 7faab6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffe0e77890 0x55ffe0e7f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57288/0 (socket says 172.21.15.38:57288) 2024-09-16T00:32:50.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.938+0000 7faab6ffd640 1 -- 172.21.15.38:0/220740277 learned_addr learned my addr 172.21.15.38:0/220740277 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:50.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.938+0000 7faab67fc640 1 --2- 172.21.15.38:0/220740277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffe0e82150 0x55ffe0e7fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:50.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.938+0000 7faab67fc640 1 --2- 172.21.15.38:0/220740277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffe0e76f70 0x55ffe0e7eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:50.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.938+0000 7faab6ffd640 1 -- 172.21.15.38:0/220740277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffe0e76f70 msgr2=0x55ffe0e7eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:50.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.938+0000 7faab6ffd640 1 --2- 172.21.15.38:0/220740277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffe0e76f70 0x55ffe0e7eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.938+0000 7faab6ffd640 1 -- 172.21.15.38:0/220740277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffe0e82150 msgr2=0x55ffe0e7fa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:50.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.938+0000 7faab6ffd640 1 --2- 172.21.15.38:0/220740277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffe0e82150 0x55ffe0e7fa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.938+0000 7faab6ffd640 1 -- 172.21.15.38:0/220740277 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffe0b496a0 con 0x55ffe0e77890 2024-09-16T00:32:50.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.938+0000 7faab6ffd640 1 --2- 172.21.15.38:0/220740277 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffe0e77890 0x55ffe0e7f530 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7faaa8010c30 tx=0x7faaa800f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=723c4cb760b1d0f7 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:50.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.938+0000 7faa96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:32:50.946 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.938+0000 7faa96ffd640 1 -- 172.21.15.38:0/220740277 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa800e5c0 con 0x55ffe0e77890 2024-09-16T00:32:50.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.938+0000 7faabfd26740 1 -- 172.21.15.38:0/220740277 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaac005020 con 0x55ffe0e77890 2024-09-16T00:32:50.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.938+0000 7faa96ffd640 1 -- 172.21.15.38:0/220740277 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7faaa800e760 con 0x55ffe0e77890 2024-09-16T00:32:50.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.938+0000 7faa96ffd640 1 -- 172.21.15.38:0/220740277 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faaa800e960 con 0x55ffe0e77890 2024-09-16T00:32:50.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.938+0000 7faabfd26740 10 client.15321 fetch_fsmap learned FSMap version 26 2024-09-16T00:32:50.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.938+0000 7faabfd26740 1 -- 172.21.15.38:0/220740277 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ffe0e909a0 con 0x55ffe0e77890 2024-09-16T00:32:50.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.939+0000 7faa96ffd640 1 -- 172.21.15.38:0/220740277 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaac005020 con 0x55ffe0e77890 2024-09-16T00:32:50.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.939+0000 7faa96ffd640 1 -- 172.21.15.38:0/220740277 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7faaa801acd0 con 0x55ffe0e77890 2024-09-16T00:32:50.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.939+0000 7faabfd26740 10 client.15321 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:32:50.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.939+0000 7faabfd26740 10 client.15321 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:32:50.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.939+0000 7faabfd26740 20 client.15321 populate_metadata read hostname 'smithi038' 2024-09-16T00:32:50.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.939+0000 7faabfd26740 1 --2- 172.21.15.38:0/220740277 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55ffe0e91e50 0x55ffe0e94270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:50.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.939+0000 7faabfd26740 4 client.15321 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:32:50.947 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.939+0000 7faabfd26740 1 -- 172.21.15.38:0/220740277 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffe0de57f0 con 0x55ffe0e91e50 2024-09-16T00:32:50.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.939+0000 7faab67fc640 1 --2- 172.21.15.38:0/220740277 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55ffe0e91e50 0x55ffe0e94270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:50.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.940+0000 7faa96ffd640 10 client.15321 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:32:50.948 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.940+0000 7faab67fc640 1 --2- 172.21.15.38:0/220740277 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55ffe0e91e50 0x55ffe0e94270 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2bf2a9dfe13f036 server_cookie=897b70dce8720f73 in_seq=0 out_seq=0 2024-09-16T00:32:50.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.940+0000 7faa96ffd640 1 -- 172.21.15.38:0/220740277 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ffe0de57f0 con 0x55ffe0e91e50 2024-09-16T00:32:50.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.940+0000 7faa96ffd640 10 client.15321 handle_command_reply: tid=0 2024-09-16T00:32:50.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.940+0000 7faabfd26740 1 client.15321 shutdown 2024-09-16T00:32:50.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.941+0000 7faabfd26740 1 -- 172.21.15.38:0/220740277 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffe0e77890 msgr2=0x55ffe0e7f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:50.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.941+0000 7faabfd26740 1 --2- 172.21.15.38:0/220740277 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffe0e77890 0x55ffe0e7f530 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7faaa8010c30 tx=0x7faaa800f7f0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.941+0000 7faabfd26740 1 -- 172.21.15.38:0/220740277 shutdown_connections 2024-09-16T00:32:50.949 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.941+0000 7faabfd26740 1 --2- 172.21.15.38:0/220740277 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffe0e82150 0x55ffe0e7fa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.941+0000 7faabfd26740 1 --2- 172.21.15.38:0/220740277 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55ffe0e91e50 0x55ffe0e94270 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.941+0000 7faabfd26740 1 --2- 172.21.15.38:0/220740277 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffe0e77890 0x55ffe0e7f530 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.941+0000 7faabfd26740 1 --2- 172.21.15.38:0/220740277 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffe0e76f70 0x55ffe0e7eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.941+0000 7faa96ffd640 0 client.15321 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:32:50.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.941+0000 7faabfd26740 1 -- 172.21.15.38:0/220740277 >> 172.21.15.38:0/220740277 conn(0x55ffe0e73500 msgr2=0x55ffe0e79230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:50.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.941+0000 7faabfd26740 1 -- 172.21.15.38:0/220740277 shutdown_connections 2024-09-16T00:32:50.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.941+0000 7faabfd26740 1 -- 172.21.15.38:0/220740277 wait complete. 2024-09-16T00:32:50.950 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.941+0000 7faabfd26740 20 client.15321 trim_cache size 0 max 16384 2024-09-16T00:32:50.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.958+0000 7faabfd26740 1 Processor -- start 2024-09-16T00:32:50.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.958+0000 7faabfd26740 1 -- start start 2024-09-16T00:32:50.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.958+0000 7faabfd26740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffe0e76f70 0x55ffe0e7f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:50.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.958+0000 7faabfd26740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffe0e77890 0x55ffe0e99b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:50.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.958+0000 7faabfd26740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffe0e82150 0x55ffe0e9a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:50.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.958+0000 7faabfd26740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7faab81af270 con 0x55ffe0e77890 2024-09-16T00:32:50.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.958+0000 7faabfd26740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7faab81acb40 con 0x55ffe0e82150 2024-09-16T00:32:50.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.958+0000 7faabfd26740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faab81aa990 con 0x55ffe0e76f70 2024-09-16T00:32:50.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.959+0000 7faab6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffe0e77890 0x55ffe0e99b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:50.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.959+0000 7faab6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffe0e77890 0x55ffe0e99b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57302/0 (socket says 172.21.15.38:57302) 2024-09-16T00:32:50.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.959+0000 7faab6ffd640 1 -- 172.21.15.38:0/3367152704 learned_addr learned my addr 172.21.15.38:0/3367152704 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:50.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.959+0000 7faab67fc640 1 --2- 172.21.15.38:0/3367152704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffe0e82150 0x55ffe0e9a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:50.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.959+0000 7faab67fc640 1 --2- 172.21.15.38:0/3367152704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffe0e76f70 0x55ffe0e7f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:50.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.959+0000 7faab6ffd640 1 -- 172.21.15.38:0/3367152704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffe0e76f70 msgr2=0x55ffe0e7f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:50.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.959+0000 7faab6ffd640 1 --2- 172.21.15.38:0/3367152704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffe0e76f70 0x55ffe0e7f8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.959+0000 7faab6ffd640 1 -- 172.21.15.38:0/3367152704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffe0e82150 msgr2=0x55ffe0e9a060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:50.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.959+0000 7faab6ffd640 1 --2- 172.21.15.38:0/3367152704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffe0e82150 0x55ffe0e9a060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.959+0000 7faab6ffd640 1 -- 172.21.15.38:0/3367152704 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffe0e73330 con 0x55ffe0e77890 2024-09-16T00:32:50.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.959+0000 7faab6ffd640 1 --2- 172.21.15.38:0/3367152704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffe0e77890 0x55ffe0e99b20 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7faaa8002230 tx=0x7faaa80104e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d1a427159cd416 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:50.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.959+0000 7faa9d7fa640 1 -- 172.21.15.38:0/3367152704 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa800e5c0 con 0x55ffe0e77890 2024-09-16T00:32:50.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.959+0000 7faa9d7fa640 1 -- 172.21.15.38:0/3367152704 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaa800e760 con 0x55ffe0e77890 2024-09-16T00:32:50.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.959+0000 7faa9d7fa640 1 -- 172.21.15.38:0/3367152704 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaa80197d0 con 0x55ffe0e77890 2024-09-16T00:32:50.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.960+0000 7faabfd26740 1 -- 172.21.15.38:0/3367152704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffe0e77890 msgr2=0x55ffe0e99b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:50.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.960+0000 7faabfd26740 1 --2- 172.21.15.38:0/3367152704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffe0e77890 0x55ffe0e99b20 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7faaa8002230 tx=0x7faaa80104e0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.960+0000 7faabfd26740 1 -- 172.21.15.38:0/3367152704 shutdown_connections 2024-09-16T00:32:50.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.960+0000 7faabfd26740 1 --2- 172.21.15.38:0/3367152704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffe0e82150 0x55ffe0e9a060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.960+0000 7faabfd26740 1 --2- 172.21.15.38:0/3367152704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffe0e77890 0x55ffe0e99b20 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.960+0000 7faabfd26740 1 --2- 172.21.15.38:0/3367152704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffe0e76f70 0x55ffe0e7f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.960+0000 7faabfd26740 1 -- 172.21.15.38:0/3367152704 >> 172.21.15.38:0/3367152704 conn(0x55ffe0e73500 msgr2=0x55ffe0e80f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:50.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.960+0000 7faabfd26740 1 -- 172.21.15.38:0/3367152704 shutdown_connections 2024-09-16T00:32:50.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.960+0000 7faabfd26740 1 -- 172.21.15.38:0/3367152704 wait complete. 2024-09-16T00:32:50.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.960+0000 7faabfd26740 1 -- start start 2024-09-16T00:32:50.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.961+0000 7faabfd26740 1 Processor -- start 2024-09-16T00:32:50.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.961+0000 7faabfd26740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffe0e76f70 0x55ffe0e85ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:50.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.961+0000 7faabfd26740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffe0e77890 0x55ffe0e86430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:50.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.961+0000 7faabfd26740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffe0e82150 0x55ffe0e86970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:50.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.961+0000 7faabfd26740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7faab81ab000 con 0x55ffe0e76f70 2024-09-16T00:32:50.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.961+0000 7faabfd26740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7faab81ac470 con 0x55ffe0e82150 2024-09-16T00:32:50.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.961+0000 7faabfd26740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7faab81ab6d0 con 0x55ffe0e77890 2024-09-16T00:32:50.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.961+0000 7faabfd26740 10 client.0 _ll_register_callbacks cb 0x7faaa8007ae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:50.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.961+0000 7faab67fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffe0e76f70 0x55ffe0e85ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:50.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.961+0000 7faab67fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffe0e76f70 0x55ffe0e85ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57304/0 (socket says 172.21.15.38:57304) 2024-09-16T00:32:50.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.961+0000 7faab67fc640 1 -- 172.21.15.38:0/2689004750 learned_addr learned my addr 172.21.15.38:0/2689004750 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:50.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.961+0000 7faab6ffd640 1 --2- 172.21.15.38:0/2689004750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffe0e77890 0x55ffe0e86430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:50.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.961+0000 7faab67fc640 1 --2- 172.21.15.38:0/2689004750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffe0e82150 0x55ffe0e86970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:50.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.961+0000 7faab67fc640 1 -- 172.21.15.38:0/2689004750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffe0e77890 msgr2=0x55ffe0e86430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:50.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.961+0000 7faab67fc640 1 --2- 172.21.15.38:0/2689004750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffe0e77890 0x55ffe0e86430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.961+0000 7faab67fc640 1 -- 172.21.15.38:0/2689004750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffe0e82150 msgr2=0x55ffe0e86970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:50.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.961+0000 7faab67fc640 1 --2- 172.21.15.38:0/2689004750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffe0e82150 0x55ffe0e86970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.961+0000 7faab67fc640 1 -- 172.21.15.38:0/2689004750 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffe0efbba0 con 0x55ffe0e76f70 2024-09-16T00:32:50.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.962+0000 7faab67fc640 1 --2- 172.21.15.38:0/2689004750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffe0e76f70 0x55ffe0e85ef0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7faaac009930 tx=0x7faaac005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=134a3c7490656c9d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:50.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.962+0000 7faa96ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:32:50.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.962+0000 7faa96ffd640 1 -- 172.21.15.38:0/2689004750 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaac035040 con 0x55ffe0e76f70 2024-09-16T00:32:50.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.962+0000 7faa96ffd640 1 -- 172.21.15.38:0/2689004750 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7faaac058e80 con 0x55ffe0e76f70 2024-09-16T00:32:50.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.962+0000 7faabfd26740 1 -- 172.21.15.38:0/2689004750 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faaa800e760 con 0x55ffe0e76f70 2024-09-16T00:32:50.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.962+0000 7faa96ffd640 1 -- 172.21.15.38:0/2689004750 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7faaac065050 con 0x55ffe0e76f70 2024-09-16T00:32:50.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.962+0000 7faabfd26740 10 client.15327 fetch_fsmap learned FSMap version 26 2024-09-16T00:32:50.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.962+0000 7faabfd26740 1 -- 172.21.15.38:0/2689004750 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ffe0efcec0 con 0x55ffe0e76f70 2024-09-16T00:32:50.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.962+0000 7faa96ffd640 1 -- 172.21.15.38:0/2689004750 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7faaa800e760 con 0x55ffe0e76f70 2024-09-16T00:32:50.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.962+0000 7faa96ffd640 1 -- 172.21.15.38:0/2689004750 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7faaac0047a0 con 0x55ffe0e76f70 2024-09-16T00:32:50.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.962+0000 7faabfd26740 10 client.15327 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:32:50.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.963+0000 7faabfd26740 10 client.15327 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:32:50.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.963+0000 7faabfd26740 20 client.15327 populate_metadata read hostname 'smithi038' 2024-09-16T00:32:50.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.963+0000 7faabfd26740 1 --2- 172.21.15.38:0/2689004750 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55ffe0efd1a0 0x55ffe0eff590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:50.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.963+0000 7faabfd26740 4 client.15327 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:32:50.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.963+0000 7faabfd26740 1 -- 172.21.15.38:0/2689004750 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ffe0ddfd00 con 0x55ffe0efd1a0 2024-09-16T00:32:50.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.963+0000 7faab6ffd640 1 --2- 172.21.15.38:0/2689004750 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55ffe0efd1a0 0x55ffe0eff590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:50.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.963+0000 7faab6ffd640 1 --2- 172.21.15.38:0/2689004750 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55ffe0efd1a0 0x55ffe0eff590 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bea9f3a214cf653d server_cookie=81d6e1137d4c5add in_seq=0 out_seq=0 2024-09-16T00:32:50.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.963+0000 7faa96ffd640 10 client.15327 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:32:50.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.964+0000 7faa96ffd640 1 -- 172.21.15.38:0/2689004750 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ffe0ddfd00 con 0x55ffe0efd1a0 2024-09-16T00:32:50.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.964+0000 7faa96ffd640 10 client.15327 handle_command_reply: tid=0 2024-09-16T00:32:50.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.964+0000 7faabfd26740 1 client.15327 shutdown 2024-09-16T00:32:50.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.965+0000 7faabfd26740 1 -- 172.21.15.38:0/2689004750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffe0e76f70 msgr2=0x55ffe0e85ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:50.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.965+0000 7faabfd26740 1 --2- 172.21.15.38:0/2689004750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffe0e76f70 0x55ffe0e85ef0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7faaac009930 tx=0x7faaac005fd0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.965+0000 7faabfd26740 1 -- 172.21.15.38:0/2689004750 shutdown_connections 2024-09-16T00:32:50.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.965+0000 7faabfd26740 1 --2- 172.21.15.38:0/2689004750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffe0e82150 0x55ffe0e86970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.971 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.965+0000 7faabfd26740 1 --2- 172.21.15.38:0/2689004750 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffe0e77890 0x55ffe0e86430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.965+0000 7faabfd26740 1 --2- 172.21.15.38:0/2689004750 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55ffe0efd1a0 0x55ffe0eff590 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.965+0000 7faabfd26740 1 --2- 172.21.15.38:0/2689004750 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffe0e76f70 0x55ffe0e85ef0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.965+0000 7faa96ffd640 0 client.15327 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:32:50.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.965+0000 7faabfd26740 1 -- 172.21.15.38:0/2689004750 >> 172.21.15.38:0/2689004750 conn(0x55ffe0e73500 msgr2=0x55ffe0e8cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:50.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.965+0000 7faabfd26740 1 -- 172.21.15.38:0/2689004750 shutdown_connections 2024-09-16T00:32:50.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.965+0000 7faabfd26740 1 -- 172.21.15.38:0/2689004750 wait complete. 2024-09-16T00:32:50.972 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.965+0000 7faabfd26740 20 client.15327 trim_cache size 0 max 16384 2024-09-16T00:32:50.972 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-16T00:32:50.972 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0, 2024-09-16T00:32:50.973 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_tag": "d060406b-650d-482a-a169-f5de2c01d192", 2024-09-16T00:32:50.973 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": "asynchronous" 2024-09-16T00:32:50.973 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-16T00:32:50.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.965+0000 7faabd94d640 1 -- 172.21.15.38:0/1629952980 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7faaa00a32d0 msgr2=0x7faaa00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:50.973 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.965+0000 7faabd94d640 1 --2- 172.21.15.38:0/1629952980 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7faaa00a32d0 0x7faaa00a5790 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7faab00979c0 tx=0x7faab006e3e0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.965+0000 7faabd94d640 1 -- 172.21.15.38:0/1629952980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faab81082a0 msgr2=0x7faab819ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:50.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.965+0000 7faabd94d640 1 --2- 172.21.15.38:0/1629952980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faab81082a0 0x7faab819ced0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7faab0009fd0 tx=0x7faab000b8e0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.965+0000 7faabd94d640 1 -- 172.21.15.38:0/1629952980 shutdown_connections 2024-09-16T00:32:50.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.965+0000 7faabd94d640 1 --2- 172.21.15.38:0/1629952980 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7faaa00a32d0 0x7faaa00a5790 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.965+0000 7faabd94d640 1 --2- 172.21.15.38:0/1629952980 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7faab81082a0 0x7faab819ced0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.965+0000 7faabd94d640 1 --2- 172.21.15.38:0/1629952980 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7faab81036e0 0x7faab8198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.965+0000 7faabd94d640 1 --2- 172.21.15.38:0/1629952980 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faab8102d30 0x7faab8198600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:50.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.965+0000 7faabd94d640 1 -- 172.21.15.38:0/1629952980 >> 172.21.15.38:0/1629952980 conn(0x7faab80fc7d0 msgr2=0x7faab8106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:50.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.966+0000 7faabd94d640 1 -- 172.21.15.38:0/1629952980 shutdown_connections 2024-09-16T00:32:50.974 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:50.966+0000 7faabd94d640 1 -- 172.21.15.38:0/1629952980 wait complete. 2024-09-16T00:32:51.037 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:50 smithi038 ceph-mon[30945]: pgmap v366: 129 pgs: 129 active+clean; 650 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-09-16T00:32:51.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.117+0000 7f0e2ec19640 1 -- 172.21.15.38:0/4109969991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e28102d40 msgr2=0x7f0e28103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:51.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.117+0000 7f0e2ec19640 1 --2- 172.21.15.38:0/4109969991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e28102d40 0x7f0e28103120 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f0e18009900 tx=0x7f0e1802a480 comp rx=0 tx=0).stop 2024-09-16T00:32:51.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.117+0000 7f0e2ec19640 1 -- 172.21.15.38:0/4109969991 shutdown_connections 2024-09-16T00:32:51.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.117+0000 7f0e2ec19640 1 --2- 172.21.15.38:0/4109969991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e2810c170 0x7f0e2810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.117+0000 7f0e2ec19640 1 --2- 172.21.15.38:0/4109969991 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e28103660 0x7f0e2810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.117+0000 7f0e2ec19640 1 --2- 172.21.15.38:0/4109969991 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e28102d40 0x7f0e28103120 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.117+0000 7f0e2ec19640 1 -- 172.21.15.38:0/4109969991 >> 172.21.15.38:0/4109969991 conn(0x7f0e280fc7a0 msgr2=0x7f0e280febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:51.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.117+0000 7f0e2ec19640 1 -- 172.21.15.38:0/4109969991 shutdown_connections 2024-09-16T00:32:51.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.118+0000 7f0e2ec19640 1 -- 172.21.15.38:0/4109969991 wait complete. 2024-09-16T00:32:51.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.118+0000 7f0e2ec19640 1 Processor -- start 2024-09-16T00:32:51.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.118+0000 7f0e2ec19640 1 -- start start 2024-09-16T00:32:51.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.118+0000 7f0e2ec19640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e28102d40 0x7f0e281009a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:51.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.118+0000 7f0e2ec19640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e28103660 0x7f0e28100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:51.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.118+0000 7f0e2ec19640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e2810c170 0x7f0e28101420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:51.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.118+0000 7f0e2ec19640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0e28111f70 con 0x7f0e28103660 2024-09-16T00:32:51.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.118+0000 7f0e2ec19640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0e28111df0 con 0x7f0e28102d40 2024-09-16T00:32:51.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.118+0000 7f0e2ec19640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0e281120f0 con 0x7f0e2810c170 2024-09-16T00:32:51.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.119+0000 7f0e27fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e28103660 0x7f0e28100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:51.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.119+0000 7f0e27fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e28103660 0x7f0e28100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57310/0 (socket says 172.21.15.38:57310) 2024-09-16T00:32:51.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.119+0000 7f0e27fff640 1 -- 172.21.15.38:0/2625550735 learned_addr learned my addr 172.21.15.38:0/2625550735 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:51.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.119+0000 7f0e2d1b6640 1 --2- 172.21.15.38:0/2625550735 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e2810c170 0x7f0e28101420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:51.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.119+0000 7f0e2c9b5640 1 --2- 172.21.15.38:0/2625550735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e28102d40 0x7f0e281009a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:51.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.119+0000 7f0e27fff640 1 -- 172.21.15.38:0/2625550735 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e2810c170 msgr2=0x7f0e28101420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:51.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.119+0000 7f0e27fff640 1 --2- 172.21.15.38:0/2625550735 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e2810c170 0x7f0e28101420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.119+0000 7f0e27fff640 1 -- 172.21.15.38:0/2625550735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e28102d40 msgr2=0x7f0e281009a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:51.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.119+0000 7f0e27fff640 1 --2- 172.21.15.38:0/2625550735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e28102d40 0x7f0e281009a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.119+0000 7f0e27fff640 1 -- 172.21.15.38:0/2625550735 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e281a68e0 con 0x7f0e28103660 2024-09-16T00:32:51.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.119+0000 7f0e27fff640 1 --2- 172.21.15.38:0/2625550735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e28103660 0x7f0e28100ee0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c00d750 tx=0x7f0e1c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:51.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.119+0000 7f0e25ffb640 1 -- 172.21.15.38:0/2625550735 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c017070 con 0x7f0e28103660 2024-09-16T00:32:51.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.119+0000 7f0e25ffb640 1 -- 172.21.15.38:0/2625550735 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0e1c005020 con 0x7f0e28103660 2024-09-16T00:32:51.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.119+0000 7f0e2ec19640 1 -- 172.21.15.38:0/2625550735 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e281a6bd0 con 0x7f0e28103660 2024-09-16T00:32:51.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.119+0000 7f0e25ffb640 1 -- 172.21.15.38:0/2625550735 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c004d00 con 0x7f0e28103660 2024-09-16T00:32:51.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.119+0000 7f0e2ec19640 1 -- 172.21.15.38:0/2625550735 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e281a70c0 con 0x7f0e28103660 2024-09-16T00:32:51.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.120+0000 7f0e2ec19640 1 -- 172.21.15.38:0/2625550735 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e28103120 con 0x7f0e28103660 2024-09-16T00:32:51.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.121+0000 7f0e25ffb640 1 -- 172.21.15.38:0/2625550735 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f0e1c009a30 con 0x7f0e28103660 2024-09-16T00:32:51.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.122+0000 7f0e25ffb640 1 --2- 172.21.15.38:0/2625550735 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f0e100a32d0 0x7f0e100a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:51.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.122+0000 7f0e2c9b5640 1 --2- 172.21.15.38:0/2625550735 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f0e100a32d0 0x7f0e100a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:51.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.122+0000 7f0e25ffb640 1 -- 172.21.15.38:0/2625550735 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f0e1c0cf830 con 0x7f0e28103660 2024-09-16T00:32:51.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.123+0000 7f0e2c9b5640 1 --2- 172.21.15.38:0/2625550735 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f0e100a32d0 0x7f0e100a5790 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0e18005ab0 tx=0x7f0e18005a20 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:51.201 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.126+0000 7f0e25ffb640 1 -- 172.21.15.38:0/2625550735 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0e1c09c070 con 0x7f0e28103660 2024-09-16T00:32:51.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:50 smithi122 ceph-mon[35412]: pgmap v366: 129 pgs: 129 active+clean; 650 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-09-16T00:32:51.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:50 smithi073 ceph-mon[35254]: pgmap v366: 129 pgs: 129 active+clean; 650 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.3 KiB/s wr, 8 op/s 2024-09-16T00:32:51.303 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.302+0000 7f0e2ec19640 1 -- 172.21.15.38:0/2625550735 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0e28062630 con 0x7f0e100a32d0 2024-09-16T00:32:51.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.310+0000 7f0e25ffb640 1 -- 172.21.15.38:0/2625550735 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340493 (secure 0 0 0) 0x7f0e28062630 con 0x7f0e100a32d0 2024-09-16T00:32:51.315 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:32:51.316 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-16T00:32:51.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.313+0000 7f0e2ec19640 1 -- 172.21.15.38:0/2625550735 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f0e100a32d0 msgr2=0x7f0e100a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:51.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.313+0000 7f0e2ec19640 1 --2- 172.21.15.38:0/2625550735 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f0e100a32d0 0x7f0e100a5790 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0e18005ab0 tx=0x7f0e18005a20 comp rx=0 tx=0).stop 2024-09-16T00:32:51.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.313+0000 7f0e2ec19640 1 -- 172.21.15.38:0/2625550735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e28103660 msgr2=0x7f0e28100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:51.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.313+0000 7f0e2ec19640 1 --2- 172.21.15.38:0/2625550735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e28103660 0x7f0e28100ee0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c00d750 tx=0x7f0e1c00dc10 comp rx=0 tx=0).stop 2024-09-16T00:32:51.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.313+0000 7f0e2ec19640 1 -- 172.21.15.38:0/2625550735 shutdown_connections 2024-09-16T00:32:51.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.313+0000 7f0e2ec19640 1 --2- 172.21.15.38:0/2625550735 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f0e100a32d0 0x7f0e100a5790 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.313+0000 7f0e2ec19640 1 --2- 172.21.15.38:0/2625550735 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0e2810c170 0x7f0e28101420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.313+0000 7f0e2ec19640 1 --2- 172.21.15.38:0/2625550735 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0e28103660 0x7f0e28100ee0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.313+0000 7f0e2ec19640 1 --2- 172.21.15.38:0/2625550735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e28102d40 0x7f0e281009a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:51.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.313+0000 7f0e2ec19640 1 -- 172.21.15.38:0/2625550735 >> 172.21.15.38:0/2625550735 conn(0x7f0e280fc7a0 msgr2=0x7f0e280fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:51.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.313+0000 7f0e2ec19640 1 -- 172.21.15.38:0/2625550735 shutdown_connections 2024-09-16T00:32:51.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:51.313+0000 7f0e2ec19640 1 -- 172.21.15.38:0/2625550735 wait complete. 2024-09-16T00:32:52.088 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:51 smithi038 ceph-mon[30945]: scrub queued for path: / 2024-09-16T00:32:52.088 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:51 smithi038 ceph-mon[30945]: scrub summary: idle+waiting paths [/] 2024-09-16T00:32:52.088 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:51 smithi038 ceph-mon[30945]: scrub summary: active paths [/] 2024-09-16T00:32:52.088 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:51 smithi038 ceph-mon[30945]: scrub summary: idle+waiting paths [/] 2024-09-16T00:32:52.088 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:51 smithi038 ceph-mon[30945]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:32:52.120 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":366,"stamp":"2024-09-16T00:32:49.544325+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666058,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":789,"num_read_kb":521,"num_write":367,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1180792,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":796,"ondisk_log_size":796,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380368,"kb_used_data":14448,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561456,"statfs":{"total":1151940427776,"available":1151550930944,"internally_reserved":0,"allocated":14794752,"data_stored":8246912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8987,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":89,"num_read_kb":51,"num_write":11,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003288"},"pg_stats":[{"pgid":"2.1f","version":"88'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.337120+0000","last_change":"2024-09-16T00:30:23.933355+0000","last_active":"2024-09-16T00:32:24.337120+0000","last_peered":"2024-09-16T00:32:24.337120+0000","last_clean":"2024-09-16T00:32:24.337120+0000","last_became_active":"2024-09-16T00:30:23.933057+0000","last_became_peered":"2024-09-16T00:30:23.933057+0000","last_unstale":"2024-09-16T00:32:24.337120+0000","last_undegraded":"2024-09-16T00:32:24.337120+0000","last_fullsized":"2024-09-16T00:32:24.337120+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:23:01.713440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.596165+0000","last_change":"2024-09-16T00:30:26.084537+0000","last_active":"2024-09-16T00:32:24.596165+0000","last_peered":"2024-09-16T00:32:24.596165+0000","last_clean":"2024-09-16T00:32:24.596165+0000","last_became_active":"2024-09-16T00:30:26.084226+0000","last_became_peered":"2024-09-16T00:30:26.084226+0000","last_unstale":"2024-09-16T00:32:24.596165+0000","last_undegraded":"2024-09-16T00:32:24.596165+0000","last_fullsized":"2024-09-16T00:32:24.596165+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:43:41.628877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594728+0000","last_change":"2024-09-16T00:30:33.154873+0000","last_active":"2024-09-16T00:32:24.594728+0000","last_peered":"2024-09-16T00:32:24.594728+0000","last_clean":"2024-09-16T00:32:24.594728+0000","last_became_active":"2024-09-16T00:30:33.154433+0000","last_became_peered":"2024-09-16T00:30:33.154433+0000","last_unstale":"2024-09-16T00:32:24.594728+0000","last_undegraded":"2024-09-16T00:32:24.594728+0000","last_fullsized":"2024-09-16T00:32:24.594728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:50.805806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.596255+0000","last_change":"2024-09-16T00:30:24.422274+0000","last_active":"2024-09-16T00:32:24.596255+0000","last_peered":"2024-09-16T00:32:24.596255+0000","last_clean":"2024-09-16T00:32:24.596255+0000","last_became_active":"2024-09-16T00:30:24.421902+0000","last_became_peered":"2024-09-16T00:30:24.421902+0000","last_unstale":"2024-09-16T00:32:24.596255+0000","last_undegraded":"2024-09-16T00:32:24.596255+0000","last_fullsized":"2024-09-16T00:32:24.596255+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:01.910187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595568+0000","last_change":"2024-09-16T00:30:26.088165+0000","last_active":"2024-09-16T00:32:24.595568+0000","last_peered":"2024-09-16T00:32:24.595568+0000","last_clean":"2024-09-16T00:32:24.595568+0000","last_became_active":"2024-09-16T00:30:26.087593+0000","last_became_peered":"2024-09-16T00:30:26.087593+0000","last_unstale":"2024-09-16T00:32:24.595568+0000","last_undegraded":"2024-09-16T00:32:24.595568+0000","last_fullsized":"2024-09-16T00:32:24.595568+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:54.149930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454083+0000","last_change":"2024-09-16T00:30:33.154586+0000","last_active":"2024-09-16T00:32:24.454083+0000","last_peered":"2024-09-16T00:32:24.454083+0000","last_clean":"2024-09-16T00:32:24.454083+0000","last_became_active":"2024-09-16T00:30:33.154209+0000","last_became_peered":"2024-09-16T00:30:33.154209+0000","last_unstale":"2024-09-16T00:32:24.454083+0000","last_undegraded":"2024-09-16T00:32:24.454083+0000","last_fullsized":"2024-09-16T00:32:24.454083+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:45.776895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339220+0000","last_change":"2024-09-16T00:30:23.932442+0000","last_active":"2024-09-16T00:32:24.339220+0000","last_peered":"2024-09-16T00:32:24.339220+0000","last_clean":"2024-09-16T00:32:24.339220+0000","last_became_active":"2024-09-16T00:30:23.932009+0000","last_became_peered":"2024-09-16T00:30:23.932009+0000","last_unstale":"2024-09-16T00:32:24.339220+0000","last_undegraded":"2024-09-16T00:32:24.339220+0000","last_fullsized":"2024-09-16T00:32:24.339220+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:09:27.078187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561572+0000","last_change":"2024-09-16T00:30:26.087715+0000","last_active":"2024-09-16T00:32:24.561572+0000","last_peered":"2024-09-16T00:32:24.561572+0000","last_clean":"2024-09-16T00:32:24.561572+0000","last_became_active":"2024-09-16T00:30:26.087415+0000","last_became_peered":"2024-09-16T00:30:26.087415+0000","last_unstale":"2024-09-16T00:32:24.561572+0000","last_undegraded":"2024-09-16T00:32:24.561572+0000","last_fullsized":"2024-09-16T00:32:24.561572+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:52.373992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454447+0000","last_change":"2024-09-16T00:30:33.152328+0000","last_active":"2024-09-16T00:32:24.454447+0000","last_peered":"2024-09-16T00:32:24.454447+0000","last_clean":"2024-09-16T00:32:24.454447+0000","last_became_active":"2024-09-16T00:30:33.151954+0000","last_became_peered":"2024-09-16T00:30:33.151954+0000","last_unstale":"2024-09-16T00:32:24.454447+0000","last_undegraded":"2024-09-16T00:32:24.454447+0000","last_fullsized":"2024-09-16T00:32:24.454447+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:36.546729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339346+0000","last_change":"2024-09-16T00:30:23.928988+0000","last_active":"2024-09-16T00:32:24.339346+0000","last_peered":"2024-09-16T00:32:24.339346+0000","last_clean":"2024-09-16T00:32:24.339346+0000","last_became_active":"2024-09-16T00:30:23.928638+0000","last_became_peered":"2024-09-16T00:30:23.928638+0000","last_unstale":"2024-09-16T00:32:24.339346+0000","last_undegraded":"2024-09-16T00:32:24.339346+0000","last_fullsized":"2024-09-16T00:32:24.339346+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:45:58.203455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.336619+0000","last_change":"2024-09-16T00:31:23.446550+0000","last_active":"2024-09-16T00:32:24.336619+0000","last_peered":"2024-09-16T00:32:24.336619+0000","last_clean":"2024-09-16T00:32:24.336619+0000","last_became_active":"2024-09-16T00:31:23.446179+0000","last_became_peered":"2024-09-16T00:31:23.446179+0000","last_unstale":"2024-09-16T00:32:24.336619+0000","last_undegraded":"2024-09-16T00:32:24.336619+0000","last_fullsized":"2024-09-16T00:32:24.336619+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:04:21.280174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454185+0000","last_change":"2024-09-16T00:30:33.147564+0000","last_active":"2024-09-16T00:32:24.454185+0000","last_peered":"2024-09-16T00:32:24.454185+0000","last_clean":"2024-09-16T00:32:24.454185+0000","last_became_active":"2024-09-16T00:30:33.147119+0000","last_became_peered":"2024-09-16T00:30:33.147119+0000","last_unstale":"2024-09-16T00:32:24.454185+0000","last_undegraded":"2024-09-16T00:32:24.454185+0000","last_fullsized":"2024-09-16T00:32:24.454185+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:00.509122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595076+0000","last_change":"2024-09-16T00:30:26.080954+0000","last_active":"2024-09-16T00:32:24.595076+0000","last_peered":"2024-09-16T00:32:24.595076+0000","last_clean":"2024-09-16T00:32:24.595076+0000","last_became_active":"2024-09-16T00:30:26.080624+0000","last_became_peered":"2024-09-16T00:30:26.080624+0000","last_unstale":"2024-09-16T00:32:24.595076+0000","last_undegraded":"2024-09-16T00:32:24.595076+0000","last_fullsized":"2024-09-16T00:32:24.595076+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:05:10.440980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339169+0000","last_change":"2024-09-16T00:30:23.933715+0000","last_active":"2024-09-16T00:32:24.339169+0000","last_peered":"2024-09-16T00:32:24.339169+0000","last_clean":"2024-09-16T00:32:24.339169+0000","last_became_active":"2024-09-16T00:30:23.933275+0000","last_became_peered":"2024-09-16T00:30:23.933275+0000","last_unstale":"2024-09-16T00:32:24.339169+0000","last_undegraded":"2024-09-16T00:32:24.339169+0000","last_fullsized":"2024-09-16T00:32:24.339169+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:40:44.846053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454634+0000","last_change":"2024-09-16T00:30:33.604602+0000","last_active":"2024-09-16T00:32:24.454634+0000","last_peered":"2024-09-16T00:32:24.454634+0000","last_clean":"2024-09-16T00:32:24.454634+0000","last_became_active":"2024-09-16T00:30:33.604107+0000","last_became_peered":"2024-09-16T00:30:33.604107+0000","last_unstale":"2024-09-16T00:32:24.454634+0000","last_undegraded":"2024-09-16T00:32:24.454634+0000","last_fullsized":"2024-09-16T00:32:24.454634+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:41.254445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595459+0000","last_change":"2024-09-16T00:30:26.080612+0000","last_active":"2024-09-16T00:32:24.595459+0000","last_peered":"2024-09-16T00:32:24.595459+0000","last_clean":"2024-09-16T00:32:24.595459+0000","last_became_active":"2024-09-16T00:30:26.080317+0000","last_became_peered":"2024-09-16T00:30:26.080317+0000","last_unstale":"2024-09-16T00:32:24.595459+0000","last_undegraded":"2024-09-16T00:32:24.595459+0000","last_fullsized":"2024-09-16T00:32:24.595459+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:23.814241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594893+0000","last_change":"2024-09-16T00:30:23.936778+0000","last_active":"2024-09-16T00:32:24.594893+0000","last_peered":"2024-09-16T00:32:24.594893+0000","last_clean":"2024-09-16T00:32:24.594893+0000","last_became_active":"2024-09-16T00:30:23.936265+0000","last_became_peered":"2024-09-16T00:30:23.936265+0000","last_unstale":"2024-09-16T00:32:24.594893+0000","last_undegraded":"2024-09-16T00:32:24.594893+0000","last_fullsized":"2024-09-16T00:32:24.594893+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:35:28.288476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454751+0000","last_change":"2024-09-16T00:30:33.148340+0000","last_active":"2024-09-16T00:32:24.454751+0000","last_peered":"2024-09-16T00:32:24.454751+0000","last_clean":"2024-09-16T00:32:24.454751+0000","last_became_active":"2024-09-16T00:30:33.147289+0000","last_became_peered":"2024-09-16T00:30:33.147289+0000","last_unstale":"2024-09-16T00:32:24.454751+0000","last_undegraded":"2024-09-16T00:32:24.454751+0000","last_fullsized":"2024-09-16T00:32:24.454751+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:21.259254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.338936+0000","last_change":"2024-09-16T00:30:26.077325+0000","last_active":"2024-09-16T00:32:24.338936+0000","last_peered":"2024-09-16T00:32:24.338936+0000","last_clean":"2024-09-16T00:32:24.338936+0000","last_became_active":"2024-09-16T00:30:26.077002+0000","last_became_peered":"2024-09-16T00:30:26.077002+0000","last_unstale":"2024-09-16T00:32:24.338936+0000","last_undegraded":"2024-09-16T00:32:24.338936+0000","last_fullsized":"2024-09-16T00:32:24.338936+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:12:38.569247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595646+0000","last_change":"2024-09-16T00:30:23.933025+0000","last_active":"2024-09-16T00:32:24.595646+0000","last_peered":"2024-09-16T00:32:24.595646+0000","last_clean":"2024-09-16T00:32:24.595646+0000","last_became_active":"2024-09-16T00:30:23.932669+0000","last_became_peered":"2024-09-16T00:30:23.932669+0000","last_unstale":"2024-09-16T00:32:24.595646+0000","last_undegraded":"2024-09-16T00:32:24.595646+0000","last_fullsized":"2024-09-16T00:32:24.595646+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:20.593665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.597881+0000","last_change":"2024-09-16T00:32:24.597170+0000","last_active":"2024-09-16T00:32:24.597881+0000","last_peered":"2024-09-16T00:32:24.597881+0000","last_clean":"2024-09-16T00:32:24.597881+0000","last_became_active":"2024-09-16T00:32:24.596810+0000","last_became_peered":"2024-09-16T00:32:24.596810+0000","last_unstale":"2024-09-16T00:32:24.597881+0000","last_undegraded":"2024-09-16T00:32:24.597881+0000","last_fullsized":"2024-09-16T00:32:24.597881+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:26:18.737332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.338711+0000","last_change":"2024-09-16T00:30:26.076104+0000","last_active":"2024-09-16T00:32:24.338711+0000","last_peered":"2024-09-16T00:32:24.338711+0000","last_clean":"2024-09-16T00:32:24.338711+0000","last_became_active":"2024-09-16T00:30:26.075764+0000","last_became_peered":"2024-09-16T00:30:26.075764+0000","last_unstale":"2024-09-16T00:32:24.338711+0000","last_undegraded":"2024-09-16T00:32:24.338711+0000","last_fullsized":"2024-09-16T00:32:24.338711+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:31.228533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561411+0000","last_change":"2024-09-16T00:30:24.422411+0000","last_active":"2024-09-16T00:32:24.561411+0000","last_peered":"2024-09-16T00:32:24.561411+0000","last_clean":"2024-09-16T00:32:24.561411+0000","last_became_active":"2024-09-16T00:30:24.422102+0000","last_became_peered":"2024-09-16T00:30:24.422102+0000","last_unstale":"2024-09-16T00:32:24.561411+0000","last_undegraded":"2024-09-16T00:32:24.561411+0000","last_fullsized":"2024-09-16T00:32:24.561411+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:06:00.572070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454739+0000","last_change":"2024-09-16T00:30:33.605397+0000","last_active":"2024-09-16T00:32:24.454739+0000","last_peered":"2024-09-16T00:32:24.454739+0000","last_clean":"2024-09-16T00:32:24.454739+0000","last_became_active":"2024-09-16T00:30:33.604978+0000","last_became_peered":"2024-09-16T00:30:33.604978+0000","last_unstale":"2024-09-16T00:32:24.454739+0000","last_undegraded":"2024-09-16T00:32:24.454739+0000","last_fullsized":"2024-09-16T00:32:24.454739+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:35:16.766056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595380+0000","last_change":"2024-09-16T00:30:26.082684+0000","last_active":"2024-09-16T00:32:24.595380+0000","last_peered":"2024-09-16T00:32:24.595380+0000","last_clean":"2024-09-16T00:32:24.595380+0000","last_became_active":"2024-09-16T00:30:26.082335+0000","last_became_peered":"2024-09-16T00:30:26.082335+0000","last_unstale":"2024-09-16T00:32:24.595380+0000","last_undegraded":"2024-09-16T00:32:24.595380+0000","last_fullsized":"2024-09-16T00:32:24.595380+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:51:21.389627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454909+0000","last_change":"2024-09-16T00:30:24.587840+0000","last_active":"2024-09-16T00:32:24.454909+0000","last_peered":"2024-09-16T00:32:24.454909+0000","last_clean":"2024-09-16T00:32:24.454909+0000","last_became_active":"2024-09-16T00:30:24.587436+0000","last_became_peered":"2024-09-16T00:30:24.587436+0000","last_unstale":"2024-09-16T00:32:24.454909+0000","last_undegraded":"2024-09-16T00:32:24.454909+0000","last_fullsized":"2024-09-16T00:32:24.454909+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:06.758585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594628+0000","last_change":"2024-09-16T00:30:33.145051+0000","last_active":"2024-09-16T00:32:24.594628+0000","last_peered":"2024-09-16T00:32:24.594628+0000","last_clean":"2024-09-16T00:32:24.594628+0000","last_became_active":"2024-09-16T00:30:33.144585+0000","last_became_peered":"2024-09-16T00:30:33.144585+0000","last_unstale":"2024-09-16T00:32:24.594628+0000","last_undegraded":"2024-09-16T00:32:24.594628+0000","last_fullsized":"2024-09-16T00:32:24.594628+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:48:23.588495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.336844+0000","last_change":"2024-09-16T00:31:23.444802+0000","last_active":"2024-09-16T00:32:24.336844+0000","last_peered":"2024-09-16T00:32:24.336844+0000","last_clean":"2024-09-16T00:32:24.336844+0000","last_became_active":"2024-09-16T00:31:23.444447+0000","last_became_peered":"2024-09-16T00:31:23.444447+0000","last_unstale":"2024-09-16T00:32:24.336844+0000","last_undegraded":"2024-09-16T00:32:24.336844+0000","last_fullsized":"2024-09-16T00:32:24.336844+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:46:37.285250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.6","version":"92'14","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:41.088268+0000","last_change":"2024-09-16T00:30:23.933284+0000","last_active":"2024-09-16T00:32:41.088268+0000","last_peered":"2024-09-16T00:32:41.088268+0000","last_clean":"2024-09-16T00:32:41.088268+0000","last_became_active":"2024-09-16T00:30:23.932920+0000","last_became_peered":"2024-09-16T00:30:23.932920+0000","last_unstale":"2024-09-16T00:32:41.088268+0000","last_undegraded":"2024-09-16T00:32:41.088268+0000","last_fullsized":"2024-09-16T00:32:41.088268+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:38.199391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594942+0000","last_change":"2024-09-16T00:30:33.604308+0000","last_active":"2024-09-16T00:32:24.594942+0000","last_peered":"2024-09-16T00:32:24.594942+0000","last_clean":"2024-09-16T00:32:24.594942+0000","last_became_active":"2024-09-16T00:30:33.603889+0000","last_became_peered":"2024-09-16T00:30:33.603889+0000","last_unstale":"2024-09-16T00:32:24.594942+0000","last_undegraded":"2024-09-16T00:32:24.594942+0000","last_fullsized":"2024-09-16T00:32:24.594942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:32:55.173063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452512+0000","last_change":"2024-09-16T00:30:26.078515+0000","last_active":"2024-09-16T00:32:24.452512+0000","last_peered":"2024-09-16T00:32:24.452512+0000","last_clean":"2024-09-16T00:32:24.452512+0000","last_became_active":"2024-09-16T00:30:26.078173+0000","last_became_peered":"2024-09-16T00:30:26.078173+0000","last_unstale":"2024-09-16T00:32:24.452512+0000","last_undegraded":"2024-09-16T00:32:24.452512+0000","last_fullsized":"2024-09-16T00:32:24.452512+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:40:31.471154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339304+0000","last_change":"2024-09-16T00:30:24.423209+0000","last_active":"2024-09-16T00:32:24.339304+0000","last_peered":"2024-09-16T00:32:24.339304+0000","last_clean":"2024-09-16T00:32:24.339304+0000","last_became_active":"2024-09-16T00:30:24.422157+0000","last_became_peered":"2024-09-16T00:30:24.422157+0000","last_unstale":"2024-09-16T00:32:24.339304+0000","last_undegraded":"2024-09-16T00:32:24.339304+0000","last_fullsized":"2024-09-16T00:32:24.339304+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:28.262392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339327+0000","last_change":"2024-09-16T00:30:33.146204+0000","last_active":"2024-09-16T00:32:24.339327+0000","last_peered":"2024-09-16T00:32:24.339327+0000","last_clean":"2024-09-16T00:32:24.339327+0000","last_became_active":"2024-09-16T00:30:33.145553+0000","last_became_peered":"2024-09-16T00:30:33.145553+0000","last_unstale":"2024-09-16T00:32:24.339327+0000","last_undegraded":"2024-09-16T00:32:24.339327+0000","last_fullsized":"2024-09-16T00:32:24.339327+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:22.096050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.337437+0000","last_change":"2024-09-16T00:30:26.070153+0000","last_active":"2024-09-16T00:32:24.337437+0000","last_peered":"2024-09-16T00:32:24.337437+0000","last_clean":"2024-09-16T00:32:24.337437+0000","last_became_active":"2024-09-16T00:30:26.069792+0000","last_became_peered":"2024-09-16T00:30:26.069792+0000","last_unstale":"2024-09-16T00:32:24.337437+0000","last_undegraded":"2024-09-16T00:32:24.337437+0000","last_fullsized":"2024-09-16T00:32:24.337437+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:28.164242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.336815+0000","last_change":"2024-09-16T00:30:24.423261+0000","last_active":"2024-09-16T00:32:24.336815+0000","last_peered":"2024-09-16T00:32:24.336815+0000","last_clean":"2024-09-16T00:32:24.336815+0000","last_became_active":"2024-09-16T00:30:24.422650+0000","last_became_peered":"2024-09-16T00:30:24.422650+0000","last_unstale":"2024-09-16T00:32:24.336815+0000","last_undegraded":"2024-09-16T00:32:24.336815+0000","last_fullsized":"2024-09-16T00:32:24.336815+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:00.211744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339762+0000","last_change":"2024-09-16T00:30:33.604492+0000","last_active":"2024-09-16T00:32:24.339762+0000","last_peered":"2024-09-16T00:32:24.339762+0000","last_clean":"2024-09-16T00:32:24.339762+0000","last_became_active":"2024-09-16T00:30:33.604015+0000","last_became_peered":"2024-09-16T00:30:33.604015+0000","last_unstale":"2024-09-16T00:32:24.339762+0000","last_undegraded":"2024-09-16T00:32:24.339762+0000","last_fullsized":"2024-09-16T00:32:24.339762+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:40:22.516583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.337389+0000","last_change":"2024-09-16T00:30:26.075517+0000","last_active":"2024-09-16T00:32:24.337389+0000","last_peered":"2024-09-16T00:32:24.337389+0000","last_clean":"2024-09-16T00:32:24.337389+0000","last_became_active":"2024-09-16T00:30:26.075113+0000","last_became_peered":"2024-09-16T00:30:26.075113+0000","last_unstale":"2024-09-16T00:32:24.337389+0000","last_undegraded":"2024-09-16T00:32:24.337389+0000","last_fullsized":"2024-09-16T00:32:24.337389+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:11.427010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595204+0000","last_change":"2024-09-16T00:30:23.933639+0000","last_active":"2024-09-16T00:32:24.595204+0000","last_peered":"2024-09-16T00:32:24.595204+0000","last_clean":"2024-09-16T00:32:24.595204+0000","last_became_active":"2024-09-16T00:30:23.933322+0000","last_became_peered":"2024-09-16T00:30:23.933322+0000","last_unstale":"2024-09-16T00:32:24.595204+0000","last_undegraded":"2024-09-16T00:32:24.595204+0000","last_fullsized":"2024-09-16T00:32:24.595204+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:59:59.082702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452139+0000","last_change":"2024-09-16T00:30:33.148508+0000","last_active":"2024-09-16T00:32:24.452139+0000","last_peered":"2024-09-16T00:32:24.452139+0000","last_clean":"2024-09-16T00:32:24.452139+0000","last_became_active":"2024-09-16T00:30:33.148091+0000","last_became_peered":"2024-09-16T00:30:33.148091+0000","last_unstale":"2024-09-16T00:32:24.452139+0000","last_undegraded":"2024-09-16T00:32:24.452139+0000","last_fullsized":"2024-09-16T00:32:24.452139+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:50.052526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454239+0000","last_change":"2024-09-16T00:30:26.080374+0000","last_active":"2024-09-16T00:32:24.454239+0000","last_peered":"2024-09-16T00:32:24.454239+0000","last_clean":"2024-09-16T00:32:24.454239+0000","last_became_active":"2024-09-16T00:30:26.080044+0000","last_became_peered":"2024-09-16T00:30:26.080044+0000","last_unstale":"2024-09-16T00:32:24.454239+0000","last_undegraded":"2024-09-16T00:32:24.454239+0000","last_fullsized":"2024-09-16T00:32:24.454239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:38:36.034903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":145,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:43.480306+0000","last_change":"2024-09-16T00:30:24.422137+0000","last_active":"2024-09-16T00:32:43.480306+0000","last_peered":"2024-09-16T00:32:43.480306+0000","last_clean":"2024-09-16T00:32:43.480306+0000","last_became_active":"2024-09-16T00:30:24.421778+0000","last_became_peered":"2024-09-16T00:30:24.421778+0000","last_unstale":"2024-09-16T00:32:43.480306+0000","last_undegraded":"2024-09-16T00:32:43.480306+0000","last_fullsized":"2024-09-16T00:32:43.480306+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:54:21.964099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452309+0000","last_change":"2024-09-16T00:30:33.604263+0000","last_active":"2024-09-16T00:32:24.452309+0000","last_peered":"2024-09-16T00:32:24.452309+0000","last_clean":"2024-09-16T00:32:24.452309+0000","last_became_active":"2024-09-16T00:30:33.603804+0000","last_became_peered":"2024-09-16T00:30:33.603804+0000","last_unstale":"2024-09-16T00:32:24.452309+0000","last_undegraded":"2024-09-16T00:32:24.452309+0000","last_fullsized":"2024-09-16T00:32:24.452309+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:08:37.590827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.451792+0000","last_change":"2024-09-16T00:30:26.082916+0000","last_active":"2024-09-16T00:32:24.451792+0000","last_peered":"2024-09-16T00:32:24.451792+0000","last_clean":"2024-09-16T00:32:24.451792+0000","last_became_active":"2024-09-16T00:30:26.082606+0000","last_became_peered":"2024-09-16T00:30:26.082606+0000","last_unstale":"2024-09-16T00:32:24.451792+0000","last_undegraded":"2024-09-16T00:32:24.451792+0000","last_fullsized":"2024-09-16T00:32:24.451792+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:56.777640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594991+0000","last_change":"2024-09-16T00:30:23.937602+0000","last_active":"2024-09-16T00:32:24.594991+0000","last_peered":"2024-09-16T00:32:24.594991+0000","last_clean":"2024-09-16T00:32:24.594991+0000","last_became_active":"2024-09-16T00:30:23.937261+0000","last_became_peered":"2024-09-16T00:30:23.937261+0000","last_unstale":"2024-09-16T00:32:24.594991+0000","last_undegraded":"2024-09-16T00:32:24.594991+0000","last_fullsized":"2024-09-16T00:32:24.594991+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:24.698758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454567+0000","last_change":"2024-09-16T00:30:33.605536+0000","last_active":"2024-09-16T00:32:24.454567+0000","last_peered":"2024-09-16T00:32:24.454567+0000","last_clean":"2024-09-16T00:32:24.454567+0000","last_became_active":"2024-09-16T00:30:33.605073+0000","last_became_peered":"2024-09-16T00:30:33.605073+0000","last_unstale":"2024-09-16T00:32:24.454567+0000","last_undegraded":"2024-09-16T00:32:24.454567+0000","last_fullsized":"2024-09-16T00:32:24.454567+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:11.143052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595152+0000","last_change":"2024-09-16T00:30:26.084507+0000","last_active":"2024-09-16T00:32:24.595152+0000","last_peered":"2024-09-16T00:32:24.595152+0000","last_clean":"2024-09-16T00:32:24.595152+0000","last_became_active":"2024-09-16T00:30:26.084152+0000","last_became_peered":"2024-09-16T00:30:26.084152+0000","last_unstale":"2024-09-16T00:32:24.595152+0000","last_undegraded":"2024-09-16T00:32:24.595152+0000","last_fullsized":"2024-09-16T00:32:24.595152+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:27:21.557409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454701+0000","last_change":"2024-09-16T00:30:23.932686+0000","last_active":"2024-09-16T00:32:24.454701+0000","last_peered":"2024-09-16T00:32:24.454701+0000","last_clean":"2024-09-16T00:32:24.454701+0000","last_became_active":"2024-09-16T00:30:23.932343+0000","last_became_peered":"2024-09-16T00:30:23.932343+0000","last_unstale":"2024-09-16T00:32:24.454701+0000","last_undegraded":"2024-09-16T00:32:24.454701+0000","last_fullsized":"2024-09-16T00:32:24.454701+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:13:44.185712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":351,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595072+0000","last_change":"2024-09-16T00:29:11.143642+0000","last_active":"2024-09-16T00:32:24.595072+0000","last_peered":"2024-09-16T00:32:24.595072+0000","last_clean":"2024-09-16T00:32:24.595072+0000","last_became_active":"2024-09-16T00:29:11.143227+0000","last_became_peered":"2024-09-16T00:29:11.143227+0000","last_unstale":"2024-09-16T00:32:24.595072+0000","last_undegraded":"2024-09-16T00:32:24.595072+0000","last_fullsized":"2024-09-16T00:32:24.595072+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:25:42.361316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:25:42.361316+0000","last_clean_scrub_stamp":"2024-09-16T00:25:42.361316+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:29:12.740474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595397+0000","last_change":"2024-09-16T00:30:33.155555+0000","last_active":"2024-09-16T00:32:24.595397+0000","last_peered":"2024-09-16T00:32:24.595397+0000","last_clean":"2024-09-16T00:32:24.595397+0000","last_became_active":"2024-09-16T00:30:33.155170+0000","last_became_peered":"2024-09-16T00:30:33.155170+0000","last_unstale":"2024-09-16T00:32:24.595397+0000","last_undegraded":"2024-09-16T00:32:24.595397+0000","last_fullsized":"2024-09-16T00:32:24.595397+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:34:46.214422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595573+0000","last_change":"2024-09-16T00:30:26.076003+0000","last_active":"2024-09-16T00:32:24.595573+0000","last_peered":"2024-09-16T00:32:24.595573+0000","last_clean":"2024-09-16T00:32:24.595573+0000","last_became_active":"2024-09-16T00:30:26.075666+0000","last_became_peered":"2024-09-16T00:30:26.075666+0000","last_unstale":"2024-09-16T00:32:24.595573+0000","last_undegraded":"2024-09-16T00:32:24.595573+0000","last_fullsized":"2024-09-16T00:32:24.595573+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:30.113380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594420+0000","last_change":"2024-09-16T00:30:24.587469+0000","last_active":"2024-09-16T00:32:24.594420+0000","last_peered":"2024-09-16T00:32:24.594420+0000","last_clean":"2024-09-16T00:32:24.594420+0000","last_became_active":"2024-09-16T00:30:24.587049+0000","last_became_peered":"2024-09-16T00:30:24.587049+0000","last_unstale":"2024-09-16T00:32:24.594420+0000","last_undegraded":"2024-09-16T00:32:24.594420+0000","last_fullsized":"2024-09-16T00:32:24.594420+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:17:45.934505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561433+0000","last_change":"2024-09-16T00:30:33.145921+0000","last_active":"2024-09-16T00:32:24.561433+0000","last_peered":"2024-09-16T00:32:24.561433+0000","last_clean":"2024-09-16T00:32:24.561433+0000","last_became_active":"2024-09-16T00:30:33.145450+0000","last_became_peered":"2024-09-16T00:30:33.145450+0000","last_unstale":"2024-09-16T00:32:24.561433+0000","last_undegraded":"2024-09-16T00:32:24.561433+0000","last_fullsized":"2024-09-16T00:32:24.561433+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:53:51.698686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595330+0000","last_change":"2024-09-16T00:30:26.087940+0000","last_active":"2024-09-16T00:32:24.595330+0000","last_peered":"2024-09-16T00:32:24.595330+0000","last_clean":"2024-09-16T00:32:24.595330+0000","last_became_active":"2024-09-16T00:30:26.087599+0000","last_became_peered":"2024-09-16T00:30:26.087599+0000","last_unstale":"2024-09-16T00:32:24.595330+0000","last_undegraded":"2024-09-16T00:32:24.595330+0000","last_fullsized":"2024-09-16T00:32:24.595330+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:43.934839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454847+0000","last_change":"2024-09-16T00:30:23.930986+0000","last_active":"2024-09-16T00:32:24.454847+0000","last_peered":"2024-09-16T00:32:24.454847+0000","last_clean":"2024-09-16T00:32:24.454847+0000","last_became_active":"2024-09-16T00:30:23.930588+0000","last_became_peered":"2024-09-16T00:30:23.930588+0000","last_unstale":"2024-09-16T00:32:24.454847+0000","last_undegraded":"2024-09-16T00:32:24.454847+0000","last_fullsized":"2024-09-16T00:32:24.454847+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:04:41.902358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594672+0000","last_change":"2024-09-16T00:30:33.154894+0000","last_active":"2024-09-16T00:32:24.594672+0000","last_peered":"2024-09-16T00:32:24.594672+0000","last_clean":"2024-09-16T00:32:24.594672+0000","last_became_active":"2024-09-16T00:30:33.154490+0000","last_became_peered":"2024-09-16T00:30:33.154490+0000","last_unstale":"2024-09-16T00:32:24.594672+0000","last_undegraded":"2024-09-16T00:32:24.594672+0000","last_fullsized":"2024-09-16T00:32:24.594672+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:59:53.854212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595549+0000","last_change":"2024-09-16T00:30:26.077069+0000","last_active":"2024-09-16T00:32:24.595549+0000","last_peered":"2024-09-16T00:32:24.595549+0000","last_clean":"2024-09-16T00:32:24.595549+0000","last_became_active":"2024-09-16T00:30:26.076718+0000","last_became_peered":"2024-09-16T00:30:26.076718+0000","last_unstale":"2024-09-16T00:32:24.595549+0000","last_undegraded":"2024-09-16T00:32:24.595549+0000","last_fullsized":"2024-09-16T00:32:24.595549+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:25:37.541704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452322+0000","last_change":"2024-09-16T00:30:23.932081+0000","last_active":"2024-09-16T00:32:24.452322+0000","last_peered":"2024-09-16T00:32:24.452322+0000","last_clean":"2024-09-16T00:32:24.452322+0000","last_became_active":"2024-09-16T00:30:23.931729+0000","last_became_peered":"2024-09-16T00:30:23.931729+0000","last_unstale":"2024-09-16T00:32:24.452322+0000","last_undegraded":"2024-09-16T00:32:24.452322+0000","last_fullsized":"2024-09-16T00:32:24.452322+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:20:05.178580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454073+0000","last_change":"2024-09-16T00:30:33.147738+0000","last_active":"2024-09-16T00:32:24.454073+0000","last_peered":"2024-09-16T00:32:24.454073+0000","last_clean":"2024-09-16T00:32:24.454073+0000","last_became_active":"2024-09-16T00:30:33.147326+0000","last_became_peered":"2024-09-16T00:30:33.147326+0000","last_unstale":"2024-09-16T00:32:24.454073+0000","last_undegraded":"2024-09-16T00:32:24.454073+0000","last_fullsized":"2024-09-16T00:32:24.454073+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:44:55.123980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339213+0000","last_change":"2024-09-16T00:30:26.079312+0000","last_active":"2024-09-16T00:32:24.339213+0000","last_peered":"2024-09-16T00:32:24.339213+0000","last_clean":"2024-09-16T00:32:24.339213+0000","last_became_active":"2024-09-16T00:30:26.078986+0000","last_became_peered":"2024-09-16T00:30:26.078986+0000","last_unstale":"2024-09-16T00:32:24.339213+0000","last_undegraded":"2024-09-16T00:32:24.339213+0000","last_fullsized":"2024-09-16T00:32:24.339213+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:02:06.602850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339614+0000","last_change":"2024-09-16T00:31:23.452102+0000","last_active":"2024-09-16T00:32:24.339614+0000","last_peered":"2024-09-16T00:32:24.339614+0000","last_clean":"2024-09-16T00:32:24.339614+0000","last_became_active":"2024-09-16T00:31:23.445974+0000","last_became_peered":"2024-09-16T00:31:23.445974+0000","last_unstale":"2024-09-16T00:32:24.339614+0000","last_undegraded":"2024-09-16T00:32:24.339614+0000","last_fullsized":"2024-09-16T00:32:24.339614+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:52.314494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454166+0000","last_change":"2024-09-16T00:30:33.149154+0000","last_active":"2024-09-16T00:32:24.454166+0000","last_peered":"2024-09-16T00:32:24.454166+0000","last_clean":"2024-09-16T00:32:24.454166+0000","last_became_active":"2024-09-16T00:30:33.148763+0000","last_became_peered":"2024-09-16T00:30:33.148763+0000","last_unstale":"2024-09-16T00:32:24.454166+0000","last_undegraded":"2024-09-16T00:32:24.454166+0000","last_fullsized":"2024-09-16T00:32:24.454166+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:58:05.884919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454295+0000","last_change":"2024-09-16T00:30:26.086905+0000","last_active":"2024-09-16T00:32:24.454295+0000","last_peered":"2024-09-16T00:32:24.454295+0000","last_clean":"2024-09-16T00:32:24.454295+0000","last_became_active":"2024-09-16T00:30:26.086519+0000","last_became_peered":"2024-09-16T00:30:26.086519+0000","last_unstale":"2024-09-16T00:32:24.454295+0000","last_undegraded":"2024-09-16T00:32:24.454295+0000","last_fullsized":"2024-09-16T00:32:24.454295+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:51:08.787679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.338826+0000","last_change":"2024-09-16T00:30:24.422893+0000","last_active":"2024-09-16T00:32:24.338826+0000","last_peered":"2024-09-16T00:32:24.338826+0000","last_clean":"2024-09-16T00:32:24.338826+0000","last_became_active":"2024-09-16T00:30:24.422073+0000","last_became_peered":"2024-09-16T00:30:24.422073+0000","last_unstale":"2024-09-16T00:32:24.338826+0000","last_undegraded":"2024-09-16T00:32:24.338826+0000","last_fullsized":"2024-09-16T00:32:24.338826+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:15:11.375957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452155+0000","last_change":"2024-09-16T00:30:33.152582+0000","last_active":"2024-09-16T00:32:24.452155+0000","last_peered":"2024-09-16T00:32:24.452155+0000","last_clean":"2024-09-16T00:32:24.452155+0000","last_became_active":"2024-09-16T00:30:33.152180+0000","last_became_peered":"2024-09-16T00:30:33.152180+0000","last_unstale":"2024-09-16T00:32:24.452155+0000","last_undegraded":"2024-09-16T00:32:24.452155+0000","last_fullsized":"2024-09-16T00:32:24.452155+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:41:10.499391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594802+0000","last_change":"2024-09-16T00:30:26.086052+0000","last_active":"2024-09-16T00:32:24.594802+0000","last_peered":"2024-09-16T00:32:24.594802+0000","last_clean":"2024-09-16T00:32:24.594802+0000","last_became_active":"2024-09-16T00:30:26.085756+0000","last_became_peered":"2024-09-16T00:30:26.085756+0000","last_unstale":"2024-09-16T00:32:24.594802+0000","last_undegraded":"2024-09-16T00:32:24.594802+0000","last_fullsized":"2024-09-16T00:32:24.594802+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:52.582166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":149,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:46.485452+0000","last_change":"2024-09-16T00:30:23.933528+0000","last_active":"2024-09-16T00:32:46.485452+0000","last_peered":"2024-09-16T00:32:46.485452+0000","last_clean":"2024-09-16T00:32:46.485452+0000","last_became_active":"2024-09-16T00:30:23.933186+0000","last_became_peered":"2024-09-16T00:30:23.933186+0000","last_unstale":"2024-09-16T00:32:46.485452+0000","last_undegraded":"2024-09-16T00:32:46.485452+0000","last_fullsized":"2024-09-16T00:32:46.485452+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:11:00.489676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.340727+0000","last_change":"2024-09-16T00:31:23.578056+0000","last_active":"2024-09-16T00:32:24.340727+0000","last_peered":"2024-09-16T00:32:24.340727+0000","last_clean":"2024-09-16T00:32:24.340727+0000","last_became_active":"2024-09-16T00:31:23.577721+0000","last_became_peered":"2024-09-16T00:31:23.577721+0000","last_unstale":"2024-09-16T00:32:24.340727+0000","last_undegraded":"2024-09-16T00:32:24.340727+0000","last_fullsized":"2024-09-16T00:32:24.340727+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:53.632440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595248+0000","last_change":"2024-09-16T00:30:26.084509+0000","last_active":"2024-09-16T00:32:24.595248+0000","last_peered":"2024-09-16T00:32:24.595248+0000","last_clean":"2024-09-16T00:32:24.595248+0000","last_became_active":"2024-09-16T00:30:26.084188+0000","last_became_peered":"2024-09-16T00:30:26.084188+0000","last_unstale":"2024-09-16T00:32:24.595248+0000","last_undegraded":"2024-09-16T00:32:24.595248+0000","last_fullsized":"2024-09-16T00:32:24.595248+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:21:48.981277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454529+0000","last_change":"2024-09-16T00:30:23.933499+0000","last_active":"2024-09-16T00:32:24.454529+0000","last_peered":"2024-09-16T00:32:24.454529+0000","last_clean":"2024-09-16T00:32:24.454529+0000","last_became_active":"2024-09-16T00:30:23.933082+0000","last_became_peered":"2024-09-16T00:30:23.933082+0000","last_unstale":"2024-09-16T00:32:24.454529+0000","last_undegraded":"2024-09-16T00:32:24.454529+0000","last_fullsized":"2024-09-16T00:32:24.454529+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:42:18.998433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.596318+0000","last_change":"2024-09-16T00:32:24.596318+0000","last_active":"2024-09-16T00:32:24.596318+0000","last_peered":"2024-09-16T00:32:24.596318+0000","last_clean":"2024-09-16T00:32:24.596318+0000","last_became_active":"2024-09-16T00:32:24.595906+0000","last_became_peered":"2024-09-16T00:32:24.595906+0000","last_unstale":"2024-09-16T00:32:24.596318+0000","last_undegraded":"2024-09-16T00:32:24.596318+0000","last_fullsized":"2024-09-16T00:32:24.596318+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:13:56.601745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452018+0000","last_change":"2024-09-16T00:30:26.080517+0000","last_active":"2024-09-16T00:32:24.452018+0000","last_peered":"2024-09-16T00:32:24.452018+0000","last_clean":"2024-09-16T00:32:24.452018+0000","last_became_active":"2024-09-16T00:30:26.080197+0000","last_became_peered":"2024-09-16T00:30:26.080197+0000","last_unstale":"2024-09-16T00:32:24.452018+0000","last_undegraded":"2024-09-16T00:32:24.452018+0000","last_fullsized":"2024-09-16T00:32:24.452018+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:15.508562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339267+0000","last_change":"2024-09-16T00:31:23.457047+0000","last_active":"2024-09-16T00:32:24.339267+0000","last_peered":"2024-09-16T00:32:24.339267+0000","last_clean":"2024-09-16T00:32:24.339267+0000","last_became_active":"2024-09-16T00:31:23.447014+0000","last_became_peered":"2024-09-16T00:31:23.447014+0000","last_unstale":"2024-09-16T00:32:24.339267+0000","last_undegraded":"2024-09-16T00:32:24.339267+0000","last_fullsized":"2024-09-16T00:32:24.339267+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:34:42.040228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454641+0000","last_change":"2024-09-16T00:30:33.145803+0000","last_active":"2024-09-16T00:32:24.454641+0000","last_peered":"2024-09-16T00:32:24.454641+0000","last_clean":"2024-09-16T00:32:24.454641+0000","last_became_active":"2024-09-16T00:30:33.145366+0000","last_became_peered":"2024-09-16T00:30:33.145366+0000","last_unstale":"2024-09-16T00:32:24.454641+0000","last_undegraded":"2024-09-16T00:32:24.454641+0000","last_fullsized":"2024-09-16T00:32:24.454641+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:47:28.689305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339085+0000","last_change":"2024-09-16T00:30:26.079415+0000","last_active":"2024-09-16T00:32:24.339085+0000","last_peered":"2024-09-16T00:32:24.339085+0000","last_clean":"2024-09-16T00:32:24.339085+0000","last_became_active":"2024-09-16T00:30:26.079075+0000","last_became_peered":"2024-09-16T00:30:26.079075+0000","last_unstale":"2024-09-16T00:32:24.339085+0000","last_undegraded":"2024-09-16T00:32:24.339085+0000","last_fullsized":"2024-09-16T00:32:24.339085+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:18:18.236342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"92'40","reported_seq":303,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:48.491764+0000","last_change":"2024-09-16T00:30:23.933172+0000","last_active":"2024-09-16T00:32:48.491764+0000","last_peered":"2024-09-16T00:32:48.491764+0000","last_clean":"2024-09-16T00:32:48.491764+0000","last_became_active":"2024-09-16T00:30:23.932492+0000","last_became_peered":"2024-09-16T00:30:23.932492+0000","last_unstale":"2024-09-16T00:32:48.491764+0000","last_undegraded":"2024-09-16T00:32:48.491764+0000","last_fullsized":"2024-09-16T00:32:48.491764+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:29.169448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70919,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":224,"num_read_kb":183,"num_write":41,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.337053+0000","last_change":"2024-09-16T00:31:23.445673+0000","last_active":"2024-09-16T00:32:24.337053+0000","last_peered":"2024-09-16T00:32:24.337053+0000","last_clean":"2024-09-16T00:32:24.337053+0000","last_became_active":"2024-09-16T00:31:23.445299+0000","last_became_peered":"2024-09-16T00:31:23.445299+0000","last_unstale":"2024-09-16T00:32:24.337053+0000","last_undegraded":"2024-09-16T00:32:24.337053+0000","last_fullsized":"2024-09-16T00:32:24.337053+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:07:44.235340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561729+0000","last_change":"2024-09-16T00:30:26.085419+0000","last_active":"2024-09-16T00:32:24.561729+0000","last_peered":"2024-09-16T00:32:24.561729+0000","last_clean":"2024-09-16T00:32:24.561729+0000","last_became_active":"2024-09-16T00:30:26.085103+0000","last_became_peered":"2024-09-16T00:30:26.085103+0000","last_unstale":"2024-09-16T00:32:24.561729+0000","last_undegraded":"2024-09-16T00:32:24.561729+0000","last_fullsized":"2024-09-16T00:32:24.561729+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:56:26.352779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.453958+0000","last_change":"2024-09-16T00:30:24.422646+0000","last_active":"2024-09-16T00:32:24.453958+0000","last_peered":"2024-09-16T00:32:24.453958+0000","last_clean":"2024-09-16T00:32:24.453958+0000","last_became_active":"2024-09-16T00:30:24.422065+0000","last_became_peered":"2024-09-16T00:30:24.422065+0000","last_unstale":"2024-09-16T00:32:24.453958+0000","last_undegraded":"2024-09-16T00:32:24.453958+0000","last_fullsized":"2024-09-16T00:32:24.453958+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:53:12.401613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454317+0000","last_change":"2024-09-16T00:30:33.147692+0000","last_active":"2024-09-16T00:32:24.454317+0000","last_peered":"2024-09-16T00:32:24.454317+0000","last_clean":"2024-09-16T00:32:24.454317+0000","last_became_active":"2024-09-16T00:30:33.147115+0000","last_became_peered":"2024-09-16T00:30:33.147115+0000","last_unstale":"2024-09-16T00:32:24.454317+0000","last_undegraded":"2024-09-16T00:32:24.454317+0000","last_fullsized":"2024-09-16T00:32:24.454317+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:44:04.536894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595596+0000","last_change":"2024-09-16T00:30:26.088427+0000","last_active":"2024-09-16T00:32:24.595596+0000","last_peered":"2024-09-16T00:32:24.595596+0000","last_clean":"2024-09-16T00:32:24.595596+0000","last_became_active":"2024-09-16T00:30:26.086454+0000","last_became_peered":"2024-09-16T00:30:26.086454+0000","last_unstale":"2024-09-16T00:32:24.595596+0000","last_undegraded":"2024-09-16T00:32:24.595596+0000","last_fullsized":"2024-09-16T00:32:24.595596+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:54:29.679959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595460+0000","last_change":"2024-09-16T00:31:23.453484+0000","last_active":"2024-09-16T00:32:24.595460+0000","last_peered":"2024-09-16T00:32:24.595460+0000","last_clean":"2024-09-16T00:32:24.595460+0000","last_became_active":"2024-09-16T00:31:23.446700+0000","last_became_peered":"2024-09-16T00:31:23.446700+0000","last_unstale":"2024-09-16T00:32:24.595460+0000","last_undegraded":"2024-09-16T00:32:24.595460+0000","last_fullsized":"2024-09-16T00:32:24.595460+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:43:26.465657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.336273+0000","last_change":"2024-09-16T00:30:33.156510+0000","last_active":"2024-09-16T00:32:24.336273+0000","last_peered":"2024-09-16T00:32:24.336273+0000","last_clean":"2024-09-16T00:32:24.336273+0000","last_became_active":"2024-09-16T00:30:33.156088+0000","last_became_peered":"2024-09-16T00:30:33.156088+0000","last_unstale":"2024-09-16T00:32:24.336273+0000","last_undegraded":"2024-09-16T00:32:24.336273+0000","last_fullsized":"2024-09-16T00:32:24.336273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:34.230901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561872+0000","last_change":"2024-09-16T00:30:26.082598+0000","last_active":"2024-09-16T00:32:24.561872+0000","last_peered":"2024-09-16T00:32:24.561872+0000","last_clean":"2024-09-16T00:32:24.561872+0000","last_became_active":"2024-09-16T00:30:26.082286+0000","last_became_peered":"2024-09-16T00:30:26.082286+0000","last_unstale":"2024-09-16T00:32:24.561872+0000","last_undegraded":"2024-09-16T00:32:24.561872+0000","last_fullsized":"2024-09-16T00:32:24.561872+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:22:18.579149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"92'5","reported_seq":150,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:48.491617+0000","last_change":"2024-09-16T00:30:23.931097+0000","last_active":"2024-09-16T00:32:48.491617+0000","last_peered":"2024-09-16T00:32:48.491617+0000","last_clean":"2024-09-16T00:32:48.491617+0000","last_became_active":"2024-09-16T00:30:23.930719+0000","last_became_peered":"2024-09-16T00:30:23.930719+0000","last_unstale":"2024-09-16T00:32:48.491617+0000","last_undegraded":"2024-09-16T00:32:48.491617+0000","last_fullsized":"2024-09-16T00:32:48.491617+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:13:47.887754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595241+0000","last_change":"2024-09-16T00:30:33.142581+0000","last_active":"2024-09-16T00:32:24.595241+0000","last_peered":"2024-09-16T00:32:24.595241+0000","last_clean":"2024-09-16T00:32:24.595241+0000","last_became_active":"2024-09-16T00:30:33.142263+0000","last_became_peered":"2024-09-16T00:30:33.142263+0000","last_unstale":"2024-09-16T00:32:24.595241+0000","last_undegraded":"2024-09-16T00:32:24.595241+0000","last_fullsized":"2024-09-16T00:32:24.595241+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:10:06.456011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595181+0000","last_change":"2024-09-16T00:30:33.152084+0000","last_active":"2024-09-16T00:32:24.595181+0000","last_peered":"2024-09-16T00:32:24.595181+0000","last_clean":"2024-09-16T00:32:24.595181+0000","last_became_active":"2024-09-16T00:30:33.151681+0000","last_became_peered":"2024-09-16T00:30:33.151681+0000","last_unstale":"2024-09-16T00:32:24.595181+0000","last_undegraded":"2024-09-16T00:32:24.595181+0000","last_fullsized":"2024-09-16T00:32:24.595181+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:36.966498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.336370+0000","last_change":"2024-09-16T00:30:26.084200+0000","last_active":"2024-09-16T00:32:24.336370+0000","last_peered":"2024-09-16T00:32:24.336370+0000","last_clean":"2024-09-16T00:32:24.336370+0000","last_became_active":"2024-09-16T00:30:26.083858+0000","last_became_peered":"2024-09-16T00:30:26.083858+0000","last_unstale":"2024-09-16T00:32:24.336370+0000","last_undegraded":"2024-09-16T00:32:24.336370+0000","last_fullsized":"2024-09-16T00:32:24.336370+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:42.712624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454400+0000","last_change":"2024-09-16T00:30:23.932399+0000","last_active":"2024-09-16T00:32:24.454400+0000","last_peered":"2024-09-16T00:32:24.454400+0000","last_clean":"2024-09-16T00:32:24.454400+0000","last_became_active":"2024-09-16T00:30:23.931996+0000","last_became_peered":"2024-09-16T00:30:23.931996+0000","last_unstale":"2024-09-16T00:32:24.454400+0000","last_undegraded":"2024-09-16T00:32:24.454400+0000","last_fullsized":"2024-09-16T00:32:24.454400+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:47:09.634507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339553+0000","last_change":"2024-09-16T00:30:33.154563+0000","last_active":"2024-09-16T00:32:24.339553+0000","last_peered":"2024-09-16T00:32:24.339553+0000","last_clean":"2024-09-16T00:32:24.339553+0000","last_became_active":"2024-09-16T00:30:33.154105+0000","last_became_peered":"2024-09-16T00:30:33.154105+0000","last_unstale":"2024-09-16T00:32:24.339553+0000","last_undegraded":"2024-09-16T00:32:24.339553+0000","last_fullsized":"2024-09-16T00:32:24.339553+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:59.966255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595005+0000","last_change":"2024-09-16T00:30:26.075126+0000","last_active":"2024-09-16T00:32:24.595005+0000","last_peered":"2024-09-16T00:32:24.595005+0000","last_clean":"2024-09-16T00:32:24.595005+0000","last_became_active":"2024-09-16T00:30:26.074819+0000","last_became_peered":"2024-09-16T00:30:26.074819+0000","last_unstale":"2024-09-16T00:32:24.595005+0000","last_undegraded":"2024-09-16T00:32:24.595005+0000","last_fullsized":"2024-09-16T00:32:24.595005+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:36.996346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.451802+0000","last_change":"2024-09-16T00:30:23.929943+0000","last_active":"2024-09-16T00:32:24.451802+0000","last_peered":"2024-09-16T00:32:24.451802+0000","last_clean":"2024-09-16T00:32:24.451802+0000","last_became_active":"2024-09-16T00:30:23.929578+0000","last_became_peered":"2024-09-16T00:30:23.929578+0000","last_unstale":"2024-09-16T00:32:24.451802+0000","last_undegraded":"2024-09-16T00:32:24.451802+0000","last_fullsized":"2024-09-16T00:32:24.451802+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:05:12.505543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.337224+0000","last_change":"2024-09-16T00:30:23.935758+0000","last_active":"2024-09-16T00:32:24.337224+0000","last_peered":"2024-09-16T00:32:24.337224+0000","last_clean":"2024-09-16T00:32:24.337224+0000","last_became_active":"2024-09-16T00:30:23.935321+0000","last_became_peered":"2024-09-16T00:30:23.935321+0000","last_unstale":"2024-09-16T00:32:24.337224+0000","last_undegraded":"2024-09-16T00:32:24.337224+0000","last_fullsized":"2024-09-16T00:32:24.337224+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:35:40.513435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454389+0000","last_change":"2024-09-16T00:30:26.086316+0000","last_active":"2024-09-16T00:32:24.454389+0000","last_peered":"2024-09-16T00:32:24.454389+0000","last_clean":"2024-09-16T00:32:24.454389+0000","last_became_active":"2024-09-16T00:30:26.085800+0000","last_became_peered":"2024-09-16T00:30:26.085800+0000","last_unstale":"2024-09-16T00:32:24.454389+0000","last_undegraded":"2024-09-16T00:32:24.454389+0000","last_fullsized":"2024-09-16T00:32:24.454389+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:05:36.277127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595014+0000","last_change":"2024-09-16T00:30:33.155665+0000","last_active":"2024-09-16T00:32:24.595014+0000","last_peered":"2024-09-16T00:32:24.595014+0000","last_clean":"2024-09-16T00:32:24.595014+0000","last_became_active":"2024-09-16T00:30:33.155253+0000","last_became_peered":"2024-09-16T00:30:33.155253+0000","last_unstale":"2024-09-16T00:32:24.595014+0000","last_undegraded":"2024-09-16T00:32:24.595014+0000","last_fullsized":"2024-09-16T00:32:24.595014+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:37.545232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'8","reported_seq":156,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:46.488890+0000","last_change":"2024-09-16T00:30:23.934487+0000","last_active":"2024-09-16T00:32:46.488890+0000","last_peered":"2024-09-16T00:32:46.488890+0000","last_clean":"2024-09-16T00:32:46.488890+0000","last_became_active":"2024-09-16T00:30:23.934137+0000","last_became_peered":"2024-09-16T00:30:23.934137+0000","last_unstale":"2024-09-16T00:32:46.488890+0000","last_undegraded":"2024-09-16T00:32:46.488890+0000","last_fullsized":"2024-09-16T00:32:46.488890+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:14:33.088776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1549,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.338839+0000","last_change":"2024-09-16T00:30:26.077533+0000","last_active":"2024-09-16T00:32:24.338839+0000","last_peered":"2024-09-16T00:32:24.338839+0000","last_clean":"2024-09-16T00:32:24.338839+0000","last_became_active":"2024-09-16T00:30:26.076443+0000","last_became_peered":"2024-09-16T00:30:26.076443+0000","last_unstale":"2024-09-16T00:32:24.338839+0000","last_undegraded":"2024-09-16T00:32:24.338839+0000","last_fullsized":"2024-09-16T00:32:24.338839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:39:17.495521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595151+0000","last_change":"2024-09-16T00:30:33.146123+0000","last_active":"2024-09-16T00:32:24.595151+0000","last_peered":"2024-09-16T00:32:24.595151+0000","last_clean":"2024-09-16T00:32:24.595151+0000","last_became_active":"2024-09-16T00:30:33.145716+0000","last_became_peered":"2024-09-16T00:30:33.145716+0000","last_unstale":"2024-09-16T00:32:24.595151+0000","last_undegraded":"2024-09-16T00:32:24.595151+0000","last_fullsized":"2024-09-16T00:32:24.595151+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:14:27.281367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.338818+0000","last_change":"2024-09-16T00:30:26.083155+0000","last_active":"2024-09-16T00:32:24.338818+0000","last_peered":"2024-09-16T00:32:24.338818+0000","last_clean":"2024-09-16T00:32:24.338818+0000","last_became_active":"2024-09-16T00:30:26.082782+0000","last_became_peered":"2024-09-16T00:30:26.082782+0000","last_unstale":"2024-09-16T00:32:24.338818+0000","last_undegraded":"2024-09-16T00:32:24.338818+0000","last_fullsized":"2024-09-16T00:32:24.338818+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:15.929343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594891+0000","last_change":"2024-09-16T00:31:23.447453+0000","last_active":"2024-09-16T00:32:24.594891+0000","last_peered":"2024-09-16T00:32:24.594891+0000","last_clean":"2024-09-16T00:32:24.594891+0000","last_became_active":"2024-09-16T00:31:23.445507+0000","last_became_peered":"2024-09-16T00:31:23.445507+0000","last_unstale":"2024-09-16T00:32:24.594891+0000","last_undegraded":"2024-09-16T00:32:24.594891+0000","last_fullsized":"2024-09-16T00:32:24.594891+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:18:50.885809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594798+0000","last_change":"2024-09-16T00:30:26.076252+0000","last_active":"2024-09-16T00:32:24.594798+0000","last_peered":"2024-09-16T00:32:24.594798+0000","last_clean":"2024-09-16T00:32:24.594798+0000","last_became_active":"2024-09-16T00:30:26.075922+0000","last_became_peered":"2024-09-16T00:30:26.075922+0000","last_unstale":"2024-09-16T00:32:24.594798+0000","last_undegraded":"2024-09-16T00:32:24.594798+0000","last_fullsized":"2024-09-16T00:32:24.594798+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:28:04.822817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594821+0000","last_change":"2024-09-16T00:30:26.085123+0000","last_active":"2024-09-16T00:32:24.594821+0000","last_peered":"2024-09-16T00:32:24.594821+0000","last_clean":"2024-09-16T00:32:24.594821+0000","last_became_active":"2024-09-16T00:30:26.084782+0000","last_became_peered":"2024-09-16T00:30:26.084782+0000","last_unstale":"2024-09-16T00:32:24.594821+0000","last_undegraded":"2024-09-16T00:32:24.594821+0000","last_fullsized":"2024-09-16T00:32:24.594821+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:09:33.766340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452404+0000","last_change":"2024-09-16T00:30:26.077048+0000","last_active":"2024-09-16T00:32:24.452404+0000","last_peered":"2024-09-16T00:32:24.452404+0000","last_clean":"2024-09-16T00:32:24.452404+0000","last_became_active":"2024-09-16T00:30:26.076719+0000","last_became_peered":"2024-09-16T00:30:26.076719+0000","last_unstale":"2024-09-16T00:32:24.452404+0000","last_undegraded":"2024-09-16T00:32:24.452404+0000","last_fullsized":"2024-09-16T00:32:24.452404+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:58.584246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339277+0000","last_change":"2024-09-16T00:30:26.078446+0000","last_active":"2024-09-16T00:32:24.339277+0000","last_peered":"2024-09-16T00:32:24.339277+0000","last_clean":"2024-09-16T00:32:24.339277+0000","last_became_active":"2024-09-16T00:30:26.078145+0000","last_became_peered":"2024-09-16T00:30:26.078145+0000","last_unstale":"2024-09-16T00:32:24.339277+0000","last_undegraded":"2024-09-16T00:32:24.339277+0000","last_fullsized":"2024-09-16T00:32:24.339277+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:52:28.739834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595347+0000","last_change":"2024-09-16T00:30:26.075875+0000","last_active":"2024-09-16T00:32:24.595347+0000","last_peered":"2024-09-16T00:32:24.595347+0000","last_clean":"2024-09-16T00:32:24.595347+0000","last_became_active":"2024-09-16T00:30:26.075424+0000","last_became_peered":"2024-09-16T00:30:26.075424+0000","last_unstale":"2024-09-16T00:32:24.595347+0000","last_undegraded":"2024-09-16T00:32:24.595347+0000","last_fullsized":"2024-09-16T00:32:24.595347+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:21:07.958507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595253+0000","last_change":"2024-09-16T00:30:26.077694+0000","last_active":"2024-09-16T00:32:24.595253+0000","last_peered":"2024-09-16T00:32:24.595253+0000","last_clean":"2024-09-16T00:32:24.595253+0000","last_became_active":"2024-09-16T00:30:26.077374+0000","last_became_peered":"2024-09-16T00:30:26.077374+0000","last_unstale":"2024-09-16T00:32:24.595253+0000","last_undegraded":"2024-09-16T00:32:24.595253+0000","last_fullsized":"2024-09-16T00:32:24.595253+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:16:30.822050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454143+0000","last_change":"2024-09-16T00:30:26.085931+0000","last_active":"2024-09-16T00:32:24.454143+0000","last_peered":"2024-09-16T00:32:24.454143+0000","last_clean":"2024-09-16T00:32:24.454143+0000","last_became_active":"2024-09-16T00:30:26.085420+0000","last_became_peered":"2024-09-16T00:30:26.085420+0000","last_unstale":"2024-09-16T00:32:24.454143+0000","last_undegraded":"2024-09-16T00:32:24.454143+0000","last_fullsized":"2024-09-16T00:32:24.454143+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:20:36.019203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595470+0000","last_change":"2024-09-16T00:30:26.074217+0000","last_active":"2024-09-16T00:32:24.595470+0000","last_peered":"2024-09-16T00:32:24.595470+0000","last_clean":"2024-09-16T00:32:24.595470+0000","last_became_active":"2024-09-16T00:30:26.073856+0000","last_became_peered":"2024-09-16T00:30:26.073856+0000","last_unstale":"2024-09-16T00:32:24.595470+0000","last_undegraded":"2024-09-16T00:32:24.595470+0000","last_fullsized":"2024-09-16T00:32:24.595470+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:53:57.078465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561639+0000","last_change":"2024-09-16T00:30:26.084248+0000","last_active":"2024-09-16T00:32:24.561639+0000","last_peered":"2024-09-16T00:32:24.561639+0000","last_clean":"2024-09-16T00:32:24.561639+0000","last_became_active":"2024-09-16T00:30:26.083530+0000","last_became_peered":"2024-09-16T00:30:26.083530+0000","last_unstale":"2024-09-16T00:32:24.561639+0000","last_undegraded":"2024-09-16T00:32:24.561639+0000","last_fullsized":"2024-09-16T00:32:24.561639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:16:49.483881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452162+0000","last_change":"2024-09-16T00:30:26.080803+0000","last_active":"2024-09-16T00:32:24.452162+0000","last_peered":"2024-09-16T00:32:24.452162+0000","last_clean":"2024-09-16T00:32:24.452162+0000","last_became_active":"2024-09-16T00:30:26.080482+0000","last_became_peered":"2024-09-16T00:30:26.080482+0000","last_unstale":"2024-09-16T00:32:24.452162+0000","last_undegraded":"2024-09-16T00:32:24.452162+0000","last_fullsized":"2024-09-16T00:32:24.452162+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:35:41.741025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339770+0000","last_change":"2024-09-16T00:30:26.077680+0000","last_active":"2024-09-16T00:32:24.339770+0000","last_peered":"2024-09-16T00:32:24.339770+0000","last_clean":"2024-09-16T00:32:24.339770+0000","last_became_active":"2024-09-16T00:30:26.077371+0000","last_became_peered":"2024-09-16T00:30:26.077371+0000","last_unstale":"2024-09-16T00:32:24.339770+0000","last_undegraded":"2024-09-16T00:32:24.339770+0000","last_fullsized":"2024-09-16T00:32:24.339770+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:31:34.338617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595364+0000","last_change":"2024-09-16T00:30:26.071531+0000","last_active":"2024-09-16T00:32:24.595364+0000","last_peered":"2024-09-16T00:32:24.595364+0000","last_clean":"2024-09-16T00:32:24.595364+0000","last_became_active":"2024-09-16T00:30:26.071189+0000","last_became_peered":"2024-09-16T00:30:26.071189+0000","last_unstale":"2024-09-16T00:32:24.595364+0000","last_undegraded":"2024-09-16T00:32:24.595364+0000","last_fullsized":"2024-09-16T00:32:24.595364+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:29:18.406272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.336922+0000","last_change":"2024-09-16T00:30:26.085398+0000","last_active":"2024-09-16T00:32:24.336922+0000","last_peered":"2024-09-16T00:32:24.336922+0000","last_clean":"2024-09-16T00:32:24.336922+0000","last_became_active":"2024-09-16T00:30:26.085076+0000","last_became_peered":"2024-09-16T00:30:26.085076+0000","last_unstale":"2024-09-16T00:32:24.336922+0000","last_undegraded":"2024-09-16T00:32:24.336922+0000","last_fullsized":"2024-09-16T00:32:24.336922+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:20:20.864660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.340070+0000","last_change":"2024-09-16T00:30:26.080953+0000","last_active":"2024-09-16T00:32:24.340070+0000","last_peered":"2024-09-16T00:32:24.340070+0000","last_clean":"2024-09-16T00:32:24.340070+0000","last_became_active":"2024-09-16T00:30:26.080185+0000","last_became_peered":"2024-09-16T00:30:26.080185+0000","last_unstale":"2024-09-16T00:32:24.340070+0000","last_undegraded":"2024-09-16T00:32:24.340070+0000","last_fullsized":"2024-09-16T00:32:24.340070+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:56.769182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561297+0000","last_change":"2024-09-16T00:30:26.079320+0000","last_active":"2024-09-16T00:32:24.561297+0000","last_peered":"2024-09-16T00:32:24.561297+0000","last_clean":"2024-09-16T00:32:24.561297+0000","last_became_active":"2024-09-16T00:30:26.078980+0000","last_became_peered":"2024-09-16T00:30:26.078980+0000","last_unstale":"2024-09-16T00:32:24.561297+0000","last_undegraded":"2024-09-16T00:32:24.561297+0000","last_fullsized":"2024-09-16T00:32:24.561297+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:03:15.386657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594661+0000","last_change":"2024-09-16T00:30:26.084948+0000","last_active":"2024-09-16T00:32:24.594661+0000","last_peered":"2024-09-16T00:32:24.594661+0000","last_clean":"2024-09-16T00:32:24.594661+0000","last_became_active":"2024-09-16T00:30:26.084602+0000","last_became_peered":"2024-09-16T00:30:26.084602+0000","last_unstale":"2024-09-16T00:32:24.594661+0000","last_undegraded":"2024-09-16T00:32:24.594661+0000","last_fullsized":"2024-09-16T00:32:24.594661+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:01.709332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594560+0000","last_change":"2024-09-16T00:30:26.080491+0000","last_active":"2024-09-16T00:32:24.594560+0000","last_peered":"2024-09-16T00:32:24.594560+0000","last_clean":"2024-09-16T00:32:24.594560+0000","last_became_active":"2024-09-16T00:30:26.080189+0000","last_became_peered":"2024-09-16T00:30:26.080189+0000","last_unstale":"2024-09-16T00:32:24.594560+0000","last_undegraded":"2024-09-16T00:32:24.594560+0000","last_fullsized":"2024-09-16T00:32:24.594560+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:38.358919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452035+0000","last_change":"2024-09-16T00:30:26.077410+0000","last_active":"2024-09-16T00:32:24.452035+0000","last_peered":"2024-09-16T00:32:24.452035+0000","last_clean":"2024-09-16T00:32:24.452035+0000","last_became_active":"2024-09-16T00:30:26.076498+0000","last_became_peered":"2024-09-16T00:30:26.076498+0000","last_unstale":"2024-09-16T00:32:24.452035+0000","last_undegraded":"2024-09-16T00:32:24.452035+0000","last_fullsized":"2024-09-16T00:32:24.452035+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:54:17.712774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454822+0000","last_change":"2024-09-16T00:30:26.074500+0000","last_active":"2024-09-16T00:32:24.454822+0000","last_peered":"2024-09-16T00:32:24.454822+0000","last_clean":"2024-09-16T00:32:24.454822+0000","last_became_active":"2024-09-16T00:30:26.074177+0000","last_became_peered":"2024-09-16T00:30:26.074177+0000","last_unstale":"2024-09-16T00:32:24.454822+0000","last_undegraded":"2024-09-16T00:32:24.454822+0000","last_fullsized":"2024-09-16T00:32:24.454822+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:43:13.792231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595503+0000","last_change":"2024-09-16T00:30:26.078358+0000","last_active":"2024-09-16T00:32:24.595503+0000","last_peered":"2024-09-16T00:32:24.595503+0000","last_clean":"2024-09-16T00:32:24.595503+0000","last_became_active":"2024-09-16T00:30:26.077993+0000","last_became_peered":"2024-09-16T00:30:26.077993+0000","last_unstale":"2024-09-16T00:32:24.595503+0000","last_undegraded":"2024-09-16T00:32:24.595503+0000","last_fullsized":"2024-09-16T00:32:24.595503+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:25:10.711394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.337671+0000","last_change":"2024-09-16T00:31:23.446836+0000","last_active":"2024-09-16T00:32:24.337671+0000","last_peered":"2024-09-16T00:32:24.337671+0000","last_clean":"2024-09-16T00:32:24.337671+0000","last_became_active":"2024-09-16T00:31:23.446471+0000","last_became_peered":"2024-09-16T00:31:23.446471+0000","last_unstale":"2024-09-16T00:32:24.337671+0000","last_undegraded":"2024-09-16T00:32:24.337671+0000","last_fullsized":"2024-09-16T00:32:24.337671+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:08:34.887618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561867+0000","last_change":"2024-09-16T00:30:26.086313+0000","last_active":"2024-09-16T00:32:24.561867+0000","last_peered":"2024-09-16T00:32:24.561867+0000","last_clean":"2024-09-16T00:32:24.561867+0000","last_became_active":"2024-09-16T00:30:26.085760+0000","last_became_peered":"2024-09-16T00:30:26.085760+0000","last_unstale":"2024-09-16T00:32:24.561867+0000","last_undegraded":"2024-09-16T00:32:24.561867+0000","last_fullsized":"2024-09-16T00:32:24.561867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:40:38.236789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594968+0000","last_change":"2024-09-16T00:30:26.085289+0000","last_active":"2024-09-16T00:32:24.594968+0000","last_peered":"2024-09-16T00:32:24.594968+0000","last_clean":"2024-09-16T00:32:24.594968+0000","last_became_active":"2024-09-16T00:30:26.084995+0000","last_became_peered":"2024-09-16T00:30:26.084995+0000","last_unstale":"2024-09-16T00:32:24.594968+0000","last_undegraded":"2024-09-16T00:32:24.594968+0000","last_fullsized":"2024-09-16T00:32:24.594968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:45.667892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594850+0000","last_change":"2024-09-16T00:30:26.088690+0000","last_active":"2024-09-16T00:32:24.594850+0000","last_peered":"2024-09-16T00:32:24.594850+0000","last_clean":"2024-09-16T00:32:24.594850+0000","last_became_active":"2024-09-16T00:30:26.087700+0000","last_became_peered":"2024-09-16T00:30:26.087700+0000","last_unstale":"2024-09-16T00:32:24.594850+0000","last_undegraded":"2024-09-16T00:32:24.594850+0000","last_fullsized":"2024-09-16T00:32:24.594850+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:39:51.445324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.337214+0000","last_change":"2024-09-16T00:30:26.080215+0000","last_active":"2024-09-16T00:32:24.337214+0000","last_peered":"2024-09-16T00:32:24.337214+0000","last_clean":"2024-09-16T00:32:24.337214+0000","last_became_active":"2024-09-16T00:30:26.079038+0000","last_became_peered":"2024-09-16T00:30:26.079038+0000","last_unstale":"2024-09-16T00:32:24.337214+0000","last_undegraded":"2024-09-16T00:32:24.337214+0000","last_fullsized":"2024-09-16T00:32:24.337214+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:01.573960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339169+0000","last_change":"2024-09-16T00:30:26.080966+0000","last_active":"2024-09-16T00:32:24.339169+0000","last_peered":"2024-09-16T00:32:24.339169+0000","last_clean":"2024-09-16T00:32:24.339169+0000","last_became_active":"2024-09-16T00:30:26.080641+0000","last_became_peered":"2024-09-16T00:30:26.080641+0000","last_unstale":"2024-09-16T00:32:24.339169+0000","last_undegraded":"2024-09-16T00:32:24.339169+0000","last_fullsized":"2024-09-16T00:32:24.339169+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:33:01.538169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454762+0000","last_change":"2024-09-16T00:30:26.072644+0000","last_active":"2024-09-16T00:32:24.454762+0000","last_peered":"2024-09-16T00:32:24.454762+0000","last_clean":"2024-09-16T00:32:24.454762+0000","last_became_active":"2024-09-16T00:30:26.072336+0000","last_became_peered":"2024-09-16T00:30:26.072336+0000","last_unstale":"2024-09-16T00:32:24.454762+0000","last_undegraded":"2024-09-16T00:32:24.454762+0000","last_fullsized":"2024-09-16T00:32:24.454762+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:44:45.760057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595215+0000","last_change":"2024-09-16T00:30:26.079155+0000","last_active":"2024-09-16T00:32:24.595215+0000","last_peered":"2024-09-16T00:32:24.595215+0000","last_clean":"2024-09-16T00:32:24.595215+0000","last_became_active":"2024-09-16T00:30:26.078820+0000","last_became_peered":"2024-09-16T00:30:26.078820+0000","last_unstale":"2024-09-16T00:32:24.595215+0000","last_undegraded":"2024-09-16T00:32:24.595215+0000","last_fullsized":"2024-09-16T00:32:24.595215+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:35.071143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595361+0000","last_change":"2024-09-16T00:30:26.081537+0000","last_active":"2024-09-16T00:32:24.595361+0000","last_peered":"2024-09-16T00:32:24.595361+0000","last_clean":"2024-09-16T00:32:24.595361+0000","last_became_active":"2024-09-16T00:30:26.080744+0000","last_became_peered":"2024-09-16T00:30:26.080744+0000","last_unstale":"2024-09-16T00:32:24.595361+0000","last_undegraded":"2024-09-16T00:32:24.595361+0000","last_fullsized":"2024-09-16T00:32:24.595361+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:46:38.179888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.340007+0000","last_change":"2024-09-16T00:30:26.081066+0000","last_active":"2024-09-16T00:32:24.340007+0000","last_peered":"2024-09-16T00:32:24.340007+0000","last_clean":"2024-09-16T00:32:24.340007+0000","last_became_active":"2024-09-16T00:30:26.080630+0000","last_became_peered":"2024-09-16T00:30:26.080630+0000","last_unstale":"2024-09-16T00:32:24.340007+0000","last_undegraded":"2024-09-16T00:32:24.340007+0000","last_fullsized":"2024-09-16T00:32:24.340007+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:44:42.644792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75456,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":663,"num_read_kb":412,"num_write":184,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":446464,"data_stored":221328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514538,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1060864,"data_stored":523210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743472,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32896,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712256,"statfs":{"total":95995035648,"available":95961350144,"internally_reserved":0,"allocated":1753088,"data_stored":1181341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906997,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":527486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135930,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1064960,"data_stored":523288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397566,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1662976,"data_stored":1119755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561092,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1048576,"data_stored":518566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790025,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28656,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716496,"statfs":{"total":95995035648,"available":95965691904,"internally_reserved":0,"allocated":1671168,"data_stored":1115343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953551,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1060864,"data_stored":523216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182482,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":518760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411417,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1116,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1142784,"data_stored":589118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574942,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28120,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717032,"statfs":{"total":95995035648,"available":95966240768,"internally_reserved":0,"allocated":1122304,"data_stored":586875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738467,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1073152,"data_stored":519954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":68399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":70642,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":72497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:32:52.125 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T00:32:52.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:51 smithi122 ceph-mon[35412]: scrub queued for path: / 2024-09-16T00:32:52.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:51 smithi122 ceph-mon[35412]: scrub summary: idle+waiting paths [/] 2024-09-16T00:32:52.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:51 smithi122 ceph-mon[35412]: scrub summary: active paths [/] 2024-09-16T00:32:52.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:51 smithi122 ceph-mon[35412]: scrub summary: idle+waiting paths [/] 2024-09-16T00:32:52.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:51 smithi122 ceph-mon[35412]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:32:52.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:51 smithi073 ceph-mon[35254]: scrub queued for path: / 2024-09-16T00:32:52.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:51 smithi073 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-09-16T00:32:52.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:51 smithi073 ceph-mon[35254]: scrub summary: active paths [/] 2024-09-16T00:32:52.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:51 smithi073 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-09-16T00:32:52.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:51 smithi073 ceph-mon[35254]: from='client.15333 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:32:52.603 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:32:52.725 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:53.146 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:52 smithi038 ceph-mon[30945]: pgmap v367: 129 pgs: 129 active+clean; 650 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-09-16T00:32:53.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:52 smithi122 ceph-mon[35412]: pgmap v367: 129 pgs: 129 active+clean; 650 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-09-16T00:32:53.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.228+0000 7f6e7233a640 1 -- 172.21.15.38:0/3523666741 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e6c102790 msgr2=0x7f6e6c1067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:53.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.228+0000 7f6e7233a640 1 --2- 172.21.15.38:0/3523666741 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e6c102790 0x7f6e6c1067e0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6e60009900 tx=0x7f6e6002a410 comp rx=0 tx=0).stop 2024-09-16T00:32:53.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.228+0000 7f6e7233a640 1 -- 172.21.15.38:0/3523666741 shutdown_connections 2024-09-16T00:32:53.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.228+0000 7f6e7233a640 1 --2- 172.21.15.38:0/3523666741 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e6c106f10 0x7f6e6c10ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.228+0000 7f6e7233a640 1 --2- 172.21.15.38:0/3523666741 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e6c102790 0x7f6e6c1067e0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.228+0000 7f6e7233a640 1 --2- 172.21.15.38:0/3523666741 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e6c101de0 0x7f6e6c1021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.228+0000 7f6e7233a640 1 -- 172.21.15.38:0/3523666741 >> 172.21.15.38:0/3523666741 conn(0x7f6e6c0fd8c0 msgr2=0x7f6e6c0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:53.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.228+0000 7f6e7233a640 1 -- 172.21.15.38:0/3523666741 shutdown_connections 2024-09-16T00:32:53.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.228+0000 7f6e7233a640 1 -- 172.21.15.38:0/3523666741 wait complete. 2024-09-16T00:32:53.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.229+0000 7f6e7233a640 1 Processor -- start 2024-09-16T00:32:53.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.229+0000 7f6e7233a640 1 -- start start 2024-09-16T00:32:53.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.229+0000 7f6e7233a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e6c101de0 0x7f6e6c198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:53.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.229+0000 7f6e7233a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e6c102790 0x7f6e6c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:53.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.229+0000 7f6e7233a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e6c106f10 0x7f6e6c19cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:53.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.229+0000 7f6e7233a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6c111f70 con 0x7f6e6c101de0 2024-09-16T00:32:53.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.229+0000 7f6e7233a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6c111df0 con 0x7f6e6c106f10 2024-09-16T00:32:53.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.229+0000 7f6e7233a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6c1120f0 con 0x7f6e6c102790 2024-09-16T00:32:53.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.229+0000 7f6e6bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e6c101de0 0x7f6e6c198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:53.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.229+0000 7f6e6bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e6c101de0 0x7f6e6c198510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57320/0 (socket says 172.21.15.38:57320) 2024-09-16T00:32:53.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.229+0000 7f6e6bfff640 1 -- 172.21.15.38:0/1654646336 learned_addr learned my addr 172.21.15.38:0/1654646336 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:53.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.230+0000 7f6e6b7fe640 1 --2- 172.21.15.38:0/1654646336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e6c102790 0x7f6e6c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:53.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.230+0000 7f6e708d7640 1 --2- 172.21.15.38:0/1654646336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e6c106f10 0x7f6e6c19cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:53.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.230+0000 7f6e6bfff640 1 -- 172.21.15.38:0/1654646336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e6c102790 msgr2=0x7f6e6c198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:53.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.230+0000 7f6e6bfff640 1 --2- 172.21.15.38:0/1654646336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e6c102790 0x7f6e6c198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.230+0000 7f6e6bfff640 1 -- 172.21.15.38:0/1654646336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e6c106f10 msgr2=0x7f6e6c19cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:53.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.230+0000 7f6e6bfff640 1 --2- 172.21.15.38:0/1654646336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e6c106f10 0x7f6e6c19cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.230+0000 7f6e6bfff640 1 -- 172.21.15.38:0/1654646336 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e6c19d4c0 con 0x7f6e6c101de0 2024-09-16T00:32:53.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.230+0000 7f6e6bfff640 1 --2- 172.21.15.38:0/1654646336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e6c101de0 0x7f6e6c198510 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c00d770 tx=0x7f6e5c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:53.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.230+0000 7f6e697fa640 1 -- 172.21.15.38:0/1654646336 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c017070 con 0x7f6e6c101de0 2024-09-16T00:32:53.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.230+0000 7f6e697fa640 1 -- 172.21.15.38:0/1654646336 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6e5c005500 con 0x7f6e6c101de0 2024-09-16T00:32:53.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.230+0000 7f6e7233a640 1 -- 172.21.15.38:0/1654646336 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e6c19d7b0 con 0x7f6e6c101de0 2024-09-16T00:32:53.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.230+0000 7f6e697fa640 1 -- 172.21.15.38:0/1654646336 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c004e00 con 0x7f6e6c101de0 2024-09-16T00:32:53.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.230+0000 7f6e7233a640 1 -- 172.21.15.38:0/1654646336 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e6c1a5090 con 0x7f6e6c101de0 2024-09-16T00:32:53.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.231+0000 7f6e7233a640 1 -- 172.21.15.38:0/1654646336 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e6c1021c0 con 0x7f6e6c101de0 2024-09-16T00:32:53.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.233+0000 7f6e697fa640 1 -- 172.21.15.38:0/1654646336 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f6e5c004fa0 con 0x7f6e6c101de0 2024-09-16T00:32:53.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.233+0000 7f6e697fa640 1 --2- 172.21.15.38:0/1654646336 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f6e540a3350 0x7f6e540a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:53.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.234+0000 7f6e6b7fe640 1 --2- 172.21.15.38:0/1654646336 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f6e540a3350 0x7f6e540a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:53.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.234+0000 7f6e697fa640 1 -- 172.21.15.38:0/1654646336 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f6e5c0cf7f0 con 0x7f6e6c101de0 2024-09-16T00:32:53.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.234+0000 7f6e6b7fe640 1 --2- 172.21.15.38:0/1654646336 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f6e540a3350 0x7f6e540a5810 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6e60009230 tx=0x7f6e600058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:53.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.238+0000 7f6e697fa640 1 -- 172.21.15.38:0/1654646336 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6e5c09c030 con 0x7f6e6c101de0 2024-09-16T00:32:53.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:52 smithi073 ceph-mon[35254]: pgmap v367: 129 pgs: 129 active+clean; 650 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.3 KiB/s wr, 6 op/s 2024-09-16T00:32:53.294 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:53.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.422+0000 7f6e7233a640 1 -- 172.21.15.38:0/1654646336 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6e6c10a1c0 con 0x7f6e540a3350 2024-09-16T00:32:53.430 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.430+0000 7f6e697fa640 1 -- 172.21.15.38:0/1654646336 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340494 (secure 0 0 0) 0x7f6e60031630 con 0x7f6e540a3350 2024-09-16T00:32:53.433 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:32:53.435 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-16T00:32:53.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.433+0000 7f6e7233a640 1 -- 172.21.15.38:0/1654646336 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f6e540a3350 msgr2=0x7f6e540a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:53.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.433+0000 7f6e7233a640 1 --2- 172.21.15.38:0/1654646336 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f6e540a3350 0x7f6e540a5810 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6e60009230 tx=0x7f6e600058d0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.433+0000 7f6e7233a640 1 -- 172.21.15.38:0/1654646336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e6c101de0 msgr2=0x7f6e6c198510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:53.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.433+0000 7f6e7233a640 1 --2- 172.21.15.38:0/1654646336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e6c101de0 0x7f6e6c198510 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c00d770 tx=0x7f6e5c00dc30 comp rx=0 tx=0).stop 2024-09-16T00:32:53.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.433+0000 7f6e7233a640 1 -- 172.21.15.38:0/1654646336 shutdown_connections 2024-09-16T00:32:53.438 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.433+0000 7f6e7233a640 1 --2- 172.21.15.38:0/1654646336 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f6e540a3350 0x7f6e540a5810 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.433+0000 7f6e7233a640 1 --2- 172.21.15.38:0/1654646336 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e6c106f10 0x7f6e6c19cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.433+0000 7f6e7233a640 1 --2- 172.21.15.38:0/1654646336 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6e6c102790 0x7f6e6c198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.433+0000 7f6e7233a640 1 --2- 172.21.15.38:0/1654646336 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6e6c101de0 0x7f6e6c198510 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:53.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.433+0000 7f6e7233a640 1 -- 172.21.15.38:0/1654646336 >> 172.21.15.38:0/1654646336 conn(0x7f6e6c0fd8c0 msgr2=0x7f6e6c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:53.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.433+0000 7f6e7233a640 1 -- 172.21.15.38:0/1654646336 shutdown_connections 2024-09-16T00:32:53.439 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:53.433+0000 7f6e7233a640 1 -- 172.21.15.38:0/1654646336 wait complete. 2024-09-16T00:32:54.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:53 smithi038 ceph-mon[30945]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:32:54.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:53 smithi122 ceph-mon[35412]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:32:54.268 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":367,"stamp":"2024-09-16T00:32:51.544837+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666058,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":794,"num_read_kb":526,"num_write":367,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1180792,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":796,"ondisk_log_size":796,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380376,"kb_used_data":14456,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561448,"statfs":{"total":1151940427776,"available":1151550922752,"internally_reserved":0,"allocated":14802944,"data_stored":8251952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8987,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":69,"num_read_kb":46,"num_write":11,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003297"},"pg_stats":[{"pgid":"2.1f","version":"88'9","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.337120+0000","last_change":"2024-09-16T00:30:23.933355+0000","last_active":"2024-09-16T00:32:24.337120+0000","last_peered":"2024-09-16T00:32:24.337120+0000","last_clean":"2024-09-16T00:32:24.337120+0000","last_became_active":"2024-09-16T00:30:23.933057+0000","last_became_peered":"2024-09-16T00:30:23.933057+0000","last_unstale":"2024-09-16T00:32:24.337120+0000","last_undegraded":"2024-09-16T00:32:24.337120+0000","last_fullsized":"2024-09-16T00:32:24.337120+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:23:01.713440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.596165+0000","last_change":"2024-09-16T00:30:26.084537+0000","last_active":"2024-09-16T00:32:24.596165+0000","last_peered":"2024-09-16T00:32:24.596165+0000","last_clean":"2024-09-16T00:32:24.596165+0000","last_became_active":"2024-09-16T00:30:26.084226+0000","last_became_peered":"2024-09-16T00:30:26.084226+0000","last_unstale":"2024-09-16T00:32:24.596165+0000","last_undegraded":"2024-09-16T00:32:24.596165+0000","last_fullsized":"2024-09-16T00:32:24.596165+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:43:41.628877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594728+0000","last_change":"2024-09-16T00:30:33.154873+0000","last_active":"2024-09-16T00:32:24.594728+0000","last_peered":"2024-09-16T00:32:24.594728+0000","last_clean":"2024-09-16T00:32:24.594728+0000","last_became_active":"2024-09-16T00:30:33.154433+0000","last_became_peered":"2024-09-16T00:30:33.154433+0000","last_unstale":"2024-09-16T00:32:24.594728+0000","last_undegraded":"2024-09-16T00:32:24.594728+0000","last_fullsized":"2024-09-16T00:32:24.594728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:50.805806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.596255+0000","last_change":"2024-09-16T00:30:24.422274+0000","last_active":"2024-09-16T00:32:24.596255+0000","last_peered":"2024-09-16T00:32:24.596255+0000","last_clean":"2024-09-16T00:32:24.596255+0000","last_became_active":"2024-09-16T00:30:24.421902+0000","last_became_peered":"2024-09-16T00:30:24.421902+0000","last_unstale":"2024-09-16T00:32:24.596255+0000","last_undegraded":"2024-09-16T00:32:24.596255+0000","last_fullsized":"2024-09-16T00:32:24.596255+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:01.910187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595568+0000","last_change":"2024-09-16T00:30:26.088165+0000","last_active":"2024-09-16T00:32:24.595568+0000","last_peered":"2024-09-16T00:32:24.595568+0000","last_clean":"2024-09-16T00:32:24.595568+0000","last_became_active":"2024-09-16T00:30:26.087593+0000","last_became_peered":"2024-09-16T00:30:26.087593+0000","last_unstale":"2024-09-16T00:32:24.595568+0000","last_undegraded":"2024-09-16T00:32:24.595568+0000","last_fullsized":"2024-09-16T00:32:24.595568+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:54.149930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454083+0000","last_change":"2024-09-16T00:30:33.154586+0000","last_active":"2024-09-16T00:32:24.454083+0000","last_peered":"2024-09-16T00:32:24.454083+0000","last_clean":"2024-09-16T00:32:24.454083+0000","last_became_active":"2024-09-16T00:30:33.154209+0000","last_became_peered":"2024-09-16T00:30:33.154209+0000","last_unstale":"2024-09-16T00:32:24.454083+0000","last_undegraded":"2024-09-16T00:32:24.454083+0000","last_fullsized":"2024-09-16T00:32:24.454083+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:45.776895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339220+0000","last_change":"2024-09-16T00:30:23.932442+0000","last_active":"2024-09-16T00:32:24.339220+0000","last_peered":"2024-09-16T00:32:24.339220+0000","last_clean":"2024-09-16T00:32:24.339220+0000","last_became_active":"2024-09-16T00:30:23.932009+0000","last_became_peered":"2024-09-16T00:30:23.932009+0000","last_unstale":"2024-09-16T00:32:24.339220+0000","last_undegraded":"2024-09-16T00:32:24.339220+0000","last_fullsized":"2024-09-16T00:32:24.339220+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:09:27.078187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561572+0000","last_change":"2024-09-16T00:30:26.087715+0000","last_active":"2024-09-16T00:32:24.561572+0000","last_peered":"2024-09-16T00:32:24.561572+0000","last_clean":"2024-09-16T00:32:24.561572+0000","last_became_active":"2024-09-16T00:30:26.087415+0000","last_became_peered":"2024-09-16T00:30:26.087415+0000","last_unstale":"2024-09-16T00:32:24.561572+0000","last_undegraded":"2024-09-16T00:32:24.561572+0000","last_fullsized":"2024-09-16T00:32:24.561572+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:52.373992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454447+0000","last_change":"2024-09-16T00:30:33.152328+0000","last_active":"2024-09-16T00:32:24.454447+0000","last_peered":"2024-09-16T00:32:24.454447+0000","last_clean":"2024-09-16T00:32:24.454447+0000","last_became_active":"2024-09-16T00:30:33.151954+0000","last_became_peered":"2024-09-16T00:30:33.151954+0000","last_unstale":"2024-09-16T00:32:24.454447+0000","last_undegraded":"2024-09-16T00:32:24.454447+0000","last_fullsized":"2024-09-16T00:32:24.454447+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:36.546729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339346+0000","last_change":"2024-09-16T00:30:23.928988+0000","last_active":"2024-09-16T00:32:24.339346+0000","last_peered":"2024-09-16T00:32:24.339346+0000","last_clean":"2024-09-16T00:32:24.339346+0000","last_became_active":"2024-09-16T00:30:23.928638+0000","last_became_peered":"2024-09-16T00:30:23.928638+0000","last_unstale":"2024-09-16T00:32:24.339346+0000","last_undegraded":"2024-09-16T00:32:24.339346+0000","last_fullsized":"2024-09-16T00:32:24.339346+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:45:58.203455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.336619+0000","last_change":"2024-09-16T00:31:23.446550+0000","last_active":"2024-09-16T00:32:24.336619+0000","last_peered":"2024-09-16T00:32:24.336619+0000","last_clean":"2024-09-16T00:32:24.336619+0000","last_became_active":"2024-09-16T00:31:23.446179+0000","last_became_peered":"2024-09-16T00:31:23.446179+0000","last_unstale":"2024-09-16T00:32:24.336619+0000","last_undegraded":"2024-09-16T00:32:24.336619+0000","last_fullsized":"2024-09-16T00:32:24.336619+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:04:21.280174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454185+0000","last_change":"2024-09-16T00:30:33.147564+0000","last_active":"2024-09-16T00:32:24.454185+0000","last_peered":"2024-09-16T00:32:24.454185+0000","last_clean":"2024-09-16T00:32:24.454185+0000","last_became_active":"2024-09-16T00:30:33.147119+0000","last_became_peered":"2024-09-16T00:30:33.147119+0000","last_unstale":"2024-09-16T00:32:24.454185+0000","last_undegraded":"2024-09-16T00:32:24.454185+0000","last_fullsized":"2024-09-16T00:32:24.454185+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:00.509122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595076+0000","last_change":"2024-09-16T00:30:26.080954+0000","last_active":"2024-09-16T00:32:24.595076+0000","last_peered":"2024-09-16T00:32:24.595076+0000","last_clean":"2024-09-16T00:32:24.595076+0000","last_became_active":"2024-09-16T00:30:26.080624+0000","last_became_peered":"2024-09-16T00:30:26.080624+0000","last_unstale":"2024-09-16T00:32:24.595076+0000","last_undegraded":"2024-09-16T00:32:24.595076+0000","last_fullsized":"2024-09-16T00:32:24.595076+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:05:10.440980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339169+0000","last_change":"2024-09-16T00:30:23.933715+0000","last_active":"2024-09-16T00:32:24.339169+0000","last_peered":"2024-09-16T00:32:24.339169+0000","last_clean":"2024-09-16T00:32:24.339169+0000","last_became_active":"2024-09-16T00:30:23.933275+0000","last_became_peered":"2024-09-16T00:30:23.933275+0000","last_unstale":"2024-09-16T00:32:24.339169+0000","last_undegraded":"2024-09-16T00:32:24.339169+0000","last_fullsized":"2024-09-16T00:32:24.339169+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:40:44.846053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454634+0000","last_change":"2024-09-16T00:30:33.604602+0000","last_active":"2024-09-16T00:32:24.454634+0000","last_peered":"2024-09-16T00:32:24.454634+0000","last_clean":"2024-09-16T00:32:24.454634+0000","last_became_active":"2024-09-16T00:30:33.604107+0000","last_became_peered":"2024-09-16T00:30:33.604107+0000","last_unstale":"2024-09-16T00:32:24.454634+0000","last_undegraded":"2024-09-16T00:32:24.454634+0000","last_fullsized":"2024-09-16T00:32:24.454634+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:41.254445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595459+0000","last_change":"2024-09-16T00:30:26.080612+0000","last_active":"2024-09-16T00:32:24.595459+0000","last_peered":"2024-09-16T00:32:24.595459+0000","last_clean":"2024-09-16T00:32:24.595459+0000","last_became_active":"2024-09-16T00:30:26.080317+0000","last_became_peered":"2024-09-16T00:30:26.080317+0000","last_unstale":"2024-09-16T00:32:24.595459+0000","last_undegraded":"2024-09-16T00:32:24.595459+0000","last_fullsized":"2024-09-16T00:32:24.595459+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:23.814241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594893+0000","last_change":"2024-09-16T00:30:23.936778+0000","last_active":"2024-09-16T00:32:24.594893+0000","last_peered":"2024-09-16T00:32:24.594893+0000","last_clean":"2024-09-16T00:32:24.594893+0000","last_became_active":"2024-09-16T00:30:23.936265+0000","last_became_peered":"2024-09-16T00:30:23.936265+0000","last_unstale":"2024-09-16T00:32:24.594893+0000","last_undegraded":"2024-09-16T00:32:24.594893+0000","last_fullsized":"2024-09-16T00:32:24.594893+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:35:28.288476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454751+0000","last_change":"2024-09-16T00:30:33.148340+0000","last_active":"2024-09-16T00:32:24.454751+0000","last_peered":"2024-09-16T00:32:24.454751+0000","last_clean":"2024-09-16T00:32:24.454751+0000","last_became_active":"2024-09-16T00:30:33.147289+0000","last_became_peered":"2024-09-16T00:30:33.147289+0000","last_unstale":"2024-09-16T00:32:24.454751+0000","last_undegraded":"2024-09-16T00:32:24.454751+0000","last_fullsized":"2024-09-16T00:32:24.454751+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:21.259254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.338936+0000","last_change":"2024-09-16T00:30:26.077325+0000","last_active":"2024-09-16T00:32:24.338936+0000","last_peered":"2024-09-16T00:32:24.338936+0000","last_clean":"2024-09-16T00:32:24.338936+0000","last_became_active":"2024-09-16T00:30:26.077002+0000","last_became_peered":"2024-09-16T00:30:26.077002+0000","last_unstale":"2024-09-16T00:32:24.338936+0000","last_undegraded":"2024-09-16T00:32:24.338936+0000","last_fullsized":"2024-09-16T00:32:24.338936+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:12:38.569247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595646+0000","last_change":"2024-09-16T00:30:23.933025+0000","last_active":"2024-09-16T00:32:24.595646+0000","last_peered":"2024-09-16T00:32:24.595646+0000","last_clean":"2024-09-16T00:32:24.595646+0000","last_became_active":"2024-09-16T00:30:23.932669+0000","last_became_peered":"2024-09-16T00:30:23.932669+0000","last_unstale":"2024-09-16T00:32:24.595646+0000","last_undegraded":"2024-09-16T00:32:24.595646+0000","last_fullsized":"2024-09-16T00:32:24.595646+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:20.593665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.597881+0000","last_change":"2024-09-16T00:32:24.597170+0000","last_active":"2024-09-16T00:32:24.597881+0000","last_peered":"2024-09-16T00:32:24.597881+0000","last_clean":"2024-09-16T00:32:24.597881+0000","last_became_active":"2024-09-16T00:32:24.596810+0000","last_became_peered":"2024-09-16T00:32:24.596810+0000","last_unstale":"2024-09-16T00:32:24.597881+0000","last_undegraded":"2024-09-16T00:32:24.597881+0000","last_fullsized":"2024-09-16T00:32:24.597881+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:26:18.737332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.338711+0000","last_change":"2024-09-16T00:30:26.076104+0000","last_active":"2024-09-16T00:32:24.338711+0000","last_peered":"2024-09-16T00:32:24.338711+0000","last_clean":"2024-09-16T00:32:24.338711+0000","last_became_active":"2024-09-16T00:30:26.075764+0000","last_became_peered":"2024-09-16T00:30:26.075764+0000","last_unstale":"2024-09-16T00:32:24.338711+0000","last_undegraded":"2024-09-16T00:32:24.338711+0000","last_fullsized":"2024-09-16T00:32:24.338711+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:31.228533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561411+0000","last_change":"2024-09-16T00:30:24.422411+0000","last_active":"2024-09-16T00:32:24.561411+0000","last_peered":"2024-09-16T00:32:24.561411+0000","last_clean":"2024-09-16T00:32:24.561411+0000","last_became_active":"2024-09-16T00:30:24.422102+0000","last_became_peered":"2024-09-16T00:30:24.422102+0000","last_unstale":"2024-09-16T00:32:24.561411+0000","last_undegraded":"2024-09-16T00:32:24.561411+0000","last_fullsized":"2024-09-16T00:32:24.561411+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:06:00.572070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454739+0000","last_change":"2024-09-16T00:30:33.605397+0000","last_active":"2024-09-16T00:32:24.454739+0000","last_peered":"2024-09-16T00:32:24.454739+0000","last_clean":"2024-09-16T00:32:24.454739+0000","last_became_active":"2024-09-16T00:30:33.604978+0000","last_became_peered":"2024-09-16T00:30:33.604978+0000","last_unstale":"2024-09-16T00:32:24.454739+0000","last_undegraded":"2024-09-16T00:32:24.454739+0000","last_fullsized":"2024-09-16T00:32:24.454739+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:35:16.766056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595380+0000","last_change":"2024-09-16T00:30:26.082684+0000","last_active":"2024-09-16T00:32:24.595380+0000","last_peered":"2024-09-16T00:32:24.595380+0000","last_clean":"2024-09-16T00:32:24.595380+0000","last_became_active":"2024-09-16T00:30:26.082335+0000","last_became_peered":"2024-09-16T00:30:26.082335+0000","last_unstale":"2024-09-16T00:32:24.595380+0000","last_undegraded":"2024-09-16T00:32:24.595380+0000","last_fullsized":"2024-09-16T00:32:24.595380+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:51:21.389627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454909+0000","last_change":"2024-09-16T00:30:24.587840+0000","last_active":"2024-09-16T00:32:24.454909+0000","last_peered":"2024-09-16T00:32:24.454909+0000","last_clean":"2024-09-16T00:32:24.454909+0000","last_became_active":"2024-09-16T00:30:24.587436+0000","last_became_peered":"2024-09-16T00:30:24.587436+0000","last_unstale":"2024-09-16T00:32:24.454909+0000","last_undegraded":"2024-09-16T00:32:24.454909+0000","last_fullsized":"2024-09-16T00:32:24.454909+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:06.758585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594628+0000","last_change":"2024-09-16T00:30:33.145051+0000","last_active":"2024-09-16T00:32:24.594628+0000","last_peered":"2024-09-16T00:32:24.594628+0000","last_clean":"2024-09-16T00:32:24.594628+0000","last_became_active":"2024-09-16T00:30:33.144585+0000","last_became_peered":"2024-09-16T00:30:33.144585+0000","last_unstale":"2024-09-16T00:32:24.594628+0000","last_undegraded":"2024-09-16T00:32:24.594628+0000","last_fullsized":"2024-09-16T00:32:24.594628+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:48:23.588495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.336844+0000","last_change":"2024-09-16T00:31:23.444802+0000","last_active":"2024-09-16T00:32:24.336844+0000","last_peered":"2024-09-16T00:32:24.336844+0000","last_clean":"2024-09-16T00:32:24.336844+0000","last_became_active":"2024-09-16T00:31:23.444447+0000","last_became_peered":"2024-09-16T00:31:23.444447+0000","last_unstale":"2024-09-16T00:32:24.336844+0000","last_undegraded":"2024-09-16T00:32:24.336844+0000","last_fullsized":"2024-09-16T00:32:24.336844+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:46:37.285250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.6","version":"92'14","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:41.088268+0000","last_change":"2024-09-16T00:30:23.933284+0000","last_active":"2024-09-16T00:32:41.088268+0000","last_peered":"2024-09-16T00:32:41.088268+0000","last_clean":"2024-09-16T00:32:41.088268+0000","last_became_active":"2024-09-16T00:30:23.932920+0000","last_became_peered":"2024-09-16T00:30:23.932920+0000","last_unstale":"2024-09-16T00:32:41.088268+0000","last_undegraded":"2024-09-16T00:32:41.088268+0000","last_fullsized":"2024-09-16T00:32:41.088268+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:38.199391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594942+0000","last_change":"2024-09-16T00:30:33.604308+0000","last_active":"2024-09-16T00:32:24.594942+0000","last_peered":"2024-09-16T00:32:24.594942+0000","last_clean":"2024-09-16T00:32:24.594942+0000","last_became_active":"2024-09-16T00:30:33.603889+0000","last_became_peered":"2024-09-16T00:30:33.603889+0000","last_unstale":"2024-09-16T00:32:24.594942+0000","last_undegraded":"2024-09-16T00:32:24.594942+0000","last_fullsized":"2024-09-16T00:32:24.594942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:32:55.173063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452512+0000","last_change":"2024-09-16T00:30:26.078515+0000","last_active":"2024-09-16T00:32:24.452512+0000","last_peered":"2024-09-16T00:32:24.452512+0000","last_clean":"2024-09-16T00:32:24.452512+0000","last_became_active":"2024-09-16T00:30:26.078173+0000","last_became_peered":"2024-09-16T00:30:26.078173+0000","last_unstale":"2024-09-16T00:32:24.452512+0000","last_undegraded":"2024-09-16T00:32:24.452512+0000","last_fullsized":"2024-09-16T00:32:24.452512+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:40:31.471154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339304+0000","last_change":"2024-09-16T00:30:24.423209+0000","last_active":"2024-09-16T00:32:24.339304+0000","last_peered":"2024-09-16T00:32:24.339304+0000","last_clean":"2024-09-16T00:32:24.339304+0000","last_became_active":"2024-09-16T00:30:24.422157+0000","last_became_peered":"2024-09-16T00:30:24.422157+0000","last_unstale":"2024-09-16T00:32:24.339304+0000","last_undegraded":"2024-09-16T00:32:24.339304+0000","last_fullsized":"2024-09-16T00:32:24.339304+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:28.262392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339327+0000","last_change":"2024-09-16T00:30:33.146204+0000","last_active":"2024-09-16T00:32:24.339327+0000","last_peered":"2024-09-16T00:32:24.339327+0000","last_clean":"2024-09-16T00:32:24.339327+0000","last_became_active":"2024-09-16T00:30:33.145553+0000","last_became_peered":"2024-09-16T00:30:33.145553+0000","last_unstale":"2024-09-16T00:32:24.339327+0000","last_undegraded":"2024-09-16T00:32:24.339327+0000","last_fullsized":"2024-09-16T00:32:24.339327+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:22.096050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.337437+0000","last_change":"2024-09-16T00:30:26.070153+0000","last_active":"2024-09-16T00:32:24.337437+0000","last_peered":"2024-09-16T00:32:24.337437+0000","last_clean":"2024-09-16T00:32:24.337437+0000","last_became_active":"2024-09-16T00:30:26.069792+0000","last_became_peered":"2024-09-16T00:30:26.069792+0000","last_unstale":"2024-09-16T00:32:24.337437+0000","last_undegraded":"2024-09-16T00:32:24.337437+0000","last_fullsized":"2024-09-16T00:32:24.337437+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:28.164242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.336815+0000","last_change":"2024-09-16T00:30:24.423261+0000","last_active":"2024-09-16T00:32:24.336815+0000","last_peered":"2024-09-16T00:32:24.336815+0000","last_clean":"2024-09-16T00:32:24.336815+0000","last_became_active":"2024-09-16T00:30:24.422650+0000","last_became_peered":"2024-09-16T00:30:24.422650+0000","last_unstale":"2024-09-16T00:32:24.336815+0000","last_undegraded":"2024-09-16T00:32:24.336815+0000","last_fullsized":"2024-09-16T00:32:24.336815+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:00.211744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339762+0000","last_change":"2024-09-16T00:30:33.604492+0000","last_active":"2024-09-16T00:32:24.339762+0000","last_peered":"2024-09-16T00:32:24.339762+0000","last_clean":"2024-09-16T00:32:24.339762+0000","last_became_active":"2024-09-16T00:30:33.604015+0000","last_became_peered":"2024-09-16T00:30:33.604015+0000","last_unstale":"2024-09-16T00:32:24.339762+0000","last_undegraded":"2024-09-16T00:32:24.339762+0000","last_fullsized":"2024-09-16T00:32:24.339762+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:40:22.516583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.337389+0000","last_change":"2024-09-16T00:30:26.075517+0000","last_active":"2024-09-16T00:32:24.337389+0000","last_peered":"2024-09-16T00:32:24.337389+0000","last_clean":"2024-09-16T00:32:24.337389+0000","last_became_active":"2024-09-16T00:30:26.075113+0000","last_became_peered":"2024-09-16T00:30:26.075113+0000","last_unstale":"2024-09-16T00:32:24.337389+0000","last_undegraded":"2024-09-16T00:32:24.337389+0000","last_fullsized":"2024-09-16T00:32:24.337389+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:11.427010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595204+0000","last_change":"2024-09-16T00:30:23.933639+0000","last_active":"2024-09-16T00:32:24.595204+0000","last_peered":"2024-09-16T00:32:24.595204+0000","last_clean":"2024-09-16T00:32:24.595204+0000","last_became_active":"2024-09-16T00:30:23.933322+0000","last_became_peered":"2024-09-16T00:30:23.933322+0000","last_unstale":"2024-09-16T00:32:24.595204+0000","last_undegraded":"2024-09-16T00:32:24.595204+0000","last_fullsized":"2024-09-16T00:32:24.595204+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:59:59.082702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452139+0000","last_change":"2024-09-16T00:30:33.148508+0000","last_active":"2024-09-16T00:32:24.452139+0000","last_peered":"2024-09-16T00:32:24.452139+0000","last_clean":"2024-09-16T00:32:24.452139+0000","last_became_active":"2024-09-16T00:30:33.148091+0000","last_became_peered":"2024-09-16T00:30:33.148091+0000","last_unstale":"2024-09-16T00:32:24.452139+0000","last_undegraded":"2024-09-16T00:32:24.452139+0000","last_fullsized":"2024-09-16T00:32:24.452139+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:50.052526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454239+0000","last_change":"2024-09-16T00:30:26.080374+0000","last_active":"2024-09-16T00:32:24.454239+0000","last_peered":"2024-09-16T00:32:24.454239+0000","last_clean":"2024-09-16T00:32:24.454239+0000","last_became_active":"2024-09-16T00:30:26.080044+0000","last_became_peered":"2024-09-16T00:30:26.080044+0000","last_unstale":"2024-09-16T00:32:24.454239+0000","last_undegraded":"2024-09-16T00:32:24.454239+0000","last_fullsized":"2024-09-16T00:32:24.454239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:38:36.034903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":150,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:48.487735+0000","last_change":"2024-09-16T00:30:24.422137+0000","last_active":"2024-09-16T00:32:48.487735+0000","last_peered":"2024-09-16T00:32:48.487735+0000","last_clean":"2024-09-16T00:32:48.487735+0000","last_became_active":"2024-09-16T00:30:24.421778+0000","last_became_peered":"2024-09-16T00:30:24.421778+0000","last_unstale":"2024-09-16T00:32:48.487735+0000","last_undegraded":"2024-09-16T00:32:48.487735+0000","last_fullsized":"2024-09-16T00:32:48.487735+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:54:21.964099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":107,"num_read_kb":107,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452309+0000","last_change":"2024-09-16T00:30:33.604263+0000","last_active":"2024-09-16T00:32:24.452309+0000","last_peered":"2024-09-16T00:32:24.452309+0000","last_clean":"2024-09-16T00:32:24.452309+0000","last_became_active":"2024-09-16T00:30:33.603804+0000","last_became_peered":"2024-09-16T00:30:33.603804+0000","last_unstale":"2024-09-16T00:32:24.452309+0000","last_undegraded":"2024-09-16T00:32:24.452309+0000","last_fullsized":"2024-09-16T00:32:24.452309+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:08:37.590827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.451792+0000","last_change":"2024-09-16T00:30:26.082916+0000","last_active":"2024-09-16T00:32:24.451792+0000","last_peered":"2024-09-16T00:32:24.451792+0000","last_clean":"2024-09-16T00:32:24.451792+0000","last_became_active":"2024-09-16T00:30:26.082606+0000","last_became_peered":"2024-09-16T00:30:26.082606+0000","last_unstale":"2024-09-16T00:32:24.451792+0000","last_undegraded":"2024-09-16T00:32:24.451792+0000","last_fullsized":"2024-09-16T00:32:24.451792+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:56.777640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594991+0000","last_change":"2024-09-16T00:30:23.937602+0000","last_active":"2024-09-16T00:32:24.594991+0000","last_peered":"2024-09-16T00:32:24.594991+0000","last_clean":"2024-09-16T00:32:24.594991+0000","last_became_active":"2024-09-16T00:30:23.937261+0000","last_became_peered":"2024-09-16T00:30:23.937261+0000","last_unstale":"2024-09-16T00:32:24.594991+0000","last_undegraded":"2024-09-16T00:32:24.594991+0000","last_fullsized":"2024-09-16T00:32:24.594991+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:24.698758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454567+0000","last_change":"2024-09-16T00:30:33.605536+0000","last_active":"2024-09-16T00:32:24.454567+0000","last_peered":"2024-09-16T00:32:24.454567+0000","last_clean":"2024-09-16T00:32:24.454567+0000","last_became_active":"2024-09-16T00:30:33.605073+0000","last_became_peered":"2024-09-16T00:30:33.605073+0000","last_unstale":"2024-09-16T00:32:24.454567+0000","last_undegraded":"2024-09-16T00:32:24.454567+0000","last_fullsized":"2024-09-16T00:32:24.454567+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:11.143052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595152+0000","last_change":"2024-09-16T00:30:26.084507+0000","last_active":"2024-09-16T00:32:24.595152+0000","last_peered":"2024-09-16T00:32:24.595152+0000","last_clean":"2024-09-16T00:32:24.595152+0000","last_became_active":"2024-09-16T00:30:26.084152+0000","last_became_peered":"2024-09-16T00:30:26.084152+0000","last_unstale":"2024-09-16T00:32:24.595152+0000","last_undegraded":"2024-09-16T00:32:24.595152+0000","last_fullsized":"2024-09-16T00:32:24.595152+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:27:21.557409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454701+0000","last_change":"2024-09-16T00:30:23.932686+0000","last_active":"2024-09-16T00:32:24.454701+0000","last_peered":"2024-09-16T00:32:24.454701+0000","last_clean":"2024-09-16T00:32:24.454701+0000","last_became_active":"2024-09-16T00:30:23.932343+0000","last_became_peered":"2024-09-16T00:30:23.932343+0000","last_unstale":"2024-09-16T00:32:24.454701+0000","last_undegraded":"2024-09-16T00:32:24.454701+0000","last_fullsized":"2024-09-16T00:32:24.454701+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:13:44.185712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":351,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595072+0000","last_change":"2024-09-16T00:29:11.143642+0000","last_active":"2024-09-16T00:32:24.595072+0000","last_peered":"2024-09-16T00:32:24.595072+0000","last_clean":"2024-09-16T00:32:24.595072+0000","last_became_active":"2024-09-16T00:29:11.143227+0000","last_became_peered":"2024-09-16T00:29:11.143227+0000","last_unstale":"2024-09-16T00:32:24.595072+0000","last_undegraded":"2024-09-16T00:32:24.595072+0000","last_fullsized":"2024-09-16T00:32:24.595072+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:25:42.361316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:25:42.361316+0000","last_clean_scrub_stamp":"2024-09-16T00:25:42.361316+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:29:12.740474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595397+0000","last_change":"2024-09-16T00:30:33.155555+0000","last_active":"2024-09-16T00:32:24.595397+0000","last_peered":"2024-09-16T00:32:24.595397+0000","last_clean":"2024-09-16T00:32:24.595397+0000","last_became_active":"2024-09-16T00:30:33.155170+0000","last_became_peered":"2024-09-16T00:30:33.155170+0000","last_unstale":"2024-09-16T00:32:24.595397+0000","last_undegraded":"2024-09-16T00:32:24.595397+0000","last_fullsized":"2024-09-16T00:32:24.595397+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:34:46.214422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595573+0000","last_change":"2024-09-16T00:30:26.076003+0000","last_active":"2024-09-16T00:32:24.595573+0000","last_peered":"2024-09-16T00:32:24.595573+0000","last_clean":"2024-09-16T00:32:24.595573+0000","last_became_active":"2024-09-16T00:30:26.075666+0000","last_became_peered":"2024-09-16T00:30:26.075666+0000","last_unstale":"2024-09-16T00:32:24.595573+0000","last_undegraded":"2024-09-16T00:32:24.595573+0000","last_fullsized":"2024-09-16T00:32:24.595573+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:30.113380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594420+0000","last_change":"2024-09-16T00:30:24.587469+0000","last_active":"2024-09-16T00:32:24.594420+0000","last_peered":"2024-09-16T00:32:24.594420+0000","last_clean":"2024-09-16T00:32:24.594420+0000","last_became_active":"2024-09-16T00:30:24.587049+0000","last_became_peered":"2024-09-16T00:30:24.587049+0000","last_unstale":"2024-09-16T00:32:24.594420+0000","last_undegraded":"2024-09-16T00:32:24.594420+0000","last_fullsized":"2024-09-16T00:32:24.594420+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:17:45.934505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561433+0000","last_change":"2024-09-16T00:30:33.145921+0000","last_active":"2024-09-16T00:32:24.561433+0000","last_peered":"2024-09-16T00:32:24.561433+0000","last_clean":"2024-09-16T00:32:24.561433+0000","last_became_active":"2024-09-16T00:30:33.145450+0000","last_became_peered":"2024-09-16T00:30:33.145450+0000","last_unstale":"2024-09-16T00:32:24.561433+0000","last_undegraded":"2024-09-16T00:32:24.561433+0000","last_fullsized":"2024-09-16T00:32:24.561433+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:53:51.698686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595330+0000","last_change":"2024-09-16T00:30:26.087940+0000","last_active":"2024-09-16T00:32:24.595330+0000","last_peered":"2024-09-16T00:32:24.595330+0000","last_clean":"2024-09-16T00:32:24.595330+0000","last_became_active":"2024-09-16T00:30:26.087599+0000","last_became_peered":"2024-09-16T00:30:26.087599+0000","last_unstale":"2024-09-16T00:32:24.595330+0000","last_undegraded":"2024-09-16T00:32:24.595330+0000","last_fullsized":"2024-09-16T00:32:24.595330+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:43.934839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454847+0000","last_change":"2024-09-16T00:30:23.930986+0000","last_active":"2024-09-16T00:32:24.454847+0000","last_peered":"2024-09-16T00:32:24.454847+0000","last_clean":"2024-09-16T00:32:24.454847+0000","last_became_active":"2024-09-16T00:30:23.930588+0000","last_became_peered":"2024-09-16T00:30:23.930588+0000","last_unstale":"2024-09-16T00:32:24.454847+0000","last_undegraded":"2024-09-16T00:32:24.454847+0000","last_fullsized":"2024-09-16T00:32:24.454847+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:04:41.902358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594719+0000","last_change":"2024-09-16T00:30:33.154894+0000","last_active":"2024-09-16T00:32:24.594719+0000","last_peered":"2024-09-16T00:32:24.594719+0000","last_clean":"2024-09-16T00:32:24.594719+0000","last_became_active":"2024-09-16T00:30:33.154490+0000","last_became_peered":"2024-09-16T00:30:33.154490+0000","last_unstale":"2024-09-16T00:32:24.594719+0000","last_undegraded":"2024-09-16T00:32:24.594719+0000","last_fullsized":"2024-09-16T00:32:24.594719+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:59:53.854212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595549+0000","last_change":"2024-09-16T00:30:26.077069+0000","last_active":"2024-09-16T00:32:24.595549+0000","last_peered":"2024-09-16T00:32:24.595549+0000","last_clean":"2024-09-16T00:32:24.595549+0000","last_became_active":"2024-09-16T00:30:26.076718+0000","last_became_peered":"2024-09-16T00:30:26.076718+0000","last_unstale":"2024-09-16T00:32:24.595549+0000","last_undegraded":"2024-09-16T00:32:24.595549+0000","last_fullsized":"2024-09-16T00:32:24.595549+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:25:37.541704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452322+0000","last_change":"2024-09-16T00:30:23.932081+0000","last_active":"2024-09-16T00:32:24.452322+0000","last_peered":"2024-09-16T00:32:24.452322+0000","last_clean":"2024-09-16T00:32:24.452322+0000","last_became_active":"2024-09-16T00:30:23.931729+0000","last_became_peered":"2024-09-16T00:30:23.931729+0000","last_unstale":"2024-09-16T00:32:24.452322+0000","last_undegraded":"2024-09-16T00:32:24.452322+0000","last_fullsized":"2024-09-16T00:32:24.452322+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:20:05.178580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454073+0000","last_change":"2024-09-16T00:30:33.147738+0000","last_active":"2024-09-16T00:32:24.454073+0000","last_peered":"2024-09-16T00:32:24.454073+0000","last_clean":"2024-09-16T00:32:24.454073+0000","last_became_active":"2024-09-16T00:30:33.147326+0000","last_became_peered":"2024-09-16T00:30:33.147326+0000","last_unstale":"2024-09-16T00:32:24.454073+0000","last_undegraded":"2024-09-16T00:32:24.454073+0000","last_fullsized":"2024-09-16T00:32:24.454073+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:44:55.123980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339213+0000","last_change":"2024-09-16T00:30:26.079312+0000","last_active":"2024-09-16T00:32:24.339213+0000","last_peered":"2024-09-16T00:32:24.339213+0000","last_clean":"2024-09-16T00:32:24.339213+0000","last_became_active":"2024-09-16T00:30:26.078986+0000","last_became_peered":"2024-09-16T00:30:26.078986+0000","last_unstale":"2024-09-16T00:32:24.339213+0000","last_undegraded":"2024-09-16T00:32:24.339213+0000","last_fullsized":"2024-09-16T00:32:24.339213+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:02:06.602850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339614+0000","last_change":"2024-09-16T00:31:23.452102+0000","last_active":"2024-09-16T00:32:24.339614+0000","last_peered":"2024-09-16T00:32:24.339614+0000","last_clean":"2024-09-16T00:32:24.339614+0000","last_became_active":"2024-09-16T00:31:23.445974+0000","last_became_peered":"2024-09-16T00:31:23.445974+0000","last_unstale":"2024-09-16T00:32:24.339614+0000","last_undegraded":"2024-09-16T00:32:24.339614+0000","last_fullsized":"2024-09-16T00:32:24.339614+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:52.314494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454166+0000","last_change":"2024-09-16T00:30:33.149154+0000","last_active":"2024-09-16T00:32:24.454166+0000","last_peered":"2024-09-16T00:32:24.454166+0000","last_clean":"2024-09-16T00:32:24.454166+0000","last_became_active":"2024-09-16T00:30:33.148763+0000","last_became_peered":"2024-09-16T00:30:33.148763+0000","last_unstale":"2024-09-16T00:32:24.454166+0000","last_undegraded":"2024-09-16T00:32:24.454166+0000","last_fullsized":"2024-09-16T00:32:24.454166+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:58:05.884919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454295+0000","last_change":"2024-09-16T00:30:26.086905+0000","last_active":"2024-09-16T00:32:24.454295+0000","last_peered":"2024-09-16T00:32:24.454295+0000","last_clean":"2024-09-16T00:32:24.454295+0000","last_became_active":"2024-09-16T00:30:26.086519+0000","last_became_peered":"2024-09-16T00:30:26.086519+0000","last_unstale":"2024-09-16T00:32:24.454295+0000","last_undegraded":"2024-09-16T00:32:24.454295+0000","last_fullsized":"2024-09-16T00:32:24.454295+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:51:08.787679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.338826+0000","last_change":"2024-09-16T00:30:24.422893+0000","last_active":"2024-09-16T00:32:24.338826+0000","last_peered":"2024-09-16T00:32:24.338826+0000","last_clean":"2024-09-16T00:32:24.338826+0000","last_became_active":"2024-09-16T00:30:24.422073+0000","last_became_peered":"2024-09-16T00:30:24.422073+0000","last_unstale":"2024-09-16T00:32:24.338826+0000","last_undegraded":"2024-09-16T00:32:24.338826+0000","last_fullsized":"2024-09-16T00:32:24.338826+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:15:11.375957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452155+0000","last_change":"2024-09-16T00:30:33.152582+0000","last_active":"2024-09-16T00:32:24.452155+0000","last_peered":"2024-09-16T00:32:24.452155+0000","last_clean":"2024-09-16T00:32:24.452155+0000","last_became_active":"2024-09-16T00:30:33.152180+0000","last_became_peered":"2024-09-16T00:30:33.152180+0000","last_unstale":"2024-09-16T00:32:24.452155+0000","last_undegraded":"2024-09-16T00:32:24.452155+0000","last_fullsized":"2024-09-16T00:32:24.452155+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:41:10.499391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594802+0000","last_change":"2024-09-16T00:30:26.086052+0000","last_active":"2024-09-16T00:32:24.594802+0000","last_peered":"2024-09-16T00:32:24.594802+0000","last_clean":"2024-09-16T00:32:24.594802+0000","last_became_active":"2024-09-16T00:30:26.085756+0000","last_became_peered":"2024-09-16T00:30:26.085756+0000","last_unstale":"2024-09-16T00:32:24.594802+0000","last_undegraded":"2024-09-16T00:32:24.594802+0000","last_fullsized":"2024-09-16T00:32:24.594802+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:52.582166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":149,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:46.485452+0000","last_change":"2024-09-16T00:30:23.933528+0000","last_active":"2024-09-16T00:32:46.485452+0000","last_peered":"2024-09-16T00:32:46.485452+0000","last_clean":"2024-09-16T00:32:46.485452+0000","last_became_active":"2024-09-16T00:30:23.933186+0000","last_became_peered":"2024-09-16T00:30:23.933186+0000","last_unstale":"2024-09-16T00:32:46.485452+0000","last_undegraded":"2024-09-16T00:32:46.485452+0000","last_fullsized":"2024-09-16T00:32:46.485452+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:11:00.489676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":108,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.340727+0000","last_change":"2024-09-16T00:31:23.578056+0000","last_active":"2024-09-16T00:32:24.340727+0000","last_peered":"2024-09-16T00:32:24.340727+0000","last_clean":"2024-09-16T00:32:24.340727+0000","last_became_active":"2024-09-16T00:31:23.577721+0000","last_became_peered":"2024-09-16T00:31:23.577721+0000","last_unstale":"2024-09-16T00:32:24.340727+0000","last_undegraded":"2024-09-16T00:32:24.340727+0000","last_fullsized":"2024-09-16T00:32:24.340727+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:53.632440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595248+0000","last_change":"2024-09-16T00:30:26.084509+0000","last_active":"2024-09-16T00:32:24.595248+0000","last_peered":"2024-09-16T00:32:24.595248+0000","last_clean":"2024-09-16T00:32:24.595248+0000","last_became_active":"2024-09-16T00:30:26.084188+0000","last_became_peered":"2024-09-16T00:30:26.084188+0000","last_unstale":"2024-09-16T00:32:24.595248+0000","last_undegraded":"2024-09-16T00:32:24.595248+0000","last_fullsized":"2024-09-16T00:32:24.595248+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:21:48.981277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454529+0000","last_change":"2024-09-16T00:30:23.933499+0000","last_active":"2024-09-16T00:32:24.454529+0000","last_peered":"2024-09-16T00:32:24.454529+0000","last_clean":"2024-09-16T00:32:24.454529+0000","last_became_active":"2024-09-16T00:30:23.933082+0000","last_became_peered":"2024-09-16T00:30:23.933082+0000","last_unstale":"2024-09-16T00:32:24.454529+0000","last_undegraded":"2024-09-16T00:32:24.454529+0000","last_fullsized":"2024-09-16T00:32:24.454529+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:42:18.998433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":32,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.596318+0000","last_change":"2024-09-16T00:32:24.596318+0000","last_active":"2024-09-16T00:32:24.596318+0000","last_peered":"2024-09-16T00:32:24.596318+0000","last_clean":"2024-09-16T00:32:24.596318+0000","last_became_active":"2024-09-16T00:32:24.595906+0000","last_became_peered":"2024-09-16T00:32:24.595906+0000","last_unstale":"2024-09-16T00:32:24.596318+0000","last_undegraded":"2024-09-16T00:32:24.596318+0000","last_fullsized":"2024-09-16T00:32:24.596318+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:13:56.601745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452018+0000","last_change":"2024-09-16T00:30:26.080517+0000","last_active":"2024-09-16T00:32:24.452018+0000","last_peered":"2024-09-16T00:32:24.452018+0000","last_clean":"2024-09-16T00:32:24.452018+0000","last_became_active":"2024-09-16T00:30:26.080197+0000","last_became_peered":"2024-09-16T00:30:26.080197+0000","last_unstale":"2024-09-16T00:32:24.452018+0000","last_undegraded":"2024-09-16T00:32:24.452018+0000","last_fullsized":"2024-09-16T00:32:24.452018+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:15.508562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339267+0000","last_change":"2024-09-16T00:31:23.457047+0000","last_active":"2024-09-16T00:32:24.339267+0000","last_peered":"2024-09-16T00:32:24.339267+0000","last_clean":"2024-09-16T00:32:24.339267+0000","last_became_active":"2024-09-16T00:31:23.447014+0000","last_became_peered":"2024-09-16T00:31:23.447014+0000","last_unstale":"2024-09-16T00:32:24.339267+0000","last_undegraded":"2024-09-16T00:32:24.339267+0000","last_fullsized":"2024-09-16T00:32:24.339267+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:34:42.040228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454641+0000","last_change":"2024-09-16T00:30:33.145803+0000","last_active":"2024-09-16T00:32:24.454641+0000","last_peered":"2024-09-16T00:32:24.454641+0000","last_clean":"2024-09-16T00:32:24.454641+0000","last_became_active":"2024-09-16T00:30:33.145366+0000","last_became_peered":"2024-09-16T00:30:33.145366+0000","last_unstale":"2024-09-16T00:32:24.454641+0000","last_undegraded":"2024-09-16T00:32:24.454641+0000","last_fullsized":"2024-09-16T00:32:24.454641+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:47:28.689305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339138+0000","last_change":"2024-09-16T00:30:26.079415+0000","last_active":"2024-09-16T00:32:24.339138+0000","last_peered":"2024-09-16T00:32:24.339138+0000","last_clean":"2024-09-16T00:32:24.339138+0000","last_became_active":"2024-09-16T00:30:26.079075+0000","last_became_peered":"2024-09-16T00:30:26.079075+0000","last_unstale":"2024-09-16T00:32:24.339138+0000","last_undegraded":"2024-09-16T00:32:24.339138+0000","last_fullsized":"2024-09-16T00:32:24.339138+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:18:18.236342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"92'40","reported_seq":303,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:48.491764+0000","last_change":"2024-09-16T00:30:23.933172+0000","last_active":"2024-09-16T00:32:48.491764+0000","last_peered":"2024-09-16T00:32:48.491764+0000","last_clean":"2024-09-16T00:32:48.491764+0000","last_became_active":"2024-09-16T00:30:23.932492+0000","last_became_peered":"2024-09-16T00:30:23.932492+0000","last_unstale":"2024-09-16T00:32:48.491764+0000","last_undegraded":"2024-09-16T00:32:48.491764+0000","last_fullsized":"2024-09-16T00:32:48.491764+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:29.169448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70919,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":224,"num_read_kb":183,"num_write":41,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.337053+0000","last_change":"2024-09-16T00:31:23.445673+0000","last_active":"2024-09-16T00:32:24.337053+0000","last_peered":"2024-09-16T00:32:24.337053+0000","last_clean":"2024-09-16T00:32:24.337053+0000","last_became_active":"2024-09-16T00:31:23.445299+0000","last_became_peered":"2024-09-16T00:31:23.445299+0000","last_unstale":"2024-09-16T00:32:24.337053+0000","last_undegraded":"2024-09-16T00:32:24.337053+0000","last_fullsized":"2024-09-16T00:32:24.337053+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:07:44.235340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561729+0000","last_change":"2024-09-16T00:30:26.085419+0000","last_active":"2024-09-16T00:32:24.561729+0000","last_peered":"2024-09-16T00:32:24.561729+0000","last_clean":"2024-09-16T00:32:24.561729+0000","last_became_active":"2024-09-16T00:30:26.085103+0000","last_became_peered":"2024-09-16T00:30:26.085103+0000","last_unstale":"2024-09-16T00:32:24.561729+0000","last_undegraded":"2024-09-16T00:32:24.561729+0000","last_fullsized":"2024-09-16T00:32:24.561729+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:56:26.352779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.453958+0000","last_change":"2024-09-16T00:30:24.422646+0000","last_active":"2024-09-16T00:32:24.453958+0000","last_peered":"2024-09-16T00:32:24.453958+0000","last_clean":"2024-09-16T00:32:24.453958+0000","last_became_active":"2024-09-16T00:30:24.422065+0000","last_became_peered":"2024-09-16T00:30:24.422065+0000","last_unstale":"2024-09-16T00:32:24.453958+0000","last_undegraded":"2024-09-16T00:32:24.453958+0000","last_fullsized":"2024-09-16T00:32:24.453958+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:53:12.401613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454317+0000","last_change":"2024-09-16T00:30:33.147692+0000","last_active":"2024-09-16T00:32:24.454317+0000","last_peered":"2024-09-16T00:32:24.454317+0000","last_clean":"2024-09-16T00:32:24.454317+0000","last_became_active":"2024-09-16T00:30:33.147115+0000","last_became_peered":"2024-09-16T00:30:33.147115+0000","last_unstale":"2024-09-16T00:32:24.454317+0000","last_undegraded":"2024-09-16T00:32:24.454317+0000","last_fullsized":"2024-09-16T00:32:24.454317+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:44:04.536894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595645+0000","last_change":"2024-09-16T00:30:26.088427+0000","last_active":"2024-09-16T00:32:24.595645+0000","last_peered":"2024-09-16T00:32:24.595645+0000","last_clean":"2024-09-16T00:32:24.595645+0000","last_became_active":"2024-09-16T00:30:26.086454+0000","last_became_peered":"2024-09-16T00:30:26.086454+0000","last_unstale":"2024-09-16T00:32:24.595645+0000","last_undegraded":"2024-09-16T00:32:24.595645+0000","last_fullsized":"2024-09-16T00:32:24.595645+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:54:29.679959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595506+0000","last_change":"2024-09-16T00:31:23.453484+0000","last_active":"2024-09-16T00:32:24.595506+0000","last_peered":"2024-09-16T00:32:24.595506+0000","last_clean":"2024-09-16T00:32:24.595506+0000","last_became_active":"2024-09-16T00:31:23.446700+0000","last_became_peered":"2024-09-16T00:31:23.446700+0000","last_unstale":"2024-09-16T00:32:24.595506+0000","last_undegraded":"2024-09-16T00:32:24.595506+0000","last_fullsized":"2024-09-16T00:32:24.595506+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:43:26.465657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.336273+0000","last_change":"2024-09-16T00:30:33.156510+0000","last_active":"2024-09-16T00:32:24.336273+0000","last_peered":"2024-09-16T00:32:24.336273+0000","last_clean":"2024-09-16T00:32:24.336273+0000","last_became_active":"2024-09-16T00:30:33.156088+0000","last_became_peered":"2024-09-16T00:30:33.156088+0000","last_unstale":"2024-09-16T00:32:24.336273+0000","last_undegraded":"2024-09-16T00:32:24.336273+0000","last_fullsized":"2024-09-16T00:32:24.336273+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:34.230901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561930+0000","last_change":"2024-09-16T00:30:26.082598+0000","last_active":"2024-09-16T00:32:24.561930+0000","last_peered":"2024-09-16T00:32:24.561930+0000","last_clean":"2024-09-16T00:32:24.561930+0000","last_became_active":"2024-09-16T00:30:26.082286+0000","last_became_peered":"2024-09-16T00:30:26.082286+0000","last_unstale":"2024-09-16T00:32:24.561930+0000","last_undegraded":"2024-09-16T00:32:24.561930+0000","last_fullsized":"2024-09-16T00:32:24.561930+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:22:18.579149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"92'5","reported_seq":150,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:48.491617+0000","last_change":"2024-09-16T00:30:23.931097+0000","last_active":"2024-09-16T00:32:48.491617+0000","last_peered":"2024-09-16T00:32:48.491617+0000","last_clean":"2024-09-16T00:32:48.491617+0000","last_became_active":"2024-09-16T00:30:23.930719+0000","last_became_peered":"2024-09-16T00:30:23.930719+0000","last_unstale":"2024-09-16T00:32:48.491617+0000","last_undegraded":"2024-09-16T00:32:48.491617+0000","last_fullsized":"2024-09-16T00:32:48.491617+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:13:47.887754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":106,"num_read_kb":106,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595241+0000","last_change":"2024-09-16T00:30:33.142581+0000","last_active":"2024-09-16T00:32:24.595241+0000","last_peered":"2024-09-16T00:32:24.595241+0000","last_clean":"2024-09-16T00:32:24.595241+0000","last_became_active":"2024-09-16T00:30:33.142263+0000","last_became_peered":"2024-09-16T00:30:33.142263+0000","last_unstale":"2024-09-16T00:32:24.595241+0000","last_undegraded":"2024-09-16T00:32:24.595241+0000","last_fullsized":"2024-09-16T00:32:24.595241+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:10:06.456011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595181+0000","last_change":"2024-09-16T00:30:33.152084+0000","last_active":"2024-09-16T00:32:24.595181+0000","last_peered":"2024-09-16T00:32:24.595181+0000","last_clean":"2024-09-16T00:32:24.595181+0000","last_became_active":"2024-09-16T00:30:33.151681+0000","last_became_peered":"2024-09-16T00:30:33.151681+0000","last_unstale":"2024-09-16T00:32:24.595181+0000","last_undegraded":"2024-09-16T00:32:24.595181+0000","last_fullsized":"2024-09-16T00:32:24.595181+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:36.966498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.336370+0000","last_change":"2024-09-16T00:30:26.084200+0000","last_active":"2024-09-16T00:32:24.336370+0000","last_peered":"2024-09-16T00:32:24.336370+0000","last_clean":"2024-09-16T00:32:24.336370+0000","last_became_active":"2024-09-16T00:30:26.083858+0000","last_became_peered":"2024-09-16T00:30:26.083858+0000","last_unstale":"2024-09-16T00:32:24.336370+0000","last_undegraded":"2024-09-16T00:32:24.336370+0000","last_fullsized":"2024-09-16T00:32:24.336370+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:42.712624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454400+0000","last_change":"2024-09-16T00:30:23.932399+0000","last_active":"2024-09-16T00:32:24.454400+0000","last_peered":"2024-09-16T00:32:24.454400+0000","last_clean":"2024-09-16T00:32:24.454400+0000","last_became_active":"2024-09-16T00:30:23.931996+0000","last_became_peered":"2024-09-16T00:30:23.931996+0000","last_unstale":"2024-09-16T00:32:24.454400+0000","last_undegraded":"2024-09-16T00:32:24.454400+0000","last_fullsized":"2024-09-16T00:32:24.454400+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:47:09.634507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339553+0000","last_change":"2024-09-16T00:30:33.154563+0000","last_active":"2024-09-16T00:32:24.339553+0000","last_peered":"2024-09-16T00:32:24.339553+0000","last_clean":"2024-09-16T00:32:24.339553+0000","last_became_active":"2024-09-16T00:30:33.154105+0000","last_became_peered":"2024-09-16T00:30:33.154105+0000","last_unstale":"2024-09-16T00:32:24.339553+0000","last_undegraded":"2024-09-16T00:32:24.339553+0000","last_fullsized":"2024-09-16T00:32:24.339553+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:59.966255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595005+0000","last_change":"2024-09-16T00:30:26.075126+0000","last_active":"2024-09-16T00:32:24.595005+0000","last_peered":"2024-09-16T00:32:24.595005+0000","last_clean":"2024-09-16T00:32:24.595005+0000","last_became_active":"2024-09-16T00:30:26.074819+0000","last_became_peered":"2024-09-16T00:30:26.074819+0000","last_unstale":"2024-09-16T00:32:24.595005+0000","last_undegraded":"2024-09-16T00:32:24.595005+0000","last_fullsized":"2024-09-16T00:32:24.595005+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:36.996346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.451861+0000","last_change":"2024-09-16T00:30:23.929943+0000","last_active":"2024-09-16T00:32:24.451861+0000","last_peered":"2024-09-16T00:32:24.451861+0000","last_clean":"2024-09-16T00:32:24.451861+0000","last_became_active":"2024-09-16T00:30:23.929578+0000","last_became_peered":"2024-09-16T00:30:23.929578+0000","last_unstale":"2024-09-16T00:32:24.451861+0000","last_undegraded":"2024-09-16T00:32:24.451861+0000","last_fullsized":"2024-09-16T00:32:24.451861+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:05:12.505543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.337224+0000","last_change":"2024-09-16T00:30:23.935758+0000","last_active":"2024-09-16T00:32:24.337224+0000","last_peered":"2024-09-16T00:32:24.337224+0000","last_clean":"2024-09-16T00:32:24.337224+0000","last_became_active":"2024-09-16T00:30:23.935321+0000","last_became_peered":"2024-09-16T00:30:23.935321+0000","last_unstale":"2024-09-16T00:32:24.337224+0000","last_undegraded":"2024-09-16T00:32:24.337224+0000","last_fullsized":"2024-09-16T00:32:24.337224+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:35:40.513435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454389+0000","last_change":"2024-09-16T00:30:26.086316+0000","last_active":"2024-09-16T00:32:24.454389+0000","last_peered":"2024-09-16T00:32:24.454389+0000","last_clean":"2024-09-16T00:32:24.454389+0000","last_became_active":"2024-09-16T00:30:26.085800+0000","last_became_peered":"2024-09-16T00:30:26.085800+0000","last_unstale":"2024-09-16T00:32:24.454389+0000","last_undegraded":"2024-09-16T00:32:24.454389+0000","last_fullsized":"2024-09-16T00:32:24.454389+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:05:36.277127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595014+0000","last_change":"2024-09-16T00:30:33.155665+0000","last_active":"2024-09-16T00:32:24.595014+0000","last_peered":"2024-09-16T00:32:24.595014+0000","last_clean":"2024-09-16T00:32:24.595014+0000","last_became_active":"2024-09-16T00:30:33.155253+0000","last_became_peered":"2024-09-16T00:30:33.155253+0000","last_unstale":"2024-09-16T00:32:24.595014+0000","last_undegraded":"2024-09-16T00:32:24.595014+0000","last_fullsized":"2024-09-16T00:32:24.595014+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:37.545232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'8","reported_seq":156,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:46.488890+0000","last_change":"2024-09-16T00:30:23.934487+0000","last_active":"2024-09-16T00:32:46.488890+0000","last_peered":"2024-09-16T00:32:46.488890+0000","last_clean":"2024-09-16T00:32:46.488890+0000","last_became_active":"2024-09-16T00:30:23.934137+0000","last_became_peered":"2024-09-16T00:30:23.934137+0000","last_unstale":"2024-09-16T00:32:46.488890+0000","last_undegraded":"2024-09-16T00:32:46.488890+0000","last_fullsized":"2024-09-16T00:32:46.488890+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:14:33.088776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1549,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":110,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.338839+0000","last_change":"2024-09-16T00:30:26.077533+0000","last_active":"2024-09-16T00:32:24.338839+0000","last_peered":"2024-09-16T00:32:24.338839+0000","last_clean":"2024-09-16T00:32:24.338839+0000","last_became_active":"2024-09-16T00:30:26.076443+0000","last_became_peered":"2024-09-16T00:30:26.076443+0000","last_unstale":"2024-09-16T00:32:24.338839+0000","last_undegraded":"2024-09-16T00:32:24.338839+0000","last_fullsized":"2024-09-16T00:32:24.338839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:39:17.495521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595151+0000","last_change":"2024-09-16T00:30:33.146123+0000","last_active":"2024-09-16T00:32:24.595151+0000","last_peered":"2024-09-16T00:32:24.595151+0000","last_clean":"2024-09-16T00:32:24.595151+0000","last_became_active":"2024-09-16T00:30:33.145716+0000","last_became_peered":"2024-09-16T00:30:33.145716+0000","last_unstale":"2024-09-16T00:32:24.595151+0000","last_undegraded":"2024-09-16T00:32:24.595151+0000","last_fullsized":"2024-09-16T00:32:24.595151+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:14:27.281367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.338818+0000","last_change":"2024-09-16T00:30:26.083155+0000","last_active":"2024-09-16T00:32:24.338818+0000","last_peered":"2024-09-16T00:32:24.338818+0000","last_clean":"2024-09-16T00:32:24.338818+0000","last_became_active":"2024-09-16T00:30:26.082782+0000","last_became_peered":"2024-09-16T00:30:26.082782+0000","last_unstale":"2024-09-16T00:32:24.338818+0000","last_undegraded":"2024-09-16T00:32:24.338818+0000","last_fullsized":"2024-09-16T00:32:24.338818+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:15.929343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594891+0000","last_change":"2024-09-16T00:31:23.447453+0000","last_active":"2024-09-16T00:32:24.594891+0000","last_peered":"2024-09-16T00:32:24.594891+0000","last_clean":"2024-09-16T00:32:24.594891+0000","last_became_active":"2024-09-16T00:31:23.445507+0000","last_became_peered":"2024-09-16T00:31:23.445507+0000","last_unstale":"2024-09-16T00:32:24.594891+0000","last_undegraded":"2024-09-16T00:32:24.594891+0000","last_fullsized":"2024-09-16T00:32:24.594891+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:18:50.885809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594798+0000","last_change":"2024-09-16T00:30:26.076252+0000","last_active":"2024-09-16T00:32:24.594798+0000","last_peered":"2024-09-16T00:32:24.594798+0000","last_clean":"2024-09-16T00:32:24.594798+0000","last_became_active":"2024-09-16T00:30:26.075922+0000","last_became_peered":"2024-09-16T00:30:26.075922+0000","last_unstale":"2024-09-16T00:32:24.594798+0000","last_undegraded":"2024-09-16T00:32:24.594798+0000","last_fullsized":"2024-09-16T00:32:24.594798+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:28:04.822817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594821+0000","last_change":"2024-09-16T00:30:26.085123+0000","last_active":"2024-09-16T00:32:24.594821+0000","last_peered":"2024-09-16T00:32:24.594821+0000","last_clean":"2024-09-16T00:32:24.594821+0000","last_became_active":"2024-09-16T00:30:26.084782+0000","last_became_peered":"2024-09-16T00:30:26.084782+0000","last_unstale":"2024-09-16T00:32:24.594821+0000","last_undegraded":"2024-09-16T00:32:24.594821+0000","last_fullsized":"2024-09-16T00:32:24.594821+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:09:33.766340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452404+0000","last_change":"2024-09-16T00:30:26.077048+0000","last_active":"2024-09-16T00:32:24.452404+0000","last_peered":"2024-09-16T00:32:24.452404+0000","last_clean":"2024-09-16T00:32:24.452404+0000","last_became_active":"2024-09-16T00:30:26.076719+0000","last_became_peered":"2024-09-16T00:30:26.076719+0000","last_unstale":"2024-09-16T00:32:24.452404+0000","last_undegraded":"2024-09-16T00:32:24.452404+0000","last_fullsized":"2024-09-16T00:32:24.452404+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:58.584246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339277+0000","last_change":"2024-09-16T00:30:26.078446+0000","last_active":"2024-09-16T00:32:24.339277+0000","last_peered":"2024-09-16T00:32:24.339277+0000","last_clean":"2024-09-16T00:32:24.339277+0000","last_became_active":"2024-09-16T00:30:26.078145+0000","last_became_peered":"2024-09-16T00:30:26.078145+0000","last_unstale":"2024-09-16T00:32:24.339277+0000","last_undegraded":"2024-09-16T00:32:24.339277+0000","last_fullsized":"2024-09-16T00:32:24.339277+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:52:28.739834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595347+0000","last_change":"2024-09-16T00:30:26.075875+0000","last_active":"2024-09-16T00:32:24.595347+0000","last_peered":"2024-09-16T00:32:24.595347+0000","last_clean":"2024-09-16T00:32:24.595347+0000","last_became_active":"2024-09-16T00:30:26.075424+0000","last_became_peered":"2024-09-16T00:30:26.075424+0000","last_unstale":"2024-09-16T00:32:24.595347+0000","last_undegraded":"2024-09-16T00:32:24.595347+0000","last_fullsized":"2024-09-16T00:32:24.595347+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:21:07.958507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595253+0000","last_change":"2024-09-16T00:30:26.077694+0000","last_active":"2024-09-16T00:32:24.595253+0000","last_peered":"2024-09-16T00:32:24.595253+0000","last_clean":"2024-09-16T00:32:24.595253+0000","last_became_active":"2024-09-16T00:30:26.077374+0000","last_became_peered":"2024-09-16T00:30:26.077374+0000","last_unstale":"2024-09-16T00:32:24.595253+0000","last_undegraded":"2024-09-16T00:32:24.595253+0000","last_fullsized":"2024-09-16T00:32:24.595253+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:16:30.822050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454143+0000","last_change":"2024-09-16T00:30:26.085931+0000","last_active":"2024-09-16T00:32:24.454143+0000","last_peered":"2024-09-16T00:32:24.454143+0000","last_clean":"2024-09-16T00:32:24.454143+0000","last_became_active":"2024-09-16T00:30:26.085420+0000","last_became_peered":"2024-09-16T00:30:26.085420+0000","last_unstale":"2024-09-16T00:32:24.454143+0000","last_undegraded":"2024-09-16T00:32:24.454143+0000","last_fullsized":"2024-09-16T00:32:24.454143+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:20:36.019203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595470+0000","last_change":"2024-09-16T00:30:26.074217+0000","last_active":"2024-09-16T00:32:24.595470+0000","last_peered":"2024-09-16T00:32:24.595470+0000","last_clean":"2024-09-16T00:32:24.595470+0000","last_became_active":"2024-09-16T00:30:26.073856+0000","last_became_peered":"2024-09-16T00:30:26.073856+0000","last_unstale":"2024-09-16T00:32:24.595470+0000","last_undegraded":"2024-09-16T00:32:24.595470+0000","last_fullsized":"2024-09-16T00:32:24.595470+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:53:57.078465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561639+0000","last_change":"2024-09-16T00:30:26.084248+0000","last_active":"2024-09-16T00:32:24.561639+0000","last_peered":"2024-09-16T00:32:24.561639+0000","last_clean":"2024-09-16T00:32:24.561639+0000","last_became_active":"2024-09-16T00:30:26.083530+0000","last_became_peered":"2024-09-16T00:30:26.083530+0000","last_unstale":"2024-09-16T00:32:24.561639+0000","last_undegraded":"2024-09-16T00:32:24.561639+0000","last_fullsized":"2024-09-16T00:32:24.561639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:16:49.483881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452162+0000","last_change":"2024-09-16T00:30:26.080803+0000","last_active":"2024-09-16T00:32:24.452162+0000","last_peered":"2024-09-16T00:32:24.452162+0000","last_clean":"2024-09-16T00:32:24.452162+0000","last_became_active":"2024-09-16T00:30:26.080482+0000","last_became_peered":"2024-09-16T00:30:26.080482+0000","last_unstale":"2024-09-16T00:32:24.452162+0000","last_undegraded":"2024-09-16T00:32:24.452162+0000","last_fullsized":"2024-09-16T00:32:24.452162+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:35:41.741025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339770+0000","last_change":"2024-09-16T00:30:26.077680+0000","last_active":"2024-09-16T00:32:24.339770+0000","last_peered":"2024-09-16T00:32:24.339770+0000","last_clean":"2024-09-16T00:32:24.339770+0000","last_became_active":"2024-09-16T00:30:26.077371+0000","last_became_peered":"2024-09-16T00:30:26.077371+0000","last_unstale":"2024-09-16T00:32:24.339770+0000","last_undegraded":"2024-09-16T00:32:24.339770+0000","last_fullsized":"2024-09-16T00:32:24.339770+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:31:34.338617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595364+0000","last_change":"2024-09-16T00:30:26.071531+0000","last_active":"2024-09-16T00:32:24.595364+0000","last_peered":"2024-09-16T00:32:24.595364+0000","last_clean":"2024-09-16T00:32:24.595364+0000","last_became_active":"2024-09-16T00:30:26.071189+0000","last_became_peered":"2024-09-16T00:30:26.071189+0000","last_unstale":"2024-09-16T00:32:24.595364+0000","last_undegraded":"2024-09-16T00:32:24.595364+0000","last_fullsized":"2024-09-16T00:32:24.595364+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:29:18.406272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.336922+0000","last_change":"2024-09-16T00:30:26.085398+0000","last_active":"2024-09-16T00:32:24.336922+0000","last_peered":"2024-09-16T00:32:24.336922+0000","last_clean":"2024-09-16T00:32:24.336922+0000","last_became_active":"2024-09-16T00:30:26.085076+0000","last_became_peered":"2024-09-16T00:30:26.085076+0000","last_unstale":"2024-09-16T00:32:24.336922+0000","last_undegraded":"2024-09-16T00:32:24.336922+0000","last_fullsized":"2024-09-16T00:32:24.336922+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:20:20.864660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.340119+0000","last_change":"2024-09-16T00:30:26.080953+0000","last_active":"2024-09-16T00:32:24.340119+0000","last_peered":"2024-09-16T00:32:24.340119+0000","last_clean":"2024-09-16T00:32:24.340119+0000","last_became_active":"2024-09-16T00:30:26.080185+0000","last_became_peered":"2024-09-16T00:30:26.080185+0000","last_unstale":"2024-09-16T00:32:24.340119+0000","last_undegraded":"2024-09-16T00:32:24.340119+0000","last_fullsized":"2024-09-16T00:32:24.340119+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:56.769182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561297+0000","last_change":"2024-09-16T00:30:26.079320+0000","last_active":"2024-09-16T00:32:24.561297+0000","last_peered":"2024-09-16T00:32:24.561297+0000","last_clean":"2024-09-16T00:32:24.561297+0000","last_became_active":"2024-09-16T00:30:26.078980+0000","last_became_peered":"2024-09-16T00:30:26.078980+0000","last_unstale":"2024-09-16T00:32:24.561297+0000","last_undegraded":"2024-09-16T00:32:24.561297+0000","last_fullsized":"2024-09-16T00:32:24.561297+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:03:15.386657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594661+0000","last_change":"2024-09-16T00:30:26.084948+0000","last_active":"2024-09-16T00:32:24.594661+0000","last_peered":"2024-09-16T00:32:24.594661+0000","last_clean":"2024-09-16T00:32:24.594661+0000","last_became_active":"2024-09-16T00:30:26.084602+0000","last_became_peered":"2024-09-16T00:30:26.084602+0000","last_unstale":"2024-09-16T00:32:24.594661+0000","last_undegraded":"2024-09-16T00:32:24.594661+0000","last_fullsized":"2024-09-16T00:32:24.594661+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:01.709332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594560+0000","last_change":"2024-09-16T00:30:26.080491+0000","last_active":"2024-09-16T00:32:24.594560+0000","last_peered":"2024-09-16T00:32:24.594560+0000","last_clean":"2024-09-16T00:32:24.594560+0000","last_became_active":"2024-09-16T00:30:26.080189+0000","last_became_peered":"2024-09-16T00:30:26.080189+0000","last_unstale":"2024-09-16T00:32:24.594560+0000","last_undegraded":"2024-09-16T00:32:24.594560+0000","last_fullsized":"2024-09-16T00:32:24.594560+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:38.358919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452086+0000","last_change":"2024-09-16T00:30:26.077410+0000","last_active":"2024-09-16T00:32:24.452086+0000","last_peered":"2024-09-16T00:32:24.452086+0000","last_clean":"2024-09-16T00:32:24.452086+0000","last_became_active":"2024-09-16T00:30:26.076498+0000","last_became_peered":"2024-09-16T00:30:26.076498+0000","last_unstale":"2024-09-16T00:32:24.452086+0000","last_undegraded":"2024-09-16T00:32:24.452086+0000","last_fullsized":"2024-09-16T00:32:24.452086+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:54:17.712774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454822+0000","last_change":"2024-09-16T00:30:26.074500+0000","last_active":"2024-09-16T00:32:24.454822+0000","last_peered":"2024-09-16T00:32:24.454822+0000","last_clean":"2024-09-16T00:32:24.454822+0000","last_became_active":"2024-09-16T00:30:26.074177+0000","last_became_peered":"2024-09-16T00:30:26.074177+0000","last_unstale":"2024-09-16T00:32:24.454822+0000","last_undegraded":"2024-09-16T00:32:24.454822+0000","last_fullsized":"2024-09-16T00:32:24.454822+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:43:13.792231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595561+0000","last_change":"2024-09-16T00:30:26.078358+0000","last_active":"2024-09-16T00:32:24.595561+0000","last_peered":"2024-09-16T00:32:24.595561+0000","last_clean":"2024-09-16T00:32:24.595561+0000","last_became_active":"2024-09-16T00:30:26.077993+0000","last_became_peered":"2024-09-16T00:30:26.077993+0000","last_unstale":"2024-09-16T00:32:24.595561+0000","last_undegraded":"2024-09-16T00:32:24.595561+0000","last_fullsized":"2024-09-16T00:32:24.595561+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:25:10.711394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.337671+0000","last_change":"2024-09-16T00:31:23.446836+0000","last_active":"2024-09-16T00:32:24.337671+0000","last_peered":"2024-09-16T00:32:24.337671+0000","last_clean":"2024-09-16T00:32:24.337671+0000","last_became_active":"2024-09-16T00:31:23.446471+0000","last_became_peered":"2024-09-16T00:31:23.446471+0000","last_unstale":"2024-09-16T00:32:24.337671+0000","last_undegraded":"2024-09-16T00:32:24.337671+0000","last_fullsized":"2024-09-16T00:32:24.337671+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:08:34.887618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561901+0000","last_change":"2024-09-16T00:30:26.086313+0000","last_active":"2024-09-16T00:32:24.561901+0000","last_peered":"2024-09-16T00:32:24.561901+0000","last_clean":"2024-09-16T00:32:24.561901+0000","last_became_active":"2024-09-16T00:30:26.085760+0000","last_became_peered":"2024-09-16T00:30:26.085760+0000","last_unstale":"2024-09-16T00:32:24.561901+0000","last_undegraded":"2024-09-16T00:32:24.561901+0000","last_fullsized":"2024-09-16T00:32:24.561901+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:40:38.236789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594968+0000","last_change":"2024-09-16T00:30:26.085289+0000","last_active":"2024-09-16T00:32:24.594968+0000","last_peered":"2024-09-16T00:32:24.594968+0000","last_clean":"2024-09-16T00:32:24.594968+0000","last_became_active":"2024-09-16T00:30:26.084995+0000","last_became_peered":"2024-09-16T00:30:26.084995+0000","last_unstale":"2024-09-16T00:32:24.594968+0000","last_undegraded":"2024-09-16T00:32:24.594968+0000","last_fullsized":"2024-09-16T00:32:24.594968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:45.667892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594850+0000","last_change":"2024-09-16T00:30:26.088690+0000","last_active":"2024-09-16T00:32:24.594850+0000","last_peered":"2024-09-16T00:32:24.594850+0000","last_clean":"2024-09-16T00:32:24.594850+0000","last_became_active":"2024-09-16T00:30:26.087700+0000","last_became_peered":"2024-09-16T00:30:26.087700+0000","last_unstale":"2024-09-16T00:32:24.594850+0000","last_undegraded":"2024-09-16T00:32:24.594850+0000","last_fullsized":"2024-09-16T00:32:24.594850+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:39:51.445324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.337214+0000","last_change":"2024-09-16T00:30:26.080215+0000","last_active":"2024-09-16T00:32:24.337214+0000","last_peered":"2024-09-16T00:32:24.337214+0000","last_clean":"2024-09-16T00:32:24.337214+0000","last_became_active":"2024-09-16T00:30:26.079038+0000","last_became_peered":"2024-09-16T00:30:26.079038+0000","last_unstale":"2024-09-16T00:32:24.337214+0000","last_undegraded":"2024-09-16T00:32:24.337214+0000","last_fullsized":"2024-09-16T00:32:24.337214+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:01.573960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339169+0000","last_change":"2024-09-16T00:30:26.080966+0000","last_active":"2024-09-16T00:32:24.339169+0000","last_peered":"2024-09-16T00:32:24.339169+0000","last_clean":"2024-09-16T00:32:24.339169+0000","last_became_active":"2024-09-16T00:30:26.080641+0000","last_became_peered":"2024-09-16T00:30:26.080641+0000","last_unstale":"2024-09-16T00:32:24.339169+0000","last_undegraded":"2024-09-16T00:32:24.339169+0000","last_fullsized":"2024-09-16T00:32:24.339169+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:33:01.538169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454762+0000","last_change":"2024-09-16T00:30:26.072644+0000","last_active":"2024-09-16T00:32:24.454762+0000","last_peered":"2024-09-16T00:32:24.454762+0000","last_clean":"2024-09-16T00:32:24.454762+0000","last_became_active":"2024-09-16T00:30:26.072336+0000","last_became_peered":"2024-09-16T00:30:26.072336+0000","last_unstale":"2024-09-16T00:32:24.454762+0000","last_undegraded":"2024-09-16T00:32:24.454762+0000","last_fullsized":"2024-09-16T00:32:24.454762+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:44:45.760057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595215+0000","last_change":"2024-09-16T00:30:26.079155+0000","last_active":"2024-09-16T00:32:24.595215+0000","last_peered":"2024-09-16T00:32:24.595215+0000","last_clean":"2024-09-16T00:32:24.595215+0000","last_became_active":"2024-09-16T00:30:26.078820+0000","last_became_peered":"2024-09-16T00:30:26.078820+0000","last_unstale":"2024-09-16T00:32:24.595215+0000","last_undegraded":"2024-09-16T00:32:24.595215+0000","last_fullsized":"2024-09-16T00:32:24.595215+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:35.071143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595361+0000","last_change":"2024-09-16T00:30:26.081537+0000","last_active":"2024-09-16T00:32:24.595361+0000","last_peered":"2024-09-16T00:32:24.595361+0000","last_clean":"2024-09-16T00:32:24.595361+0000","last_became_active":"2024-09-16T00:30:26.080744+0000","last_became_peered":"2024-09-16T00:30:26.080744+0000","last_unstale":"2024-09-16T00:32:24.595361+0000","last_undegraded":"2024-09-16T00:32:24.595361+0000","last_fullsized":"2024-09-16T00:32:24.595361+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:46:38.179888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.340007+0000","last_change":"2024-09-16T00:30:26.081066+0000","last_active":"2024-09-16T00:32:24.340007+0000","last_peered":"2024-09-16T00:32:24.340007+0000","last_clean":"2024-09-16T00:32:24.340007+0000","last_became_active":"2024-09-16T00:30:26.080630+0000","last_became_peered":"2024-09-16T00:30:26.080630+0000","last_unstale":"2024-09-16T00:32:24.340007+0000","last_undegraded":"2024-09-16T00:32:24.340007+0000","last_fullsized":"2024-09-16T00:32:24.340007+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:44:42.644792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75456,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":668,"num_read_kb":417,"num_write":184,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":226368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514539,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1060864,"data_stored":523210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743472,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32896,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712256,"statfs":{"total":95995035648,"available":95961350144,"internally_reserved":0,"allocated":1753088,"data_stored":1181341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906997,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":527486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135931,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1064960,"data_stored":523288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397567,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1662976,"data_stored":1119755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561093,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1048576,"data_stored":518566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790025,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28656,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716496,"statfs":{"total":95995035648,"available":95965691904,"internally_reserved":0,"allocated":1671168,"data_stored":1115343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953551,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1060864,"data_stored":523216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182482,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":518760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411418,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1146880,"data_stored":591638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574943,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1126400,"data_stored":589395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738468,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1073152,"data_stored":519954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":70919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":73162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":72497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:32:54.274 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-16T00:32:54.274 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph pg dump --format=json 2024-09-16T00:32:54.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:53 smithi073 ceph-mon[35254]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:32:54.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.457+0000 7fab3c0e7640 1 -- 172.21.15.38:0/1962958479 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab34106f10 msgr2=0x7fab3410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:54.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.457+0000 7fab3c0e7640 1 --2- 172.21.15.38:0/1962958479 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab34106f10 0x7fab3410ea90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009ca0 tx=0x7fab2c02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.457+0000 7fab3c0e7640 1 -- 172.21.15.38:0/1962958479 shutdown_connections 2024-09-16T00:32:54.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.457+0000 7fab3c0e7640 1 --2- 172.21.15.38:0/1962958479 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab34106f10 0x7fab3410ea90 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.457+0000 7fab3c0e7640 1 --2- 172.21.15.38:0/1962958479 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab34102790 0x7fab341067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.457+0000 7fab3c0e7640 1 --2- 172.21.15.38:0/1962958479 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab34101de0 0x7fab341021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.457+0000 7fab3c0e7640 1 -- 172.21.15.38:0/1962958479 >> 172.21.15.38:0/1962958479 conn(0x7fab340fd8c0 msgr2=0x7fab340ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:54.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.458+0000 7fab3c0e7640 1 -- 172.21.15.38:0/1962958479 shutdown_connections 2024-09-16T00:32:54.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.458+0000 7fab3c0e7640 1 -- 172.21.15.38:0/1962958479 wait complete. 2024-09-16T00:32:54.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.458+0000 7fab3c0e7640 1 Processor -- start 2024-09-16T00:32:54.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.458+0000 7fab3c0e7640 1 -- start start 2024-09-16T00:32:54.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.458+0000 7fab3c0e7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab34101de0 0x7fab340744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:54.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.458+0000 7fab3c0e7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab34102790 0x7fab34074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:54.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.458+0000 7fab3c0e7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab34106f10 0x7fab34074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:54.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.458+0000 7fab3c0e7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fab34111fc0 con 0x7fab34102790 2024-09-16T00:32:54.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.459+0000 7fab3c0e7640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fab34111e40 con 0x7fab34106f10 2024-09-16T00:32:54.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.459+0000 7fab3c0e7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fab34112140 con 0x7fab34101de0 2024-09-16T00:32:54.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.459+0000 7fab39682640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab34102790 0x7fab34074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:54.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.459+0000 7fab39682640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab34102790 0x7fab34074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57338/0 (socket says 172.21.15.38:57338) 2024-09-16T00:32:54.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.459+0000 7fab39e83640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab34101de0 0x7fab340744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:54.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.459+0000 7fab39682640 1 -- 172.21.15.38:0/3583310166 learned_addr learned my addr 172.21.15.38:0/3583310166 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:54.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.459+0000 7fab3a684640 1 --2- 172.21.15.38:0/3583310166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab34106f10 0x7fab34074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:54.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.459+0000 7fab39682640 1 -- 172.21.15.38:0/3583310166 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab34101de0 msgr2=0x7fab340744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:54.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.459+0000 7fab39682640 1 --2- 172.21.15.38:0/3583310166 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab34101de0 0x7fab340744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.459+0000 7fab39682640 1 -- 172.21.15.38:0/3583310166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab34106f10 msgr2=0x7fab34074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:54.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.459+0000 7fab39682640 1 --2- 172.21.15.38:0/3583310166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab34106f10 0x7fab34074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.459+0000 7fab39682640 1 -- 172.21.15.38:0/3583310166 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab341a68c0 con 0x7fab34102790 2024-09-16T00:32:54.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.459+0000 7fab3a684640 1 --2- 172.21.15.38:0/3583310166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab34106f10 0x7fab34074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:54.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.459+0000 7fab39e83640 1 --2- 172.21.15.38:0/3583310166 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab34101de0 0x7fab340744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:32:54.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.459+0000 7fab39682640 1 --2- 172.21.15.38:0/3583310166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab34102790 0x7fab34074a30 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fab2400d950 tx=0x7fab2400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:54.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.459+0000 7fab2affd640 1 -- 172.21.15.38:0/3583310166 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab24017030 con 0x7fab34102790 2024-09-16T00:32:54.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.460+0000 7fab3c0e7640 1 -- 172.21.15.38:0/3583310166 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab341a6bb0 con 0x7fab34102790 2024-09-16T00:32:54.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.460+0000 7fab2affd640 1 -- 172.21.15.38:0/3583310166 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab24004020 con 0x7fab34102790 2024-09-16T00:32:54.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.460+0000 7fab2affd640 1 -- 172.21.15.38:0/3583310166 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab240047a0 con 0x7fab34102790 2024-09-16T00:32:54.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.460+0000 7fab3c0e7640 1 -- 172.21.15.38:0/3583310166 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab341a7140 con 0x7fab34102790 2024-09-16T00:32:54.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.462+0000 7fab2affd640 1 -- 172.21.15.38:0/3583310166 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7fab240041d0 con 0x7fab34102790 2024-09-16T00:32:54.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.462+0000 7fab3e4c0740 1 Processor -- start 2024-09-16T00:32:54.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.462+0000 7fab3e4c0740 1 -- start start 2024-09-16T00:32:54.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.463+0000 7fab3e4c0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f7a2e14a0 0x561f7a2e1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:54.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.463+0000 7fab3e4c0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7a2e1dc0 0x561f7a2ec140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:54.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.463+0000 7fab3e4c0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f7a2ec680 0x561f7a2eea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:54.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.463+0000 7fab3e4c0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561f7a2472e0 con 0x561f7a2ec680 2024-09-16T00:32:54.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.463+0000 7fab3e4c0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561f7a1eece0 con 0x561f7a2e1dc0 2024-09-16T00:32:54.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.463+0000 7fab3e4c0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561f7a2432a0 con 0x561f7a2e14a0 2024-09-16T00:32:54.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.463+0000 7fab39682640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f7a2ec680 0x561f7a2eea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:54.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.463+0000 7fab39682640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f7a2ec680 0x561f7a2eea70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57352/0 (socket says 172.21.15.38:57352) 2024-09-16T00:32:54.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.463+0000 7fab39682640 1 -- 172.21.15.38:0/4193258356 learned_addr learned my addr 172.21.15.38:0/4193258356 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:54.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.463+0000 7fab39e83640 1 --2- 172.21.15.38:0/4193258356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7a2e1dc0 0x561f7a2ec140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:54.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.463+0000 7fab3a684640 1 --2- 172.21.15.38:0/4193258356 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f7a2e14a0 0x561f7a2e1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:54.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.463+0000 7fab2affd640 1 --2- 172.21.15.38:0/3583310166 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fab200a32d0 0x7fab200a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:54.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.463+0000 7fab39e83640 1 -- 172.21.15.38:0/4193258356 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f7a2e14a0 msgr2=0x561f7a2e1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:54.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.463+0000 7fab39e83640 1 --2- 172.21.15.38:0/4193258356 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f7a2e14a0 0x561f7a2e1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.463+0000 7fab39e83640 1 -- 172.21.15.38:0/4193258356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f7a2ec680 msgr2=0x561f7a2eea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:54.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.463+0000 7fab39e83640 1 --2- 172.21.15.38:0/4193258356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f7a2ec680 0x561f7a2eea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.463+0000 7fab39e83640 1 -- 172.21.15.38:0/4193258356 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f7a1d3440 con 0x561f7a2e1dc0 2024-09-16T00:32:54.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.463+0000 7fab3a684640 1 --2- 172.21.15.38:0/3583310166 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fab200a32d0 0x7fab200a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:54.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.463+0000 7fab2affd640 1 -- 172.21.15.38:0/3583310166 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7fab240cf760 con 0x7fab34102790 2024-09-16T00:32:54.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.464+0000 7fab39e83640 1 --2- 172.21.15.38:0/4193258356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7a2e1dc0 0x561f7a2ec140 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fab30009fd0 tx=0x7fab300118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46c9db93c1b74258 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:54.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.464+0000 7fab1ffff640 1 -- 172.21.15.38:0/4193258356 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab30004030 con 0x561f7a2e1dc0 2024-09-16T00:32:54.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.464+0000 7fab1ffff640 1 -- 172.21.15.38:0/4193258356 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab30005500 con 0x561f7a2e1dc0 2024-09-16T00:32:54.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.464+0000 7fab1ffff640 1 -- 172.21.15.38:0/4193258356 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab30004db0 con 0x561f7a2e1dc0 2024-09-16T00:32:54.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.465+0000 7fab3e4c0740 1 -- 172.21.15.38:0/4193258356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7a2e1dc0 msgr2=0x561f7a2ec140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:54.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.465+0000 7fab3e4c0740 1 --2- 172.21.15.38:0/4193258356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7a2e1dc0 0x561f7a2ec140 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fab30009fd0 tx=0x7fab300118d0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.465+0000 7fab3a684640 1 --2- 172.21.15.38:0/3583310166 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fab200a32d0 0x7fab200a5790 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009430 tx=0x7fab2c0095c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:54.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.465+0000 7fab3e4c0740 1 -- 172.21.15.38:0/4193258356 shutdown_connections 2024-09-16T00:32:54.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.465+0000 7fab3e4c0740 1 --2- 172.21.15.38:0/4193258356 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f7a2ec680 0x561f7a2eea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.465+0000 7fab3e4c0740 1 --2- 172.21.15.38:0/4193258356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7a2e1dc0 0x561f7a2ec140 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.465+0000 7fab3e4c0740 1 --2- 172.21.15.38:0/4193258356 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f7a2e14a0 0x561f7a2e1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.465+0000 7fab3e4c0740 1 -- 172.21.15.38:0/4193258356 >> 172.21.15.38:0/4193258356 conn(0x561f7a2dda30 msgr2=0x561f7a2dde10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:54.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.465+0000 7fab3e4c0740 1 -- 172.21.15.38:0/4193258356 shutdown_connections 2024-09-16T00:32:54.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.465+0000 7fab3e4c0740 1 -- 172.21.15.38:0/4193258356 wait complete. 2024-09-16T00:32:54.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.465+0000 7fab3e4c0740 1 -- start start 2024-09-16T00:32:54.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.465+0000 7fab3e4c0740 1 Processor -- start 2024-09-16T00:32:54.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.466+0000 7fab3e4c0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7a2e14a0 0x561f7a2ebf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:54.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.466+0000 7fab3e4c0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f7a2e1dc0 0x561f7a2e91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:54.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.466+0000 7fab3e4c0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f7a2ec680 0x561f7a2e9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:54.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.466+0000 7fab3e4c0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561f7a2ef250 con 0x561f7a2ec680 2024-09-16T00:32:54.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.466+0000 7fab3e4c0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561f7a202050 con 0x561f7a2e14a0 2024-09-16T00:32:54.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.466+0000 7fab3e4c0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561f7a2ef3d0 con 0x561f7a2e1dc0 2024-09-16T00:32:54.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.466+0000 7fab3e4c0740 10 client.0 _ll_register_callbacks cb 0x561f7a242160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:54.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.466+0000 7fab3a684640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f7a2ec680 0x561f7a2e9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:54.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.466+0000 7fab3a684640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f7a2ec680 0x561f7a2e9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57358/0 (socket says 172.21.15.38:57358) 2024-09-16T00:32:54.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.466+0000 7fab3a684640 1 -- 172.21.15.38:0/139868791 learned_addr learned my addr 172.21.15.38:0/139868791 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:54.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.466+0000 7fab39682640 1 --2- 172.21.15.38:0/139868791 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f7a2e1dc0 0x561f7a2e91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:54.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.466+0000 7fab39e83640 1 --2- 172.21.15.38:0/139868791 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7a2e14a0 0x561f7a2ebf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:54.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.466+0000 7fab3a684640 1 -- 172.21.15.38:0/139868791 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f7a2e1dc0 msgr2=0x561f7a2e91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:54.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.466+0000 7fab3a684640 1 --2- 172.21.15.38:0/139868791 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f7a2e1dc0 0x561f7a2e91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.466+0000 7fab3a684640 1 -- 172.21.15.38:0/139868791 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7a2e14a0 msgr2=0x561f7a2ebf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:54.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.466+0000 7fab3a684640 1 --2- 172.21.15.38:0/139868791 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7a2e14a0 0x561f7a2ebf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.466+0000 7fab3a684640 1 -- 172.21.15.38:0/139868791 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f79fb83c0 con 0x561f7a2ec680 2024-09-16T00:32:54.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.466+0000 7fab39e83640 1 --2- 172.21.15.38:0/139868791 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7a2e14a0 0x561f7a2ebf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:54.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.467+0000 7fab3a684640 1 --2- 172.21.15.38:0/139868791 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f7a2ec680 0x561f7a2e9720 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fab2c035000 tx=0x7fab2c02a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=314e40634b923d23 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:54.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.467+0000 7fab1d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:32:54.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.467+0000 7fab1d7fa640 1 -- 172.21.15.38:0/139868791 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2c02fa20 con 0x561f7a2ec680 2024-09-16T00:32:54.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.467+0000 7fab1d7fa640 1 -- 172.21.15.38:0/139868791 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7fab2c02fbc0 con 0x561f7a2ec680 2024-09-16T00:32:54.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.467+0000 7fab3e4c0740 1 -- 172.21.15.38:0/139868791 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab30005500 con 0x561f7a2ec680 2024-09-16T00:32:54.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.467+0000 7fab1d7fa640 1 -- 172.21.15.38:0/139868791 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fab2c02fdc0 con 0x561f7a2ec680 2024-09-16T00:32:54.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.467+0000 7fab3e4c0740 10 client.15351 fetch_fsmap learned FSMap version 26 2024-09-16T00:32:54.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.467+0000 7fab3e4c0740 1 -- 172.21.15.38:0/139868791 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561f7a2faba0 con 0x561f7a2ec680 2024-09-16T00:32:54.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.467+0000 7fab1d7fa640 1 -- 172.21.15.38:0/139868791 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab30005500 con 0x561f7a2ec680 2024-09-16T00:32:54.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.467+0000 7fab1d7fa640 1 -- 172.21.15.38:0/139868791 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fab2c052470 con 0x561f7a2ec680 2024-09-16T00:32:54.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.467+0000 7fab3e4c0740 10 client.15351 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:32:54.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.467+0000 7fab3e4c0740 10 client.15351 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:32:54.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.467+0000 7fab3e4c0740 20 client.15351 populate_metadata read hostname 'smithi038' 2024-09-16T00:32:54.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.467+0000 7fab3e4c0740 1 --2- 172.21.15.38:0/139868791 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x561f7a2fbfb0 0x561f7a2fe450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:54.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.467+0000 7fab3e4c0740 4 client.15351 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:32:54.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.467+0000 7fab3e4c0740 1 -- 172.21.15.38:0/139868791 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f7a254850 con 0x561f7a2fbfb0 2024-09-16T00:32:54.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.468+0000 7fab39e83640 1 --2- 172.21.15.38:0/139868791 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x561f7a2fbfb0 0x561f7a2fe450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:54.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.468+0000 7fab39e83640 1 --2- 172.21.15.38:0/139868791 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x561f7a2fbfb0 0x561f7a2fe450 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dff0c962f2c0e05 server_cookie=57ecb9d90d892483 in_seq=0 out_seq=0 2024-09-16T00:32:54.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.468+0000 7fab1d7fa640 10 client.15351 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:32:54.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.469+0000 7fab1d7fa640 1 -- 172.21.15.38:0/139868791 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x561f7a254850 con 0x561f7a2fbfb0 2024-09-16T00:32:54.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.469+0000 7fab1d7fa640 10 client.15351 handle_command_reply: tid=0 2024-09-16T00:32:54.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.469+0000 7fab3e4c0740 1 client.15351 shutdown 2024-09-16T00:32:54.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.469+0000 7fab3e4c0740 1 -- 172.21.15.38:0/139868791 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f7a2ec680 msgr2=0x561f7a2e9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:54.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.469+0000 7fab3e4c0740 1 --2- 172.21.15.38:0/139868791 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f7a2ec680 0x561f7a2e9720 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fab2c035000 tx=0x7fab2c02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.469+0000 7fab3e4c0740 1 -- 172.21.15.38:0/139868791 shutdown_connections 2024-09-16T00:32:54.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.469+0000 7fab3e4c0740 1 --2- 172.21.15.38:0/139868791 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x561f7a2fbfb0 0x561f7a2fe450 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.469+0000 7fab3e4c0740 1 --2- 172.21.15.38:0/139868791 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f7a2ec680 0x561f7a2e9720 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.469+0000 7fab1d7fa640 0 client.15351 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:32:54.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.469+0000 7fab3e4c0740 1 --2- 172.21.15.38:0/139868791 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f7a2e1dc0 0x561f7a2e91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.469+0000 7fab3e4c0740 1 --2- 172.21.15.38:0/139868791 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7a2e14a0 0x561f7a2ebf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.469+0000 7fab3e4c0740 1 -- 172.21.15.38:0/139868791 >> 172.21.15.38:0/139868791 conn(0x561f7a2dda30 msgr2=0x561f7a2e3710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:54.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.469+0000 7fab3e4c0740 1 -- 172.21.15.38:0/139868791 shutdown_connections 2024-09-16T00:32:54.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.469+0000 7fab3e4c0740 1 -- 172.21.15.38:0/139868791 wait complete. 2024-09-16T00:32:54.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.469+0000 7fab3e4c0740 20 client.15351 trim_cache size 0 max 16384 2024-09-16T00:32:54.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.487+0000 7fab3e4c0740 1 Processor -- start 2024-09-16T00:32:54.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.487+0000 7fab3e4c0740 1 -- start start 2024-09-16T00:32:54.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.488+0000 7fab3e4c0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7a2e14a0 0x561f7a303d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:54.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.488+0000 7fab3e4c0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f7a2e1dc0 0x561f7a304260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:54.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.488+0000 7fab3e4c0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f7a2ec680 0x561f7a2e6440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:54.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.488+0000 7fab3e4c0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fab341b3850 con 0x561f7a2e1dc0 2024-09-16T00:32:54.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.488+0000 7fab3e4c0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fab341b1120 con 0x561f7a2e14a0 2024-09-16T00:32:54.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.488+0000 7fab3e4c0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fab341aef70 con 0x561f7a2ec680 2024-09-16T00:32:54.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.488+0000 7fab39682640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f7a2e1dc0 0x561f7a304260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:54.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.488+0000 7fab39682640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f7a2e1dc0 0x561f7a304260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57366/0 (socket says 172.21.15.38:57366) 2024-09-16T00:32:54.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.488+0000 7fab39682640 1 -- 172.21.15.38:0/395478474 learned_addr learned my addr 172.21.15.38:0/395478474 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:54.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.488+0000 7fab39e83640 1 --2- 172.21.15.38:0/395478474 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7a2e14a0 0x561f7a303d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:54.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.488+0000 7fab3a684640 1 --2- 172.21.15.38:0/395478474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f7a2ec680 0x561f7a2e6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:54.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.488+0000 7fab39682640 1 -- 172.21.15.38:0/395478474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f7a2ec680 msgr2=0x561f7a2e6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:54.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.488+0000 7fab39682640 1 --2- 172.21.15.38:0/395478474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f7a2ec680 0x561f7a2e6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.488+0000 7fab39682640 1 -- 172.21.15.38:0/395478474 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7a2e14a0 msgr2=0x561f7a303d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:54.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.488+0000 7fab39682640 1 --2- 172.21.15.38:0/395478474 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7a2e14a0 0x561f7a303d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.488+0000 7fab39682640 1 -- 172.21.15.38:0/395478474 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f7a2ec2a0 con 0x561f7a2e1dc0 2024-09-16T00:32:54.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.489+0000 7fab39682640 1 --2- 172.21.15.38:0/395478474 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f7a2e1dc0 0x561f7a304260 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fab24009610 tx=0x7fab240d5530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a17f31a7c52fc3c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:54.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.489+0000 7fab1f7fe640 1 -- 172.21.15.38:0/395478474 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2409b670 con 0x561f7a2e1dc0 2024-09-16T00:32:54.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.489+0000 7fab1f7fe640 1 -- 172.21.15.38:0/395478474 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab2409bfa0 con 0x561f7a2e1dc0 2024-09-16T00:32:54.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.489+0000 7fab1f7fe640 1 -- 172.21.15.38:0/395478474 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab240860b0 con 0x561f7a2e1dc0 2024-09-16T00:32:54.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.490+0000 7fab3e4c0740 1 -- 172.21.15.38:0/395478474 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f7a2e1dc0 msgr2=0x561f7a304260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:54.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.490+0000 7fab3e4c0740 1 --2- 172.21.15.38:0/395478474 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f7a2e1dc0 0x561f7a304260 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fab24009610 tx=0x7fab240d5530 comp rx=0 tx=0).stop 2024-09-16T00:32:54.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.490+0000 7fab3e4c0740 1 -- 172.21.15.38:0/395478474 shutdown_connections 2024-09-16T00:32:54.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.490+0000 7fab3e4c0740 1 --2- 172.21.15.38:0/395478474 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f7a2ec680 0x561f7a2e6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.490+0000 7fab3e4c0740 1 --2- 172.21.15.38:0/395478474 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f7a2e1dc0 0x561f7a304260 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.490+0000 7fab3e4c0740 1 --2- 172.21.15.38:0/395478474 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7a2e14a0 0x561f7a303d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.490+0000 7fab3e4c0740 1 -- 172.21.15.38:0/395478474 >> 172.21.15.38:0/395478474 conn(0x561f7a2dda30 msgr2=0x561f7a2eb260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:54.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.490+0000 7fab3e4c0740 1 -- 172.21.15.38:0/395478474 shutdown_connections 2024-09-16T00:32:54.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.490+0000 7fab3e4c0740 1 -- 172.21.15.38:0/395478474 wait complete. 2024-09-16T00:32:54.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.491+0000 7fab3e4c0740 1 -- start start 2024-09-16T00:32:54.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.491+0000 7fab3e4c0740 1 Processor -- start 2024-09-16T00:32:54.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.491+0000 7fab3e4c0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f7a2e14a0 0x561f7a2f2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:54.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.491+0000 7fab3e4c0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7a2e1dc0 0x561f7a2f3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:54.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.492+0000 7fab3e4c0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f7a2ec680 0x561f7a365d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:54.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.492+0000 7fab3e4c0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fab341af5e0 con 0x561f7a2e14a0 2024-09-16T00:32:54.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.492+0000 7fab3e4c0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fab341b0a50 con 0x561f7a2e1dc0 2024-09-16T00:32:54.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.492+0000 7fab3e4c0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fab341afcb0 con 0x561f7a2ec680 2024-09-16T00:32:54.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.492+0000 7fab3e4c0740 10 client.0 _ll_register_callbacks cb 0x7fab2c009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:32:54.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.492+0000 7fab39e83640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f7a2e14a0 0x561f7a2f2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:54.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.492+0000 7fab39682640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7a2e1dc0 0x561f7a2f3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:54.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.492+0000 7fab39e83640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f7a2e14a0 0x561f7a2f2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57382/0 (socket says 172.21.15.38:57382) 2024-09-16T00:32:54.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.492+0000 7fab39e83640 1 -- 172.21.15.38:0/1606646194 learned_addr learned my addr 172.21.15.38:0/1606646194 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:54.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.492+0000 7fab3a684640 1 --2- 172.21.15.38:0/1606646194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f7a2ec680 0x561f7a365d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:54.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.492+0000 7fab39682640 1 -- 172.21.15.38:0/1606646194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f7a2ec680 msgr2=0x561f7a365d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:54.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.492+0000 7fab39682640 1 --2- 172.21.15.38:0/1606646194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f7a2ec680 0x561f7a365d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.492+0000 7fab39682640 1 -- 172.21.15.38:0/1606646194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f7a2e14a0 msgr2=0x561f7a2f2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:32:54.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.492+0000 7fab39682640 1 --2- 172.21.15.38:0/1606646194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f7a2e14a0 0x561f7a2f2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.492+0000 7fab39682640 1 -- 172.21.15.38:0/1606646194 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f7a366340 con 0x561f7a2e1dc0 2024-09-16T00:32:54.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.492+0000 7fab39e83640 1 --2- 172.21.15.38:0/1606646194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f7a2e14a0 0x561f7a2f2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:32:54.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.493+0000 7fab39682640 1 --2- 172.21.15.38:0/1606646194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7a2e1dc0 0x561f7a2f3100 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x561f7a2de130 tx=0x7fab24097c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b0c51d67f71bf15 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:54.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.493+0000 7fab1d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-09-16T00:32:54.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.493+0000 7fab1d7fa640 1 -- 172.21.15.38:0/1606646194 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab2409b200 con 0x561f7a2e1dc0 2024-09-16T00:32:54.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.493+0000 7fab3e4c0740 1 -- 172.21.15.38:0/1606646194 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab2409bfa0 con 0x561f7a2e1dc0 2024-09-16T00:32:54.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.493+0000 7fab1d7fa640 1 -- 172.21.15.38:0/1606646194 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7fab2409cbd0 con 0x561f7a2e1dc0 2024-09-16T00:32:54.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.493+0000 7fab1d7fa640 1 -- 172.21.15.38:0/1606646194 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fab240b9050 con 0x561f7a2e1dc0 2024-09-16T00:32:54.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.493+0000 7fab3e4c0740 10 client.25028 fetch_fsmap learned FSMap version 26 2024-09-16T00:32:54.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.493+0000 7fab3e4c0740 1 -- 172.21.15.38:0/1606646194 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x561f7a367610 con 0x561f7a2e1dc0 2024-09-16T00:32:54.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.493+0000 7fab1d7fa640 1 -- 172.21.15.38:0/1606646194 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fab2409bfa0 con 0x561f7a2e1dc0 2024-09-16T00:32:54.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.494+0000 7fab1d7fa640 1 -- 172.21.15.38:0/1606646194 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fab240cfbd0 con 0x561f7a2e1dc0 2024-09-16T00:32:54.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.494+0000 7fab3e4c0740 10 client.25028 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:32:54.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.494+0000 7fab3e4c0740 10 client.25028 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:32:54.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.494+0000 7fab3e4c0740 20 client.25028 populate_metadata read hostname 'smithi038' 2024-09-16T00:32:54.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.494+0000 7fab3e4c0740 1 --2- 172.21.15.38:0/1606646194 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x561f7a367be0 0x561f7a369fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:54.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.494+0000 7fab3e4c0740 4 client.25028 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:32:54.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.494+0000 7fab3e4c0740 1 -- 172.21.15.38:0/1606646194 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561f7a24ed60 con 0x561f7a367be0 2024-09-16T00:32:54.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.494+0000 7fab39e83640 1 --2- 172.21.15.38:0/1606646194 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x561f7a367be0 0x561f7a369fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:54.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.494+0000 7fab39e83640 1 --2- 172.21.15.38:0/1606646194 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x561f7a367be0 0x561f7a369fd0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=689141400e625001 server_cookie=128565341d5931ae in_seq=0 out_seq=0 2024-09-16T00:32:54.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.494+0000 7fab1d7fa640 10 client.25028 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:32:54.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.495+0000 7fab1d7fa640 1 -- 172.21.15.38:0/1606646194 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x561f7a24ed60 con 0x561f7a367be0 2024-09-16T00:32:54.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.495+0000 7fab1d7fa640 10 client.25028 handle_command_reply: tid=0 2024-09-16T00:32:54.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.495+0000 7fab3e4c0740 1 client.25028 shutdown 2024-09-16T00:32:54.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.495+0000 7fab3e4c0740 1 -- 172.21.15.38:0/1606646194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7a2e1dc0 msgr2=0x561f7a2f3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:54.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.495+0000 7fab3e4c0740 1 --2- 172.21.15.38:0/1606646194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7a2e1dc0 0x561f7a2f3100 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x561f7a2de130 tx=0x7fab24097c00 comp rx=0 tx=0).stop 2024-09-16T00:32:54.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.495+0000 7fab3e4c0740 1 -- 172.21.15.38:0/1606646194 shutdown_connections 2024-09-16T00:32:54.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.495+0000 7fab3e4c0740 1 --2- 172.21.15.38:0/1606646194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561f7a2ec680 0x561f7a365d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.495+0000 7fab3e4c0740 1 --2- 172.21.15.38:0/1606646194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561f7a2e1dc0 0x561f7a2f3100 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.495+0000 7fab3e4c0740 1 --2- 172.21.15.38:0/1606646194 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x561f7a367be0 0x561f7a369fd0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.495+0000 7fab3e4c0740 1 --2- 172.21.15.38:0/1606646194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561f7a2e14a0 0x561f7a2f2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.495+0000 7fab3e4c0740 1 -- 172.21.15.38:0/1606646194 >> 172.21.15.38:0/1606646194 conn(0x561f7a2dda30 msgr2=0x561f7a2dc350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:54.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.495+0000 7fab1d7fa640 0 client.25028 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:32:54.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.496+0000 7fab3e4c0740 1 -- 172.21.15.38:0/1606646194 shutdown_connections 2024-09-16T00:32:54.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.496+0000 7fab3e4c0740 1 -- 172.21.15.38:0/1606646194 wait complete. 2024-09-16T00:32:54.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.496+0000 7fab3e4c0740 20 client.25028 trim_cache size 0 max 16384 2024-09-16T00:32:54.500 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-16T00:32:54.500 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:32:54.500 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": { 2024-09-16T00:32:54.501 INFO:teuthology.orchestra.run.smithi038.stdout: "d060406b-650d-482a-a169-f5de2c01d192": { 2024-09-16T00:32:54.501 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "/", 2024-09-16T00:32:54.501 INFO:teuthology.orchestra.run.smithi038.stdout: "tag": "d060406b-650d-482a-a169-f5de2c01d192", 2024-09-16T00:32:54.501 INFO:teuthology.orchestra.run.smithi038.stdout: "options": "recursive,force" 2024-09-16T00:32:54.501 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:32:54.501 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:32:54.501 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-16T00:32:54.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.497+0000 7fab3c0e7640 1 -- 172.21.15.38:0/3583310166 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fab200a32d0 msgr2=0x7fab200a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:54.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.497+0000 7fab3c0e7640 1 --2- 172.21.15.38:0/3583310166 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fab200a32d0 0x7fab200a5790 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fab2c009430 tx=0x7fab2c0095c0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.497+0000 7fab3c0e7640 1 -- 172.21.15.38:0/3583310166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab34102790 msgr2=0x7fab34074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:54.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.497+0000 7fab3c0e7640 1 --2- 172.21.15.38:0/3583310166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab34102790 0x7fab34074a30 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fab2400d950 tx=0x7fab2400de10 comp rx=0 tx=0).stop 2024-09-16T00:32:54.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.497+0000 7fab3c0e7640 1 -- 172.21.15.38:0/3583310166 shutdown_connections 2024-09-16T00:32:54.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.497+0000 7fab3c0e7640 1 --2- 172.21.15.38:0/3583310166 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fab200a32d0 0x7fab200a5790 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.497+0000 7fab3c0e7640 1 --2- 172.21.15.38:0/3583310166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab34106f10 0x7fab34074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.497+0000 7fab3c0e7640 1 --2- 172.21.15.38:0/3583310166 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fab34102790 0x7fab34074a30 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.497+0000 7fab3c0e7640 1 --2- 172.21.15.38:0/3583310166 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fab34101de0 0x7fab340744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:54.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.497+0000 7fab3c0e7640 1 -- 172.21.15.38:0/3583310166 >> 172.21.15.38:0/3583310166 conn(0x7fab340fd8c0 msgr2=0x7fab340feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:54.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.498+0000 7fab3c0e7640 1 -- 172.21.15.38:0/3583310166 shutdown_connections 2024-09-16T00:32:54.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:54.498+0000 7fab3c0e7640 1 -- 172.21.15.38:0/3583310166 wait complete. 2024-09-16T00:32:55.162 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:54 smithi038 ceph-mon[30945]: pgmap v368: 129 pgs: 129 active+clean; 650 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-16T00:32:55.193 INFO:tasks.cephfs.filesystem:scrub status for tag:d060406b-650d-482a-a169-f5de2c01d192 - {'path': '/', 'tag': 'd060406b-650d-482a-a169-f5de2c01d192', 'options': 'recursive,force'} 2024-09-16T00:32:55.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:54 smithi122 ceph-mon[35412]: pgmap v368: 129 pgs: 129 active+clean; 650 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-16T00:32:55.275 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:55.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:54 smithi073 ceph-mon[35254]: pgmap v368: 129 pgs: 129 active+clean; 650 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2024-09-16T00:32:55.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:55.829+0000 7f6c86dd6640 1 -- 172.21.15.38:0/4001520875 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c80102ef0 msgr2=0x7f6c8010da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:55.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:55.829+0000 7f6c86dd6640 1 --2- 172.21.15.38:0/4001520875 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c80102ef0 0x7f6c8010da40 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6c78009900 tx=0x7f6c7802a480 comp rx=0 tx=0).stop 2024-09-16T00:32:55.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:55.829+0000 7f6c86dd6640 1 -- 172.21.15.38:0/4001520875 shutdown_connections 2024-09-16T00:32:55.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:55.829+0000 7f6c86dd6640 1 --2- 172.21.15.38:0/4001520875 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c80102ef0 0x7f6c8010da40 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:55.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:55.829+0000 7f6c86dd6640 1 --2- 172.21.15.38:0/4001520875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c80102570 0x7f6c801029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:55.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:55.829+0000 7f6c86dd6640 1 --2- 172.21.15.38:0/4001520875 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c80100e20 0x7f6c80101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:55.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:55.829+0000 7f6c86dd6640 1 -- 172.21.15.38:0/4001520875 >> 172.21.15.38:0/4001520875 conn(0x7f6c800fc7d0 msgr2=0x7f6c800febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:55.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:55.829+0000 7f6c86dd6640 1 -- 172.21.15.38:0/4001520875 shutdown_connections 2024-09-16T00:32:55.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:55.829+0000 7f6c86dd6640 1 -- 172.21.15.38:0/4001520875 wait complete. 2024-09-16T00:32:55.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:55.829+0000 7f6c86dd6640 1 Processor -- start 2024-09-16T00:32:55.935 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:55.829+0000 7f6c86dd6640 1 -- start start 2024-09-16T00:32:55.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:55.830+0000 7f6c86dd6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c80100e20 0x7f6c80197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:55.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:55.830+0000 7f6c86dd6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c80102570 0x7f6c801977a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:55.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:55.830+0000 7f6c86dd6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c80102ef0 0x7f6c8019bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:55.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:55.830+0000 7f6c86dd6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6c80110f80 con 0x7f6c80100e20 2024-09-16T00:32:55.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:55.830+0000 7f6c86dd6640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6c80110e00 con 0x7f6c80102570 2024-09-16T00:32:55.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:55.830+0000 7f6c86dd6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6c80111100 con 0x7f6c80102ef0 2024-09-16T00:32:55.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:55.830+0000 7f6c84b72640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c80100e20 0x7f6c80197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:55.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:55.830+0000 7f6c84b72640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c80100e20 0x7f6c80197260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57398/0 (socket says 172.21.15.38:57398) 2024-09-16T00:32:55.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:55.830+0000 7f6c84b72640 1 -- 172.21.15.38:0/2103854744 learned_addr learned my addr 172.21.15.38:0/2103854744 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:55.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:55.830+0000 7f6c7ffff640 1 --2- 172.21.15.38:0/2103854744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c80102570 0x7f6c801977a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:55.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:55.830+0000 7f6c85373640 1 --2- 172.21.15.38:0/2103854744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c80102ef0 0x7f6c8019bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:55.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:55.830+0000 7f6c84b72640 1 -- 172.21.15.38:0/2103854744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c80102ef0 msgr2=0x7f6c8019bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:55.936 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:55.830+0000 7f6c84b72640 1 --2- 172.21.15.38:0/2103854744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c80102ef0 0x7f6c8019bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:55.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:55.830+0000 7f6c84b72640 1 -- 172.21.15.38:0/2103854744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c80102570 msgr2=0x7f6c801977a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:55.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:55.830+0000 7f6c84b72640 1 --2- 172.21.15.38:0/2103854744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c80102570 0x7f6c801977a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:55.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:55.830+0000 7f6c84b72640 1 -- 172.21.15.38:0/2103854744 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c8019c210 con 0x7f6c80100e20 2024-09-16T00:32:55.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:55.830+0000 7f6c84b72640 1 --2- 172.21.15.38:0/2103854744 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c80100e20 0x7f6c80197260 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6c6800cce0 tx=0x7f6c6800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:55.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:55.831+0000 7f6c7dffb640 1 -- 172.21.15.38:0/2103854744 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6800f450 con 0x7f6c80100e20 2024-09-16T00:32:55.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:55.831+0000 7f6c86dd6640 1 -- 172.21.15.38:0/2103854744 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c8019c500 con 0x7f6c80100e20 2024-09-16T00:32:55.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:55.831+0000 7f6c7dffb640 1 -- 172.21.15.38:0/2103854744 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6c6800fad0 con 0x7f6c80100e20 2024-09-16T00:32:55.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:55.831+0000 7f6c7dffb640 1 -- 172.21.15.38:0/2103854744 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c68018b80 con 0x7f6c80100e20 2024-09-16T00:32:55.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:55.831+0000 7f6c86dd6640 1 -- 172.21.15.38:0/2103854744 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c801a3de0 con 0x7f6c80100e20 2024-09-16T00:32:55.937 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:55.832+0000 7f6c86dd6640 1 -- 172.21.15.38:0/2103854744 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c80101200 con 0x7f6c80100e20 2024-09-16T00:32:55.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:55.833+0000 7f6c7dffb640 1 -- 172.21.15.38:0/2103854744 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f6c6800f5f0 con 0x7f6c80100e20 2024-09-16T00:32:55.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:55.834+0000 7f6c7dffb640 1 --2- 172.21.15.38:0/2103854744 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f6c6c0a32d0 0x7f6c6c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:55.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:55.834+0000 7f6c7ffff640 1 --2- 172.21.15.38:0/2103854744 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f6c6c0a32d0 0x7f6c6c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:55.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:55.834+0000 7f6c7dffb640 1 -- 172.21.15.38:0/2103854744 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f6c680d04d0 con 0x7f6c80100e20 2024-09-16T00:32:55.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:55.834+0000 7f6c7ffff640 1 --2- 172.21.15.38:0/2103854744 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f6c6c0a32d0 0x7f6c6c0a5790 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6c800fec10 tx=0x7f6c7000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:55.938 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:55.838+0000 7f6c7dffb640 1 -- 172.21.15.38:0/2103854744 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f6c68014030 con 0x7f6c80100e20 2024-09-16T00:32:56.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:56.012+0000 7f6c86dd6640 1 -- 172.21.15.38:0/2103854744 --> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6c80106e00 con 0x7f6c6c0a32d0 2024-09-16T00:32:56.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:56.020+0000 7f6c7dffb640 1 -- 172.21.15.38:0/2103854744 <== mgr.14150 v2:172.21.15.38:6800/1461245815 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340493 (secure 0 0 0) 0x7f6c80106e00 con 0x7f6c6c0a32d0 2024-09-16T00:32:56.024 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:32:56.026 INFO:teuthology.orchestra.run.smithi038.stderr:dumped all 2024-09-16T00:32:56.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:56.024+0000 7f6c86dd6640 1 -- 172.21.15.38:0/2103854744 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f6c6c0a32d0 msgr2=0x7f6c6c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:56.028 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:56.024+0000 7f6c86dd6640 1 --2- 172.21.15.38:0/2103854744 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f6c6c0a32d0 0x7f6c6c0a5790 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f6c800fec10 tx=0x7f6c7000a2d0 comp rx=0 tx=0).stop 2024-09-16T00:32:56.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:56.024+0000 7f6c86dd6640 1 -- 172.21.15.38:0/2103854744 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c80100e20 msgr2=0x7f6c80197260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:56.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:56.024+0000 7f6c86dd6640 1 --2- 172.21.15.38:0/2103854744 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c80100e20 0x7f6c80197260 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f6c6800cce0 tx=0x7f6c6800a620 comp rx=0 tx=0).stop 2024-09-16T00:32:56.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:56.024+0000 7f6c86dd6640 1 -- 172.21.15.38:0/2103854744 shutdown_connections 2024-09-16T00:32:56.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:56.024+0000 7f6c86dd6640 1 --2- 172.21.15.38:0/2103854744 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f6c6c0a32d0 0x7f6c6c0a5790 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:56.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:56.024+0000 7f6c86dd6640 1 --2- 172.21.15.38:0/2103854744 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f6c80102ef0 0x7f6c8019bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:56.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:56.024+0000 7f6c86dd6640 1 --2- 172.21.15.38:0/2103854744 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6c80102570 0x7f6c801977a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:56.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:56.024+0000 7f6c86dd6640 1 --2- 172.21.15.38:0/2103854744 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6c80100e20 0x7f6c80197260 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:56.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:56.024+0000 7f6c86dd6640 1 -- 172.21.15.38:0/2103854744 >> 172.21.15.38:0/2103854744 conn(0x7f6c800fc7d0 msgr2=0x7f6c801056e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:56.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:56.024+0000 7f6c86dd6640 1 -- 172.21.15.38:0/2103854744 shutdown_connections 2024-09-16T00:32:56.029 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:56.024+0000 7f6c86dd6640 1 -- 172.21.15.38:0/2103854744 wait complete. 2024-09-16T00:32:56.687 INFO:teuthology.orchestra.run.smithi038.stdout:{"pg_ready":true,"pg_map":{"version":369,"stamp":"2024-09-16T00:32:55.546119+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":666058,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":826,"num_read_kb":543,"num_write":367,"num_write_kb":3076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1180792,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":796,"ondisk_log_size":796,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":380376,"kb_used_data":14456,"kb_used_omap":18,"kb_used_meta":365677,"kb_avail":1124561448,"statfs":{"total":1151940427776,"available":1151550922752,"internally_reserved":0,"allocated":14802944,"data_stored":8251952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":374453561},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":8594,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":57,"num_write":6,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003236"},"pg_stats":[{"pgid":"2.1f","version":"88'9","reported_seq":50,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:28.176004+0000","last_change":"2024-09-16T00:30:23.933355+0000","last_active":"2024-09-16T00:32:28.176004+0000","last_peered":"2024-09-16T00:32:28.176004+0000","last_clean":"2024-09-16T00:32:28.176004+0000","last_became_active":"2024-09-16T00:30:23.933057+0000","last_became_peered":"2024-09-16T00:30:23.933057+0000","last_unstale":"2024-09-16T00:32:28.176004+0000","last_undegraded":"2024-09-16T00:32:28.176004+0000","last_fullsized":"2024-09-16T00:32:28.176004+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:23:01.713440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.596165+0000","last_change":"2024-09-16T00:30:26.084537+0000","last_active":"2024-09-16T00:32:24.596165+0000","last_peered":"2024-09-16T00:32:24.596165+0000","last_clean":"2024-09-16T00:32:24.596165+0000","last_became_active":"2024-09-16T00:30:26.084226+0000","last_became_peered":"2024-09-16T00:30:26.084226+0000","last_unstale":"2024-09-16T00:32:24.596165+0000","last_undegraded":"2024-09-16T00:32:24.596165+0000","last_fullsized":"2024-09-16T00:32:24.596165+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:43:41.628877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594728+0000","last_change":"2024-09-16T00:30:33.154873+0000","last_active":"2024-09-16T00:32:24.594728+0000","last_peered":"2024-09-16T00:32:24.594728+0000","last_clean":"2024-09-16T00:32:24.594728+0000","last_became_active":"2024-09-16T00:30:33.154433+0000","last_became_peered":"2024-09-16T00:30:33.154433+0000","last_unstale":"2024-09-16T00:32:24.594728+0000","last_undegraded":"2024-09-16T00:32:24.594728+0000","last_fullsized":"2024-09-16T00:32:24.594728+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:50.805806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.596255+0000","last_change":"2024-09-16T00:30:24.422274+0000","last_active":"2024-09-16T00:32:24.596255+0000","last_peered":"2024-09-16T00:32:24.596255+0000","last_clean":"2024-09-16T00:32:24.596255+0000","last_became_active":"2024-09-16T00:30:24.421902+0000","last_became_peered":"2024-09-16T00:30:24.421902+0000","last_unstale":"2024-09-16T00:32:24.596255+0000","last_undegraded":"2024-09-16T00:32:24.596255+0000","last_fullsized":"2024-09-16T00:32:24.596255+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:01.910187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595568+0000","last_change":"2024-09-16T00:30:26.088165+0000","last_active":"2024-09-16T00:32:24.595568+0000","last_peered":"2024-09-16T00:32:24.595568+0000","last_clean":"2024-09-16T00:32:24.595568+0000","last_became_active":"2024-09-16T00:30:26.087593+0000","last_became_peered":"2024-09-16T00:30:26.087593+0000","last_unstale":"2024-09-16T00:32:24.595568+0000","last_undegraded":"2024-09-16T00:32:24.595568+0000","last_fullsized":"2024-09-16T00:32:24.595568+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:46:54.149930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454083+0000","last_change":"2024-09-16T00:30:33.154586+0000","last_active":"2024-09-16T00:32:24.454083+0000","last_peered":"2024-09-16T00:32:24.454083+0000","last_clean":"2024-09-16T00:32:24.454083+0000","last_became_active":"2024-09-16T00:30:33.154209+0000","last_became_peered":"2024-09-16T00:30:33.154209+0000","last_unstale":"2024-09-16T00:32:24.454083+0000","last_undegraded":"2024-09-16T00:32:24.454083+0000","last_fullsized":"2024-09-16T00:32:24.454083+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:37:45.776895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'4","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339220+0000","last_change":"2024-09-16T00:30:23.932442+0000","last_active":"2024-09-16T00:32:24.339220+0000","last_peered":"2024-09-16T00:32:24.339220+0000","last_clean":"2024-09-16T00:32:24.339220+0000","last_became_active":"2024-09-16T00:30:23.932009+0000","last_became_peered":"2024-09-16T00:30:23.932009+0000","last_unstale":"2024-09-16T00:32:24.339220+0000","last_undegraded":"2024-09-16T00:32:24.339220+0000","last_fullsized":"2024-09-16T00:32:24.339220+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:09:27.078187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561669+0000","last_change":"2024-09-16T00:30:26.087715+0000","last_active":"2024-09-16T00:32:24.561669+0000","last_peered":"2024-09-16T00:32:24.561669+0000","last_clean":"2024-09-16T00:32:24.561669+0000","last_became_active":"2024-09-16T00:30:26.087415+0000","last_became_peered":"2024-09-16T00:30:26.087415+0000","last_unstale":"2024-09-16T00:32:24.561669+0000","last_undegraded":"2024-09-16T00:32:24.561669+0000","last_fullsized":"2024-09-16T00:32:24.561669+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:01:52.373992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454447+0000","last_change":"2024-09-16T00:30:33.152328+0000","last_active":"2024-09-16T00:32:24.454447+0000","last_peered":"2024-09-16T00:32:24.454447+0000","last_clean":"2024-09-16T00:32:24.454447+0000","last_became_active":"2024-09-16T00:30:33.151954+0000","last_became_peered":"2024-09-16T00:30:33.151954+0000","last_unstale":"2024-09-16T00:32:24.454447+0000","last_undegraded":"2024-09-16T00:32:24.454447+0000","last_fullsized":"2024-09-16T00:32:24.454447+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:24:36.546729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"82'2","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339346+0000","last_change":"2024-09-16T00:30:23.928988+0000","last_active":"2024-09-16T00:32:24.339346+0000","last_peered":"2024-09-16T00:32:24.339346+0000","last_clean":"2024-09-16T00:32:24.339346+0000","last_became_active":"2024-09-16T00:30:23.928638+0000","last_became_peered":"2024-09-16T00:30:23.928638+0000","last_unstale":"2024-09-16T00:32:24.339346+0000","last_undegraded":"2024-09-16T00:32:24.339346+0000","last_fullsized":"2024-09-16T00:32:24.339346+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:45:58.203455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":30,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.336619+0000","last_change":"2024-09-16T00:31:23.446550+0000","last_active":"2024-09-16T00:32:24.336619+0000","last_peered":"2024-09-16T00:32:24.336619+0000","last_clean":"2024-09-16T00:32:24.336619+0000","last_became_active":"2024-09-16T00:31:23.446179+0000","last_became_peered":"2024-09-16T00:31:23.446179+0000","last_unstale":"2024-09-16T00:32:24.336619+0000","last_undegraded":"2024-09-16T00:32:24.336619+0000","last_fullsized":"2024-09-16T00:32:24.336619+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:04:21.280174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,1],"acting":[9,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454185+0000","last_change":"2024-09-16T00:30:33.147564+0000","last_active":"2024-09-16T00:32:24.454185+0000","last_peered":"2024-09-16T00:32:24.454185+0000","last_clean":"2024-09-16T00:32:24.454185+0000","last_became_active":"2024-09-16T00:30:33.147119+0000","last_became_peered":"2024-09-16T00:30:33.147119+0000","last_unstale":"2024-09-16T00:32:24.454185+0000","last_undegraded":"2024-09-16T00:32:24.454185+0000","last_fullsized":"2024-09-16T00:32:24.454185+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:00.509122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595076+0000","last_change":"2024-09-16T00:30:26.080954+0000","last_active":"2024-09-16T00:32:24.595076+0000","last_peered":"2024-09-16T00:32:24.595076+0000","last_clean":"2024-09-16T00:32:24.595076+0000","last_became_active":"2024-09-16T00:30:26.080624+0000","last_became_peered":"2024-09-16T00:30:26.080624+0000","last_unstale":"2024-09-16T00:32:24.595076+0000","last_undegraded":"2024-09-16T00:32:24.595076+0000","last_fullsized":"2024-09-16T00:32:24.595076+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:05:10.440980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339169+0000","last_change":"2024-09-16T00:30:23.933715+0000","last_active":"2024-09-16T00:32:24.339169+0000","last_peered":"2024-09-16T00:32:24.339169+0000","last_clean":"2024-09-16T00:32:24.339169+0000","last_became_active":"2024-09-16T00:30:23.933275+0000","last_became_peered":"2024-09-16T00:30:23.933275+0000","last_unstale":"2024-09-16T00:32:24.339169+0000","last_undegraded":"2024-09-16T00:32:24.339169+0000","last_fullsized":"2024-09-16T00:32:24.339169+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:40:44.846053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454634+0000","last_change":"2024-09-16T00:30:33.604602+0000","last_active":"2024-09-16T00:32:24.454634+0000","last_peered":"2024-09-16T00:32:24.454634+0000","last_clean":"2024-09-16T00:32:24.454634+0000","last_became_active":"2024-09-16T00:30:33.604107+0000","last_became_peered":"2024-09-16T00:30:33.604107+0000","last_unstale":"2024-09-16T00:32:24.454634+0000","last_undegraded":"2024-09-16T00:32:24.454634+0000","last_fullsized":"2024-09-16T00:32:24.454634+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:41.254445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595459+0000","last_change":"2024-09-16T00:30:26.080612+0000","last_active":"2024-09-16T00:32:24.595459+0000","last_peered":"2024-09-16T00:32:24.595459+0000","last_clean":"2024-09-16T00:32:24.595459+0000","last_became_active":"2024-09-16T00:30:26.080317+0000","last_became_peered":"2024-09-16T00:30:26.080317+0000","last_unstale":"2024-09-16T00:32:24.595459+0000","last_undegraded":"2024-09-16T00:32:24.595459+0000","last_fullsized":"2024-09-16T00:32:24.595459+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:23.814241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"88'1","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594893+0000","last_change":"2024-09-16T00:30:23.936778+0000","last_active":"2024-09-16T00:32:24.594893+0000","last_peered":"2024-09-16T00:32:24.594893+0000","last_clean":"2024-09-16T00:32:24.594893+0000","last_became_active":"2024-09-16T00:30:23.936265+0000","last_became_peered":"2024-09-16T00:30:23.936265+0000","last_unstale":"2024-09-16T00:32:24.594893+0000","last_undegraded":"2024-09-16T00:32:24.594893+0000","last_fullsized":"2024-09-16T00:32:24.594893+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:35:28.288476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454751+0000","last_change":"2024-09-16T00:30:33.148340+0000","last_active":"2024-09-16T00:32:24.454751+0000","last_peered":"2024-09-16T00:32:24.454751+0000","last_clean":"2024-09-16T00:32:24.454751+0000","last_became_active":"2024-09-16T00:30:33.147289+0000","last_became_peered":"2024-09-16T00:30:33.147289+0000","last_unstale":"2024-09-16T00:32:24.454751+0000","last_undegraded":"2024-09-16T00:32:24.454751+0000","last_fullsized":"2024-09-16T00:32:24.454751+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:17:21.259254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.338936+0000","last_change":"2024-09-16T00:30:26.077325+0000","last_active":"2024-09-16T00:32:24.338936+0000","last_peered":"2024-09-16T00:32:24.338936+0000","last_clean":"2024-09-16T00:32:24.338936+0000","last_became_active":"2024-09-16T00:30:26.077002+0000","last_became_peered":"2024-09-16T00:30:26.077002+0000","last_unstale":"2024-09-16T00:32:24.338936+0000","last_undegraded":"2024-09-16T00:32:24.338936+0000","last_fullsized":"2024-09-16T00:32:24.338936+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:12:38.569247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.9","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595646+0000","last_change":"2024-09-16T00:30:23.933025+0000","last_active":"2024-09-16T00:32:24.595646+0000","last_peered":"2024-09-16T00:32:24.595646+0000","last_clean":"2024-09-16T00:32:24.595646+0000","last_became_active":"2024-09-16T00:30:23.932669+0000","last_became_peered":"2024-09-16T00:30:23.932669+0000","last_unstale":"2024-09-16T00:32:24.595646+0000","last_undegraded":"2024-09-16T00:32:24.595646+0000","last_fullsized":"2024-09-16T00:32:24.595646+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:48:20.593665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.597881+0000","last_change":"2024-09-16T00:32:24.597170+0000","last_active":"2024-09-16T00:32:24.597881+0000","last_peered":"2024-09-16T00:32:24.597881+0000","last_clean":"2024-09-16T00:32:24.597881+0000","last_became_active":"2024-09-16T00:32:24.596810+0000","last_became_peered":"2024-09-16T00:32:24.596810+0000","last_unstale":"2024-09-16T00:32:24.597881+0000","last_undegraded":"2024-09-16T00:32:24.597881+0000","last_fullsized":"2024-09-16T00:32:24.597881+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:26:18.737332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,8],"acting":[9,3,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.338711+0000","last_change":"2024-09-16T00:30:26.076104+0000","last_active":"2024-09-16T00:32:24.338711+0000","last_peered":"2024-09-16T00:32:24.338711+0000","last_clean":"2024-09-16T00:32:24.338711+0000","last_became_active":"2024-09-16T00:30:26.075764+0000","last_became_peered":"2024-09-16T00:30:26.075764+0000","last_unstale":"2024-09-16T00:32:24.338711+0000","last_undegraded":"2024-09-16T00:32:24.338711+0000","last_fullsized":"2024-09-16T00:32:24.338711+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:36:31.228533+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561411+0000","last_change":"2024-09-16T00:30:24.422411+0000","last_active":"2024-09-16T00:32:24.561411+0000","last_peered":"2024-09-16T00:32:24.561411+0000","last_clean":"2024-09-16T00:32:24.561411+0000","last_became_active":"2024-09-16T00:30:24.422102+0000","last_became_peered":"2024-09-16T00:30:24.422102+0000","last_unstale":"2024-09-16T00:32:24.561411+0000","last_undegraded":"2024-09-16T00:32:24.561411+0000","last_fullsized":"2024-09-16T00:32:24.561411+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:06:00.572070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454739+0000","last_change":"2024-09-16T00:30:33.605397+0000","last_active":"2024-09-16T00:32:24.454739+0000","last_peered":"2024-09-16T00:32:24.454739+0000","last_clean":"2024-09-16T00:32:24.454739+0000","last_became_active":"2024-09-16T00:30:33.604978+0000","last_became_peered":"2024-09-16T00:30:33.604978+0000","last_unstale":"2024-09-16T00:32:24.454739+0000","last_undegraded":"2024-09-16T00:32:24.454739+0000","last_fullsized":"2024-09-16T00:32:24.454739+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:35:16.766056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595380+0000","last_change":"2024-09-16T00:30:26.082684+0000","last_active":"2024-09-16T00:32:24.595380+0000","last_peered":"2024-09-16T00:32:24.595380+0000","last_clean":"2024-09-16T00:32:24.595380+0000","last_became_active":"2024-09-16T00:30:26.082335+0000","last_became_peered":"2024-09-16T00:30:26.082335+0000","last_unstale":"2024-09-16T00:32:24.595380+0000","last_undegraded":"2024-09-16T00:32:24.595380+0000","last_fullsized":"2024-09-16T00:32:24.595380+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:51:21.389627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"88'4","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454909+0000","last_change":"2024-09-16T00:30:24.587840+0000","last_active":"2024-09-16T00:32:24.454909+0000","last_peered":"2024-09-16T00:32:24.454909+0000","last_clean":"2024-09-16T00:32:24.454909+0000","last_became_active":"2024-09-16T00:30:24.587436+0000","last_became_peered":"2024-09-16T00:30:24.587436+0000","last_unstale":"2024-09-16T00:32:24.454909+0000","last_undegraded":"2024-09-16T00:32:24.454909+0000","last_fullsized":"2024-09-16T00:32:24.454909+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:23:06.758585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594628+0000","last_change":"2024-09-16T00:30:33.145051+0000","last_active":"2024-09-16T00:32:24.594628+0000","last_peered":"2024-09-16T00:32:24.594628+0000","last_clean":"2024-09-16T00:32:24.594628+0000","last_became_active":"2024-09-16T00:30:33.144585+0000","last_became_peered":"2024-09-16T00:30:33.144585+0000","last_unstale":"2024-09-16T00:32:24.594628+0000","last_undegraded":"2024-09-16T00:32:24.594628+0000","last_fullsized":"2024-09-16T00:32:24.594628+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:48:23.588495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":20,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.336844+0000","last_change":"2024-09-16T00:31:23.444802+0000","last_active":"2024-09-16T00:32:24.336844+0000","last_peered":"2024-09-16T00:32:24.336844+0000","last_clean":"2024-09-16T00:32:24.336844+0000","last_became_active":"2024-09-16T00:31:23.444447+0000","last_became_peered":"2024-09-16T00:31:23.444447+0000","last_unstale":"2024-09-16T00:32:24.336844+0000","last_undegraded":"2024-09-16T00:32:24.336844+0000","last_fullsized":"2024-09-16T00:32:24.336844+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:46:37.285250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,7],"acting":[9,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.6","version":"92'14","reported_seq":53,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:41.088268+0000","last_change":"2024-09-16T00:30:23.933284+0000","last_active":"2024-09-16T00:32:41.088268+0000","last_peered":"2024-09-16T00:32:41.088268+0000","last_clean":"2024-09-16T00:32:41.088268+0000","last_became_active":"2024-09-16T00:30:23.932920+0000","last_became_peered":"2024-09-16T00:30:23.932920+0000","last_unstale":"2024-09-16T00:32:41.088268+0000","last_undegraded":"2024-09-16T00:32:41.088268+0000","last_fullsized":"2024-09-16T00:32:41.088268+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:12:38.199391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594942+0000","last_change":"2024-09-16T00:30:33.604308+0000","last_active":"2024-09-16T00:32:24.594942+0000","last_peered":"2024-09-16T00:32:24.594942+0000","last_clean":"2024-09-16T00:32:24.594942+0000","last_became_active":"2024-09-16T00:30:33.603889+0000","last_became_peered":"2024-09-16T00:30:33.603889+0000","last_unstale":"2024-09-16T00:32:24.594942+0000","last_undegraded":"2024-09-16T00:32:24.594942+0000","last_fullsized":"2024-09-16T00:32:24.594942+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:32:55.173063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452512+0000","last_change":"2024-09-16T00:30:26.078515+0000","last_active":"2024-09-16T00:32:24.452512+0000","last_peered":"2024-09-16T00:32:24.452512+0000","last_clean":"2024-09-16T00:32:24.452512+0000","last_became_active":"2024-09-16T00:30:26.078173+0000","last_became_peered":"2024-09-16T00:30:26.078173+0000","last_unstale":"2024-09-16T00:32:24.452512+0000","last_undegraded":"2024-09-16T00:32:24.452512+0000","last_fullsized":"2024-09-16T00:32:24.452512+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:40:31.471154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"88'1","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339304+0000","last_change":"2024-09-16T00:30:24.423209+0000","last_active":"2024-09-16T00:32:24.339304+0000","last_peered":"2024-09-16T00:32:24.339304+0000","last_clean":"2024-09-16T00:32:24.339304+0000","last_became_active":"2024-09-16T00:30:24.422157+0000","last_became_peered":"2024-09-16T00:30:24.422157+0000","last_unstale":"2024-09-16T00:32:24.339304+0000","last_undegraded":"2024-09-16T00:32:24.339304+0000","last_fullsized":"2024-09-16T00:32:24.339304+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:28.262392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339327+0000","last_change":"2024-09-16T00:30:33.146204+0000","last_active":"2024-09-16T00:32:24.339327+0000","last_peered":"2024-09-16T00:32:24.339327+0000","last_clean":"2024-09-16T00:32:24.339327+0000","last_became_active":"2024-09-16T00:30:33.145553+0000","last_became_peered":"2024-09-16T00:30:33.145553+0000","last_unstale":"2024-09-16T00:32:24.339327+0000","last_undegraded":"2024-09-16T00:32:24.339327+0000","last_fullsized":"2024-09-16T00:32:24.339327+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:32:22.096050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.337437+0000","last_change":"2024-09-16T00:30:26.070153+0000","last_active":"2024-09-16T00:32:24.337437+0000","last_peered":"2024-09-16T00:32:24.337437+0000","last_clean":"2024-09-16T00:32:24.337437+0000","last_became_active":"2024-09-16T00:30:26.069792+0000","last_became_peered":"2024-09-16T00:30:26.069792+0000","last_unstale":"2024-09-16T00:32:24.337437+0000","last_undegraded":"2024-09-16T00:32:24.337437+0000","last_fullsized":"2024-09-16T00:32:24.337437+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:15:28.164242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.336815+0000","last_change":"2024-09-16T00:30:24.423261+0000","last_active":"2024-09-16T00:32:24.336815+0000","last_peered":"2024-09-16T00:32:24.336815+0000","last_clean":"2024-09-16T00:32:24.336815+0000","last_became_active":"2024-09-16T00:30:24.422650+0000","last_became_peered":"2024-09-16T00:30:24.422650+0000","last_unstale":"2024-09-16T00:32:24.336815+0000","last_undegraded":"2024-09-16T00:32:24.336815+0000","last_fullsized":"2024-09-16T00:32:24.336815+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:00:00.211744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339762+0000","last_change":"2024-09-16T00:30:33.604492+0000","last_active":"2024-09-16T00:32:24.339762+0000","last_peered":"2024-09-16T00:32:24.339762+0000","last_clean":"2024-09-16T00:32:24.339762+0000","last_became_active":"2024-09-16T00:30:33.604015+0000","last_became_peered":"2024-09-16T00:30:33.604015+0000","last_unstale":"2024-09-16T00:32:24.339762+0000","last_undegraded":"2024-09-16T00:32:24.339762+0000","last_fullsized":"2024-09-16T00:32:24.339762+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:40:22.516583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.337389+0000","last_change":"2024-09-16T00:30:26.075517+0000","last_active":"2024-09-16T00:32:24.337389+0000","last_peered":"2024-09-16T00:32:24.337389+0000","last_clean":"2024-09-16T00:32:24.337389+0000","last_became_active":"2024-09-16T00:30:26.075113+0000","last_became_peered":"2024-09-16T00:30:26.075113+0000","last_unstale":"2024-09-16T00:32:24.337389+0000","last_undegraded":"2024-09-16T00:32:24.337389+0000","last_fullsized":"2024-09-16T00:32:24.337389+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:07:11.427010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"88'1","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595204+0000","last_change":"2024-09-16T00:30:23.933639+0000","last_active":"2024-09-16T00:32:24.595204+0000","last_peered":"2024-09-16T00:32:24.595204+0000","last_clean":"2024-09-16T00:32:24.595204+0000","last_became_active":"2024-09-16T00:30:23.933322+0000","last_became_peered":"2024-09-16T00:30:23.933322+0000","last_unstale":"2024-09-16T00:32:24.595204+0000","last_undegraded":"2024-09-16T00:32:24.595204+0000","last_fullsized":"2024-09-16T00:32:24.595204+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:59:59.082702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452139+0000","last_change":"2024-09-16T00:30:33.148508+0000","last_active":"2024-09-16T00:32:24.452139+0000","last_peered":"2024-09-16T00:32:24.452139+0000","last_clean":"2024-09-16T00:32:24.452139+0000","last_became_active":"2024-09-16T00:30:33.148091+0000","last_became_peered":"2024-09-16T00:30:33.148091+0000","last_unstale":"2024-09-16T00:32:24.452139+0000","last_undegraded":"2024-09-16T00:32:24.452139+0000","last_fullsized":"2024-09-16T00:32:24.452139+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:43:50.052526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454239+0000","last_change":"2024-09-16T00:30:26.080374+0000","last_active":"2024-09-16T00:32:24.454239+0000","last_peered":"2024-09-16T00:32:24.454239+0000","last_clean":"2024-09-16T00:32:24.454239+0000","last_became_active":"2024-09-16T00:30:26.080044+0000","last_became_peered":"2024-09-16T00:30:26.080044+0000","last_unstale":"2024-09-16T00:32:24.454239+0000","last_undegraded":"2024-09-16T00:32:24.454239+0000","last_fullsized":"2024-09-16T00:32:24.454239+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:38:36.034903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'4","reported_seq":155,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:53.495131+0000","last_change":"2024-09-16T00:30:24.422137+0000","last_active":"2024-09-16T00:32:53.495131+0000","last_peered":"2024-09-16T00:32:53.495131+0000","last_clean":"2024-09-16T00:32:53.495131+0000","last_became_active":"2024-09-16T00:30:24.421778+0000","last_became_peered":"2024-09-16T00:30:24.421778+0000","last_unstale":"2024-09-16T00:32:53.495131+0000","last_undegraded":"2024-09-16T00:32:53.495131+0000","last_fullsized":"2024-09-16T00:32:53.495131+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:54:21.964099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":112,"num_read_kb":112,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452309+0000","last_change":"2024-09-16T00:30:33.604263+0000","last_active":"2024-09-16T00:32:24.452309+0000","last_peered":"2024-09-16T00:32:24.452309+0000","last_clean":"2024-09-16T00:32:24.452309+0000","last_became_active":"2024-09-16T00:30:33.603804+0000","last_became_peered":"2024-09-16T00:30:33.603804+0000","last_unstale":"2024-09-16T00:32:24.452309+0000","last_undegraded":"2024-09-16T00:32:24.452309+0000","last_fullsized":"2024-09-16T00:32:24.452309+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:08:37.590827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.451792+0000","last_change":"2024-09-16T00:30:26.082916+0000","last_active":"2024-09-16T00:32:24.451792+0000","last_peered":"2024-09-16T00:32:24.451792+0000","last_clean":"2024-09-16T00:32:24.451792+0000","last_became_active":"2024-09-16T00:30:26.082606+0000","last_became_peered":"2024-09-16T00:30:26.082606+0000","last_unstale":"2024-09-16T00:32:24.451792+0000","last_undegraded":"2024-09-16T00:32:24.451792+0000","last_fullsized":"2024-09-16T00:32:24.451792+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:11:56.777640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594991+0000","last_change":"2024-09-16T00:30:23.937602+0000","last_active":"2024-09-16T00:32:24.594991+0000","last_peered":"2024-09-16T00:32:24.594991+0000","last_clean":"2024-09-16T00:32:24.594991+0000","last_became_active":"2024-09-16T00:30:23.937261+0000","last_became_peered":"2024-09-16T00:30:23.937261+0000","last_unstale":"2024-09-16T00:32:24.594991+0000","last_undegraded":"2024-09-16T00:32:24.594991+0000","last_fullsized":"2024-09-16T00:32:24.594991+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:55:24.698758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454567+0000","last_change":"2024-09-16T00:30:33.605536+0000","last_active":"2024-09-16T00:32:24.454567+0000","last_peered":"2024-09-16T00:32:24.454567+0000","last_clean":"2024-09-16T00:32:24.454567+0000","last_became_active":"2024-09-16T00:30:33.605073+0000","last_became_peered":"2024-09-16T00:30:33.605073+0000","last_unstale":"2024-09-16T00:32:24.454567+0000","last_undegraded":"2024-09-16T00:32:24.454567+0000","last_fullsized":"2024-09-16T00:32:24.454567+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:48:11.143052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595152+0000","last_change":"2024-09-16T00:30:26.084507+0000","last_active":"2024-09-16T00:32:24.595152+0000","last_peered":"2024-09-16T00:32:24.595152+0000","last_clean":"2024-09-16T00:32:24.595152+0000","last_became_active":"2024-09-16T00:30:26.084152+0000","last_became_peered":"2024-09-16T00:30:26.084152+0000","last_unstale":"2024-09-16T00:32:24.595152+0000","last_undegraded":"2024-09-16T00:32:24.595152+0000","last_fullsized":"2024-09-16T00:32:24.595152+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:27:21.557409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'3","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454701+0000","last_change":"2024-09-16T00:30:23.932686+0000","last_active":"2024-09-16T00:32:24.454701+0000","last_peered":"2024-09-16T00:32:24.454701+0000","last_clean":"2024-09-16T00:32:24.454701+0000","last_became_active":"2024-09-16T00:30:23.932343+0000","last_became_peered":"2024-09-16T00:30:23.932343+0000","last_unstale":"2024-09-16T00:32:24.454701+0000","last_undegraded":"2024-09-16T00:32:24.454701+0000","last_fullsized":"2024-09-16T00:32:24.454701+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:13:44.185712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":351,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595072+0000","last_change":"2024-09-16T00:29:11.143642+0000","last_active":"2024-09-16T00:32:24.595072+0000","last_peered":"2024-09-16T00:32:24.595072+0000","last_clean":"2024-09-16T00:32:24.595072+0000","last_became_active":"2024-09-16T00:29:11.143227+0000","last_became_peered":"2024-09-16T00:29:11.143227+0000","last_unstale":"2024-09-16T00:32:24.595072+0000","last_undegraded":"2024-09-16T00:32:24.595072+0000","last_fullsized":"2024-09-16T00:32:24.595072+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:25:42.361316+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:25:42.361316+0000","last_clean_scrub_stamp":"2024-09-16T00:25:42.361316+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:29:12.740474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":27,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595397+0000","last_change":"2024-09-16T00:30:33.155555+0000","last_active":"2024-09-16T00:32:24.595397+0000","last_peered":"2024-09-16T00:32:24.595397+0000","last_clean":"2024-09-16T00:32:24.595397+0000","last_became_active":"2024-09-16T00:30:33.155170+0000","last_became_peered":"2024-09-16T00:30:33.155170+0000","last_unstale":"2024-09-16T00:32:24.595397+0000","last_undegraded":"2024-09-16T00:32:24.595397+0000","last_fullsized":"2024-09-16T00:32:24.595397+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:34:46.214422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595573+0000","last_change":"2024-09-16T00:30:26.076003+0000","last_active":"2024-09-16T00:32:24.595573+0000","last_peered":"2024-09-16T00:32:24.595573+0000","last_clean":"2024-09-16T00:32:24.595573+0000","last_became_active":"2024-09-16T00:30:26.075666+0000","last_became_peered":"2024-09-16T00:30:26.075666+0000","last_unstale":"2024-09-16T00:32:24.595573+0000","last_undegraded":"2024-09-16T00:32:24.595573+0000","last_fullsized":"2024-09-16T00:32:24.595573+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:58:30.113380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"88'3","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594420+0000","last_change":"2024-09-16T00:30:24.587469+0000","last_active":"2024-09-16T00:32:24.594420+0000","last_peered":"2024-09-16T00:32:24.594420+0000","last_clean":"2024-09-16T00:32:24.594420+0000","last_became_active":"2024-09-16T00:30:24.587049+0000","last_became_peered":"2024-09-16T00:30:24.587049+0000","last_unstale":"2024-09-16T00:32:24.594420+0000","last_undegraded":"2024-09-16T00:32:24.594420+0000","last_fullsized":"2024-09-16T00:32:24.594420+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:17:45.934505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561433+0000","last_change":"2024-09-16T00:30:33.145921+0000","last_active":"2024-09-16T00:32:24.561433+0000","last_peered":"2024-09-16T00:32:24.561433+0000","last_clean":"2024-09-16T00:32:24.561433+0000","last_became_active":"2024-09-16T00:30:33.145450+0000","last_became_peered":"2024-09-16T00:30:33.145450+0000","last_unstale":"2024-09-16T00:32:24.561433+0000","last_undegraded":"2024-09-16T00:32:24.561433+0000","last_fullsized":"2024-09-16T00:32:24.561433+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:53:51.698686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595330+0000","last_change":"2024-09-16T00:30:26.087940+0000","last_active":"2024-09-16T00:32:24.595330+0000","last_peered":"2024-09-16T00:32:24.595330+0000","last_clean":"2024-09-16T00:32:24.595330+0000","last_became_active":"2024-09-16T00:30:26.087599+0000","last_became_peered":"2024-09-16T00:30:26.087599+0000","last_unstale":"2024-09-16T00:32:24.595330+0000","last_undegraded":"2024-09-16T00:32:24.595330+0000","last_fullsized":"2024-09-16T00:32:24.595330+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:36:43.934839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"82'1","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454847+0000","last_change":"2024-09-16T00:30:23.930986+0000","last_active":"2024-09-16T00:32:24.454847+0000","last_peered":"2024-09-16T00:32:24.454847+0000","last_clean":"2024-09-16T00:32:24.454847+0000","last_became_active":"2024-09-16T00:30:23.930588+0000","last_became_peered":"2024-09-16T00:30:23.930588+0000","last_unstale":"2024-09-16T00:32:24.454847+0000","last_undegraded":"2024-09-16T00:32:24.454847+0000","last_fullsized":"2024-09-16T00:32:24.454847+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:04:41.902358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594719+0000","last_change":"2024-09-16T00:30:33.154894+0000","last_active":"2024-09-16T00:32:24.594719+0000","last_peered":"2024-09-16T00:32:24.594719+0000","last_clean":"2024-09-16T00:32:24.594719+0000","last_became_active":"2024-09-16T00:30:33.154490+0000","last_became_peered":"2024-09-16T00:30:33.154490+0000","last_unstale":"2024-09-16T00:32:24.594719+0000","last_undegraded":"2024-09-16T00:32:24.594719+0000","last_fullsized":"2024-09-16T00:32:24.594719+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:59:53.854212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595549+0000","last_change":"2024-09-16T00:30:26.077069+0000","last_active":"2024-09-16T00:32:24.595549+0000","last_peered":"2024-09-16T00:32:24.595549+0000","last_clean":"2024-09-16T00:32:24.595549+0000","last_became_active":"2024-09-16T00:30:26.076718+0000","last_became_peered":"2024-09-16T00:30:26.076718+0000","last_unstale":"2024-09-16T00:32:24.595549+0000","last_undegraded":"2024-09-16T00:32:24.595549+0000","last_fullsized":"2024-09-16T00:32:24.595549+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:25:37.541704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"82'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452322+0000","last_change":"2024-09-16T00:30:23.932081+0000","last_active":"2024-09-16T00:32:24.452322+0000","last_peered":"2024-09-16T00:32:24.452322+0000","last_clean":"2024-09-16T00:32:24.452322+0000","last_became_active":"2024-09-16T00:30:23.931729+0000","last_became_peered":"2024-09-16T00:30:23.931729+0000","last_unstale":"2024-09-16T00:32:24.452322+0000","last_undegraded":"2024-09-16T00:32:24.452322+0000","last_fullsized":"2024-09-16T00:32:24.452322+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:20:05.178580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454073+0000","last_change":"2024-09-16T00:30:33.147738+0000","last_active":"2024-09-16T00:32:24.454073+0000","last_peered":"2024-09-16T00:32:24.454073+0000","last_clean":"2024-09-16T00:32:24.454073+0000","last_became_active":"2024-09-16T00:30:33.147326+0000","last_became_peered":"2024-09-16T00:30:33.147326+0000","last_unstale":"2024-09-16T00:32:24.454073+0000","last_undegraded":"2024-09-16T00:32:24.454073+0000","last_fullsized":"2024-09-16T00:32:24.454073+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:44:55.123980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339213+0000","last_change":"2024-09-16T00:30:26.079312+0000","last_active":"2024-09-16T00:32:24.339213+0000","last_peered":"2024-09-16T00:32:24.339213+0000","last_clean":"2024-09-16T00:32:24.339213+0000","last_became_active":"2024-09-16T00:30:26.078986+0000","last_became_peered":"2024-09-16T00:30:26.078986+0000","last_unstale":"2024-09-16T00:32:24.339213+0000","last_undegraded":"2024-09-16T00:32:24.339213+0000","last_fullsized":"2024-09-16T00:32:24.339213+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:02:06.602850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":21,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339614+0000","last_change":"2024-09-16T00:31:23.452102+0000","last_active":"2024-09-16T00:32:24.339614+0000","last_peered":"2024-09-16T00:32:24.339614+0000","last_clean":"2024-09-16T00:32:24.339614+0000","last_became_active":"2024-09-16T00:31:23.445974+0000","last_became_peered":"2024-09-16T00:31:23.445974+0000","last_unstale":"2024-09-16T00:32:24.339614+0000","last_undegraded":"2024-09-16T00:32:24.339614+0000","last_fullsized":"2024-09-16T00:32:24.339614+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:13:52.314494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454166+0000","last_change":"2024-09-16T00:30:33.149154+0000","last_active":"2024-09-16T00:32:24.454166+0000","last_peered":"2024-09-16T00:32:24.454166+0000","last_clean":"2024-09-16T00:32:24.454166+0000","last_became_active":"2024-09-16T00:30:33.148763+0000","last_became_peered":"2024-09-16T00:30:33.148763+0000","last_unstale":"2024-09-16T00:32:24.454166+0000","last_undegraded":"2024-09-16T00:32:24.454166+0000","last_fullsized":"2024-09-16T00:32:24.454166+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:58:05.884919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454295+0000","last_change":"2024-09-16T00:30:26.086905+0000","last_active":"2024-09-16T00:32:24.454295+0000","last_peered":"2024-09-16T00:32:24.454295+0000","last_clean":"2024-09-16T00:32:24.454295+0000","last_became_active":"2024-09-16T00:30:26.086519+0000","last_became_peered":"2024-09-16T00:30:26.086519+0000","last_unstale":"2024-09-16T00:32:24.454295+0000","last_undegraded":"2024-09-16T00:32:24.454295+0000","last_fullsized":"2024-09-16T00:32:24.454295+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:51:08.787679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.10","version":"88'15","reported_seq":55,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.338826+0000","last_change":"2024-09-16T00:30:24.422893+0000","last_active":"2024-09-16T00:32:24.338826+0000","last_peered":"2024-09-16T00:32:24.338826+0000","last_clean":"2024-09-16T00:32:24.338826+0000","last_became_active":"2024-09-16T00:30:24.422073+0000","last_became_peered":"2024-09-16T00:30:24.422073+0000","last_unstale":"2024-09-16T00:32:24.338826+0000","last_undegraded":"2024-09-16T00:32:24.338826+0000","last_fullsized":"2024-09-16T00:32:24.338826+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:15:11.375957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452155+0000","last_change":"2024-09-16T00:30:33.152582+0000","last_active":"2024-09-16T00:32:24.452155+0000","last_peered":"2024-09-16T00:32:24.452155+0000","last_clean":"2024-09-16T00:32:24.452155+0000","last_became_active":"2024-09-16T00:30:33.152180+0000","last_became_peered":"2024-09-16T00:30:33.152180+0000","last_unstale":"2024-09-16T00:32:24.452155+0000","last_undegraded":"2024-09-16T00:32:24.452155+0000","last_fullsized":"2024-09-16T00:32:24.452155+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:41:10.499391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594802+0000","last_change":"2024-09-16T00:30:26.086052+0000","last_active":"2024-09-16T00:32:24.594802+0000","last_peered":"2024-09-16T00:32:24.594802+0000","last_clean":"2024-09-16T00:32:24.594802+0000","last_became_active":"2024-09-16T00:30:26.085756+0000","last_became_peered":"2024-09-16T00:30:26.085756+0000","last_unstale":"2024-09-16T00:32:24.594802+0000","last_undegraded":"2024-09-16T00:32:24.594802+0000","last_fullsized":"2024-09-16T00:32:24.594802+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:31:52.582166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"88'3","reported_seq":154,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:51.492404+0000","last_change":"2024-09-16T00:30:23.933528+0000","last_active":"2024-09-16T00:32:51.492404+0000","last_peered":"2024-09-16T00:32:51.492404+0000","last_clean":"2024-09-16T00:32:51.492404+0000","last_became_active":"2024-09-16T00:30:23.933186+0000","last_became_peered":"2024-09-16T00:30:23.933186+0000","last_unstale":"2024-09-16T00:32:51.492404+0000","last_undegraded":"2024-09-16T00:32:51.492404+0000","last_fullsized":"2024-09-16T00:32:51.492404+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:11:00.489676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":113,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.340727+0000","last_change":"2024-09-16T00:31:23.578056+0000","last_active":"2024-09-16T00:32:24.340727+0000","last_peered":"2024-09-16T00:32:24.340727+0000","last_clean":"2024-09-16T00:32:24.340727+0000","last_became_active":"2024-09-16T00:31:23.577721+0000","last_became_peered":"2024-09-16T00:31:23.577721+0000","last_unstale":"2024-09-16T00:32:24.340727+0000","last_undegraded":"2024-09-16T00:32:24.340727+0000","last_fullsized":"2024-09-16T00:32:24.340727+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:53.632440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,7,0,3],"acting":[8,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595248+0000","last_change":"2024-09-16T00:30:26.084509+0000","last_active":"2024-09-16T00:32:24.595248+0000","last_peered":"2024-09-16T00:32:24.595248+0000","last_clean":"2024-09-16T00:32:24.595248+0000","last_became_active":"2024-09-16T00:30:26.084188+0000","last_became_peered":"2024-09-16T00:30:26.084188+0000","last_unstale":"2024-09-16T00:32:24.595248+0000","last_undegraded":"2024-09-16T00:32:24.595248+0000","last_fullsized":"2024-09-16T00:32:24.595248+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:21:48.981277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.12","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454529+0000","last_change":"2024-09-16T00:30:23.933499+0000","last_active":"2024-09-16T00:32:24.454529+0000","last_peered":"2024-09-16T00:32:24.454529+0000","last_clean":"2024-09-16T00:32:24.454529+0000","last_became_active":"2024-09-16T00:30:23.933082+0000","last_became_peered":"2024-09-16T00:30:23.933082+0000","last_unstale":"2024-09-16T00:32:24.454529+0000","last_undegraded":"2024-09-16T00:32:24.454529+0000","last_fullsized":"2024-09-16T00:32:24.454529+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:42:18.998433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":33,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.596742+0000","last_change":"2024-09-16T00:32:24.596318+0000","last_active":"2024-09-16T00:32:24.596742+0000","last_peered":"2024-09-16T00:32:24.596742+0000","last_clean":"2024-09-16T00:32:24.596742+0000","last_became_active":"2024-09-16T00:32:24.595906+0000","last_became_peered":"2024-09-16T00:32:24.595906+0000","last_unstale":"2024-09-16T00:32:24.596742+0000","last_undegraded":"2024-09-16T00:32:24.596742+0000","last_fullsized":"2024-09-16T00:32:24.596742+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:13:56.601745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452018+0000","last_change":"2024-09-16T00:30:26.080517+0000","last_active":"2024-09-16T00:32:24.452018+0000","last_peered":"2024-09-16T00:32:24.452018+0000","last_clean":"2024-09-16T00:32:24.452018+0000","last_became_active":"2024-09-16T00:30:26.080197+0000","last_became_peered":"2024-09-16T00:30:26.080197+0000","last_unstale":"2024-09-16T00:32:24.452018+0000","last_undegraded":"2024-09-16T00:32:24.452018+0000","last_fullsized":"2024-09-16T00:32:24.452018+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:23:15.508562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.13","version":"88'6","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339267+0000","last_change":"2024-09-16T00:31:23.457047+0000","last_active":"2024-09-16T00:32:24.339267+0000","last_peered":"2024-09-16T00:32:24.339267+0000","last_clean":"2024-09-16T00:32:24.339267+0000","last_became_active":"2024-09-16T00:31:23.447014+0000","last_became_peered":"2024-09-16T00:31:23.447014+0000","last_unstale":"2024-09-16T00:32:24.339267+0000","last_undegraded":"2024-09-16T00:32:24.339267+0000","last_fullsized":"2024-09-16T00:32:24.339267+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:34:42.040228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":34,"num_keys_recovered":10,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,10,3],"acting":[11,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454641+0000","last_change":"2024-09-16T00:30:33.145803+0000","last_active":"2024-09-16T00:32:24.454641+0000","last_peered":"2024-09-16T00:32:24.454641+0000","last_clean":"2024-09-16T00:32:24.454641+0000","last_became_active":"2024-09-16T00:30:33.145366+0000","last_became_peered":"2024-09-16T00:30:33.145366+0000","last_unstale":"2024-09-16T00:32:24.454641+0000","last_undegraded":"2024-09-16T00:32:24.454641+0000","last_fullsized":"2024-09-16T00:32:24.454641+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:47:28.689305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339138+0000","last_change":"2024-09-16T00:30:26.079415+0000","last_active":"2024-09-16T00:32:24.339138+0000","last_peered":"2024-09-16T00:32:24.339138+0000","last_clean":"2024-09-16T00:32:24.339138+0000","last_became_active":"2024-09-16T00:30:26.079075+0000","last_became_peered":"2024-09-16T00:30:26.079075+0000","last_unstale":"2024-09-16T00:32:24.339138+0000","last_undegraded":"2024-09-16T00:32:24.339138+0000","last_fullsized":"2024-09-16T00:32:24.339138+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:18:18.236342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.14","version":"92'40","reported_seq":314,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:53.498804+0000","last_change":"2024-09-16T00:30:23.933172+0000","last_active":"2024-09-16T00:32:53.498804+0000","last_peered":"2024-09-16T00:32:53.498804+0000","last_clean":"2024-09-16T00:32:53.498804+0000","last_became_active":"2024-09-16T00:30:23.932492+0000","last_became_peered":"2024-09-16T00:30:23.932492+0000","last_unstale":"2024-09-16T00:32:53.498804+0000","last_undegraded":"2024-09-16T00:32:53.498804+0000","last_fullsized":"2024-09-16T00:32:53.498804+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:34:29.169448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70919,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":235,"num_read_kb":189,"num_write":41,"num_write_kb":90,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.337053+0000","last_change":"2024-09-16T00:31:23.445673+0000","last_active":"2024-09-16T00:32:24.337053+0000","last_peered":"2024-09-16T00:32:24.337053+0000","last_clean":"2024-09-16T00:32:24.337053+0000","last_became_active":"2024-09-16T00:31:23.445299+0000","last_became_peered":"2024-09-16T00:31:23.445299+0000","last_unstale":"2024-09-16T00:32:24.337053+0000","last_undegraded":"2024-09-16T00:32:24.337053+0000","last_fullsized":"2024-09-16T00:32:24.337053+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:07:44.235340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561729+0000","last_change":"2024-09-16T00:30:26.085419+0000","last_active":"2024-09-16T00:32:24.561729+0000","last_peered":"2024-09-16T00:32:24.561729+0000","last_clean":"2024-09-16T00:32:24.561729+0000","last_became_active":"2024-09-16T00:30:26.085103+0000","last_became_peered":"2024-09-16T00:30:26.085103+0000","last_unstale":"2024-09-16T00:32:24.561729+0000","last_undegraded":"2024-09-16T00:32:24.561729+0000","last_fullsized":"2024-09-16T00:32:24.561729+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:56:26.352779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.15","version":"88'4","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.453958+0000","last_change":"2024-09-16T00:30:24.422646+0000","last_active":"2024-09-16T00:32:24.453958+0000","last_peered":"2024-09-16T00:32:24.453958+0000","last_clean":"2024-09-16T00:32:24.453958+0000","last_became_active":"2024-09-16T00:30:24.422065+0000","last_became_peered":"2024-09-16T00:30:24.422065+0000","last_unstale":"2024-09-16T00:32:24.453958+0000","last_undegraded":"2024-09-16T00:32:24.453958+0000","last_fullsized":"2024-09-16T00:32:24.453958+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:53:12.401613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454372+0000","last_change":"2024-09-16T00:30:33.147692+0000","last_active":"2024-09-16T00:32:24.454372+0000","last_peered":"2024-09-16T00:32:24.454372+0000","last_clean":"2024-09-16T00:32:24.454372+0000","last_became_active":"2024-09-16T00:30:33.147115+0000","last_became_peered":"2024-09-16T00:30:33.147115+0000","last_unstale":"2024-09-16T00:32:24.454372+0000","last_undegraded":"2024-09-16T00:32:24.454372+0000","last_fullsized":"2024-09-16T00:32:24.454372+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:44:04.536894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595645+0000","last_change":"2024-09-16T00:30:26.088427+0000","last_active":"2024-09-16T00:32:24.595645+0000","last_peered":"2024-09-16T00:32:24.595645+0000","last_clean":"2024-09-16T00:32:24.595645+0000","last_became_active":"2024-09-16T00:30:26.086454+0000","last_became_peered":"2024-09-16T00:30:26.086454+0000","last_unstale":"2024-09-16T00:32:24.595645+0000","last_undegraded":"2024-09-16T00:32:24.595645+0000","last_fullsized":"2024-09-16T00:32:24.595645+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:54:29.679959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.16","version":"88'2","reported_seq":51,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595506+0000","last_change":"2024-09-16T00:31:23.453484+0000","last_active":"2024-09-16T00:32:24.595506+0000","last_peered":"2024-09-16T00:32:24.595506+0000","last_clean":"2024-09-16T00:32:24.595506+0000","last_became_active":"2024-09-16T00:31:23.446700+0000","last_became_peered":"2024-09-16T00:31:23.446700+0000","last_unstale":"2024-09-16T00:32:24.595506+0000","last_undegraded":"2024-09-16T00:32:24.595506+0000","last_fullsized":"2024-09-16T00:32:24.595506+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:43:26.465657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,4],"acting":[7,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.336334+0000","last_change":"2024-09-16T00:30:33.156510+0000","last_active":"2024-09-16T00:32:24.336334+0000","last_peered":"2024-09-16T00:32:24.336334+0000","last_clean":"2024-09-16T00:32:24.336334+0000","last_became_active":"2024-09-16T00:30:33.156088+0000","last_became_peered":"2024-09-16T00:30:33.156088+0000","last_unstale":"2024-09-16T00:32:24.336334+0000","last_undegraded":"2024-09-16T00:32:24.336334+0000","last_fullsized":"2024-09-16T00:32:24.336334+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:59:34.230901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561930+0000","last_change":"2024-09-16T00:30:26.082598+0000","last_active":"2024-09-16T00:32:24.561930+0000","last_peered":"2024-09-16T00:32:24.561930+0000","last_clean":"2024-09-16T00:32:24.561930+0000","last_became_active":"2024-09-16T00:30:26.082286+0000","last_became_peered":"2024-09-16T00:30:26.082286+0000","last_unstale":"2024-09-16T00:32:24.561930+0000","last_undegraded":"2024-09-16T00:32:24.561930+0000","last_fullsized":"2024-09-16T00:32:24.561930+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:22:18.579149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"92'5","reported_seq":155,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:52.497378+0000","last_change":"2024-09-16T00:30:23.931097+0000","last_active":"2024-09-16T00:32:52.497378+0000","last_peered":"2024-09-16T00:32:52.497378+0000","last_clean":"2024-09-16T00:32:52.497378+0000","last_became_active":"2024-09-16T00:30:23.930719+0000","last_became_peered":"2024-09-16T00:30:23.930719+0000","last_unstale":"2024-09-16T00:32:52.497378+0000","last_undegraded":"2024-09-16T00:32:52.497378+0000","last_fullsized":"2024-09-16T00:32:52.497378+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:13:47.887754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":111,"num_read_kb":111,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595241+0000","last_change":"2024-09-16T00:30:33.142581+0000","last_active":"2024-09-16T00:32:24.595241+0000","last_peered":"2024-09-16T00:32:24.595241+0000","last_clean":"2024-09-16T00:32:24.595241+0000","last_became_active":"2024-09-16T00:30:33.142263+0000","last_became_peered":"2024-09-16T00:30:33.142263+0000","last_unstale":"2024-09-16T00:32:24.595241+0000","last_undegraded":"2024-09-16T00:32:24.595241+0000","last_fullsized":"2024-09-16T00:32:24.595241+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:10:06.456011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595181+0000","last_change":"2024-09-16T00:30:33.152084+0000","last_active":"2024-09-16T00:32:24.595181+0000","last_peered":"2024-09-16T00:32:24.595181+0000","last_clean":"2024-09-16T00:32:24.595181+0000","last_became_active":"2024-09-16T00:30:33.151681+0000","last_became_peered":"2024-09-16T00:30:33.151681+0000","last_unstale":"2024-09-16T00:32:24.595181+0000","last_undegraded":"2024-09-16T00:32:24.595181+0000","last_fullsized":"2024-09-16T00:32:24.595181+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:07:36.966498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.336400+0000","last_change":"2024-09-16T00:30:26.084200+0000","last_active":"2024-09-16T00:32:24.336400+0000","last_peered":"2024-09-16T00:32:24.336400+0000","last_clean":"2024-09-16T00:32:24.336400+0000","last_became_active":"2024-09-16T00:30:26.083858+0000","last_became_peered":"2024-09-16T00:30:26.083858+0000","last_unstale":"2024-09-16T00:32:24.336400+0000","last_undegraded":"2024-09-16T00:32:24.336400+0000","last_fullsized":"2024-09-16T00:32:24.336400+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:33:42.712624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.18","version":"88'1","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454400+0000","last_change":"2024-09-16T00:30:23.932399+0000","last_active":"2024-09-16T00:32:24.454400+0000","last_peered":"2024-09-16T00:32:24.454400+0000","last_clean":"2024-09-16T00:32:24.454400+0000","last_became_active":"2024-09-16T00:30:23.931996+0000","last_became_peered":"2024-09-16T00:30:23.931996+0000","last_unstale":"2024-09-16T00:32:24.454400+0000","last_undegraded":"2024-09-16T00:32:24.454400+0000","last_fullsized":"2024-09-16T00:32:24.454400+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:47:09.634507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":26,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339553+0000","last_change":"2024-09-16T00:30:33.154563+0000","last_active":"2024-09-16T00:32:24.339553+0000","last_peered":"2024-09-16T00:32:24.339553+0000","last_clean":"2024-09-16T00:32:24.339553+0000","last_became_active":"2024-09-16T00:30:33.154105+0000","last_became_peered":"2024-09-16T00:30:33.154105+0000","last_unstale":"2024-09-16T00:32:24.339553+0000","last_undegraded":"2024-09-16T00:32:24.339553+0000","last_fullsized":"2024-09-16T00:32:24.339553+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:13:59.966255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595055+0000","last_change":"2024-09-16T00:30:26.075126+0000","last_active":"2024-09-16T00:32:24.595055+0000","last_peered":"2024-09-16T00:32:24.595055+0000","last_clean":"2024-09-16T00:32:24.595055+0000","last_became_active":"2024-09-16T00:30:26.074819+0000","last_became_peered":"2024-09-16T00:30:26.074819+0000","last_unstale":"2024-09-16T00:32:24.595055+0000","last_undegraded":"2024-09-16T00:32:24.595055+0000","last_fullsized":"2024-09-16T00:32:24.595055+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:47:36.996346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"88'2","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.451861+0000","last_change":"2024-09-16T00:30:23.929943+0000","last_active":"2024-09-16T00:32:24.451861+0000","last_peered":"2024-09-16T00:32:24.451861+0000","last_clean":"2024-09-16T00:32:24.451861+0000","last_became_active":"2024-09-16T00:30:23.929578+0000","last_became_peered":"2024-09-16T00:30:23.929578+0000","last_unstale":"2024-09-16T00:32:24.451861+0000","last_undegraded":"2024-09-16T00:32:24.451861+0000","last_fullsized":"2024-09-16T00:32:24.451861+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:05:12.505543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1a","version":"88'3","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.337288+0000","last_change":"2024-09-16T00:30:23.935758+0000","last_active":"2024-09-16T00:32:24.337288+0000","last_peered":"2024-09-16T00:32:24.337288+0000","last_clean":"2024-09-16T00:32:24.337288+0000","last_became_active":"2024-09-16T00:30:23.935321+0000","last_became_peered":"2024-09-16T00:30:23.935321+0000","last_unstale":"2024-09-16T00:32:24.337288+0000","last_undegraded":"2024-09-16T00:32:24.337288+0000","last_fullsized":"2024-09-16T00:32:24.337288+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:35:40.513435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454389+0000","last_change":"2024-09-16T00:30:26.086316+0000","last_active":"2024-09-16T00:32:24.454389+0000","last_peered":"2024-09-16T00:32:24.454389+0000","last_clean":"2024-09-16T00:32:24.454389+0000","last_became_active":"2024-09-16T00:30:26.085800+0000","last_became_peered":"2024-09-16T00:30:26.085800+0000","last_unstale":"2024-09-16T00:32:24.454389+0000","last_undegraded":"2024-09-16T00:32:24.454389+0000","last_fullsized":"2024-09-16T00:32:24.454389+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:05:36.277127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595014+0000","last_change":"2024-09-16T00:30:33.155665+0000","last_active":"2024-09-16T00:32:24.595014+0000","last_peered":"2024-09-16T00:32:24.595014+0000","last_clean":"2024-09-16T00:32:24.595014+0000","last_became_active":"2024-09-16T00:30:33.155253+0000","last_became_peered":"2024-09-16T00:30:33.155253+0000","last_unstale":"2024-09-16T00:32:24.595014+0000","last_undegraded":"2024-09-16T00:32:24.595014+0000","last_fullsized":"2024-09-16T00:32:24.595014+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:30:37.545232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"92'8","reported_seq":162,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:52.498040+0000","last_change":"2024-09-16T00:30:23.934487+0000","last_active":"2024-09-16T00:32:52.498040+0000","last_peered":"2024-09-16T00:32:52.498040+0000","last_clean":"2024-09-16T00:32:52.498040+0000","last_became_active":"2024-09-16T00:30:23.934137+0000","last_became_peered":"2024-09-16T00:30:23.934137+0000","last_unstale":"2024-09-16T00:32:52.498040+0000","last_undegraded":"2024-09-16T00:32:52.498040+0000","last_fullsized":"2024-09-16T00:32:52.498040+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:22.900751+0000","last_clean_scrub_stamp":"2024-09-16T00:30:22.900751+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:14:33.088776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1549,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":115,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.338839+0000","last_change":"2024-09-16T00:30:26.077533+0000","last_active":"2024-09-16T00:32:24.338839+0000","last_peered":"2024-09-16T00:32:24.338839+0000","last_clean":"2024-09-16T00:32:24.338839+0000","last_became_active":"2024-09-16T00:30:26.076443+0000","last_became_peered":"2024-09-16T00:30:26.076443+0000","last_unstale":"2024-09-16T00:32:24.338839+0000","last_undegraded":"2024-09-16T00:32:24.338839+0000","last_fullsized":"2024-09-16T00:32:24.338839+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T11:39:17.495521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":25,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595151+0000","last_change":"2024-09-16T00:30:33.146123+0000","last_active":"2024-09-16T00:32:24.595151+0000","last_peered":"2024-09-16T00:32:24.595151+0000","last_clean":"2024-09-16T00:32:24.595151+0000","last_became_active":"2024-09-16T00:30:33.145716+0000","last_became_peered":"2024-09-16T00:30:33.145716+0000","last_unstale":"2024-09-16T00:32:24.595151+0000","last_undegraded":"2024-09-16T00:32:24.595151+0000","last_fullsized":"2024-09-16T00:32:24.595151+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":85,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:32.120570+0000","last_clean_scrub_stamp":"2024-09-16T00:30:32.120570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:14:27.281367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.338818+0000","last_change":"2024-09-16T00:30:26.083155+0000","last_active":"2024-09-16T00:32:24.338818+0000","last_peered":"2024-09-16T00:32:24.338818+0000","last_clean":"2024-09-16T00:32:24.338818+0000","last_became_active":"2024-09-16T00:30:26.082782+0000","last_became_peered":"2024-09-16T00:30:26.082782+0000","last_unstale":"2024-09-16T00:32:24.338818+0000","last_undegraded":"2024-09-16T00:32:24.338818+0000","last_fullsized":"2024-09-16T00:32:24.338818+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:44:15.929343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":52,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594891+0000","last_change":"2024-09-16T00:31:23.447453+0000","last_active":"2024-09-16T00:32:24.594891+0000","last_peered":"2024-09-16T00:32:24.594891+0000","last_clean":"2024-09-16T00:32:24.594891+0000","last_became_active":"2024-09-16T00:31:23.445507+0000","last_became_peered":"2024-09-16T00:31:23.445507+0000","last_unstale":"2024-09-16T00:32:24.594891+0000","last_undegraded":"2024-09-16T00:32:24.594891+0000","last_fullsized":"2024-09-16T00:32:24.594891+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:18:50.885809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,6],"acting":[2,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594798+0000","last_change":"2024-09-16T00:30:26.076252+0000","last_active":"2024-09-16T00:32:24.594798+0000","last_peered":"2024-09-16T00:32:24.594798+0000","last_clean":"2024-09-16T00:32:24.594798+0000","last_became_active":"2024-09-16T00:30:26.075922+0000","last_became_peered":"2024-09-16T00:30:26.075922+0000","last_unstale":"2024-09-16T00:32:24.594798+0000","last_undegraded":"2024-09-16T00:32:24.594798+0000","last_fullsized":"2024-09-16T00:32:24.594798+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:28:04.822817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594821+0000","last_change":"2024-09-16T00:30:26.085123+0000","last_active":"2024-09-16T00:32:24.594821+0000","last_peered":"2024-09-16T00:32:24.594821+0000","last_clean":"2024-09-16T00:32:24.594821+0000","last_became_active":"2024-09-16T00:30:26.084782+0000","last_became_peered":"2024-09-16T00:30:26.084782+0000","last_unstale":"2024-09-16T00:32:24.594821+0000","last_undegraded":"2024-09-16T00:32:24.594821+0000","last_fullsized":"2024-09-16T00:32:24.594821+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T05:09:33.766340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452404+0000","last_change":"2024-09-16T00:30:26.077048+0000","last_active":"2024-09-16T00:32:24.452404+0000","last_peered":"2024-09-16T00:32:24.452404+0000","last_clean":"2024-09-16T00:32:24.452404+0000","last_became_active":"2024-09-16T00:30:26.076719+0000","last_became_peered":"2024-09-16T00:30:26.076719+0000","last_unstale":"2024-09-16T00:32:24.452404+0000","last_undegraded":"2024-09-16T00:32:24.452404+0000","last_fullsized":"2024-09-16T00:32:24.452404+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:48:58.584246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339277+0000","last_change":"2024-09-16T00:30:26.078446+0000","last_active":"2024-09-16T00:32:24.339277+0000","last_peered":"2024-09-16T00:32:24.339277+0000","last_clean":"2024-09-16T00:32:24.339277+0000","last_became_active":"2024-09-16T00:30:26.078145+0000","last_became_peered":"2024-09-16T00:30:26.078145+0000","last_unstale":"2024-09-16T00:32:24.339277+0000","last_undegraded":"2024-09-16T00:32:24.339277+0000","last_fullsized":"2024-09-16T00:32:24.339277+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T04:52:28.739834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":49,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595347+0000","last_change":"2024-09-16T00:30:26.075875+0000","last_active":"2024-09-16T00:32:24.595347+0000","last_peered":"2024-09-16T00:32:24.595347+0000","last_clean":"2024-09-16T00:32:24.595347+0000","last_became_active":"2024-09-16T00:30:26.075424+0000","last_became_peered":"2024-09-16T00:30:26.075424+0000","last_unstale":"2024-09-16T00:32:24.595347+0000","last_undegraded":"2024-09-16T00:32:24.595347+0000","last_fullsized":"2024-09-16T00:32:24.595347+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:21:07.958507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":36,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595253+0000","last_change":"2024-09-16T00:30:26.077694+0000","last_active":"2024-09-16T00:32:24.595253+0000","last_peered":"2024-09-16T00:32:24.595253+0000","last_clean":"2024-09-16T00:32:24.595253+0000","last_became_active":"2024-09-16T00:30:26.077374+0000","last_became_peered":"2024-09-16T00:30:26.077374+0000","last_unstale":"2024-09-16T00:32:24.595253+0000","last_undegraded":"2024-09-16T00:32:24.595253+0000","last_fullsized":"2024-09-16T00:32:24.595253+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:16:30.822050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454143+0000","last_change":"2024-09-16T00:30:26.085931+0000","last_active":"2024-09-16T00:32:24.454143+0000","last_peered":"2024-09-16T00:32:24.454143+0000","last_clean":"2024-09-16T00:32:24.454143+0000","last_became_active":"2024-09-16T00:30:26.085420+0000","last_became_peered":"2024-09-16T00:30:26.085420+0000","last_unstale":"2024-09-16T00:32:24.454143+0000","last_undegraded":"2024-09-16T00:32:24.454143+0000","last_fullsized":"2024-09-16T00:32:24.454143+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:20:36.019203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595470+0000","last_change":"2024-09-16T00:30:26.074217+0000","last_active":"2024-09-16T00:32:24.595470+0000","last_peered":"2024-09-16T00:32:24.595470+0000","last_clean":"2024-09-16T00:32:24.595470+0000","last_became_active":"2024-09-16T00:30:26.073856+0000","last_became_peered":"2024-09-16T00:30:26.073856+0000","last_unstale":"2024-09-16T00:32:24.595470+0000","last_undegraded":"2024-09-16T00:32:24.595470+0000","last_fullsized":"2024-09-16T00:32:24.595470+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:53:57.078465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561639+0000","last_change":"2024-09-16T00:30:26.084248+0000","last_active":"2024-09-16T00:32:24.561639+0000","last_peered":"2024-09-16T00:32:24.561639+0000","last_clean":"2024-09-16T00:32:24.561639+0000","last_became_active":"2024-09-16T00:30:26.083530+0000","last_became_peered":"2024-09-16T00:30:26.083530+0000","last_unstale":"2024-09-16T00:32:24.561639+0000","last_undegraded":"2024-09-16T00:32:24.561639+0000","last_fullsized":"2024-09-16T00:32:24.561639+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:16:49.483881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452162+0000","last_change":"2024-09-16T00:30:26.080803+0000","last_active":"2024-09-16T00:32:24.452162+0000","last_peered":"2024-09-16T00:32:24.452162+0000","last_clean":"2024-09-16T00:32:24.452162+0000","last_became_active":"2024-09-16T00:30:26.080482+0000","last_became_peered":"2024-09-16T00:30:26.080482+0000","last_unstale":"2024-09-16T00:32:24.452162+0000","last_undegraded":"2024-09-16T00:32:24.452162+0000","last_fullsized":"2024-09-16T00:32:24.452162+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:35:41.741025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339770+0000","last_change":"2024-09-16T00:30:26.077680+0000","last_active":"2024-09-16T00:32:24.339770+0000","last_peered":"2024-09-16T00:32:24.339770+0000","last_clean":"2024-09-16T00:32:24.339770+0000","last_became_active":"2024-09-16T00:30:26.077371+0000","last_became_peered":"2024-09-16T00:30:26.077371+0000","last_unstale":"2024-09-16T00:32:24.339770+0000","last_undegraded":"2024-09-16T00:32:24.339770+0000","last_fullsized":"2024-09-16T00:32:24.339770+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T00:31:34.338617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595364+0000","last_change":"2024-09-16T00:30:26.071531+0000","last_active":"2024-09-16T00:32:24.595364+0000","last_peered":"2024-09-16T00:32:24.595364+0000","last_clean":"2024-09-16T00:32:24.595364+0000","last_became_active":"2024-09-16T00:30:26.071189+0000","last_became_peered":"2024-09-16T00:30:26.071189+0000","last_unstale":"2024-09-16T00:32:24.595364+0000","last_undegraded":"2024-09-16T00:32:24.595364+0000","last_fullsized":"2024-09-16T00:32:24.595364+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:29:18.406272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.336922+0000","last_change":"2024-09-16T00:30:26.085398+0000","last_active":"2024-09-16T00:32:24.336922+0000","last_peered":"2024-09-16T00:32:24.336922+0000","last_clean":"2024-09-16T00:32:24.336922+0000","last_became_active":"2024-09-16T00:30:26.085076+0000","last_became_peered":"2024-09-16T00:30:26.085076+0000","last_unstale":"2024-09-16T00:32:24.336922+0000","last_undegraded":"2024-09-16T00:32:24.336922+0000","last_fullsized":"2024-09-16T00:32:24.336922+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:20:20.864660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.340119+0000","last_change":"2024-09-16T00:30:26.080953+0000","last_active":"2024-09-16T00:32:24.340119+0000","last_peered":"2024-09-16T00:32:24.340119+0000","last_clean":"2024-09-16T00:32:24.340119+0000","last_became_active":"2024-09-16T00:30:26.080185+0000","last_became_peered":"2024-09-16T00:30:26.080185+0000","last_unstale":"2024-09-16T00:32:24.340119+0000","last_undegraded":"2024-09-16T00:32:24.340119+0000","last_fullsized":"2024-09-16T00:32:24.340119+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:23:56.769182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":43,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561297+0000","last_change":"2024-09-16T00:30:26.079320+0000","last_active":"2024-09-16T00:32:24.561297+0000","last_peered":"2024-09-16T00:32:24.561297+0000","last_clean":"2024-09-16T00:32:24.561297+0000","last_became_active":"2024-09-16T00:30:26.078980+0000","last_became_peered":"2024-09-16T00:30:26.078980+0000","last_unstale":"2024-09-16T00:32:24.561297+0000","last_undegraded":"2024-09-16T00:32:24.561297+0000","last_fullsized":"2024-09-16T00:32:24.561297+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:03:15.386657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594661+0000","last_change":"2024-09-16T00:30:26.084948+0000","last_active":"2024-09-16T00:32:24.594661+0000","last_peered":"2024-09-16T00:32:24.594661+0000","last_clean":"2024-09-16T00:32:24.594661+0000","last_became_active":"2024-09-16T00:30:26.084602+0000","last_became_peered":"2024-09-16T00:30:26.084602+0000","last_unstale":"2024-09-16T00:32:24.594661+0000","last_undegraded":"2024-09-16T00:32:24.594661+0000","last_fullsized":"2024-09-16T00:32:24.594661+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:30:01.709332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594560+0000","last_change":"2024-09-16T00:30:26.080491+0000","last_active":"2024-09-16T00:32:24.594560+0000","last_peered":"2024-09-16T00:32:24.594560+0000","last_clean":"2024-09-16T00:32:24.594560+0000","last_became_active":"2024-09-16T00:30:26.080189+0000","last_became_peered":"2024-09-16T00:30:26.080189+0000","last_unstale":"2024-09-16T00:32:24.594560+0000","last_undegraded":"2024-09-16T00:32:24.594560+0000","last_fullsized":"2024-09-16T00:32:24.594560+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:03:38.358919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":44,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.452086+0000","last_change":"2024-09-16T00:30:26.077410+0000","last_active":"2024-09-16T00:32:24.452086+0000","last_peered":"2024-09-16T00:32:24.452086+0000","last_clean":"2024-09-16T00:32:24.452086+0000","last_became_active":"2024-09-16T00:30:26.076498+0000","last_became_peered":"2024-09-16T00:30:26.076498+0000","last_unstale":"2024-09-16T00:32:24.452086+0000","last_undegraded":"2024-09-16T00:32:24.452086+0000","last_fullsized":"2024-09-16T00:32:24.452086+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:54:17.712774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454867+0000","last_change":"2024-09-16T00:30:26.074500+0000","last_active":"2024-09-16T00:32:24.454867+0000","last_peered":"2024-09-16T00:32:24.454867+0000","last_clean":"2024-09-16T00:32:24.454867+0000","last_became_active":"2024-09-16T00:30:26.074177+0000","last_became_peered":"2024-09-16T00:30:26.074177+0000","last_unstale":"2024-09-16T00:32:24.454867+0000","last_undegraded":"2024-09-16T00:32:24.454867+0000","last_fullsized":"2024-09-16T00:32:24.454867+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:43:13.792231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":40,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595561+0000","last_change":"2024-09-16T00:30:26.078358+0000","last_active":"2024-09-16T00:32:24.595561+0000","last_peered":"2024-09-16T00:32:24.595561+0000","last_clean":"2024-09-16T00:32:24.595561+0000","last_became_active":"2024-09-16T00:30:26.077993+0000","last_became_peered":"2024-09-16T00:30:26.077993+0000","last_unstale":"2024-09-16T00:32:24.595561+0000","last_undegraded":"2024-09-16T00:32:24.595561+0000","last_fullsized":"2024-09-16T00:32:24.595561+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T08:25:10.711394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":56,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.337703+0000","last_change":"2024-09-16T00:31:23.446836+0000","last_active":"2024-09-16T00:32:24.337703+0000","last_peered":"2024-09-16T00:32:24.337703+0000","last_clean":"2024-09-16T00:32:24.337703+0000","last_became_active":"2024-09-16T00:31:23.446471+0000","last_became_peered":"2024-09-16T00:31:23.446471+0000","last_unstale":"2024-09-16T00:32:24.337703+0000","last_undegraded":"2024-09-16T00:32:24.337703+0000","last_fullsized":"2024-09-16T00:32:24.337703+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:08:34.887618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":45,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.561901+0000","last_change":"2024-09-16T00:30:26.086313+0000","last_active":"2024-09-16T00:32:24.561901+0000","last_peered":"2024-09-16T00:32:24.561901+0000","last_clean":"2024-09-16T00:32:24.561901+0000","last_became_active":"2024-09-16T00:30:26.085760+0000","last_became_peered":"2024-09-16T00:30:26.085760+0000","last_unstale":"2024-09-16T00:32:24.561901+0000","last_undegraded":"2024-09-16T00:32:24.561901+0000","last_fullsized":"2024-09-16T00:32:24.561901+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T09:40:38.236789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594968+0000","last_change":"2024-09-16T00:30:26.085289+0000","last_active":"2024-09-16T00:32:24.594968+0000","last_peered":"2024-09-16T00:32:24.594968+0000","last_clean":"2024-09-16T00:32:24.594968+0000","last_became_active":"2024-09-16T00:30:26.084995+0000","last_became_peered":"2024-09-16T00:30:26.084995+0000","last_unstale":"2024-09-16T00:32:24.594968+0000","last_undegraded":"2024-09-16T00:32:24.594968+0000","last_fullsized":"2024-09-16T00:32:24.594968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T07:06:45.667892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.594921+0000","last_change":"2024-09-16T00:30:26.088690+0000","last_active":"2024-09-16T00:32:24.594921+0000","last_peered":"2024-09-16T00:32:24.594921+0000","last_clean":"2024-09-16T00:32:24.594921+0000","last_became_active":"2024-09-16T00:30:26.087700+0000","last_became_peered":"2024-09-16T00:30:26.087700+0000","last_unstale":"2024-09-16T00:32:24.594921+0000","last_undegraded":"2024-09-16T00:32:24.594921+0000","last_fullsized":"2024-09-16T00:32:24.594921+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T01:39:51.445324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":39,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.337214+0000","last_change":"2024-09-16T00:30:26.080215+0000","last_active":"2024-09-16T00:32:24.337214+0000","last_peered":"2024-09-16T00:32:24.337214+0000","last_clean":"2024-09-16T00:32:24.337214+0000","last_became_active":"2024-09-16T00:30:26.079038+0000","last_became_peered":"2024-09-16T00:30:26.079038+0000","last_unstale":"2024-09-16T00:32:24.337214+0000","last_undegraded":"2024-09-16T00:32:24.337214+0000","last_fullsized":"2024-09-16T00:32:24.337214+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T02:41:01.573960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":37,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.339169+0000","last_change":"2024-09-16T00:30:26.080966+0000","last_active":"2024-09-16T00:32:24.339169+0000","last_peered":"2024-09-16T00:32:24.339169+0000","last_clean":"2024-09-16T00:32:24.339169+0000","last_became_active":"2024-09-16T00:30:26.080641+0000","last_became_peered":"2024-09-16T00:30:26.080641+0000","last_unstale":"2024-09-16T00:32:24.339169+0000","last_undegraded":"2024-09-16T00:32:24.339169+0000","last_fullsized":"2024-09-16T00:32:24.339169+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:33:01.538169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.454762+0000","last_change":"2024-09-16T00:30:26.072644+0000","last_active":"2024-09-16T00:32:24.454762+0000","last_peered":"2024-09-16T00:32:24.454762+0000","last_clean":"2024-09-16T00:32:24.454762+0000","last_became_active":"2024-09-16T00:30:26.072336+0000","last_became_peered":"2024-09-16T00:30:26.072336+0000","last_unstale":"2024-09-16T00:32:24.454762+0000","last_undegraded":"2024-09-16T00:32:24.454762+0000","last_fullsized":"2024-09-16T00:32:24.454762+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T10:44:45.760057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":41,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595215+0000","last_change":"2024-09-16T00:30:26.079155+0000","last_active":"2024-09-16T00:32:24.595215+0000","last_peered":"2024-09-16T00:32:24.595215+0000","last_clean":"2024-09-16T00:32:24.595215+0000","last_became_active":"2024-09-16T00:30:26.078820+0000","last_became_peered":"2024-09-16T00:30:26.078820+0000","last_unstale":"2024-09-16T00:32:24.595215+0000","last_undegraded":"2024-09-16T00:32:24.595215+0000","last_fullsized":"2024-09-16T00:32:24.595215+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T12:08:35.071143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":38,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.595361+0000","last_change":"2024-09-16T00:30:26.081537+0000","last_active":"2024-09-16T00:32:24.595361+0000","last_peered":"2024-09-16T00:32:24.595361+0000","last_clean":"2024-09-16T00:32:24.595361+0000","last_became_active":"2024-09-16T00:30:26.080744+0000","last_became_peered":"2024-09-16T00:30:26.080744+0000","last_unstale":"2024-09-16T00:32:24.595361+0000","last_undegraded":"2024-09-16T00:32:24.595361+0000","last_fullsized":"2024-09-16T00:32:24.595361+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T03:46:38.179888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3f","version":"90'8","reported_seq":42,"reported_epoch":92,"state":"active+clean","last_fresh":"2024-09-16T00:32:24.340007+0000","last_change":"2024-09-16T00:30:26.081066+0000","last_active":"2024-09-16T00:32:24.340007+0000","last_peered":"2024-09-16T00:32:24.340007+0000","last_clean":"2024-09-16T00:32:24.340007+0000","last_became_active":"2024-09-16T00:30:26.080630+0000","last_became_peered":"2024-09-16T00:30:26.080630+0000","last_unstale":"2024-09-16T00:32:24.340007+0000","last_undegraded":"2024-09-16T00:32:24.340007+0000","last_fullsized":"2024-09-16T00:32:24.340007+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-16T00:30:25.045310+0000","last_clean_scrub_stamp":"2024-09-16T00:30:25.045310+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-17T06:44:42.644792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":75456,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":700,"num_read_kb":434,"num_write":184,"num_write_kb":136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":56,"num_keys_recovered":10,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":454656,"data_stored":226368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":109,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4751360,"data_stored":4722944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":76,"seq":326417514540,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36388,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708764,"statfs":{"total":95995035648,"available":95957774336,"internally_reserved":0,"allocated":1060864,"data_stored":523210,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":69,"seq":296352743473,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32896,"kb_used_data":1712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712256,"statfs":{"total":95995035648,"available":95961350144,"internally_reserved":0,"allocated":1753088,"data_stored":1181341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":64,"seq":274877906998,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28072,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717080,"statfs":{"total":95995035648,"available":95966289920,"internally_reserved":0,"allocated":1073152,"data_stored":527486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":57,"seq":244813135931,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1064960,"data_stored":523288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":49,"seq":210453397568,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28652,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716500,"statfs":{"total":95995035648,"available":95965696000,"internally_reserved":0,"allocated":1662976,"data_stored":1119755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":44,"seq":188978561094,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32216,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712936,"statfs":{"total":95995035648,"available":95962046464,"internally_reserved":0,"allocated":1048576,"data_stored":518566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":37,"seq":158913790026,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28656,"kb_used_data":1632,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716496,"statfs":{"total":95995035648,"available":95965691904,"internally_reserved":0,"allocated":1671168,"data_stored":1115343,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953552,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36384,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708768,"statfs":{"total":95995035648,"available":95957778432,"internally_reserved":0,"allocated":1060864,"data_stored":523216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182483,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32220,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712932,"statfs":{"total":95995035648,"available":95962042368,"internally_reserved":0,"allocated":1060864,"data_stored":518760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411418,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28144,"kb_used_data":1120,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717008,"statfs":{"total":95995035648,"available":95966216192,"internally_reserved":0,"allocated":1146880,"data_stored":591638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574944,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28124,"kb_used_data":1100,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717028,"statfs":{"total":95995035648,"available":95966236672,"internally_reserved":0,"allocated":1126400,"data_stored":589395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738469,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32240,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712912,"statfs":{"total":95995035648,"available":95962021888,"internally_reserved":0,"allocated":1073152,"data_stored":519954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":70919,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":73162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2403,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":72497,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-16T00:32:56.693 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:32:56.950 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:57.213 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:56 smithi038 ceph-mon[30945]: pgmap v369: 129 pgs: 129 active+clean; 650 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-16T00:32:57.213 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:56 smithi038 ceph-mon[30945]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:32:57.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:56 smithi122 ceph-mon[35412]: pgmap v369: 129 pgs: 129 active+clean; 650 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-16T00:32:57.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:56 smithi122 ceph-mon[35412]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:32:57.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:56 smithi073 ceph-mon[35254]: pgmap v369: 129 pgs: 129 active+clean; 650 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.1 KiB/s wr, 7 op/s 2024-09-16T00:32:57.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:56 smithi073 ceph-mon[35254]: from='client.15363 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-16T00:32:57.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.435+0000 7f42141dd640 1 -- 172.21.15.38:0/2590513247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f420c1082e0 msgr2=0x7f420c10ec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:57.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.435+0000 7f42141dd640 1 --2- 172.21.15.38:0/2590513247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f420c1082e0 0x7f420c10ec50 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f4204009cd0 tx=0x7f420402a910 comp rx=0 tx=0).stop 2024-09-16T00:32:57.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.435+0000 7f42141dd640 1 -- 172.21.15.38:0/2590513247 shutdown_connections 2024-09-16T00:32:57.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.435+0000 7f42141dd640 1 --2- 172.21.15.38:0/2590513247 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f420c1082e0 0x7f420c10ec50 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.435+0000 7f42141dd640 1 --2- 172.21.15.38:0/2590513247 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f420c103760 0x7f420c107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.435+0000 7f42141dd640 1 --2- 172.21.15.38:0/2590513247 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f420c102db0 0x7f420c103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.435+0000 7f42141dd640 1 -- 172.21.15.38:0/2590513247 >> 172.21.15.38:0/2590513247 conn(0x7f420c0fc7d0 msgr2=0x7f420c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:57.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.435+0000 7f42141dd640 1 -- 172.21.15.38:0/2590513247 shutdown_connections 2024-09-16T00:32:57.455 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.435+0000 7f42141dd640 1 -- 172.21.15.38:0/2590513247 wait complete. 2024-09-16T00:32:57.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.435+0000 7f42141dd640 1 Processor -- start 2024-09-16T00:32:57.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.435+0000 7f42141dd640 1 -- start start 2024-09-16T00:32:57.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.436+0000 7f42141dd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f420c102db0 0x7f420c1986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:57.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.436+0000 7f42141dd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f420c103760 0x7f420c198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:57.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.436+0000 7f42141dd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f420c1082e0 0x7f420c19cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:57.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.436+0000 7f42141dd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f420c112170 con 0x7f420c102db0 2024-09-16T00:32:57.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.436+0000 7f42141dd640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f420c111ff0 con 0x7f420c103760 2024-09-16T00:32:57.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.436+0000 7f42141dd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f420c1122f0 con 0x7f420c1082e0 2024-09-16T00:32:57.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.436+0000 7f4211f79640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f420c102db0 0x7f420c1986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:57.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.436+0000 7f4211f79640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f420c102db0 0x7f420c1986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43354/0 (socket says 172.21.15.38:43354) 2024-09-16T00:32:57.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.436+0000 7f4211f79640 1 -- 172.21.15.38:0/153463873 learned_addr learned my addr 172.21.15.38:0/153463873 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:57.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.436+0000 7f421277a640 1 --2- 172.21.15.38:0/153463873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f420c1082e0 0x7f420c19cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:57.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.436+0000 7f4211778640 1 --2- 172.21.15.38:0/153463873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f420c103760 0x7f420c198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:57.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.436+0000 7f4211f79640 1 -- 172.21.15.38:0/153463873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f420c1082e0 msgr2=0x7f420c19cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:57.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.436+0000 7f4211f79640 1 --2- 172.21.15.38:0/153463873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f420c1082e0 0x7f420c19cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.436+0000 7f4211f79640 1 -- 172.21.15.38:0/153463873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f420c103760 msgr2=0x7f420c198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:57.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.436+0000 7f4211f79640 1 --2- 172.21.15.38:0/153463873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f420c103760 0x7f420c198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.436+0000 7f4211f79640 1 -- 172.21.15.38:0/153463873 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f420c19d650 con 0x7f420c102db0 2024-09-16T00:32:57.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.436+0000 7f4211778640 1 --2- 172.21.15.38:0/153463873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f420c103760 0x7f420c198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:57.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.436+0000 7f4211f79640 1 --2- 172.21.15.38:0/153463873 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f420c102db0 0x7f420c1986a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f420800b4a0 tx=0x7f420800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:57.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.437+0000 7f4202ffd640 1 -- 172.21.15.38:0/153463873 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4208014070 con 0x7f420c102db0 2024-09-16T00:32:57.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.437+0000 7f4202ffd640 1 -- 172.21.15.38:0/153463873 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4208010af0 con 0x7f420c102db0 2024-09-16T00:32:57.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.437+0000 7f42141dd640 1 -- 172.21.15.38:0/153463873 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f420c19d940 con 0x7f420c102db0 2024-09-16T00:32:57.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.437+0000 7f4202ffd640 1 -- 172.21.15.38:0/153463873 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4208017d10 con 0x7f420c102db0 2024-09-16T00:32:57.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.437+0000 7f42141dd640 1 -- 172.21.15.38:0/153463873 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f420c1a5220 con 0x7f420c102db0 2024-09-16T00:32:57.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.438+0000 7f42141dd640 1 -- 172.21.15.38:0/153463873 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f420c06a7d0 con 0x7f420c102db0 2024-09-16T00:32:57.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.439+0000 7f4202ffd640 1 -- 172.21.15.38:0/153463873 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f4208017070 con 0x7f420c102db0 2024-09-16T00:32:57.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.440+0000 7f4202ffd640 1 --2- 172.21.15.38:0/153463873 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f41f80a3350 0x7f41f80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:57.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.440+0000 7f4211778640 1 --2- 172.21.15.38:0/153463873 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f41f80a3350 0x7f41f80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:57.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.440+0000 7f4202ffd640 1 -- 172.21.15.38:0/153463873 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f42080cf4d0 con 0x7f420c102db0 2024-09-16T00:32:57.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.440+0000 7f4211778640 1 --2- 172.21.15.38:0/153463873 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f41f80a3350 0x7f41f80a5810 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f420c0fee60 tx=0x7f41fc008040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:57.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.443+0000 7f4202ffd640 1 -- 172.21.15.38:0/153463873 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f420809bdc0 con 0x7f420c102db0 2024-09-16T00:32:57.659 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.659+0000 7f42141dd640 1 -- 172.21.15.38:0/153463873 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f420c103210 con 0x7f420c102db0 2024-09-16T00:32:57.660 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.660+0000 7f4202ffd640 1 -- 172.21.15.38:0/153463873 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f420809b6c0 con 0x7f420c102db0 2024-09-16T00:32:57.660 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:32:57.661 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"btime":"2024-09-16T00:32:41:853514+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/367974527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":367974527},{"type":"v1","addr":"172.21.15.38:6835","nonce":367974527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/335077136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":335077136},{"type":"v1","addr":"172.21.15.38:6837","nonce":335077136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/3501211160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3501211160},{"type":"v1","addr":"172.21.15.38:6839","nonce":3501211160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6841/2623130915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":2623130915},{"type":"v1","addr":"172.21.15.38:6841","nonce":2623130915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/412277519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":412277519},{"type":"v1","addr":"172.21.15.122:6833","nonce":412277519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3987234757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3987234757},{"type":"v1","addr":"172.21.15.122:6835","nonce":3987234757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:30:26.764565+0000","modified":"2024-09-16T00:32:41.355828+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24440},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1210328177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1210328177},{"type":"v1","addr":"172.21.15.73:6833","nonce":1210328177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6837/1858299680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1858299680},{"type":"v1","addr":"172.21.15.122:6837","nonce":1858299680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1983975311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1983975311},{"type":"v1","addr":"172.21.15.122:6839","nonce":1983975311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.73:6835/2866363113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2866363113},{"type":"v1","addr":"172.21.15.73:6835","nonce":2866363113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.73:6837/2681081265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2681081265},{"type":"v1","addr":"172.21.15.73:6837","nonce":2681081265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.73:6839/2396224525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2396224525},{"type":"v1","addr":"172.21.15.73:6839","nonce":2396224525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24464,24440]},"id":1}]} 2024-09-16T00:32:57.661 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 26 2024-09-16T00:32:57.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.662+0000 7f42141dd640 1 -- 172.21.15.38:0/153463873 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f41f80a3350 msgr2=0x7f41f80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:57.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.662+0000 7f42141dd640 1 --2- 172.21.15.38:0/153463873 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f41f80a3350 0x7f41f80a5810 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f420c0fee60 tx=0x7f41fc008040 comp rx=0 tx=0).stop 2024-09-16T00:32:57.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.662+0000 7f42141dd640 1 -- 172.21.15.38:0/153463873 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f420c102db0 msgr2=0x7f420c1986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:57.662 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.662+0000 7f42141dd640 1 --2- 172.21.15.38:0/153463873 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f420c102db0 0x7f420c1986a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f420800b4a0 tx=0x7f420800b960 comp rx=0 tx=0).stop 2024-09-16T00:32:57.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.662+0000 7f42141dd640 1 -- 172.21.15.38:0/153463873 shutdown_connections 2024-09-16T00:32:57.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.662+0000 7f42141dd640 1 --2- 172.21.15.38:0/153463873 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f41f80a3350 0x7f41f80a5810 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.662+0000 7f42141dd640 1 --2- 172.21.15.38:0/153463873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f420c1082e0 0x7f420c19cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.662+0000 7f42141dd640 1 --2- 172.21.15.38:0/153463873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f420c103760 0x7f420c198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.662+0000 7f42141dd640 1 --2- 172.21.15.38:0/153463873 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f420c102db0 0x7f420c1986a0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:57.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.662+0000 7f42141dd640 1 -- 172.21.15.38:0/153463873 >> 172.21.15.38:0/153463873 conn(0x7f420c0fc7d0 msgr2=0x7f420c105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:57.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.662+0000 7f42141dd640 1 -- 172.21.15.38:0/153463873 shutdown_connections 2024-09-16T00:32:57.663 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:57.662+0000 7f42141dd640 1 -- 172.21.15.38:0/153463873 wait complete. 2024-09-16T00:32:58.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:57 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/153463873' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:32:58.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:57 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/153463873' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:32:58.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:57 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/153463873' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:32:58.375 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:32:58.637 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:32:59.039 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:58 smithi038 ceph-mon[30945]: pgmap v370: 129 pgs: 129 active+clean; 650 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-16T00:32:59.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.121+0000 7f7ef5e6c640 1 -- 172.21.15.38:0/3221687883 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ef0102db0 msgr2=0x7f7ef0103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:59.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.121+0000 7f7ef5e6c640 1 --2- 172.21.15.38:0/3221687883 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ef0102db0 0x7f7ef0103190 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0009cd0 tx=0x7f7ee002a8e0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.121+0000 7f7ef5e6c640 1 -- 172.21.15.38:0/3221687883 shutdown_connections 2024-09-16T00:32:59.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.121+0000 7f7ef5e6c640 1 --2- 172.21.15.38:0/3221687883 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ef01082e0 0x7f7ef010ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.121+0000 7f7ef5e6c640 1 --2- 172.21.15.38:0/3221687883 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ef0103760 0x7f7ef0107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.121+0000 7f7ef5e6c640 1 --2- 172.21.15.38:0/3221687883 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ef0102db0 0x7f7ef0103190 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.121+0000 7f7ef5e6c640 1 -- 172.21.15.38:0/3221687883 >> 172.21.15.38:0/3221687883 conn(0x7f7ef00fc7d0 msgr2=0x7f7ef00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:59.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.121+0000 7f7ef5e6c640 1 -- 172.21.15.38:0/3221687883 shutdown_connections 2024-09-16T00:32:59.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.121+0000 7f7ef5e6c640 1 -- 172.21.15.38:0/3221687883 wait complete. 2024-09-16T00:32:59.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.121+0000 7f7ef5e6c640 1 Processor -- start 2024-09-16T00:32:59.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.121+0000 7f7ef5e6c640 1 -- start start 2024-09-16T00:32:59.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.122+0000 7f7ef5e6c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ef0102db0 0x7f7ef01985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:59.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.122+0000 7f7ef5e6c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ef0103760 0x7f7ef0198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:59.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.122+0000 7f7ef5e6c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ef01082e0 0x7f7ef019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:59.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.122+0000 7f7ef5e6c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef0112110 con 0x7f7ef01082e0 2024-09-16T00:32:59.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.122+0000 7f7ef5e6c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef0111f90 con 0x7f7ef0103760 2024-09-16T00:32:59.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.122+0000 7f7ef5e6c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7ef0112290 con 0x7f7ef0102db0 2024-09-16T00:32:59.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.122+0000 7f7eeffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ef01082e0 0x7f7ef019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:59.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.122+0000 7f7eeffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ef01082e0 0x7f7ef019ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43382/0 (socket says 172.21.15.38:43382) 2024-09-16T00:32:59.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.122+0000 7f7eeffff640 1 -- 172.21.15.38:0/3246930169 learned_addr learned my addr 172.21.15.38:0/3246930169 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:32:59.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.122+0000 7f7eef7fe640 1 --2- 172.21.15.38:0/3246930169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ef0102db0 0x7f7ef01985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:59.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.122+0000 7f7eeeffd640 1 --2- 172.21.15.38:0/3246930169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ef0103760 0x7f7ef0198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:59.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.122+0000 7f7eeffff640 1 -- 172.21.15.38:0/3246930169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ef0102db0 msgr2=0x7f7ef01985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:59.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.122+0000 7f7eeffff640 1 --2- 172.21.15.38:0/3246930169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ef0102db0 0x7f7ef01985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.122+0000 7f7eeffff640 1 -- 172.21.15.38:0/3246930169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ef0103760 msgr2=0x7f7ef0198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:59.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.122+0000 7f7eeffff640 1 --2- 172.21.15.38:0/3246930169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ef0103760 0x7f7ef0198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.122+0000 7f7eeffff640 1 -- 172.21.15.38:0/3246930169 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ef019d560 con 0x7f7ef01082e0 2024-09-16T00:32:59.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.122+0000 7f7eeeffd640 1 --2- 172.21.15.38:0/3246930169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ef0103760 0x7f7ef0198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:32:59.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.122+0000 7f7eeffff640 1 --2- 172.21.15.38:0/3246930169 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ef01082e0 0x7f7ef019ce80 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8009fd0 tx=0x7f7ee800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:59.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.123+0000 7f7eecff9640 1 -- 172.21.15.38:0/3246930169 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee8011830 con 0x7f7ef01082e0 2024-09-16T00:32:59.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.123+0000 7f7eecff9640 1 -- 172.21.15.38:0/3246930169 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7ee8016070 con 0x7f7ef01082e0 2024-09-16T00:32:59.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.123+0000 7f7ef5e6c640 1 -- 172.21.15.38:0/3246930169 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ef019d850 con 0x7f7ef01082e0 2024-09-16T00:32:59.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.123+0000 7f7eecff9640 1 -- 172.21.15.38:0/3246930169 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ee800f2e0 con 0x7f7ef01082e0 2024-09-16T00:32:59.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.123+0000 7f7ef5e6c640 1 -- 172.21.15.38:0/3246930169 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ef01a5130 con 0x7f7ef01082e0 2024-09-16T00:32:59.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.124+0000 7f7ef5e6c640 1 -- 172.21.15.38:0/3246930169 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ef006a7d0 con 0x7f7ef01082e0 2024-09-16T00:32:59.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.125+0000 7f7eecff9640 1 -- 172.21.15.38:0/3246930169 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f7ee80119d0 con 0x7f7ef01082e0 2024-09-16T00:32:59.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.126+0000 7f7eecff9640 1 --2- 172.21.15.38:0/3246930169 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f7ed80a32d0 0x7f7ed80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:32:59.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.126+0000 7f7eef7fe640 1 --2- 172.21.15.38:0/3246930169 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f7ed80a32d0 0x7f7ed80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:32:59.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.126+0000 7f7eecff9640 1 -- 172.21.15.38:0/3246930169 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f7ee80cf420 con 0x7f7ef01082e0 2024-09-16T00:32:59.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.127+0000 7f7eef7fe640 1 --2- 172.21.15.38:0/3246930169 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f7ed80a32d0 0x7f7ed80a5790 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0006010 tx=0x7f7ee0009360 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:32:59.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.130+0000 7f7eecff9640 1 -- 172.21.15.38:0/3246930169 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f7ee809bd10 con 0x7f7ef01082e0 2024-09-16T00:32:59.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:58 smithi122 ceph-mon[35412]: pgmap v370: 129 pgs: 129 active+clean; 650 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-16T00:32:59.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:58 smithi073 ceph-mon[35254]: pgmap v370: 129 pgs: 129 active+clean; 650 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 255 B/s wr, 5 op/s 2024-09-16T00:32:59.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.345+0000 7f7ef5e6c640 1 -- 172.21.15.38:0/3246930169 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ef010a2c0 con 0x7f7ef01082e0 2024-09-16T00:32:59.346 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.346+0000 7f7eecff9640 1 -- 172.21.15.38:0/3246930169 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f7ee809b610 con 0x7f7ef01082e0 2024-09-16T00:32:59.347 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:32:59.347 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"btime":"2024-09-16T00:32:41:853514+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/367974527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":367974527},{"type":"v1","addr":"172.21.15.38:6835","nonce":367974527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/335077136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":335077136},{"type":"v1","addr":"172.21.15.38:6837","nonce":335077136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/3501211160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3501211160},{"type":"v1","addr":"172.21.15.38:6839","nonce":3501211160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6841/2623130915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":2623130915},{"type":"v1","addr":"172.21.15.38:6841","nonce":2623130915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/412277519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":412277519},{"type":"v1","addr":"172.21.15.122:6833","nonce":412277519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3987234757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3987234757},{"type":"v1","addr":"172.21.15.122:6835","nonce":3987234757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:30:26.764565+0000","modified":"2024-09-16T00:32:41.355828+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24440},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1210328177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1210328177},{"type":"v1","addr":"172.21.15.73:6833","nonce":1210328177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6837/1858299680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1858299680},{"type":"v1","addr":"172.21.15.122:6837","nonce":1858299680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1983975311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1983975311},{"type":"v1","addr":"172.21.15.122:6839","nonce":1983975311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.73:6835/2866363113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2866363113},{"type":"v1","addr":"172.21.15.73:6835","nonce":2866363113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.73:6837/2681081265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2681081265},{"type":"v1","addr":"172.21.15.73:6837","nonce":2681081265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.73:6839/2396224525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2396224525},{"type":"v1","addr":"172.21.15.73:6839","nonce":2396224525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24464,24440]},"id":1}]} 2024-09-16T00:32:59.347 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 26 2024-09-16T00:32:59.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.348+0000 7f7ef5e6c640 1 -- 172.21.15.38:0/3246930169 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f7ed80a32d0 msgr2=0x7f7ed80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:59.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.348+0000 7f7ef5e6c640 1 --2- 172.21.15.38:0/3246930169 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f7ed80a32d0 0x7f7ed80a5790 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f7ee0006010 tx=0x7f7ee0009360 comp rx=0 tx=0).stop 2024-09-16T00:32:59.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.348+0000 7f7ef5e6c640 1 -- 172.21.15.38:0/3246930169 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ef01082e0 msgr2=0x7f7ef019ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:32:59.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.348+0000 7f7ef5e6c640 1 --2- 172.21.15.38:0/3246930169 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ef01082e0 0x7f7ef019ce80 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8009fd0 tx=0x7f7ee800b8e0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.348+0000 7f7ef5e6c640 1 -- 172.21.15.38:0/3246930169 shutdown_connections 2024-09-16T00:32:59.349 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.349+0000 7f7ef5e6c640 1 --2- 172.21.15.38:0/3246930169 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f7ed80a32d0 0x7f7ed80a5790 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.349+0000 7f7ef5e6c640 1 --2- 172.21.15.38:0/3246930169 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7ef01082e0 0x7f7ef019ce80 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.349+0000 7f7ef5e6c640 1 --2- 172.21.15.38:0/3246930169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ef0103760 0x7f7ef0198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.349+0000 7f7ef5e6c640 1 --2- 172.21.15.38:0/3246930169 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7ef0102db0 0x7f7ef01985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:32:59.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.349+0000 7f7ef5e6c640 1 -- 172.21.15.38:0/3246930169 >> 172.21.15.38:0/3246930169 conn(0x7f7ef00fc7d0 msgr2=0x7f7ef00fe470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:32:59.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.349+0000 7f7ef5e6c640 1 -- 172.21.15.38:0/3246930169 shutdown_connections 2024-09-16T00:32:59.350 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:32:59.349+0000 7f7ef5e6c640 1 -- 172.21.15.38:0/3246930169 wait complete. 2024-09-16T00:33:00.067 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:33:00.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:32:59 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/3246930169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:33:00.326 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:00.447 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:32:59 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/3246930169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:33:00.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:32:59 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/3246930169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:33:00.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:00.821+0000 7fbe284ed640 1 -- 172.21.15.38:0/1340256676 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe201036e0 msgr2=0x7fbe20107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:00.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:00.821+0000 7fbe284ed640 1 --2- 172.21.15.38:0/1340256676 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe201036e0 0x7fbe20107b90 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fbe14009930 tx=0x7fbe1402a420 comp rx=0 tx=0).stop 2024-09-16T00:33:00.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:00.821+0000 7fbe284ed640 1 -- 172.21.15.38:0/1340256676 shutdown_connections 2024-09-16T00:33:00.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:00.821+0000 7fbe284ed640 1 --2- 172.21.15.38:0/1340256676 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe201082a0 0x7fbe2010ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:00.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:00.821+0000 7fbe284ed640 1 --2- 172.21.15.38:0/1340256676 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe201036e0 0x7fbe20107b90 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:00.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:00.821+0000 7fbe284ed640 1 --2- 172.21.15.38:0/1340256676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe20102d30 0x7fbe20103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:00.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:00.821+0000 7fbe284ed640 1 -- 172.21.15.38:0/1340256676 >> 172.21.15.38:0/1340256676 conn(0x7fbe200fc7d0 msgr2=0x7fbe200febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:00.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:00.821+0000 7fbe284ed640 1 -- 172.21.15.38:0/1340256676 shutdown_connections 2024-09-16T00:33:00.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:00.821+0000 7fbe284ed640 1 -- 172.21.15.38:0/1340256676 wait complete. 2024-09-16T00:33:00.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:00.821+0000 7fbe284ed640 1 Processor -- start 2024-09-16T00:33:00.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:00.821+0000 7fbe284ed640 1 -- start start 2024-09-16T00:33:00.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:00.822+0000 7fbe284ed640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe20102d30 0x7fbe20198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:00.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:00.822+0000 7fbe284ed640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe201036e0 0x7fbe20198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:00.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:00.822+0000 7fbe284ed640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe201082a0 0x7fbe2019cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:00.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:00.822+0000 7fbe284ed640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fbe201120c0 con 0x7fbe201036e0 2024-09-16T00:33:00.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:00.822+0000 7fbe284ed640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fbe20111f40 con 0x7fbe201082a0 2024-09-16T00:33:00.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:00.822+0000 7fbe284ed640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fbe20112240 con 0x7fbe20102d30 2024-09-16T00:33:00.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:00.822+0000 7fbe25a88640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe201036e0 0x7fbe20198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:00.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:00.822+0000 7fbe25a88640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe201036e0 0x7fbe20198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43406/0 (socket says 172.21.15.38:43406) 2024-09-16T00:33:00.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:00.822+0000 7fbe25a88640 1 -- 172.21.15.38:0/3586630699 learned_addr learned my addr 172.21.15.38:0/3586630699 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:00.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:00.822+0000 7fbe26289640 1 --2- 172.21.15.38:0/3586630699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe20102d30 0x7fbe20198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:00.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:00.822+0000 7fbe26a8a640 1 --2- 172.21.15.38:0/3586630699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe201082a0 0x7fbe2019cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:00.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:00.822+0000 7fbe25a88640 1 -- 172.21.15.38:0/3586630699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe20102d30 msgr2=0x7fbe20198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:00.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:00.822+0000 7fbe25a88640 1 --2- 172.21.15.38:0/3586630699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe20102d30 0x7fbe20198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:00.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:00.822+0000 7fbe25a88640 1 -- 172.21.15.38:0/3586630699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe201082a0 msgr2=0x7fbe2019cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:00.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:00.822+0000 7fbe25a88640 1 --2- 172.21.15.38:0/3586630699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe201082a0 0x7fbe2019cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:00.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:00.822+0000 7fbe25a88640 1 -- 172.21.15.38:0/3586630699 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe2019d5d0 con 0x7fbe201036e0 2024-09-16T00:33:00.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:00.822+0000 7fbe25a88640 1 --2- 172.21.15.38:0/3586630699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe201036e0 0x7fbe20198b60 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fbe14005940 tx=0x7fbe140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:00.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:00.823+0000 7fbe1f7fe640 1 -- 172.21.15.38:0/3586630699 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe1403f030 con 0x7fbe201036e0 2024-09-16T00:33:00.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:00.823+0000 7fbe1f7fe640 1 -- 172.21.15.38:0/3586630699 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fbe14003ce0 con 0x7fbe201036e0 2024-09-16T00:33:00.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:00.823+0000 7fbe284ed640 1 -- 172.21.15.38:0/3586630699 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe2019d860 con 0x7fbe201036e0 2024-09-16T00:33:00.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:00.823+0000 7fbe1f7fe640 1 -- 172.21.15.38:0/3586630699 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe14035070 con 0x7fbe201036e0 2024-09-16T00:33:00.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:00.823+0000 7fbe284ed640 1 -- 172.21.15.38:0/3586630699 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe201a51a0 con 0x7fbe201036e0 2024-09-16T00:33:00.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:00.824+0000 7fbe284ed640 1 -- 172.21.15.38:0/3586630699 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe2006a850 con 0x7fbe201036e0 2024-09-16T00:33:00.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:00.825+0000 7fbe1f7fe640 1 -- 172.21.15.38:0/3586630699 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7fbe14042030 con 0x7fbe201036e0 2024-09-16T00:33:00.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:00.825+0000 7fbe1f7fe640 1 --2- 172.21.15.38:0/3586630699 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fbe080a32d0 0x7fbe080a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:00.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:00.826+0000 7fbe26289640 1 --2- 172.21.15.38:0/3586630699 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fbe080a32d0 0x7fbe080a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:00.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:00.826+0000 7fbe1f7fe640 1 -- 172.21.15.38:0/3586630699 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7fbe140f0490 con 0x7fbe201036e0 2024-09-16T00:33:00.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:00.826+0000 7fbe26289640 1 --2- 172.21.15.38:0/3586630699 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fbe080a32d0 0x7fbe080a5790 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbe180058c0 tx=0x7fbe1800a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:00.844 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:00.830+0000 7fbe1f7fe640 1 -- 172.21.15.38:0/3586630699 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fbe140bccd0 con 0x7fbe201036e0 2024-09-16T00:33:01.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:01.006+0000 7fbe284ed640 1 -- 172.21.15.38:0/3586630699 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbe2010a2b0 con 0x7fbe201036e0 2024-09-16T00:33:01.007 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:00 smithi038 ceph-mon[30945]: pgmap v371: 129 pgs: 129 active+clean; 650 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-16T00:33:01.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:01.008+0000 7fbe1f7fe640 1 -- 172.21.15.38:0/3586630699 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24767 (secure 0 0 0) 0x7fbe140bc5d0 con 0x7fbe201036e0 2024-09-16T00:33:01.009 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:33:01.009 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":92,"fsid":"983b2072-73c1-11ef-bceb-c7b262605968","created":"2024-09-16T00:22:34.897053+0000","modified":"2024-09-16T00:32:24.296695+0000","last_up_change":"2024-09-16T00:29:34.503134+0000","last_in_change":"2024-09-16T00:29:10.135656+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:25:41.931977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:30:22.741889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:30:25.042590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:30:31.554350+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3afc20e3-869f-43a9-a911-b0e8d07de86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6803","nonce":1622028921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6805","nonce":1622028921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6809","nonce":1622028921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6807","nonce":1622028921}]},"public_addr":"172.21.15.38:6803/1622028921","cluster_addr":"172.21.15.38:6805/1622028921","heartbeat_back_addr":"172.21.15.38:6809/1622028921","heartbeat_front_addr":"172.21.15.38:6807/1622028921","state":["exists","up"]},{"osd":1,"uuid":"6d9d7acb-45ed-4543-8afc-6433fc7f096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6801","nonce":3811690336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6803","nonce":3811690336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6807","nonce":3811690336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6805","nonce":3811690336}]},"public_addr":"172.21.15.73:6801/3811690336","cluster_addr":"172.21.15.73:6803/3811690336","heartbeat_back_addr":"172.21.15.73:6807/3811690336","heartbeat_front_addr":"172.21.15.73:6805/3811690336","state":["exists","up"]},{"osd":2,"uuid":"48690f35-99e9-4f45-9bee-487857caa8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6801","nonce":1754741655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6803","nonce":1754741655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6807","nonce":1754741655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6805","nonce":1754741655}]},"public_addr":"172.21.15.122:6801/1754741655","cluster_addr":"172.21.15.122:6803/1754741655","heartbeat_back_addr":"172.21.15.122:6807/1754741655","heartbeat_front_addr":"172.21.15.122:6805/1754741655","state":["exists","up"]},{"osd":3,"uuid":"28ebb4a0-92e7-4f57-82a5-b9b761eca269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6811","nonce":3523190079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6813","nonce":3523190079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6817","nonce":3523190079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6815","nonce":3523190079}]},"public_addr":"172.21.15.38:6811/3523190079","cluster_addr":"172.21.15.38:6813/3523190079","heartbeat_back_addr":"172.21.15.38:6817/3523190079","heartbeat_front_addr":"172.21.15.38:6815/3523190079","state":["exists","up"]},{"osd":4,"uuid":"5ea363f7-7ad8-45ed-b7e4-4b010fe7f8cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6809","nonce":1469927628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6811","nonce":1469927628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6815","nonce":1469927628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6813","nonce":1469927628}]},"public_addr":"172.21.15.73:6809/1469927628","cluster_addr":"172.21.15.73:6811/1469927628","heartbeat_back_addr":"172.21.15.73:6815/1469927628","heartbeat_front_addr":"172.21.15.73:6813/1469927628","state":["exists","up"]},{"osd":5,"uuid":"e3c66d43-dd28-4bb1-a886-62a869be82cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6809","nonce":2954413058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6811","nonce":2954413058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6815","nonce":2954413058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6813","nonce":2954413058}]},"public_addr":"172.21.15.122:6809/2954413058","cluster_addr":"172.21.15.122:6811/2954413058","heartbeat_back_addr":"172.21.15.122:6815/2954413058","heartbeat_front_addr":"172.21.15.122:6813/2954413058","state":["exists","up"]},{"osd":6,"uuid":"211497ac-7c48-4095-bfdb-1d4df0149f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6819","nonce":1415719550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6821","nonce":1415719550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6825","nonce":1415719550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6823","nonce":1415719550}]},"public_addr":"172.21.15.38:6819/1415719550","cluster_addr":"172.21.15.38:6821/1415719550","heartbeat_back_addr":"172.21.15.38:6825/1415719550","heartbeat_front_addr":"172.21.15.38:6823/1415719550","state":["exists","up"]},{"osd":7,"uuid":"06512315-3f5e-44f8-976d-d3f5f1ba00e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6817","nonce":2634361473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6819","nonce":2634361473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6823","nonce":2634361473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6821","nonce":2634361473}]},"public_addr":"172.21.15.73:6817/2634361473","cluster_addr":"172.21.15.73:6819/2634361473","heartbeat_back_addr":"172.21.15.73:6823/2634361473","heartbeat_front_addr":"172.21.15.73:6821/2634361473","state":["exists","up"]},{"osd":8,"uuid":"db703c0a-1f87-46f0-be5f-f826f7f9daac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6817","nonce":1800567633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6819","nonce":1800567633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6823","nonce":1800567633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6821","nonce":1800567633}]},"public_addr":"172.21.15.122:6817/1800567633","cluster_addr":"172.21.15.122:6819/1800567633","heartbeat_back_addr":"172.21.15.122:6823/1800567633","heartbeat_front_addr":"172.21.15.122:6821/1800567633","state":["exists","up"]},{"osd":9,"uuid":"f5384200-b3b4-4936-85e1-fc57a9fe2bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6827","nonce":445727833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6829","nonce":445727833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6833","nonce":445727833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6831","nonce":445727833}]},"public_addr":"172.21.15.38:6827/445727833","cluster_addr":"172.21.15.38:6829/445727833","heartbeat_back_addr":"172.21.15.38:6833/445727833","heartbeat_front_addr":"172.21.15.38:6831/445727833","state":["exists","up"]},{"osd":10,"uuid":"c22883d9-6672-4796-a961-963d21eb37fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6825","nonce":43297139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6827","nonce":43297139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6831","nonce":43297139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6829","nonce":43297139}]},"public_addr":"172.21.15.73:6825/43297139","cluster_addr":"172.21.15.73:6827/43297139","heartbeat_back_addr":"172.21.15.73:6831/43297139","heartbeat_front_addr":"172.21.15.73:6829/43297139","state":["exists","up"]},{"osd":11,"uuid":"9f98e8cb-1bff-4b8b-aa19-56b691387383","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6825","nonce":1886446061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6827","nonce":1886446061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6831","nonce":1886446061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6829","nonce":1886446061}]},"public_addr":"172.21.15.122:6825/1886446061","cluster_addr":"172.21.15.122:6827/1886446061","heartbeat_back_addr":"172.21.15.122:6831/1886446061","heartbeat_front_addr":"172.21.15.122:6829/1886446061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:47.384012+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:12.834488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:38.770055+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:04.800466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:30.808076+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:56.646967+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:27:22.721270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:27:48.510011+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:28:13.922944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:28:40.514947+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:29:06.031052+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:29:31.882062+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6801/1779324916":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/924150447":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/141237849":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/2302081750":"2024-09-17T00:22:56.844212+0000","172.21.15.38:0/2245468594":"2024-09-17T00:22:56.844212+0000","172.21.15.38:6801/583006373":"2024-09-17T00:22:56.844212+0000","172.21.15.38:0/2435492667":"2024-09-17T00:23:21.417483+0000","172.21.15.38:6800/583006373":"2024-09-17T00:22:56.844212+0000","172.21.15.38:6800/1779324916":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/1103516837":"2024-09-17T00:22:56.844212+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:33:01.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:01.010+0000 7fbe284ed640 1 -- 172.21.15.38:0/3586630699 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fbe080a32d0 msgr2=0x7fbe080a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:01.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:01.010+0000 7fbe284ed640 1 --2- 172.21.15.38:0/3586630699 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fbe080a32d0 0x7fbe080a5790 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fbe180058c0 tx=0x7fbe1800a4c0 comp rx=0 tx=0).stop 2024-09-16T00:33:01.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:01.010+0000 7fbe284ed640 1 -- 172.21.15.38:0/3586630699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe201036e0 msgr2=0x7fbe20198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:01.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:01.010+0000 7fbe284ed640 1 --2- 172.21.15.38:0/3586630699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe201036e0 0x7fbe20198b60 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fbe14005940 tx=0x7fbe140042f0 comp rx=0 tx=0).stop 2024-09-16T00:33:01.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:01.010+0000 7fbe284ed640 1 -- 172.21.15.38:0/3586630699 shutdown_connections 2024-09-16T00:33:01.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:01.010+0000 7fbe284ed640 1 --2- 172.21.15.38:0/3586630699 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fbe080a32d0 0x7fbe080a5790 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:01.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:01.010+0000 7fbe284ed640 1 --2- 172.21.15.38:0/3586630699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbe201082a0 0x7fbe2019cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:01.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:01.010+0000 7fbe284ed640 1 --2- 172.21.15.38:0/3586630699 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fbe201036e0 0x7fbe20198b60 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:01.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:01.010+0000 7fbe284ed640 1 --2- 172.21.15.38:0/3586630699 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fbe20102d30 0x7fbe20198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:01.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:01.011+0000 7fbe284ed640 1 -- 172.21.15.38:0/3586630699 >> 172.21.15.38:0/3586630699 conn(0x7fbe200fc7d0 msgr2=0x7fbe20105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:01.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:01.011+0000 7fbe284ed640 1 -- 172.21.15.38:0/3586630699 shutdown_connections 2024-09-16T00:33:01.012 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:01.011+0000 7fbe284ed640 1 -- 172.21.15.38:0/3586630699 wait complete. 2024-09-16T00:33:01.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:00 smithi073 ceph-mon[35254]: pgmap v371: 129 pgs: 129 active+clean; 650 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-16T00:33:01.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:00 smithi122 ceph-mon[35412]: pgmap v371: 129 pgs: 129 active+clean; 650 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 255 B/s wr, 7 op/s 2024-09-16T00:33:01.693 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-09-16T00:33:01.949 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:02.249 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:01 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/3586630699' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:33:02.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:01 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/3586630699' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:33:02.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:01 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/3586630699' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:33:03.114 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:02 smithi038 ceph-mon[30945]: pgmap v372: 129 pgs: 129 active+clean; 650 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-09-16T00:33:03.114 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:02 smithi038 ceph-mon[30945]: scrub summary: idle 2024-09-16T00:33:03.143 INFO:teuthology.orchestra.run.smithi038.stdout:{"run_dir":"/var/run/ceph"} 2024-09-16T00:33:03.143 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-09-16T00:33:03.144 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 3393 2024-09-16T00:33:03.144 DEBUG:tasks.quiescer:Yielding 2024-09-16T00:33:03.144 INFO:teuthology.run_tasks:Running task workunit... 2024-09-16T00:33:03.160 INFO:tasks.workunit:Pulling workunits from ref ad35db06e8924912f01e32b35576297c0f1352b4 2024-09-16T00:33:03.160 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-16T00:33:03.160 INFO:tasks.workunit:timeout=3h 2024-09-16T00:33:03.160 INFO:tasks.workunit:cleanup=True 2024-09-16T00:33:03.161 DEBUG:teuthology.orchestra.run.smithi038:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-16T00:33:03.161 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:33:03.179 INFO:teuthology.orchestra.run.smithi038.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-16T00:33:03.179 INFO:teuthology.orchestra.run.smithi038.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-16T00:33:03.179 INFO:teuthology.orchestra.run.smithi038.stdout:Device: 60h/96d Inode: 1099511628283 Links: 2 2024-09-16T00:33:03.179 INFO:teuthology.orchestra.run.smithi038.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-16T00:33:03.179 INFO:teuthology.orchestra.run.smithi038.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-16T00:33:03.179 INFO:teuthology.orchestra.run.smithi038.stdout:Access: 2024-09-16 00:31:12.238455889 +0000 2024-09-16T00:33:03.179 INFO:teuthology.orchestra.run.smithi038.stdout:Modify: 2024-09-16 00:31:12.238455889 +0000 2024-09-16T00:33:03.179 INFO:teuthology.orchestra.run.smithi038.stdout:Change: 2024-09-16 00:32:40.667095796 +0000 2024-09-16T00:33:03.179 INFO:teuthology.orchestra.run.smithi038.stdout: Birth: 2024-09-16 00:31:12.238455889 +0000 2024-09-16T00:33:03.180 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-16T00:33:03.180 DEBUG:teuthology.orchestra.run.smithi038:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-16T00:33:03.252 DEBUG:teuthology.orchestra.run.smithi038:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout ad35db06e8924912f01e32b35576297c0f1352b4 2024-09-16T00:33:03.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:02 smithi073 ceph-mon[35254]: pgmap v372: 129 pgs: 129 active+clean; 650 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-09-16T00:33:03.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:02 smithi073 ceph-mon[35254]: scrub summary: idle 2024-09-16T00:33:03.310 INFO:tasks.workunit.client.0.smithi038.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-16T00:33:03.368 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:02 smithi122 ceph-mon[35412]: pgmap v372: 129 pgs: 129 active+clean; 650 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-09-16T00:33:03.368 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:02 smithi122 ceph-mon[35412]: scrub summary: idle 2024-09-16T00:33:03.422 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:03.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:03.928+0000 7fdf34790640 1 -- 172.21.15.38:0/2360507784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf2c106c30 msgr2=0x7fdf2c107010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:03.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:03.928+0000 7fdf34790640 1 --2- 172.21.15.38:0/2360507784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf2c106c30 0x7fdf2c107010 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fdf28009900 tx=0x7fdf2802a410 comp rx=0 tx=0).stop 2024-09-16T00:33:03.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:03.928+0000 7fdf34790640 1 -- 172.21.15.38:0/2360507784 shutdown_connections 2024-09-16T00:33:03.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:03.928+0000 7fdf34790640 1 --2- 172.21.15.38:0/2360507784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf2c101b40 0x7fdf2c10da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:03.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:03.928+0000 7fdf34790640 1 --2- 172.21.15.38:0/2360507784 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf2c1011a0 0x7fdf2c101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:03.952 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:03.928+0000 7fdf34790640 1 --2- 172.21.15.38:0/2360507784 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf2c106c30 0x7fdf2c107010 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:03.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:03.928+0000 7fdf34790640 1 -- 172.21.15.38:0/2360507784 >> 172.21.15.38:0/2360507784 conn(0x7fdf2c077220 msgr2=0x7fdf2c0fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:03.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:03.928+0000 7fdf34790640 1 -- 172.21.15.38:0/2360507784 shutdown_connections 2024-09-16T00:33:03.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:03.928+0000 7fdf34790640 1 -- 172.21.15.38:0/2360507784 wait complete. 2024-09-16T00:33:03.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:03.929+0000 7fdf34790640 1 Processor -- start 2024-09-16T00:33:03.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:03.929+0000 7fdf34790640 1 -- start start 2024-09-16T00:33:03.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:03.929+0000 7fdf34790640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf2c1011a0 0x7fdf2c1027b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:03.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:03.929+0000 7fdf34790640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf2c101b40 0x7fdf2c102cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:03.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:03.929+0000 7fdf34790640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf2c106c30 0x7fdf2c103230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:03.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:03.929+0000 7fdf34790640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c110ff0 con 0x7fdf2c101b40 2024-09-16T00:33:03.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:03.929+0000 7fdf34790640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c110e70 con 0x7fdf2c1011a0 2024-09-16T00:33:03.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:03.929+0000 7fdf34790640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2c111170 con 0x7fdf2c106c30 2024-09-16T00:33:03.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:03.929+0000 7fdf31d2b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf2c101b40 0x7fdf2c102cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:03.953 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:03.929+0000 7fdf31d2b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf2c101b40 0x7fdf2c102cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43416/0 (socket says 172.21.15.38:43416) 2024-09-16T00:33:03.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:03.929+0000 7fdf31d2b640 1 -- 172.21.15.38:0/3811779273 learned_addr learned my addr 172.21.15.38:0/3811779273 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:03.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:03.929+0000 7fdf3252c640 1 --2- 172.21.15.38:0/3811779273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf2c1011a0 0x7fdf2c1027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:03.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:03.929+0000 7fdf31d2b640 1 -- 172.21.15.38:0/3811779273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf2c106c30 msgr2=0x7fdf2c103230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:03.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:03.929+0000 7fdf31d2b640 1 --2- 172.21.15.38:0/3811779273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf2c106c30 0x7fdf2c103230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:03.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:03.929+0000 7fdf31d2b640 1 -- 172.21.15.38:0/3811779273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf2c1011a0 msgr2=0x7fdf2c1027b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:03.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:03.929+0000 7fdf31d2b640 1 --2- 172.21.15.38:0/3811779273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf2c1011a0 0x7fdf2c1027b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:03.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:03.929+0000 7fdf31d2b640 1 -- 172.21.15.38:0/3811779273 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf2c1a6980 con 0x7fdf2c101b40 2024-09-16T00:33:03.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:03.929+0000 7fdf3252c640 1 --2- 172.21.15.38:0/3811779273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf2c1011a0 0x7fdf2c1027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:33:03.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:03.930+0000 7fdf31d2b640 1 --2- 172.21.15.38:0/3811779273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf2c101b40 0x7fdf2c102cf0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c00d950 tx=0x7fdf1c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:03.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:03.930+0000 7fdf237fe640 1 -- 172.21.15.38:0/3811779273 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c017070 con 0x7fdf2c101b40 2024-09-16T00:33:03.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:03.930+0000 7fdf237fe640 1 -- 172.21.15.38:0/3811779273 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fdf1c009c30 con 0x7fdf2c101b40 2024-09-16T00:33:03.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:03.930+0000 7fdf237fe640 1 -- 172.21.15.38:0/3811779273 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c00f6b0 con 0x7fdf2c101b40 2024-09-16T00:33:03.954 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:03.930+0000 7fdf34790640 1 -- 172.21.15.38:0/3811779273 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf2c1a6c70 con 0x7fdf2c101b40 2024-09-16T00:33:03.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:03.930+0000 7fdf34790640 1 -- 172.21.15.38:0/3811779273 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf2c1a7160 con 0x7fdf2c101b40 2024-09-16T00:33:03.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:03.931+0000 7fdf34790640 1 -- 172.21.15.38:0/3811779273 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf2c101600 con 0x7fdf2c101b40 2024-09-16T00:33:03.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:03.933+0000 7fdf237fe640 1 -- 172.21.15.38:0/3811779273 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7fdf1c0109d0 con 0x7fdf2c101b40 2024-09-16T00:33:03.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:03.934+0000 7fdf237fe640 1 --2- 172.21.15.38:0/3811779273 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fdf180a3350 0x7fdf180a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:03.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:03.934+0000 7fdf237fe640 1 -- 172.21.15.38:0/3811779273 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7fdf1c0cfac0 con 0x7fdf2c101b40 2024-09-16T00:33:03.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:03.934+0000 7fdf3252c640 1 --2- 172.21.15.38:0/3811779273 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fdf180a3350 0x7fdf180a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:03.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:03.935+0000 7fdf3252c640 1 --2- 172.21.15.38:0/3811779273 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fdf180a3350 0x7fdf180a5810 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdf28005ab0 tx=0x7fdf28000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:03.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:03.937+0000 7fdf237fe640 1 -- 172.21.15.38:0/3811779273 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fdf1c009de0 con 0x7fdf2c101b40 2024-09-16T00:33:04.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:04.153+0000 7fdf34790640 1 -- 172.21.15.38:0/3811779273 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf2c1a7330 con 0x7fdf2c101b40 2024-09-16T00:33:04.154 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:04.154+0000 7fdf237fe640 1 -- 172.21.15.38:0/3811779273 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7fdf1c09c320 con 0x7fdf2c101b40 2024-09-16T00:33:04.155 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:33:04.155 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"btime":"2024-09-16T00:32:41:853514+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/367974527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":367974527},{"type":"v1","addr":"172.21.15.38:6835","nonce":367974527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/335077136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":335077136},{"type":"v1","addr":"172.21.15.38:6837","nonce":335077136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/3501211160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3501211160},{"type":"v1","addr":"172.21.15.38:6839","nonce":3501211160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6841/2623130915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":2623130915},{"type":"v1","addr":"172.21.15.38:6841","nonce":2623130915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/412277519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":412277519},{"type":"v1","addr":"172.21.15.122:6833","nonce":412277519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3987234757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3987234757},{"type":"v1","addr":"172.21.15.122:6835","nonce":3987234757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:30:26.764565+0000","modified":"2024-09-16T00:32:41.355828+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24440},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1210328177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1210328177},{"type":"v1","addr":"172.21.15.73:6833","nonce":1210328177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6837/1858299680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1858299680},{"type":"v1","addr":"172.21.15.122:6837","nonce":1858299680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1983975311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1983975311},{"type":"v1","addr":"172.21.15.122:6839","nonce":1983975311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.73:6835/2866363113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2866363113},{"type":"v1","addr":"172.21.15.73:6835","nonce":2866363113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.73:6837/2681081265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2681081265},{"type":"v1","addr":"172.21.15.73:6837","nonce":2681081265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.73:6839/2396224525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2396224525},{"type":"v1","addr":"172.21.15.73:6839","nonce":2396224525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24464,24440]},"id":1}]} 2024-09-16T00:33:04.155 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 26 2024-09-16T00:33:04.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:04.157+0000 7fdf34790640 1 -- 172.21.15.38:0/3811779273 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fdf180a3350 msgr2=0x7fdf180a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:04.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:04.157+0000 7fdf34790640 1 --2- 172.21.15.38:0/3811779273 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fdf180a3350 0x7fdf180a5810 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fdf28005ab0 tx=0x7fdf28000bc0 comp rx=0 tx=0).stop 2024-09-16T00:33:04.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:04.157+0000 7fdf34790640 1 -- 172.21.15.38:0/3811779273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf2c101b40 msgr2=0x7fdf2c102cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:04.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:04.157+0000 7fdf34790640 1 --2- 172.21.15.38:0/3811779273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf2c101b40 0x7fdf2c102cf0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c00d950 tx=0x7fdf1c00de10 comp rx=0 tx=0).stop 2024-09-16T00:33:04.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:04.157+0000 7fdf34790640 1 -- 172.21.15.38:0/3811779273 shutdown_connections 2024-09-16T00:33:04.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:04.157+0000 7fdf34790640 1 --2- 172.21.15.38:0/3811779273 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fdf180a3350 0x7fdf180a5810 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:04.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:04.157+0000 7fdf34790640 1 --2- 172.21.15.38:0/3811779273 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fdf2c106c30 0x7fdf2c103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:04.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:04.157+0000 7fdf34790640 1 --2- 172.21.15.38:0/3811779273 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fdf2c101b40 0x7fdf2c102cf0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:04.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:04.157+0000 7fdf34790640 1 --2- 172.21.15.38:0/3811779273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf2c1011a0 0x7fdf2c1027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:04.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:04.157+0000 7fdf34790640 1 -- 172.21.15.38:0/3811779273 >> 172.21.15.38:0/3811779273 conn(0x7fdf2c077220 msgr2=0x7fdf2c0fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:04.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:04.157+0000 7fdf34790640 1 -- 172.21.15.38:0/3811779273 shutdown_connections 2024-09-16T00:33:04.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:04.157+0000 7fdf34790640 1 -- 172.21.15.38:0/3811779273 wait complete. 2024-09-16T00:33:04.889 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-16T00:30:26.764565+0000', 'modified': '2024-09-16T00:32:41.355828+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24494, 'mds_1': 24464, 'mds_2': 24440}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24416': {'gid': 24416, 'name': 'k', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.73:6833/1210328177', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6832', 'nonce': 1210328177}, {'type': 'v1', 'addr': '172.21.15.73:6833', 'nonce': 1210328177}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24421': {'gid': 24421, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.122:6837/1858299680', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6836', 'nonce': 1858299680}, {'type': 'v1', 'addr': '172.21.15.122:6837', 'nonce': 1858299680}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'i', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.122:6839/1983975311', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.122:6838', 'nonce': 1983975311}, {'type': 'v1', 'addr': '172.21.15.122:6839', 'nonce': 1983975311}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24440': {'gid': 24440, 'name': 'e', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 18, 'addr': '172.21.15.73:6835/2866363113', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6834', 'nonce': 2866363113}, {'type': 'v1', 'addr': '172.21.15.73:6835', 'nonce': 2866363113}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24464': {'gid': 24464, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.73:6837/2681081265', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6836', 'nonce': 2681081265}, {'type': 'v1', 'addr': '172.21.15.73:6837', 'nonce': 2681081265}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24494': {'gid': 24494, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.73:6839/2396224525', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6838', 'nonce': 2396224525}, {'type': 'v1', 'addr': '172.21.15.73:6839', 'nonce': 2396224525}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24494, 'qdb_cluster': [24494, 24464, 24440]} 2024-09-16T00:33:04.889 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2024-09-16T00:33:04.889 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.e status 2024-09-16T00:33:05.190 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:05.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:04 smithi038 ceph-mon[30945]: pgmap v373: 129 pgs: 129 active+clean; 651 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-16T00:33:05.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:04 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/3811779273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:33:05.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:04 smithi073 ceph-mon[35254]: pgmap v373: 129 pgs: 129 active+clean; 651 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-16T00:33:05.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:04 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/3811779273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:33:05.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:04 smithi122 ceph-mon[35412]: pgmap v373: 129 pgs: 129 active+clean; 651 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-16T00:33:05.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:04 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/3811779273' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:33:05.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.672+0000 7f34a92a9640 1 -- 172.21.15.38:0/682571849 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34a4102790 msgr2=0x7f34a41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:05.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.672+0000 7f34a92a9640 1 --2- 172.21.15.38:0/682571849 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34a4102790 0x7f34a41067e0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3498009900 tx=0x7f349802a3f0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.672+0000 7f34a92a9640 1 -- 172.21.15.38:0/682571849 shutdown_connections 2024-09-16T00:33:05.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.672+0000 7f34a92a9640 1 --2- 172.21.15.38:0/682571849 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34a4106f10 0x7f34a410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.672+0000 7f34a92a9640 1 --2- 172.21.15.38:0/682571849 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34a4102790 0x7f34a41067e0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.672+0000 7f34a92a9640 1 --2- 172.21.15.38:0/682571849 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34a4101de0 0x7f34a41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.672+0000 7f34a92a9640 1 -- 172.21.15.38:0/682571849 >> 172.21.15.38:0/682571849 conn(0x7f34a40fd8c0 msgr2=0x7f34a40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:05.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.672+0000 7f34a92a9640 1 -- 172.21.15.38:0/682571849 shutdown_connections 2024-09-16T00:33:05.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.672+0000 7f34a92a9640 1 -- 172.21.15.38:0/682571849 wait complete. 2024-09-16T00:33:05.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.673+0000 7f34a92a9640 1 Processor -- start 2024-09-16T00:33:05.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.673+0000 7f34a92a9640 1 -- start start 2024-09-16T00:33:05.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.674+0000 7f34a92a9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34a4101de0 0x7f34a41983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:05.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.674+0000 7f34a92a9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34a4102790 0x7f34a41988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:05.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.674+0000 7f34a92a9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34a4106f10 0x7f34a419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:05.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.674+0000 7f34a92a9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f34a4111f70 con 0x7f34a4106f10 2024-09-16T00:33:05.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.674+0000 7f34a92a9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f34a4111df0 con 0x7f34a4102790 2024-09-16T00:33:05.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.674+0000 7f34a92a9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f34a41120f0 con 0x7f34a4101de0 2024-09-16T00:33:05.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.674+0000 7f34a37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34a4106f10 0x7f34a419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:05.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.674+0000 7f34a37fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34a4106f10 0x7f34a419cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43442/0 (socket says 172.21.15.38:43442) 2024-09-16T00:33:05.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.674+0000 7f34a37fe640 1 -- 172.21.15.38:0/3426669472 learned_addr learned my addr 172.21.15.38:0/3426669472 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:05.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.674+0000 7f34a2ffd640 1 --2- 172.21.15.38:0/3426669472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34a4101de0 0x7f34a41983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:05.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.674+0000 7f34a27fc640 1 --2- 172.21.15.38:0/3426669472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34a4102790 0x7f34a41988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:05.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.674+0000 7f34a37fe640 1 -- 172.21.15.38:0/3426669472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34a4101de0 msgr2=0x7f34a41983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:05.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.674+0000 7f34a37fe640 1 --2- 172.21.15.38:0/3426669472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34a4101de0 0x7f34a41983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.674+0000 7f34a37fe640 1 -- 172.21.15.38:0/3426669472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34a4102790 msgr2=0x7f34a41988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:05.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.674+0000 7f34a37fe640 1 --2- 172.21.15.38:0/3426669472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34a4102790 0x7f34a41988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.674+0000 7f34a37fe640 1 -- 172.21.15.38:0/3426669472 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34a419d360 con 0x7f34a4106f10 2024-09-16T00:33:05.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.674+0000 7f34a27fc640 1 --2- 172.21.15.38:0/3426669472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34a4102790 0x7f34a41988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:05.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.674+0000 7f34a37fe640 1 --2- 172.21.15.38:0/3426669472 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34a4106f10 0x7f34a419cc80 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f349c00d950 tx=0x7f349c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:05.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.674+0000 7f348bfff640 1 -- 172.21.15.38:0/3426669472 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349c017030 con 0x7f34a4106f10 2024-09-16T00:33:05.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.674+0000 7f348bfff640 1 -- 172.21.15.38:0/3426669472 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f349c004020 con 0x7f34a4106f10 2024-09-16T00:33:05.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.674+0000 7f34a92a9640 1 -- 172.21.15.38:0/3426669472 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34a419d650 con 0x7f34a4106f10 2024-09-16T00:33:05.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.674+0000 7f348bfff640 1 -- 172.21.15.38:0/3426669472 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f349c003c00 con 0x7f34a4106f10 2024-09-16T00:33:05.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.674+0000 7f34a92a9640 1 -- 172.21.15.38:0/3426669472 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34a41a4f30 con 0x7f34a4106f10 2024-09-16T00:33:05.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.677+0000 7f348bfff640 1 -- 172.21.15.38:0/3426669472 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f349c01a070 con 0x7f34a4106f10 2024-09-16T00:33:05.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.677+0000 7f34ab682740 1 Processor -- start 2024-09-16T00:33:05.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.677+0000 7f34ab682740 1 -- start start 2024-09-16T00:33:05.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.677+0000 7f34ab682740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbf30bb8d0 0x55cbf30bbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:05.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.677+0000 7f34ab682740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbf30bc1f0 0x55cbf30caa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:05.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.677+0000 7f34ab682740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cbf30cafc0 0x55cbf30cd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:05.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.677+0000 7f34ab682740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55cbf3022090 con 0x55cbf30bb8d0 2024-09-16T00:33:05.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.677+0000 7f34ab682740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55cbf2fc9a70 con 0x55cbf30cafc0 2024-09-16T00:33:05.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.677+0000 7f34ab682740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cbf301e050 con 0x55cbf30bc1f0 2024-09-16T00:33:05.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.677+0000 7f348bfff640 1 --2- 172.21.15.38:0/3426669472 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f348c0a32d0 0x7f348c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:05.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.677+0000 7f34a27fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbf30bb8d0 0x55cbf30bbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:05.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.677+0000 7f34a27fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbf30bb8d0 0x55cbf30bbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43458/0 (socket says 172.21.15.38:43458) 2024-09-16T00:33:05.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.677+0000 7f34a27fc640 1 -- 172.21.15.38:0/1478937001 learned_addr learned my addr 172.21.15.38:0/1478937001 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:05.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.678+0000 7f34a37fe640 1 --2- 172.21.15.38:0/3426669472 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f348c0a32d0 0x7f348c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:05.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.678+0000 7f34a2ffd640 1 --2- 172.21.15.38:0/1478937001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbf30bc1f0 0x55cbf30caa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:05.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.678+0000 7f34a27fc640 1 --2- 172.21.15.38:0/1478937001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cbf30cafc0 0x55cbf30cd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:05.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.678+0000 7f348bfff640 1 -- 172.21.15.38:0/3426669472 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f349c0cf730 con 0x7f34a4106f10 2024-09-16T00:33:05.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.678+0000 7f34a27fc640 1 -- 172.21.15.38:0/1478937001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbf30bc1f0 msgr2=0x55cbf30caa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:05.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.678+0000 7f34a27fc640 1 --2- 172.21.15.38:0/1478937001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbf30bc1f0 0x55cbf30caa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.678+0000 7f34a27fc640 1 -- 172.21.15.38:0/1478937001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cbf30cafc0 msgr2=0x55cbf30cd3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:05.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.678+0000 7f34a27fc640 1 --2- 172.21.15.38:0/1478937001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cbf30cafc0 0x55cbf30cd3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.678+0000 7f34a27fc640 1 -- 172.21.15.38:0/1478937001 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbf2fd6130 con 0x55cbf30bb8d0 2024-09-16T00:33:05.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.678+0000 7f34a2ffd640 1 --2- 172.21.15.38:0/1478937001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbf30bc1f0 0x55cbf30caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:05.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.678+0000 7f34a27fc640 1 --2- 172.21.15.38:0/1478937001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbf30bb8d0 0x55cbf30bbcb0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3498005940 tx=0x7f3498031480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68de2fd482b544bd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:05.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.678+0000 7f34a37fe640 1 --2- 172.21.15.38:0/3426669472 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f348c0a32d0 0x7f348c0a5790 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f349c0a3fb0 tx=0x7f349c004540 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:05.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.678+0000 7f34897fa640 1 -- 172.21.15.38:0/1478937001 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3498035050 con 0x55cbf30bb8d0 2024-09-16T00:33:05.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.678+0000 7f34897fa640 1 -- 172.21.15.38:0/1478937001 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3498031890 con 0x55cbf30bb8d0 2024-09-16T00:33:05.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.678+0000 7f34897fa640 1 -- 172.21.15.38:0/1478937001 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34980046c0 con 0x55cbf30bb8d0 2024-09-16T00:33:05.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.679+0000 7f34ab682740 1 -- 172.21.15.38:0/1478937001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbf30bb8d0 msgr2=0x55cbf30bbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:05.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.679+0000 7f34ab682740 1 --2- 172.21.15.38:0/1478937001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbf30bb8d0 0x55cbf30bbcb0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f3498005940 tx=0x7f3498031480 comp rx=0 tx=0).stop 2024-09-16T00:33:05.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.679+0000 7f34ab682740 1 -- 172.21.15.38:0/1478937001 shutdown_connections 2024-09-16T00:33:05.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.679+0000 7f34ab682740 1 --2- 172.21.15.38:0/1478937001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cbf30cafc0 0x55cbf30cd3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.679+0000 7f34ab682740 1 --2- 172.21.15.38:0/1478937001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbf30bc1f0 0x55cbf30caa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.679+0000 7f34ab682740 1 --2- 172.21.15.38:0/1478937001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbf30bb8d0 0x55cbf30bbcb0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.679+0000 7f34ab682740 1 -- 172.21.15.38:0/1478937001 >> 172.21.15.38:0/1478937001 conn(0x55cbf30b7f60 msgr2=0x55cbf30b8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:05.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.679+0000 7f34ab682740 1 -- 172.21.15.38:0/1478937001 shutdown_connections 2024-09-16T00:33:05.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.679+0000 7f34ab682740 1 -- 172.21.15.38:0/1478937001 wait complete. 2024-09-16T00:33:05.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.679+0000 7f34ab682740 1 -- start start 2024-09-16T00:33:05.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.679+0000 7f34ab682740 1 Processor -- start 2024-09-16T00:33:05.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.680+0000 7f34ab682740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbf30bb8d0 0x55cbf30ca8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:05.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.680+0000 7f34ab682740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cbf30bc1f0 0x55cbf30c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:05.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.680+0000 7f34ab682740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbf30cafc0 0x55cbf30c8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:05.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.680+0000 7f34ab682740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55cbf30cdb90 con 0x55cbf30bb8d0 2024-09-16T00:33:05.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.680+0000 7f34ab682740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55cbf2fdcf20 con 0x55cbf30bc1f0 2024-09-16T00:33:05.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.680+0000 7f34ab682740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cbf30cdd10 con 0x55cbf30cafc0 2024-09-16T00:33:05.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.680+0000 7f34ab682740 10 client.0 _ll_register_callbacks cb 0x55cbf2fa6030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:05.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.680+0000 7f34a27fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbf30bb8d0 0x55cbf30ca8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:05.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.680+0000 7f34a27fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbf30bb8d0 0x55cbf30ca8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43466/0 (socket says 172.21.15.38:43466) 2024-09-16T00:33:05.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.680+0000 7f34a27fc640 1 -- 172.21.15.38:0/743702103 learned_addr learned my addr 172.21.15.38:0/743702103 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:05.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.680+0000 7f34a2ffd640 1 --2- 172.21.15.38:0/743702103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cbf30bc1f0 0x55cbf30c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:05.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.680+0000 7f34a27fc640 1 --2- 172.21.15.38:0/743702103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbf30cafc0 0x55cbf30c8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:05.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.680+0000 7f34a27fc640 1 -- 172.21.15.38:0/743702103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbf30cafc0 msgr2=0x55cbf30c8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:05.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.680+0000 7f34a27fc640 1 --2- 172.21.15.38:0/743702103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbf30cafc0 0x55cbf30c8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.680+0000 7f34a27fc640 1 -- 172.21.15.38:0/743702103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cbf30bc1f0 msgr2=0x55cbf30c7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:05.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.680+0000 7f34a27fc640 1 --2- 172.21.15.38:0/743702103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cbf30bc1f0 0x55cbf30c7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.680+0000 7f34a27fc640 1 -- 172.21.15.38:0/743702103 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbf2d932b0 con 0x55cbf30bb8d0 2024-09-16T00:33:05.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.680+0000 7f34a2ffd640 1 --2- 172.21.15.38:0/743702103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cbf30bc1f0 0x55cbf30c7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:33:05.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.680+0000 7f34a27fc640 1 --2- 172.21.15.38:0/743702103 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbf30bb8d0 0x55cbf30ca8f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3498031550 tx=0x7f34980054c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c52ae14fefc765e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:05.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.680+0000 7f3486ff5640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:33:05.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.680+0000 7f3486ff5640 1 -- 172.21.15.38:0/743702103 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3498035040 con 0x55cbf30bb8d0 2024-09-16T00:33:05.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.680+0000 7f3486ff5640 1 -- 172.21.15.38:0/743702103 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f34980035f0 con 0x55cbf30bb8d0 2024-09-16T00:33:05.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.680+0000 7f34ab682740 1 -- 172.21.15.38:0/743702103 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3498031890 con 0x55cbf30bb8d0 2024-09-16T00:33:05.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.681+0000 7f3486ff5640 1 -- 172.21.15.38:0/743702103 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3498058050 con 0x55cbf30bb8d0 2024-09-16T00:33:05.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.681+0000 7f34ab682740 10 client.15399 fetch_fsmap learned FSMap version 26 2024-09-16T00:33:05.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.681+0000 7f34ab682740 1 -- 172.21.15.38:0/743702103 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cbf30d9580 con 0x55cbf30bb8d0 2024-09-16T00:33:05.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.681+0000 7f3486ff5640 1 -- 172.21.15.38:0/743702103 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3498031890 con 0x55cbf30bb8d0 2024-09-16T00:33:05.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.681+0000 7f3486ff5640 1 -- 172.21.15.38:0/743702103 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f349803a910 con 0x55cbf30bb8d0 2024-09-16T00:33:05.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.681+0000 7f34ab682740 10 client.15399 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:33:05.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.681+0000 7f34ab682740 10 client.15399 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-16T00:33:05.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.681+0000 7f34ab682740 20 client.15399 populate_metadata read hostname 'smithi038' 2024-09-16T00:33:05.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.681+0000 7f34ab682740 1 --2- 172.21.15.38:0/743702103 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x55cbf30da8d0 0x55cbf30dcad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:05.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.681+0000 7f34ab682740 4 client.15399 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:33:05.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.681+0000 7f34ab682740 1 -- 172.21.15.38:0/743702103 --> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cbf302f600 con 0x55cbf30da8d0 2024-09-16T00:33:05.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.681+0000 7f34a2ffd640 1 --2- 172.21.15.38:0/743702103 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x55cbf30da8d0 0x55cbf30dcad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:05.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.682+0000 7f34a2ffd640 1 --2- 172.21.15.38:0/743702103 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x55cbf30da8d0 0x55cbf30dcad0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=78bc5c37c9a51447 server_cookie=8e2a8fc75fe3038a in_seq=0 out_seq=0 2024-09-16T00:33:05.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.682+0000 7f3486ff5640 10 client.15399 ms_handle_connect on v2:172.21.15.73:6834/2866363113 2024-09-16T00:33:05.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.682+0000 7f3486ff5640 1 -- 172.21.15.38:0/743702103 <== mds.2 v2:172.21.15.73:6834/2866363113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55cbf302f600 con 0x55cbf30da8d0 2024-09-16T00:33:05.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.682+0000 7f3486ff5640 10 client.15399 handle_command_reply: tid=0 2024-09-16T00:33:05.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.683+0000 7f34ab682740 1 client.15399 shutdown 2024-09-16T00:33:05.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.683+0000 7f34ab682740 1 -- 172.21.15.38:0/743702103 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbf30bb8d0 msgr2=0x55cbf30ca8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:05.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.683+0000 7f34ab682740 1 --2- 172.21.15.38:0/743702103 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbf30bb8d0 0x55cbf30ca8f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f3498031550 tx=0x7f34980054c0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.683+0000 7f34ab682740 1 -- 172.21.15.38:0/743702103 shutdown_connections 2024-09-16T00:33:05.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.683+0000 7f34ab682740 1 --2- 172.21.15.38:0/743702103 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x55cbf30da8d0 0x55cbf30dcad0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.683+0000 7f34ab682740 1 --2- 172.21.15.38:0/743702103 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbf30cafc0 0x55cbf30c8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.683+0000 7f34ab682740 1 --2- 172.21.15.38:0/743702103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cbf30bc1f0 0x55cbf30c7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.683+0000 7f34ab682740 1 --2- 172.21.15.38:0/743702103 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbf30bb8d0 0x55cbf30ca8f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.683+0000 7f34ab682740 1 -- 172.21.15.38:0/743702103 >> 172.21.15.38:0/743702103 conn(0x55cbf30b7f60 msgr2=0x55cbf30bdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:05.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.683+0000 7f3486ff5640 0 client.15399 ms_handle_reset on v2:172.21.15.73:6834/2866363113 2024-09-16T00:33:05.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.683+0000 7f34ab682740 1 -- 172.21.15.38:0/743702103 shutdown_connections 2024-09-16T00:33:05.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.683+0000 7f34ab682740 1 -- 172.21.15.38:0/743702103 wait complete. 2024-09-16T00:33:05.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.683+0000 7f34ab682740 20 client.15399 trim_cache size 0 max 16384 2024-09-16T00:33:05.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.701+0000 7f34ab682740 1 Processor -- start 2024-09-16T00:33:05.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.701+0000 7f34ab682740 1 -- start start 2024-09-16T00:33:05.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.701+0000 7f34ab682740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbf30bb8d0 0x55cbf30dc400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:05.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.701+0000 7f34ab682740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cbf30bc1f0 0x55cbf30dbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:05.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.701+0000 7f34ab682740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbf30cafc0 0x55cbf30daf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:05.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.701+0000 7f34ab682740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55cbf30dae10 con 0x55cbf30bb8d0 2024-09-16T00:33:05.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.701+0000 7f34ab682740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55cbf30db4e0 con 0x55cbf30bc1f0 2024-09-16T00:33:05.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.701+0000 7f34ab682740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cbf30dbbb0 con 0x55cbf30cafc0 2024-09-16T00:33:05.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.702+0000 7f34a27fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbf30bb8d0 0x55cbf30dc400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:05.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.702+0000 7f34a27fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbf30bb8d0 0x55cbf30dc400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43470/0 (socket says 172.21.15.38:43470) 2024-09-16T00:33:05.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.702+0000 7f34a27fc640 1 -- 172.21.15.38:0/856488354 learned_addr learned my addr 172.21.15.38:0/856488354 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:05.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.702+0000 7f34a2ffd640 1 --2- 172.21.15.38:0/856488354 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cbf30bc1f0 0x55cbf30dbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:05.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.702+0000 7f34a27fc640 1 -- 172.21.15.38:0/856488354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbf30cafc0 msgr2=0x55cbf30daf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:05.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.702+0000 7f34a27fc640 1 --2- 172.21.15.38:0/856488354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbf30cafc0 0x55cbf30daf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.702+0000 7f34a27fc640 1 -- 172.21.15.38:0/856488354 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cbf30bc1f0 msgr2=0x55cbf30dbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:05.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.702+0000 7f34a27fc640 1 --2- 172.21.15.38:0/856488354 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cbf30bc1f0 0x55cbf30dbd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.702+0000 7f34a27fc640 1 -- 172.21.15.38:0/856488354 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cbf30cac60 con 0x55cbf30bb8d0 2024-09-16T00:33:05.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.702+0000 7f34a27fc640 1 --2- 172.21.15.38:0/856488354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbf30bb8d0 0x55cbf30dc400 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3498031610 tx=0x7f349802a3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5989fb517da90c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:05.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.702+0000 7f3488ff9640 1 -- 172.21.15.38:0/856488354 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3498060070 con 0x55cbf30bb8d0 2024-09-16T00:33:05.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.702+0000 7f3488ff9640 1 -- 172.21.15.38:0/856488354 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f349804f070 con 0x55cbf30bb8d0 2024-09-16T00:33:05.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.702+0000 7f3488ff9640 1 -- 172.21.15.38:0/856488354 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3498035040 con 0x55cbf30bb8d0 2024-09-16T00:33:05.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.703+0000 7f34ab682740 1 -- 172.21.15.38:0/856488354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbf30bb8d0 msgr2=0x55cbf30dc400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:05.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.703+0000 7f34ab682740 1 --2- 172.21.15.38:0/856488354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbf30bb8d0 0x55cbf30dc400 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3498031610 tx=0x7f349802a3f0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.703+0000 7f34ab682740 1 -- 172.21.15.38:0/856488354 shutdown_connections 2024-09-16T00:33:05.712 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.703+0000 7f34ab682740 1 --2- 172.21.15.38:0/856488354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbf30cafc0 0x55cbf30daf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.703+0000 7f34ab682740 1 --2- 172.21.15.38:0/856488354 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cbf30bc1f0 0x55cbf30dbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.703+0000 7f34ab682740 1 --2- 172.21.15.38:0/856488354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbf30bb8d0 0x55cbf30dc400 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.703+0000 7f34ab682740 1 -- 172.21.15.38:0/856488354 >> 172.21.15.38:0/856488354 conn(0x55cbf30b7f60 msgr2=0x55cbf30b8340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:05.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.703+0000 7f34ab682740 1 -- 172.21.15.38:0/856488354 shutdown_connections 2024-09-16T00:33:05.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.703+0000 7f34ab682740 1 -- 172.21.15.38:0/856488354 wait complete. 2024-09-16T00:33:05.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.703+0000 7f34ab682740 1 -- start start 2024-09-16T00:33:05.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.704+0000 7f34ab682740 1 Processor -- start 2024-09-16T00:33:05.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.704+0000 7f34ab682740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbf30bb8d0 0x55cbf30c3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:05.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.704+0000 7f34ab682740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cbf30bc1f0 0x55cbf30c43f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:05.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.704+0000 7f34ab682740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbf30cafc0 0x55cbf30c4930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:05.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.704+0000 7f34ab682740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55cbf30dc950 con 0x55cbf30bb8d0 2024-09-16T00:33:05.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.704+0000 7f34ab682740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55cbf30dc280 con 0x55cbf30bc1f0 2024-09-16T00:33:05.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.704+0000 7f34ab682740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55cbf30dd020 con 0x55cbf30cafc0 2024-09-16T00:33:05.713 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.704+0000 7f34ab682740 10 client.0 _ll_register_callbacks cb 0x7f3498031640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:05.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.704+0000 7f34a27fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbf30bb8d0 0x55cbf30c3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:05.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.704+0000 7f34a27fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbf30bb8d0 0x55cbf30c3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43478/0 (socket says 172.21.15.38:43478) 2024-09-16T00:33:05.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.704+0000 7f34a27fc640 1 -- 172.21.15.38:0/267169424 learned_addr learned my addr 172.21.15.38:0/267169424 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:05.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.704+0000 7f34a2ffd640 1 --2- 172.21.15.38:0/267169424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cbf30bc1f0 0x55cbf30c43f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:05.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.704+0000 7f34a27fc640 1 -- 172.21.15.38:0/267169424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbf30cafc0 msgr2=0x55cbf30c4930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:05.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.704+0000 7f34a27fc640 1 --2- 172.21.15.38:0/267169424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbf30cafc0 0x55cbf30c4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.704+0000 7f34a27fc640 1 -- 172.21.15.38:0/267169424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cbf30bc1f0 msgr2=0x55cbf30c43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:05.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.704+0000 7f34a27fc640 1 --2- 172.21.15.38:0/267169424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cbf30bc1f0 0x55cbf30c43f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.704+0000 7f34a27fc640 1 -- 172.21.15.38:0/267169424 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cbf30e3d20 con 0x55cbf30bb8d0 2024-09-16T00:33:05.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.704+0000 7f34a2ffd640 1 --2- 172.21.15.38:0/267169424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cbf30bc1f0 0x55cbf30c43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:33:05.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.704+0000 7f34a27fc640 1 --2- 172.21.15.38:0/267169424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbf30bb8d0 0x55cbf30c3eb0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3498033000 tx=0x7f3498031470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cd077156134c8b3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:05.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.704+0000 7f3486ff5640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:33:05.714 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.705+0000 7f3486ff5640 1 -- 172.21.15.38:0/267169424 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3498060040 con 0x55cbf30bb8d0 2024-09-16T00:33:05.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.705+0000 7f3486ff5640 1 -- 172.21.15.38:0/267169424 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f349803a480 con 0x55cbf30bb8d0 2024-09-16T00:33:05.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.705+0000 7f34ab682740 1 -- 172.21.15.38:0/267169424 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f349804f070 con 0x55cbf30bb8d0 2024-09-16T00:33:05.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.705+0000 7f3486ff5640 1 -- 172.21.15.38:0/267169424 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f349804a460 con 0x55cbf30bb8d0 2024-09-16T00:33:05.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.705+0000 7f34ab682740 10 client.15405 fetch_fsmap learned FSMap version 26 2024-09-16T00:33:05.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.705+0000 7f34ab682740 1 -- 172.21.15.38:0/267169424 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55cbf30e5020 con 0x55cbf30bb8d0 2024-09-16T00:33:05.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.705+0000 7f3486ff5640 1 -- 172.21.15.38:0/267169424 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f349804f070 con 0x55cbf30bb8d0 2024-09-16T00:33:05.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.705+0000 7f3486ff5640 1 -- 172.21.15.38:0/267169424 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f34980316f0 con 0x55cbf30bb8d0 2024-09-16T00:33:05.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.705+0000 7f34ab682740 10 client.15405 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:33:05.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.705+0000 7f34ab682740 10 client.15405 resolve_mds: resolved name 'e' to daemon mds.e 2024-09-16T00:33:05.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.705+0000 7f34ab682740 20 client.15405 populate_metadata read hostname 'smithi038' 2024-09-16T00:33:05.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.705+0000 7f34ab682740 1 --2- 172.21.15.38:0/267169424 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x55cbf3149690 0x55cbf31483a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:05.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.705+0000 7f34ab682740 4 client.15405 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:33:05.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.705+0000 7f34ab682740 1 -- 172.21.15.38:0/267169424 --> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] -- command(tid 0: {"prefix": "status"}) -- 0x55cbf3029b10 con 0x55cbf3149690 2024-09-16T00:33:05.715 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.706+0000 7f34a2ffd640 1 --2- 172.21.15.38:0/267169424 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x55cbf3149690 0x55cbf31483a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:05.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.706+0000 7f3486ff5640 10 client.15405 ms_handle_connect on v2:172.21.15.73:6834/2866363113 2024-09-16T00:33:05.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.706+0000 7f34a2ffd640 1 --2- 172.21.15.38:0/267169424 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x55cbf3149690 0x55cbf31483a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e3808a0b89cf8552 server_cookie=f03933b8b6042560 in_seq=0 out_seq=0 2024-09-16T00:33:05.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.706+0000 7f3486ff5640 1 -- 172.21.15.38:0/267169424 <== mds.2 v2:172.21.15.73:6834/2866363113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55cbf3029b10 con 0x55cbf3149690 2024-09-16T00:33:05.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.706+0000 7f3486ff5640 10 client.15405 handle_command_reply: tid=0 2024-09-16T00:33:05.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.706+0000 7f34ab682740 1 client.15405 shutdown 2024-09-16T00:33:05.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.707+0000 7f34ab682740 1 -- 172.21.15.38:0/267169424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbf30bb8d0 msgr2=0x55cbf30c3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:05.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.707+0000 7f34ab682740 1 --2- 172.21.15.38:0/267169424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbf30bb8d0 0x55cbf30c3eb0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3498033000 tx=0x7f3498031470 comp rx=0 tx=0).stop 2024-09-16T00:33:05.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.707+0000 7f34ab682740 1 -- 172.21.15.38:0/267169424 shutdown_connections 2024-09-16T00:33:05.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.707+0000 7f34ab682740 1 --2- 172.21.15.38:0/267169424 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x55cbf3149690 0x55cbf31483a0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.707+0000 7f34ab682740 1 --2- 172.21.15.38:0/267169424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55cbf30cafc0 0x55cbf30c4930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.707+0000 7f34ab682740 1 --2- 172.21.15.38:0/267169424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55cbf30bc1f0 0x55cbf30c43f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.707+0000 7f3486ff5640 0 client.15405 ms_handle_reset on v2:172.21.15.73:6834/2866363113 2024-09-16T00:33:05.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.707+0000 7f34ab682740 1 --2- 172.21.15.38:0/267169424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55cbf30bb8d0 0x55cbf30c3eb0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.716 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.707+0000 7f34ab682740 1 -- 172.21.15.38:0/267169424 >> 172.21.15.38:0/267169424 conn(0x55cbf30b7f60 msgr2=0x55cbf30b7020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:05.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.707+0000 7f34ab682740 1 -- 172.21.15.38:0/267169424 shutdown_connections 2024-09-16T00:33:05.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.707+0000 7f34ab682740 1 -- 172.21.15.38:0/267169424 wait complete. 2024-09-16T00:33:05.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.707+0000 7f34ab682740 20 client.15405 trim_cache size 0 max 16384 2024-09-16T00:33:05.717 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-16T00:33:05.717 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "983b2072-73c1-11ef-bceb-c7b262605968", 2024-09-16T00:33:05.717 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 2, 2024-09-16T00:33:05.717 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24440, 2024-09-16T00:33:05.717 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-16T00:33:05.717 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-16T00:33:05.717 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-16T00:33:05.718 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 123.401871797, 2024-09-16T00:33:05.718 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 26, 2024-09-16T00:33:05.718 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 92, 2024-09-16T00:33:05.718 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 92, 2024-09-16T00:33:05.718 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 188.98237167799999 2024-09-16T00:33:05.718 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-16T00:33:05.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.707+0000 7f34a92a9640 1 -- 172.21.15.38:0/3426669472 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f348c0a32d0 msgr2=0x7f348c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:05.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.707+0000 7f34a92a9640 1 --2- 172.21.15.38:0/3426669472 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f348c0a32d0 0x7f348c0a5790 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f349c0a3fb0 tx=0x7f349c004540 comp rx=0 tx=0).stop 2024-09-16T00:33:05.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.707+0000 7f34a92a9640 1 -- 172.21.15.38:0/3426669472 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34a4106f10 msgr2=0x7f34a419cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:05.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.707+0000 7f34a92a9640 1 --2- 172.21.15.38:0/3426669472 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34a4106f10 0x7f34a419cc80 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f349c00d950 tx=0x7f349c00de10 comp rx=0 tx=0).stop 2024-09-16T00:33:05.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.707+0000 7f34a92a9640 1 -- 172.21.15.38:0/3426669472 shutdown_connections 2024-09-16T00:33:05.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.707+0000 7f34a92a9640 1 --2- 172.21.15.38:0/3426669472 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f348c0a32d0 0x7f348c0a5790 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.707+0000 7f34a92a9640 1 --2- 172.21.15.38:0/3426669472 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f34a4106f10 0x7f34a419cc80 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.707+0000 7f34a92a9640 1 --2- 172.21.15.38:0/3426669472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34a4102790 0x7f34a41988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.707+0000 7f34a92a9640 1 --2- 172.21.15.38:0/3426669472 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f34a4101de0 0x7f34a41983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:05.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.707+0000 7f34a92a9640 1 -- 172.21.15.38:0/3426669472 >> 172.21.15.38:0/3426669472 conn(0x7f34a40fd8c0 msgr2=0x7f34a40fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:05.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.708+0000 7f34a92a9640 1 -- 172.21.15.38:0/3426669472 shutdown_connections 2024-09-16T00:33:05.719 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:05.708+0000 7f34a92a9640 1 -- 172.21.15.38:0/3426669472 wait complete. 2024-09-16T00:33:06.464 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.h status 2024-09-16T00:33:06.725 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:07.004 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:06 smithi038 ceph-mon[30945]: pgmap v374: 129 pgs: 129 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-16T00:33:07.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.194+0000 7ff103b06640 1 -- 172.21.15.38:0/3622574775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0fc10e500 msgr2=0x7ff0fc110910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:07.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.194+0000 7ff103b06640 1 --2- 172.21.15.38:0/3622574775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0fc10e500 0x7ff0fc110910 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4009900 tx=0x7ff0f402a410 comp rx=0 tx=0).stop 2024-09-16T00:33:07.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.194+0000 7ff103b06640 1 -- 172.21.15.38:0/3622574775 shutdown_connections 2024-09-16T00:33:07.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.194+0000 7ff103b06640 1 --2- 172.21.15.38:0/3622574775 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0fc10e500 0x7ff0fc110910 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.194+0000 7ff103b06640 1 --2- 172.21.15.38:0/3622574775 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0fc101440 0x7ff0fc10df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.194+0000 7ff103b06640 1 --2- 172.21.15.38:0/3622574775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0fc100b20 0x7ff0fc100f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.194+0000 7ff103b06640 1 -- 172.21.15.38:0/3622574775 >> 172.21.15.38:0/3622574775 conn(0x7ff0fc0fc7d0 msgr2=0x7ff0fc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:07.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.194+0000 7ff103b06640 1 -- 172.21.15.38:0/3622574775 shutdown_connections 2024-09-16T00:33:07.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.194+0000 7ff103b06640 1 -- 172.21.15.38:0/3622574775 wait complete. 2024-09-16T00:33:07.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.194+0000 7ff103b06640 1 Processor -- start 2024-09-16T00:33:07.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.194+0000 7ff103b06640 1 -- start start 2024-09-16T00:33:07.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.195+0000 7ff103b06640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0fc100b20 0x7ff0fc19a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:07.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.195+0000 7ff103b06640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0fc101440 0x7ff0fc19ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:07.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.195+0000 7ff103b06640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0fc10e500 0x7ff0fc19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:07.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.195+0000 7ff103b06640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff0fc114230 con 0x7ff0fc10e500 2024-09-16T00:33:07.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.195+0000 7ff103b06640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff0fc1140b0 con 0x7ff0fc100b20 2024-09-16T00:33:07.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.195+0000 7ff103b06640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff0fc1143b0 con 0x7ff0fc101440 2024-09-16T00:33:07.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.195+0000 7ff1020a3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0fc10e500 0x7ff0fc19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:07.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.195+0000 7ff1020a3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0fc10e500 0x7ff0fc19efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40828/0 (socket says 172.21.15.38:40828) 2024-09-16T00:33:07.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.195+0000 7ff1020a3640 1 -- 172.21.15.38:0/1562973941 learned_addr learned my addr 172.21.15.38:0/1562973941 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:07.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.195+0000 7ff1018a2640 1 --2- 172.21.15.38:0/1562973941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0fc100b20 0x7ff0fc19a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:07.217 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.195+0000 7ff1010a1640 1 --2- 172.21.15.38:0/1562973941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0fc101440 0x7ff0fc19ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:07.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.195+0000 7ff1020a3640 1 -- 172.21.15.38:0/1562973941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0fc101440 msgr2=0x7ff0fc19ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:07.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.195+0000 7ff1020a3640 1 --2- 172.21.15.38:0/1562973941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0fc101440 0x7ff0fc19ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.195+0000 7ff1020a3640 1 -- 172.21.15.38:0/1562973941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0fc100b20 msgr2=0x7ff0fc19a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:07.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.195+0000 7ff1020a3640 1 --2- 172.21.15.38:0/1562973941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0fc100b20 0x7ff0fc19a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.195+0000 7ff1020a3640 1 -- 172.21.15.38:0/1562973941 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0fc19f6a0 con 0x7ff0fc10e500 2024-09-16T00:33:07.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.195+0000 7ff1020a3640 1 --2- 172.21.15.38:0/1562973941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0fc10e500 0x7ff0fc19efc0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4005ab0 tx=0x7ff0f40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:07.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.196+0000 7ff0eeffd640 1 -- 172.21.15.38:0/1562973941 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f403f030 con 0x7ff0fc10e500 2024-09-16T00:33:07.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.196+0000 7ff103b06640 1 -- 172.21.15.38:0/1562973941 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0fc19f930 con 0x7ff0fc10e500 2024-09-16T00:33:07.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.196+0000 7ff0eeffd640 1 -- 172.21.15.38:0/1562973941 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0f4003ce0 con 0x7ff0fc10e500 2024-09-16T00:33:07.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.196+0000 7ff0eeffd640 1 -- 172.21.15.38:0/1562973941 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f4035070 con 0x7ff0fc10e500 2024-09-16T00:33:07.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.196+0000 7ff103b06640 1 -- 172.21.15.38:0/1562973941 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0fc1a7270 con 0x7ff0fc10e500 2024-09-16T00:33:07.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.198+0000 7ff0eeffd640 1 -- 172.21.15.38:0/1562973941 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7ff0f4042030 con 0x7ff0fc10e500 2024-09-16T00:33:07.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.198+0000 7ff105edf740 1 Processor -- start 2024-09-16T00:33:07.218 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.199+0000 7ff105edf740 1 -- start start 2024-09-16T00:33:07.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.199+0000 7ff105edf740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5647fda399d0 0x5647fda39db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:07.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.199+0000 7ff105edf740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647fda3a2f0 0x5647fda44670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:07.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.199+0000 7ff105edf740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647fda44bb0 0x5647fda46fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:07.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.199+0000 7ff105edf740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5647fd9a0090 con 0x5647fda399d0 2024-09-16T00:33:07.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.199+0000 7ff105edf740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5647fd947a70 con 0x5647fda44bb0 2024-09-16T00:33:07.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.199+0000 7ff105edf740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5647fd99c050 con 0x5647fda3a2f0 2024-09-16T00:33:07.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.199+0000 7ff1010a1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5647fda399d0 0x5647fda39db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:07.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.199+0000 7ff1010a1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5647fda399d0 0x5647fda39db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40838/0 (socket says 172.21.15.38:40838) 2024-09-16T00:33:07.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.199+0000 7ff1010a1640 1 -- 172.21.15.38:0/2653200742 learned_addr learned my addr 172.21.15.38:0/2653200742 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:07.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.199+0000 7ff1018a2640 1 --2- 172.21.15.38:0/2653200742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647fda3a2f0 0x5647fda44670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:07.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.199+0000 7ff0eeffd640 1 --2- 172.21.15.38:0/1562973941 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ff0e80a32d0 0x7ff0e80a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:07.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.199+0000 7ff1010a1640 1 --2- 172.21.15.38:0/2653200742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647fda44bb0 0x5647fda46fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:07.219 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.199+0000 7ff1010a1640 1 -- 172.21.15.38:0/2653200742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647fda3a2f0 msgr2=0x5647fda44670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:07.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.199+0000 7ff1010a1640 1 --2- 172.21.15.38:0/2653200742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647fda3a2f0 0x5647fda44670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.199+0000 7ff1010a1640 1 -- 172.21.15.38:0/2653200742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647fda44bb0 msgr2=0x5647fda46fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:07.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.199+0000 7ff1010a1640 1 --2- 172.21.15.38:0/2653200742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647fda44bb0 0x5647fda46fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.199+0000 7ff1010a1640 1 -- 172.21.15.38:0/2653200742 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647fd954130 con 0x5647fda399d0 2024-09-16T00:33:07.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.199+0000 7ff0eeffd640 1 -- 172.21.15.38:0/1562973941 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7ff0f40f0470 con 0x7ff0fc10e500 2024-09-16T00:33:07.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.200+0000 7ff1020a3640 1 --2- 172.21.15.38:0/1562973941 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ff0e80a32d0 0x7ff0e80a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:07.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.200+0000 7ff1010a1640 1 --2- 172.21.15.38:0/2653200742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5647fda399d0 0x5647fda39db0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8010370 tx=0x7ff0f800c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83c6f3c8e644642b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:07.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.200+0000 7ff0e7fff640 1 -- 172.21.15.38:0/2653200742 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8018030 con 0x5647fda399d0 2024-09-16T00:33:07.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.200+0000 7ff0e7fff640 1 -- 172.21.15.38:0/2653200742 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0f8005500 con 0x5647fda399d0 2024-09-16T00:33:07.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.200+0000 7ff0e7fff640 1 -- 172.21.15.38:0/2653200742 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8012660 con 0x5647fda399d0 2024-09-16T00:33:07.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.201+0000 7ff105edf740 1 -- 172.21.15.38:0/2653200742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5647fda399d0 msgr2=0x5647fda39db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:07.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.201+0000 7ff1020a3640 1 --2- 172.21.15.38:0/1562973941 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ff0e80a32d0 0x7ff0e80a5790 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff0f40bc2b0 tx=0x7ff0f4005780 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:07.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.201+0000 7ff105edf740 1 --2- 172.21.15.38:0/2653200742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5647fda399d0 0x5647fda39db0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8010370 tx=0x7ff0f800c510 comp rx=0 tx=0).stop 2024-09-16T00:33:07.220 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.201+0000 7ff105edf740 1 -- 172.21.15.38:0/2653200742 shutdown_connections 2024-09-16T00:33:07.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.201+0000 7ff105edf740 1 --2- 172.21.15.38:0/2653200742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647fda44bb0 0x5647fda46fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.201+0000 7ff105edf740 1 --2- 172.21.15.38:0/2653200742 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647fda3a2f0 0x5647fda44670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.201+0000 7ff105edf740 1 --2- 172.21.15.38:0/2653200742 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5647fda399d0 0x5647fda39db0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.201+0000 7ff105edf740 1 -- 172.21.15.38:0/2653200742 >> 172.21.15.38:0/2653200742 conn(0x5647fda35f60 msgr2=0x5647fda36340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:07.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.201+0000 7ff105edf740 1 -- 172.21.15.38:0/2653200742 shutdown_connections 2024-09-16T00:33:07.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.201+0000 7ff105edf740 1 -- 172.21.15.38:0/2653200742 wait complete. 2024-09-16T00:33:07.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.201+0000 7ff105edf740 1 -- start start 2024-09-16T00:33:07.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.201+0000 7ff105edf740 1 Processor -- start 2024-09-16T00:33:07.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.202+0000 7ff105edf740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647fda399d0 0x5647fda44440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:07.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.202+0000 7ff105edf740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5647fda3a2f0 0x5647fda41790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:07.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.202+0000 7ff105edf740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647fda44bb0 0x5647fda41cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:07.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.202+0000 7ff105edf740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5647fda47780 con 0x5647fda3a2f0 2024-09-16T00:33:07.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.202+0000 7ff105edf740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5647fd95af20 con 0x5647fda399d0 2024-09-16T00:33:07.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.202+0000 7ff105edf740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5647fda47900 con 0x5647fda44bb0 2024-09-16T00:33:07.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.202+0000 7ff105edf740 10 client.0 _ll_register_callbacks cb 0x5647fd924030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:07.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.202+0000 7ff1018a2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5647fda3a2f0 0x5647fda41790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:07.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.202+0000 7ff1018a2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5647fda3a2f0 0x5647fda41790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40854/0 (socket says 172.21.15.38:40854) 2024-09-16T00:33:07.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.202+0000 7ff1018a2640 1 -- 172.21.15.38:0/371367347 learned_addr learned my addr 172.21.15.38:0/371367347 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:07.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.202+0000 7ff1010a1640 1 --2- 172.21.15.38:0/371367347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647fda399d0 0x5647fda44440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:07.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.202+0000 7ff1010a1640 1 --2- 172.21.15.38:0/371367347 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647fda44bb0 0x5647fda41cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:07.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.202+0000 7ff1018a2640 1 -- 172.21.15.38:0/371367347 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647fda44bb0 msgr2=0x5647fda41cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:07.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.202+0000 7ff1018a2640 1 --2- 172.21.15.38:0/371367347 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647fda44bb0 0x5647fda41cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.202+0000 7ff1018a2640 1 -- 172.21.15.38:0/371367347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647fda399d0 msgr2=0x5647fda44440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:07.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.202+0000 7ff1018a2640 1 --2- 172.21.15.38:0/371367347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647fda399d0 0x5647fda44440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.202+0000 7ff1018a2640 1 -- 172.21.15.38:0/371367347 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647fd7112b0 con 0x5647fda3a2f0 2024-09-16T00:33:07.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.202+0000 7ff1010a1640 1 --2- 172.21.15.38:0/371367347 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647fda44bb0 0x5647fda41cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:07.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.202+0000 7ff1018a2640 1 --2- 172.21.15.38:0/371367347 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5647fda3a2f0 0x5647fda41790 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0011710 tx=0x7ff0f0011bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f82e882b8d256fc8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:07.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.202+0000 7ff0e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:33:07.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.202+0000 7ff0e57fa640 1 -- 172.21.15.38:0/371367347 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f0004d20 con 0x5647fda3a2f0 2024-09-16T00:33:07.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.202+0000 7ff0e57fa640 1 -- 172.21.15.38:0/371367347 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7ff0f000f940 con 0x5647fda3a2f0 2024-09-16T00:33:07.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.202+0000 7ff105edf740 1 -- 172.21.15.38:0/371367347 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0f8005500 con 0x5647fda3a2f0 2024-09-16T00:33:07.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.203+0000 7ff0e57fa640 1 -- 172.21.15.38:0/371367347 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff0f0024050 con 0x5647fda3a2f0 2024-09-16T00:33:07.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.203+0000 7ff105edf740 10 client.15417 fetch_fsmap learned FSMap version 26 2024-09-16T00:33:07.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.203+0000 7ff105edf740 1 -- 172.21.15.38:0/371367347 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5647fda530f0 con 0x5647fda3a2f0 2024-09-16T00:33:07.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.203+0000 7ff0e57fa640 1 -- 172.21.15.38:0/371367347 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0f8005500 con 0x5647fda3a2f0 2024-09-16T00:33:07.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.203+0000 7ff0e57fa640 1 -- 172.21.15.38:0/371367347 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff0f0012450 con 0x5647fda3a2f0 2024-09-16T00:33:07.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.203+0000 7ff105edf740 10 client.15417 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:33:07.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.203+0000 7ff105edf740 10 client.15417 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-16T00:33:07.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.203+0000 7ff105edf740 20 client.15417 populate_metadata read hostname 'smithi038' 2024-09-16T00:33:07.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.203+0000 7ff105edf740 1 --2- 172.21.15.38:0/371367347 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x5647fda55770 0x5647fda56be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:07.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.203+0000 7ff105edf740 4 client.15417 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:33:07.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.203+0000 7ff105edf740 1 -- 172.21.15.38:0/371367347 --> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647fd9ad600 con 0x5647fda55770 2024-09-16T00:33:07.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.203+0000 7ff1010a1640 1 --2- 172.21.15.38:0/371367347 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x5647fda55770 0x5647fda56be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:07.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.204+0000 7ff1010a1640 1 --2- 172.21.15.38:0/371367347 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x5647fda55770 0x5647fda56be0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=38f8231f71b7dc2 server_cookie=5a761e1217898c3d in_seq=0 out_seq=0 2024-09-16T00:33:07.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.204+0000 7ff0e57fa640 10 client.15417 ms_handle_connect on v2:172.21.15.73:6836/2681081265 2024-09-16T00:33:07.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.204+0000 7ff0e57fa640 1 -- 172.21.15.38:0/371367347 <== mds.1 v2:172.21.15.73:6836/2681081265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5647fd9ad600 con 0x5647fda55770 2024-09-16T00:33:07.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.204+0000 7ff0e57fa640 10 client.15417 handle_command_reply: tid=0 2024-09-16T00:33:07.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.204+0000 7ff105edf740 1 client.15417 shutdown 2024-09-16T00:33:07.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.205+0000 7ff105edf740 1 -- 172.21.15.38:0/371367347 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5647fda3a2f0 msgr2=0x5647fda41790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:07.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.205+0000 7ff105edf740 1 --2- 172.21.15.38:0/371367347 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5647fda3a2f0 0x5647fda41790 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0011710 tx=0x7ff0f0011bd0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.205+0000 7ff105edf740 1 -- 172.21.15.38:0/371367347 shutdown_connections 2024-09-16T00:33:07.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.205+0000 7ff105edf740 1 --2- 172.21.15.38:0/371367347 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647fda44bb0 0x5647fda41cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.205+0000 7ff105edf740 1 --2- 172.21.15.38:0/371367347 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x5647fda55770 0x5647fda56be0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.224 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.205+0000 7ff105edf740 1 --2- 172.21.15.38:0/371367347 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5647fda3a2f0 0x5647fda41790 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.205+0000 7ff105edf740 1 --2- 172.21.15.38:0/371367347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647fda399d0 0x5647fda44440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.205+0000 7ff0e57fa640 0 client.15417 ms_handle_reset on v2:172.21.15.73:6836/2681081265 2024-09-16T00:33:07.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.205+0000 7ff105edf740 1 -- 172.21.15.38:0/371367347 >> 172.21.15.38:0/371367347 conn(0x5647fda35f60 msgr2=0x5647fda3bc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:07.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.205+0000 7ff105edf740 1 -- 172.21.15.38:0/371367347 shutdown_connections 2024-09-16T00:33:07.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.205+0000 7ff105edf740 1 -- 172.21.15.38:0/371367347 wait complete. 2024-09-16T00:33:07.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.205+0000 7ff105edf740 20 client.15417 trim_cache size 0 max 16384 2024-09-16T00:33:07.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.223+0000 7ff105edf740 1 Processor -- start 2024-09-16T00:33:07.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.223+0000 7ff105edf740 1 -- start start 2024-09-16T00:33:07.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.223+0000 7ff105edf740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647fda399d0 0x5647fda56be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:07.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.223+0000 7ff105edf740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647fda3a2f0 0x5647fda5eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:07.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.223+0000 7ff105edf740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5647fda44bb0 0x5647fda5f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:07.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.223+0000 7ff105edf740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5647fda54f20 con 0x5647fda44bb0 2024-09-16T00:33:07.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.223+0000 7ff105edf740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5647fda555f0 con 0x5647fda3a2f0 2024-09-16T00:33:07.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.223+0000 7ff105edf740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5647fda55cc0 con 0x5647fda399d0 2024-09-16T00:33:07.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.223+0000 7ff1010a1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5647fda44bb0 0x5647fda5f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:07.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.223+0000 7ff1010a1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5647fda44bb0 0x5647fda5f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40866/0 (socket says 172.21.15.38:40866) 2024-09-16T00:33:07.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.223+0000 7ff1010a1640 1 -- 172.21.15.38:0/2896993613 learned_addr learned my addr 172.21.15.38:0/2896993613 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:07.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.224+0000 7ff1010a1640 1 --2- 172.21.15.38:0/2896993613 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647fda399d0 0x5647fda56be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:07.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.224+0000 7ff1018a2640 1 --2- 172.21.15.38:0/2896993613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647fda3a2f0 0x5647fda5eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:07.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.224+0000 7ff1010a1640 1 -- 172.21.15.38:0/2896993613 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647fda399d0 msgr2=0x5647fda56be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:07.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.224+0000 7ff1010a1640 1 --2- 172.21.15.38:0/2896993613 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647fda399d0 0x5647fda56be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.224+0000 7ff1010a1640 1 -- 172.21.15.38:0/2896993613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647fda3a2f0 msgr2=0x5647fda5eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:07.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.224+0000 7ff1010a1640 1 --2- 172.21.15.38:0/2896993613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647fda3a2f0 0x5647fda5eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.224+0000 7ff1010a1640 1 -- 172.21.15.38:0/2896993613 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647fda44060 con 0x5647fda44bb0 2024-09-16T00:33:07.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.224+0000 7ff1018a2640 1 --2- 172.21.15.38:0/2896993613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647fda3a2f0 0x5647fda5eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:07.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.224+0000 7ff1010a1640 1 --2- 172.21.15.38:0/2896993613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5647fda44bb0 0x5647fda5f400 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8004c60 tx=0x7ff0f8004b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1f0af32393c91e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:07.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.224+0000 7ff0e77fe640 1 -- 172.21.15.38:0/2896993613 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8027040 con 0x5647fda44bb0 2024-09-16T00:33:07.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.224+0000 7ff0e77fe640 1 -- 172.21.15.38:0/2896993613 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0f8032070 con 0x5647fda44bb0 2024-09-16T00:33:07.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.224+0000 7ff0e77fe640 1 -- 172.21.15.38:0/2896993613 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f8018040 con 0x5647fda44bb0 2024-09-16T00:33:07.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.225+0000 7ff105edf740 1 -- 172.21.15.38:0/2896993613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5647fda44bb0 msgr2=0x5647fda5f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:07.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.225+0000 7ff105edf740 1 --2- 172.21.15.38:0/2896993613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5647fda44bb0 0x5647fda5f400 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff0f8004c60 tx=0x7ff0f8004b90 comp rx=0 tx=0).stop 2024-09-16T00:33:07.231 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.225+0000 7ff105edf740 1 -- 172.21.15.38:0/2896993613 shutdown_connections 2024-09-16T00:33:07.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.225+0000 7ff105edf740 1 --2- 172.21.15.38:0/2896993613 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5647fda44bb0 0x5647fda5f400 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.225+0000 7ff105edf740 1 --2- 172.21.15.38:0/2896993613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647fda3a2f0 0x5647fda5eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.225+0000 7ff105edf740 1 --2- 172.21.15.38:0/2896993613 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647fda399d0 0x5647fda56be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.225+0000 7ff105edf740 1 -- 172.21.15.38:0/2896993613 >> 172.21.15.38:0/2896993613 conn(0x5647fda35f60 msgr2=0x5647fda400a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:07.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.225+0000 7ff105edf740 1 -- 172.21.15.38:0/2896993613 shutdown_connections 2024-09-16T00:33:07.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.225+0000 7ff105edf740 1 -- 172.21.15.38:0/2896993613 wait complete. 2024-09-16T00:33:07.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.225+0000 7ff105edf740 1 -- start start 2024-09-16T00:33:07.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.226+0000 7ff105edf740 1 Processor -- start 2024-09-16T00:33:07.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.226+0000 7ff105edf740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647fda399d0 0x5647fda5f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:07.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.226+0000 7ff105edf740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5647fda3a2f0 0x5647fda5a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:07.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.226+0000 7ff105edf740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647fda44bb0 0x5647fda5adf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:07.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.226+0000 7ff105edf740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5647fda56a60 con 0x5647fda3a2f0 2024-09-16T00:33:07.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.226+0000 7ff105edf740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5647fda56390 con 0x5647fda44bb0 2024-09-16T00:33:07.232 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.226+0000 7ff105edf740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5647fda57130 con 0x5647fda399d0 2024-09-16T00:33:07.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.226+0000 7ff105edf740 10 client.0 _ll_register_callbacks cb 0x7ff0f000a630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:07.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.226+0000 7ff1018a2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5647fda3a2f0 0x5647fda5a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:07.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.226+0000 7ff1018a2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5647fda3a2f0 0x5647fda5a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40868/0 (socket says 172.21.15.38:40868) 2024-09-16T00:33:07.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.226+0000 7ff1018a2640 1 -- 172.21.15.38:0/2076064704 learned_addr learned my addr 172.21.15.38:0/2076064704 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:07.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.226+0000 7ff1010a1640 1 --2- 172.21.15.38:0/2076064704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647fda44bb0 0x5647fda5adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:07.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.226+0000 7ff1010a1640 1 --2- 172.21.15.38:0/2076064704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647fda399d0 0x5647fda5f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:07.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.226+0000 7ff1018a2640 1 -- 172.21.15.38:0/2076064704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647fda399d0 msgr2=0x5647fda5f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:07.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.226+0000 7ff1018a2640 1 --2- 172.21.15.38:0/2076064704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647fda399d0 0x5647fda5f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.227+0000 7ff1018a2640 1 -- 172.21.15.38:0/2076064704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647fda44bb0 msgr2=0x5647fda5adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:07.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.227+0000 7ff1018a2640 1 --2- 172.21.15.38:0/2076064704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647fda44bb0 0x5647fda5adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.227+0000 7ff1018a2640 1 -- 172.21.15.38:0/2076064704 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647fda5b620 con 0x5647fda3a2f0 2024-09-16T00:33:07.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.227+0000 7ff1010a1640 1 --2- 172.21.15.38:0/2076064704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647fda399d0 0x5647fda5f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:33:07.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.227+0000 7ff1018a2640 1 --2- 172.21.15.38:0/2076064704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5647fda3a2f0 0x5647fda5a8b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0005c40 tx=0x7ff0f000a9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e7d014d27cf24f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:07.233 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.227+0000 7ff0e57fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:33:07.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.227+0000 7ff0e57fa640 1 -- 172.21.15.38:0/2076064704 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0f0012ce0 con 0x5647fda3a2f0 2024-09-16T00:33:07.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.227+0000 7ff0e57fa640 1 -- 172.21.15.38:0/2076064704 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7ff0f0012450 con 0x5647fda3a2f0 2024-09-16T00:33:07.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.227+0000 7ff105edf740 1 -- 172.21.15.38:0/2076064704 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0f8032070 con 0x5647fda3a2f0 2024-09-16T00:33:07.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.227+0000 7ff0e57fa640 1 -- 172.21.15.38:0/2076064704 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ff0f0012650 con 0x5647fda3a2f0 2024-09-16T00:33:07.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.227+0000 7ff105edf740 10 client.15423 fetch_fsmap learned FSMap version 26 2024-09-16T00:33:07.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.227+0000 7ff105edf740 1 -- 172.21.15.38:0/2076064704 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5647fdabfea0 con 0x5647fda3a2f0 2024-09-16T00:33:07.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.227+0000 7ff0e57fa640 1 -- 172.21.15.38:0/2076064704 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff0f8032070 con 0x5647fda3a2f0 2024-09-16T00:33:07.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.228+0000 7ff0e57fa640 1 -- 172.21.15.38:0/2076064704 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ff0f00055b0 con 0x5647fda3a2f0 2024-09-16T00:33:07.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.228+0000 7ff105edf740 10 client.15423 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:33:07.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.228+0000 7ff105edf740 10 client.15423 resolve_mds: resolved name 'h' to daemon mds.h 2024-09-16T00:33:07.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.228+0000 7ff105edf740 20 client.15423 populate_metadata read hostname 'smithi038' 2024-09-16T00:33:07.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.228+0000 7ff105edf740 1 --2- 172.21.15.38:0/2076064704 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x5647fdac4ad0 0x5647fdac37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:07.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.228+0000 7ff105edf740 4 client.15423 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:33:07.234 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.228+0000 7ff105edf740 1 -- 172.21.15.38:0/2076064704 --> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] -- command(tid 0: {"prefix": "status"}) -- 0x5647fd9a7b10 con 0x5647fdac4ad0 2024-09-16T00:33:07.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.228+0000 7ff1010a1640 1 --2- 172.21.15.38:0/2076064704 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x5647fdac4ad0 0x5647fdac37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:07.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.228+0000 7ff1010a1640 1 --2- 172.21.15.38:0/2076064704 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x5647fdac4ad0 0x5647fdac37e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8d9949035eae424d server_cookie=bf572af1ea2f1169 in_seq=0 out_seq=0 2024-09-16T00:33:07.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.228+0000 7ff0e57fa640 10 client.15423 ms_handle_connect on v2:172.21.15.73:6836/2681081265 2024-09-16T00:33:07.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.229+0000 7ff0e57fa640 1 -- 172.21.15.38:0/2076064704 <== mds.1 v2:172.21.15.73:6836/2681081265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x5647fd9a7b10 con 0x5647fdac4ad0 2024-09-16T00:33:07.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.229+0000 7ff0e57fa640 10 client.15423 handle_command_reply: tid=0 2024-09-16T00:33:07.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.229+0000 7ff105edf740 1 client.15423 shutdown 2024-09-16T00:33:07.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.229+0000 7ff105edf740 1 -- 172.21.15.38:0/2076064704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5647fda3a2f0 msgr2=0x5647fda5a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:07.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.229+0000 7ff105edf740 1 --2- 172.21.15.38:0/2076064704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5647fda3a2f0 0x5647fda5a8b0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff0f0005c40 tx=0x7ff0f000a9d0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.229+0000 7ff105edf740 1 -- 172.21.15.38:0/2076064704 shutdown_connections 2024-09-16T00:33:07.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.229+0000 7ff105edf740 1 --2- 172.21.15.38:0/2076064704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5647fda44bb0 0x5647fda5adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.229+0000 7ff105edf740 1 --2- 172.21.15.38:0/2076064704 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x5647fdac4ad0 0x5647fdac37e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.229+0000 7ff105edf740 1 --2- 172.21.15.38:0/2076064704 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5647fda3a2f0 0x5647fda5a8b0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.229+0000 7ff105edf740 1 --2- 172.21.15.38:0/2076064704 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5647fda399d0 0x5647fda5f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.235 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.229+0000 7ff0e57fa640 0 client.15423 ms_handle_reset on v2:172.21.15.73:6836/2681081265 2024-09-16T00:33:07.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.229+0000 7ff105edf740 1 -- 172.21.15.38:0/2076064704 >> 172.21.15.38:0/2076064704 conn(0x5647fda35f60 msgr2=0x5647fda42e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:07.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.229+0000 7ff105edf740 1 -- 172.21.15.38:0/2076064704 shutdown_connections 2024-09-16T00:33:07.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.229+0000 7ff105edf740 1 -- 172.21.15.38:0/2076064704 wait complete. 2024-09-16T00:33:07.236 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.229+0000 7ff105edf740 20 client.15423 trim_cache size 0 max 16384 2024-09-16T00:33:07.236 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-16T00:33:07.236 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "983b2072-73c1-11ef-bceb-c7b262605968", 2024-09-16T00:33:07.236 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 1, 2024-09-16T00:33:07.236 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24464, 2024-09-16T00:33:07.236 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-16T00:33:07.237 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-16T00:33:07.237 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-16T00:33:07.237 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 125.93636536, 2024-09-16T00:33:07.237 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 26, 2024-09-16T00:33:07.237 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 92, 2024-09-16T00:33:07.237 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 92, 2024-09-16T00:33:07.237 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 182.57349618000001 2024-09-16T00:33:07.237 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-16T00:33:07.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.230+0000 7ff103b06640 1 -- 172.21.15.38:0/1562973941 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ff0e80a32d0 msgr2=0x7ff0e80a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:07.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.230+0000 7ff103b06640 1 --2- 172.21.15.38:0/1562973941 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ff0e80a32d0 0x7ff0e80a5790 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ff0f40bc2b0 tx=0x7ff0f4005780 comp rx=0 tx=0).stop 2024-09-16T00:33:07.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.230+0000 7ff103b06640 1 -- 172.21.15.38:0/1562973941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0fc10e500 msgr2=0x7ff0fc19efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:07.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.230+0000 7ff103b06640 1 --2- 172.21.15.38:0/1562973941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0fc10e500 0x7ff0fc19efc0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7ff0f4005ab0 tx=0x7ff0f40042f0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.230+0000 7ff103b06640 1 -- 172.21.15.38:0/1562973941 shutdown_connections 2024-09-16T00:33:07.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.230+0000 7ff103b06640 1 --2- 172.21.15.38:0/1562973941 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ff0e80a32d0 0x7ff0e80a5790 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.230+0000 7ff103b06640 1 --2- 172.21.15.38:0/1562973941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff0fc10e500 0x7ff0fc19efc0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.230+0000 7ff103b06640 1 --2- 172.21.15.38:0/1562973941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff0fc101440 0x7ff0fc19ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.230+0000 7ff103b06640 1 --2- 172.21.15.38:0/1562973941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0fc100b20 0x7ff0fc19a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:07.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.230+0000 7ff103b06640 1 -- 172.21.15.38:0/1562973941 >> 172.21.15.38:0/1562973941 conn(0x7ff0fc0fc7d0 msgr2=0x7ff0fc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:07.238 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.230+0000 7ff103b06640 1 -- 172.21.15.38:0/1562973941 shutdown_connections 2024-09-16T00:33:07.239 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:07.230+0000 7ff103b06640 1 -- 172.21.15.38:0/1562973941 wait complete. 2024-09-16T00:33:07.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:06 smithi073 ceph-mon[35254]: pgmap v374: 129 pgs: 129 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-16T00:33:07.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:07 smithi122 ceph-mon[35412]: pgmap v374: 129 pgs: 129 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-16T00:33:08.002 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.b status 2024-09-16T00:33:08.258 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:08.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.733+0000 7f198fff9640 1 -- 172.21.15.38:0/3600732759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19881027a0 msgr2=0x7f19881067f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.733+0000 7f198fff9640 1 --2- 172.21.15.38:0/3600732759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19881027a0 0x7f19881067f0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1978009900 tx=0x7f197802a3f0 comp rx=0 tx=0).stop 2024-09-16T00:33:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.733+0000 7f198fff9640 1 -- 172.21.15.38:0/3600732759 shutdown_connections 2024-09-16T00:33:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.733+0000 7f198fff9640 1 --2- 172.21.15.38:0/3600732759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1988106f20 0x7f198810eaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.733+0000 7f198fff9640 1 --2- 172.21.15.38:0/3600732759 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19881027a0 0x7f19881067f0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.733+0000 7f198fff9640 1 --2- 172.21.15.38:0/3600732759 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1988101df0 0x7f19881021d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.733+0000 7f198fff9640 1 -- 172.21.15.38:0/3600732759 >> 172.21.15.38:0/3600732759 conn(0x7f19880fd8f0 msgr2=0x7f19880ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.733+0000 7f198fff9640 1 -- 172.21.15.38:0/3600732759 shutdown_connections 2024-09-16T00:33:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.733+0000 7f198fff9640 1 -- 172.21.15.38:0/3600732759 wait complete. 2024-09-16T00:33:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.734+0000 7f198fff9640 1 Processor -- start 2024-09-16T00:33:08.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.734+0000 7f198fff9640 1 -- start start 2024-09-16T00:33:08.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.734+0000 7f198fff9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1988101df0 0x7f19880744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:08.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.734+0000 7f198fff9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19881027a0 0x7f1988074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:08.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.734+0000 7f198fff9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1988106f20 0x7f1988074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:08.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.734+0000 7f198fff9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1988111f80 con 0x7f19881027a0 2024-09-16T00:33:08.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.734+0000 7f198fff9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1988111e00 con 0x7f1988106f20 2024-09-16T00:33:08.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.734+0000 7f198fff9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1988112100 con 0x7f1988101df0 2024-09-16T00:33:08.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.734+0000 7f198d594640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19881027a0 0x7f1988074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:08.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.735+0000 7f198d594640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19881027a0 0x7f1988074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40880/0 (socket says 172.21.15.38:40880) 2024-09-16T00:33:08.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.735+0000 7f198d594640 1 -- 172.21.15.38:0/2128575666 learned_addr learned my addr 172.21.15.38:0/2128575666 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:08.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.735+0000 7f198dd95640 1 --2- 172.21.15.38:0/2128575666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1988101df0 0x7f19880744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:08.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.735+0000 7f198e596640 1 --2- 172.21.15.38:0/2128575666 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1988106f20 0x7f1988074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:08.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.735+0000 7f198d594640 1 -- 172.21.15.38:0/2128575666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1988101df0 msgr2=0x7f19880744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:08.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.735+0000 7f198d594640 1 --2- 172.21.15.38:0/2128575666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1988101df0 0x7f19880744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:08.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.735+0000 7f198d594640 1 -- 172.21.15.38:0/2128575666 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1988106f20 msgr2=0x7f1988074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:08.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.735+0000 7f198d594640 1 --2- 172.21.15.38:0/2128575666 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1988106f20 0x7f1988074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:08.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.735+0000 7f198d594640 1 -- 172.21.15.38:0/2128575666 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19881a6820 con 0x7f19881027a0 2024-09-16T00:33:08.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.735+0000 7f198d594640 1 --2- 172.21.15.38:0/2128575666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19881027a0 0x7f1988074a30 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1978005ab0 tx=0x7f19780042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:08.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.735+0000 7f197effd640 1 -- 172.21.15.38:0/2128575666 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197803f030 con 0x7f19881027a0 2024-09-16T00:33:08.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.735+0000 7f197effd640 1 -- 172.21.15.38:0/2128575666 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1978003ce0 con 0x7f19881027a0 2024-09-16T00:33:08.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.735+0000 7f198fff9640 1 -- 172.21.15.38:0/2128575666 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19881a6ab0 con 0x7f19881027a0 2024-09-16T00:33:08.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.735+0000 7f197effd640 1 -- 172.21.15.38:0/2128575666 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1978035070 con 0x7f19881027a0 2024-09-16T00:33:08.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.735+0000 7f198fff9640 1 -- 172.21.15.38:0/2128575666 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19881a6f90 con 0x7f19881027a0 2024-09-16T00:33:08.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.738+0000 7f197effd640 1 -- 172.21.15.38:0/2128575666 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f1978042030 con 0x7f19881027a0 2024-09-16T00:33:08.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.738+0000 7f19923d2740 1 Processor -- start 2024-09-16T00:33:08.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.738+0000 7f19923d2740 1 -- start start 2024-09-16T00:33:08.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.738+0000 7f19923d2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564902e2d9d0 0x564902e2ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:08.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.738+0000 7f19923d2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564902e2e2f0 0x564902e38670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:08.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.738+0000 7f19923d2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902e38bb0 0x564902e3afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:08.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.738+0000 7f19923d2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564902d94090 con 0x564902e38bb0 2024-09-16T00:33:08.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.738+0000 7f19923d2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564902d3ba70 con 0x564902e2d9d0 2024-09-16T00:33:08.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.738+0000 7f19923d2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564902d90050 con 0x564902e2e2f0 2024-09-16T00:33:08.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.739+0000 7f198d594640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902e38bb0 0x564902e3afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:08.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.739+0000 7f198d594640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902e38bb0 0x564902e3afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40896/0 (socket says 172.21.15.38:40896) 2024-09-16T00:33:08.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.739+0000 7f198d594640 1 -- 172.21.15.38:0/1408567726 learned_addr learned my addr 172.21.15.38:0/1408567726 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:08.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.739+0000 7f198e596640 1 --2- 172.21.15.38:0/1408567726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564902e2d9d0 0x564902e2ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:08.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.739+0000 7f198dd95640 1 --2- 172.21.15.38:0/1408567726 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564902e2e2f0 0x564902e38670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:08.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.739+0000 7f197effd640 1 --2- 172.21.15.38:0/2128575666 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f19740a32d0 0x7f19740a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:08.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.739+0000 7f198e596640 1 -- 172.21.15.38:0/1408567726 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564902e2e2f0 msgr2=0x564902e38670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:08.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.739+0000 7f198e596640 1 --2- 172.21.15.38:0/1408567726 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564902e2e2f0 0x564902e38670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:08.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.739+0000 7f198e596640 1 -- 172.21.15.38:0/1408567726 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902e38bb0 msgr2=0x564902e3afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:08.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.739+0000 7f198e596640 1 --2- 172.21.15.38:0/1408567726 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902e38bb0 0x564902e3afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:08.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.739+0000 7f198e596640 1 -- 172.21.15.38:0/1408567726 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564902d48130 con 0x564902e2d9d0 2024-09-16T00:33:08.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.739+0000 7f198e596640 1 --2- 172.21.15.38:0/2128575666 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f19740a32d0 0x7f19740a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:08.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.739+0000 7f197effd640 1 -- 172.21.15.38:0/2128575666 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f19780f06f0 con 0x7f19881027a0 2024-09-16T00:33:08.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.739+0000 7f198e596640 1 --2- 172.21.15.38:0/1408567726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564902e2d9d0 0x564902e2ddb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1980007ec0 tx=0x7f198000f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9c55e16b236c9fd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:08.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.740+0000 7f198e596640 1 --2- 172.21.15.38:0/2128575666 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f19740a32d0 0x7f19740a5790 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1980020000 tx=0x7f198001d000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:08.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.740+0000 7f1973fff640 1 -- 172.21.15.38:0/1408567726 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19800122d0 con 0x564902e2d9d0 2024-09-16T00:33:08.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.740+0000 7f1973fff640 1 -- 172.21.15.38:0/1408567726 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1980019a40 con 0x564902e2d9d0 2024-09-16T00:33:08.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.740+0000 7f1973fff640 1 -- 172.21.15.38:0/1408567726 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1980010920 con 0x564902e2d9d0 2024-09-16T00:33:08.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.740+0000 7f19923d2740 1 -- 172.21.15.38:0/1408567726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564902e2d9d0 msgr2=0x564902e2ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:08.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.740+0000 7f19923d2740 1 --2- 172.21.15.38:0/1408567726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564902e2d9d0 0x564902e2ddb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1980007ec0 tx=0x7f198000f790 comp rx=0 tx=0).stop 2024-09-16T00:33:08.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.740+0000 7f19923d2740 1 -- 172.21.15.38:0/1408567726 shutdown_connections 2024-09-16T00:33:08.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.740+0000 7f19923d2740 1 --2- 172.21.15.38:0/1408567726 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902e38bb0 0x564902e3afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:08.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.740+0000 7f19923d2740 1 --2- 172.21.15.38:0/1408567726 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564902e2e2f0 0x564902e38670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:08.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.740+0000 7f19923d2740 1 --2- 172.21.15.38:0/1408567726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564902e2d9d0 0x564902e2ddb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:08.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.740+0000 7f19923d2740 1 -- 172.21.15.38:0/1408567726 >> 172.21.15.38:0/1408567726 conn(0x564902e29f60 msgr2=0x564902e2a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:08.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.741+0000 7f19923d2740 1 -- 172.21.15.38:0/1408567726 shutdown_connections 2024-09-16T00:33:08.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.741+0000 7f19923d2740 1 -- 172.21.15.38:0/1408567726 wait complete. 2024-09-16T00:33:08.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.741+0000 7f19923d2740 1 -- start start 2024-09-16T00:33:08.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.741+0000 7f19923d2740 1 Processor -- start 2024-09-16T00:33:08.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.741+0000 7f19923d2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564902e2d9d0 0x564902e38480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:08.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.741+0000 7f19923d2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902e2e2f0 0x564902e357d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:08.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.741+0000 7f19923d2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564902e38bb0 0x564902e35d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:08.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.741+0000 7f19923d2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564902e3b780 con 0x564902e2e2f0 2024-09-16T00:33:08.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.741+0000 7f19923d2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564902d4ef20 con 0x564902e2d9d0 2024-09-16T00:33:08.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.741+0000 7f19923d2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564902e3b900 con 0x564902e38bb0 2024-09-16T00:33:08.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.741+0000 7f19923d2740 10 client.0 _ll_register_callbacks cb 0x564902d18030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:08.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.742+0000 7f198d594640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902e2e2f0 0x564902e357d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:08.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.742+0000 7f198d594640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902e2e2f0 0x564902e357d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40904/0 (socket says 172.21.15.38:40904) 2024-09-16T00:33:08.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.742+0000 7f198d594640 1 -- 172.21.15.38:0/1368607194 learned_addr learned my addr 172.21.15.38:0/1368607194 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:08.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.742+0000 7f198dd95640 1 --2- 172.21.15.38:0/1368607194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564902e2d9d0 0x564902e38480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:08.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.742+0000 7f198e596640 1 --2- 172.21.15.38:0/1368607194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564902e38bb0 0x564902e35d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:08.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.742+0000 7f198d594640 1 -- 172.21.15.38:0/1368607194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564902e38bb0 msgr2=0x564902e35d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:08.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.742+0000 7f198d594640 1 --2- 172.21.15.38:0/1368607194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564902e38bb0 0x564902e35d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:08.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.742+0000 7f198d594640 1 -- 172.21.15.38:0/1368607194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564902e2d9d0 msgr2=0x564902e38480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:08.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.742+0000 7f198d594640 1 --2- 172.21.15.38:0/1368607194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564902e2d9d0 0x564902e38480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:08.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.742+0000 7f198d594640 1 -- 172.21.15.38:0/1368607194 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564902b052b0 con 0x564902e2e2f0 2024-09-16T00:33:08.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.742+0000 7f198e596640 1 --2- 172.21.15.38:0/1368607194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564902e38bb0 0x564902e35d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:08.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.742+0000 7f198d594640 1 --2- 172.21.15.38:0/1368607194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902e2e2f0 0x564902e357d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f197804b000 tx=0x7f1978005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad405e0c04be125f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:08.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.742+0000 7f19717fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:33:08.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.742+0000 7f19717fa640 1 -- 172.21.15.38:0/1368607194 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19780f6020 con 0x564902e2e2f0 2024-09-16T00:33:08.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.742+0000 7f19717fa640 1 -- 172.21.15.38:0/1368607194 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f197803b460 con 0x564902e2e2f0 2024-09-16T00:33:08.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.742+0000 7f19923d2740 1 -- 172.21.15.38:0/1368607194 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1980019a40 con 0x564902e2e2f0 2024-09-16T00:33:08.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.742+0000 7f19717fa640 1 -- 172.21.15.38:0/1368607194 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f1978049070 con 0x564902e2e2f0 2024-09-16T00:33:08.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.742+0000 7f19923d2740 10 client.15435 fetch_fsmap learned FSMap version 26 2024-09-16T00:33:08.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.742+0000 7f19923d2740 1 -- 172.21.15.38:0/1368607194 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564902e470d0 con 0x564902e2e2f0 2024-09-16T00:33:08.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.743+0000 7f19717fa640 1 -- 172.21.15.38:0/1368607194 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1980019a40 con 0x564902e2e2f0 2024-09-16T00:33:08.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.743+0000 7f19717fa640 1 -- 172.21.15.38:0/1368607194 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f19780f2ce0 con 0x564902e2e2f0 2024-09-16T00:33:08.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.743+0000 7f19923d2740 10 client.15435 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:33:08.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.743+0000 7f19923d2740 10 client.15435 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T00:33:08.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.743+0000 7f19923d2740 20 client.15435 populate_metadata read hostname 'smithi038' 2024-09-16T00:33:08.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.743+0000 7f19923d2740 1 --2- 172.21.15.38:0/1368607194 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x564902e49920 0x564902e4ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:08.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.743+0000 7f19923d2740 4 client.15435 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:33:08.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.743+0000 7f19923d2740 1 -- 172.21.15.38:0/1368607194 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564902da1600 con 0x564902e49920 2024-09-16T00:33:08.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.743+0000 7f198dd95640 1 --2- 172.21.15.38:0/1368607194 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x564902e49920 0x564902e4ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:08.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.744+0000 7f19717fa640 10 client.15435 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:33:08.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.744+0000 7f198dd95640 1 --2- 172.21.15.38:0/1368607194 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x564902e49920 0x564902e4ad90 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b21b58d77a7372 server_cookie=69982f1de61bad3b in_seq=0 out_seq=0 2024-09-16T00:33:08.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.744+0000 7f19717fa640 1 -- 172.21.15.38:0/1368607194 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564902da1600 con 0x564902e49920 2024-09-16T00:33:08.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.744+0000 7f19717fa640 10 client.15435 handle_command_reply: tid=0 2024-09-16T00:33:08.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.744+0000 7f19923d2740 1 client.15435 shutdown 2024-09-16T00:33:08.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.744+0000 7f19923d2740 1 -- 172.21.15.38:0/1368607194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902e2e2f0 msgr2=0x564902e357d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:08.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.744+0000 7f19923d2740 1 --2- 172.21.15.38:0/1368607194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902e2e2f0 0x564902e357d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f197804b000 tx=0x7f1978005d50 comp rx=0 tx=0).stop 2024-09-16T00:33:08.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.745+0000 7f19923d2740 1 -- 172.21.15.38:0/1368607194 shutdown_connections 2024-09-16T00:33:08.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.745+0000 7f19923d2740 1 --2- 172.21.15.38:0/1368607194 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564902e38bb0 0x564902e35d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:08.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.745+0000 7f19923d2740 1 --2- 172.21.15.38:0/1368607194 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x564902e49920 0x564902e4ad90 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:08.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.745+0000 7f19923d2740 1 --2- 172.21.15.38:0/1368607194 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902e2e2f0 0x564902e357d0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:08.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.745+0000 7f19923d2740 1 --2- 172.21.15.38:0/1368607194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564902e2d9d0 0x564902e38480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:08.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.745+0000 7f19923d2740 1 -- 172.21.15.38:0/1368607194 >> 172.21.15.38:0/1368607194 conn(0x564902e29f60 msgr2=0x564902e2fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:08.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.745+0000 7f19717fa640 0 client.15435 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:33:08.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.745+0000 7f19923d2740 1 -- 172.21.15.38:0/1368607194 shutdown_connections 2024-09-16T00:33:08.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.745+0000 7f19923d2740 1 -- 172.21.15.38:0/1368607194 wait complete. 2024-09-16T00:33:08.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.745+0000 7f19923d2740 20 client.15435 trim_cache size 0 max 16384 2024-09-16T00:33:08.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.762+0000 7f19923d2740 1 Processor -- start 2024-09-16T00:33:08.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.762+0000 7f19923d2740 1 -- start start 2024-09-16T00:33:08.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.763+0000 7f19923d2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564902e2d9d0 0x564902e49250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:08.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.763+0000 7f19923d2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902e2e2f0 0x564902e49ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:08.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.763+0000 7f19923d2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564902e38bb0 0x564902e340a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:08.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.763+0000 7f19923d2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564902e490d0 con 0x564902e2e2f0 2024-09-16T00:33:08.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.763+0000 7f19923d2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564902e497a0 con 0x564902e38bb0 2024-09-16T00:33:08.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.763+0000 7f19923d2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564902e49e70 con 0x564902e2d9d0 2024-09-16T00:33:08.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.763+0000 7f198d594640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902e2e2f0 0x564902e49ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:08.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.763+0000 7f198d594640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902e2e2f0 0x564902e49ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40920/0 (socket says 172.21.15.38:40920) 2024-09-16T00:33:08.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.763+0000 7f198d594640 1 -- 172.21.15.38:0/2581996554 learned_addr learned my addr 172.21.15.38:0/2581996554 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:08.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.763+0000 7f198e596640 1 --2- 172.21.15.38:0/2581996554 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564902e38bb0 0x564902e340a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:08.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.763+0000 7f198dd95640 1 --2- 172.21.15.38:0/2581996554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564902e2d9d0 0x564902e49250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:08.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.763+0000 7f198d594640 1 -- 172.21.15.38:0/2581996554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564902e2d9d0 msgr2=0x564902e49250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:08.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.763+0000 7f198d594640 1 --2- 172.21.15.38:0/2581996554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564902e2d9d0 0x564902e49250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:08.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.763+0000 7f198d594640 1 -- 172.21.15.38:0/2581996554 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564902e38bb0 msgr2=0x564902e340a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:08.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.763+0000 7f198d594640 1 --2- 172.21.15.38:0/2581996554 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564902e38bb0 0x564902e340a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:08.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.763+0000 7f198d594640 1 -- 172.21.15.38:0/2581996554 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564902e380a0 con 0x564902e2e2f0 2024-09-16T00:33:08.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.763+0000 7f198e596640 1 --2- 172.21.15.38:0/2581996554 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564902e38bb0 0x564902e340a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:08.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.763+0000 7f198d594640 1 --2- 172.21.15.38:0/2581996554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902e2e2f0 0x564902e49ff0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1978005ec0 tx=0x7f19780ee180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84b278342f1baeb0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:08.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.763+0000 7f19737fe640 1 -- 172.21.15.38:0/2581996554 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19780d9030 con 0x564902e2e2f0 2024-09-16T00:33:08.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.764+0000 7f19737fe640 1 -- 172.21.15.38:0/2581996554 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19780c0be0 con 0x564902e2e2f0 2024-09-16T00:33:08.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.764+0000 7f19737fe640 1 -- 172.21.15.38:0/2581996554 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f197803bd00 con 0x564902e2e2f0 2024-09-16T00:33:08.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.764+0000 7f19923d2740 1 -- 172.21.15.38:0/2581996554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902e2e2f0 msgr2=0x564902e49ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:08.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.764+0000 7f19923d2740 1 --2- 172.21.15.38:0/2581996554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902e2e2f0 0x564902e49ff0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1978005ec0 tx=0x7f19780ee180 comp rx=0 tx=0).stop 2024-09-16T00:33:08.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.764+0000 7f19923d2740 1 -- 172.21.15.38:0/2581996554 shutdown_connections 2024-09-16T00:33:08.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.764+0000 7f19923d2740 1 --2- 172.21.15.38:0/2581996554 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564902e38bb0 0x564902e340a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:08.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.764+0000 7f19923d2740 1 --2- 172.21.15.38:0/2581996554 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902e2e2f0 0x564902e49ff0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:08.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.764+0000 7f19923d2740 1 --2- 172.21.15.38:0/2581996554 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564902e2d9d0 0x564902e49250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:08.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.764+0000 7f19923d2740 1 -- 172.21.15.38:0/2581996554 >> 172.21.15.38:0/2581996554 conn(0x564902e29f60 msgr2=0x564902e24ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:08.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.764+0000 7f19923d2740 1 -- 172.21.15.38:0/2581996554 shutdown_connections 2024-09-16T00:33:08.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.764+0000 7f19923d2740 1 -- 172.21.15.38:0/2581996554 wait complete. 2024-09-16T00:33:08.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.764+0000 7f19923d2740 1 -- start start 2024-09-16T00:33:08.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.765+0000 7f19923d2740 1 Processor -- start 2024-09-16T00:33:08.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.765+0000 7f19923d2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564902e2d9d0 0x564902e4bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:08.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.765+0000 7f19923d2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564902e2e2f0 0x564902e4c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:08.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.765+0000 7f19923d2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902e38bb0 0x564902e4ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:08.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.765+0000 7f19923d2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564902e4ac10 con 0x564902e38bb0 2024-09-16T00:33:08.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.765+0000 7f19923d2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564902e4a540 con 0x564902e2e2f0 2024-09-16T00:33:08.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.765+0000 7f19923d2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564902e4b2e0 con 0x564902e2d9d0 2024-09-16T00:33:08.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.765+0000 7f19923d2740 10 client.0 _ll_register_callbacks cb 0x7f19780bc6f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:08.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.765+0000 7f198e596640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902e38bb0 0x564902e4ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:08.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.765+0000 7f198e596640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902e38bb0 0x564902e4ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:40926/0 (socket says 172.21.15.38:40926) 2024-09-16T00:33:08.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.765+0000 7f198e596640 1 -- 172.21.15.38:0/1150154204 learned_addr learned my addr 172.21.15.38:0/1150154204 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:08.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.765+0000 7f198d594640 1 --2- 172.21.15.38:0/1150154204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564902e2e2f0 0x564902e4c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:08.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.765+0000 7f198dd95640 1 --2- 172.21.15.38:0/1150154204 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564902e2d9d0 0x564902e4bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:08.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.766+0000 7f198e596640 1 -- 172.21.15.38:0/1150154204 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564902e2d9d0 msgr2=0x564902e4bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:08.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.766+0000 7f198e596640 1 --2- 172.21.15.38:0/1150154204 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564902e2d9d0 0x564902e4bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:08.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.766+0000 7f198e596640 1 -- 172.21.15.38:0/1150154204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564902e2e2f0 msgr2=0x564902e4c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:08.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.766+0000 7f198e596640 1 --2- 172.21.15.38:0/1150154204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564902e2e2f0 0x564902e4c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:08.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.766+0000 7f198e596640 1 -- 172.21.15.38:0/1150154204 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564902eb2270 con 0x564902e38bb0 2024-09-16T00:33:08.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.766+0000 7f198e596640 1 --2- 172.21.15.38:0/1150154204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902e38bb0 0x564902e4ca00 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f198000fc90 tx=0x7f198001c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3db15803954e3fbd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:08.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.766+0000 7f19717fa640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:33:08.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.766+0000 7f19717fa640 1 -- 172.21.15.38:0/1150154204 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1980036060 con 0x564902e38bb0 2024-09-16T00:33:08.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.766+0000 7f19923d2740 1 -- 172.21.15.38:0/1150154204 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19780c0be0 con 0x564902e38bb0 2024-09-16T00:33:08.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.766+0000 7f19717fa640 1 -- 172.21.15.38:0/1150154204 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f1980019da0 con 0x564902e38bb0 2024-09-16T00:33:08.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.766+0000 7f19717fa640 1 -- 172.21.15.38:0/1150154204 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f198003d050 con 0x564902e38bb0 2024-09-16T00:33:08.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.766+0000 7f19923d2740 10 client.15441 fetch_fsmap learned FSMap version 26 2024-09-16T00:33:08.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.766+0000 7f19923d2740 1 -- 172.21.15.38:0/1150154204 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564902eb3550 con 0x564902e38bb0 2024-09-16T00:33:08.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.766+0000 7f19717fa640 1 -- 172.21.15.38:0/1150154204 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f19780c0be0 con 0x564902e38bb0 2024-09-16T00:33:08.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.767+0000 7f19717fa640 1 -- 172.21.15.38:0/1150154204 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f1980025440 con 0x564902e38bb0 2024-09-16T00:33:08.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.767+0000 7f19923d2740 10 client.15441 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:33:08.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.767+0000 7f19923d2740 10 client.15441 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-16T00:33:08.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.767+0000 7f19923d2740 20 client.15441 populate_metadata read hostname 'smithi038' 2024-09-16T00:33:08.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.767+0000 7f19923d2740 1 --2- 172.21.15.38:0/1150154204 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x564902eb7d90 0x564902eb6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:08.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.767+0000 7f19923d2740 4 client.15441 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-16T00:33:08.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.767+0000 7f19923d2740 1 -- 172.21.15.38:0/1150154204 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "status"}) -- 0x564902d9bb10 con 0x564902eb7d90 2024-09-16T00:33:08.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.767+0000 7f198dd95640 1 --2- 172.21.15.38:0/1150154204 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x564902eb7d90 0x564902eb6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:08.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.767+0000 7f198dd95640 1 --2- 172.21.15.38:0/1150154204 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x564902eb7d90 0x564902eb6aa0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee7e8e80338477c5 server_cookie=b1aef20ae2705ba1 in_seq=0 out_seq=0 2024-09-16T00:33:08.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.767+0000 7f19717fa640 10 client.15441 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:33:08.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.768+0000 7f19717fa640 1 -- 172.21.15.38:0/1150154204 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x564902d9bb10 con 0x564902eb7d90 2024-09-16T00:33:08.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.768+0000 7f19717fa640 10 client.15441 handle_command_reply: tid=0 2024-09-16T00:33:08.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.768+0000 7f19923d2740 1 client.15441 shutdown 2024-09-16T00:33:08.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.768+0000 7f19923d2740 1 -- 172.21.15.38:0/1150154204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902e38bb0 msgr2=0x564902e4ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:08.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.768+0000 7f19923d2740 1 --2- 172.21.15.38:0/1150154204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902e38bb0 0x564902e4ca00 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f198000fc90 tx=0x7f198001c040 comp rx=0 tx=0).stop 2024-09-16T00:33:08.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.768+0000 7f19923d2740 1 -- 172.21.15.38:0/1150154204 shutdown_connections 2024-09-16T00:33:08.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.768+0000 7f19923d2740 1 --2- 172.21.15.38:0/1150154204 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x564902eb7d90 0x564902eb6aa0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:08.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.768+0000 7f19923d2740 1 --2- 172.21.15.38:0/1150154204 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564902e38bb0 0x564902e4ca00 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:08.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.768+0000 7f19717fa640 0 client.15441 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:33:08.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.768+0000 7f19923d2740 1 --2- 172.21.15.38:0/1150154204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564902e2e2f0 0x564902e4c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:08.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.768+0000 7f19923d2740 1 --2- 172.21.15.38:0/1150154204 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564902e2d9d0 0x564902e4bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:08.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.768+0000 7f19923d2740 1 -- 172.21.15.38:0/1150154204 >> 172.21.15.38:0/1150154204 conn(0x564902e29f60 msgr2=0x564902e48420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:08.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.768+0000 7f19923d2740 1 -- 172.21.15.38:0/1150154204 shutdown_connections 2024-09-16T00:33:08.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.768+0000 7f19923d2740 1 -- 172.21.15.38:0/1150154204 wait complete. 2024-09-16T00:33:08.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.768+0000 7f19923d2740 20 client.15441 trim_cache size 0 max 16384 2024-09-16T00:33:08.776 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-16T00:33:08.776 INFO:teuthology.orchestra.run.smithi038.stdout: "cluster_fsid": "983b2072-73c1-11ef-bceb-c7b262605968", 2024-09-16T00:33:08.776 INFO:teuthology.orchestra.run.smithi038.stdout: "whoami": 0, 2024-09-16T00:33:08.776 INFO:teuthology.orchestra.run.smithi038.stdout: "id": 24494, 2024-09-16T00:33:08.776 INFO:teuthology.orchestra.run.smithi038.stdout: "want_state": "up:active", 2024-09-16T00:33:08.776 INFO:teuthology.orchestra.run.smithi038.stdout: "state": "up:active", 2024-09-16T00:33:08.776 INFO:teuthology.orchestra.run.smithi038.stdout: "fs_name": "cephfs", 2024-09-16T00:33:08.776 INFO:teuthology.orchestra.run.smithi038.stdout: "rank_uptime": 161.70554528900001, 2024-09-16T00:33:08.776 INFO:teuthology.orchestra.run.smithi038.stdout: "mdsmap_epoch": 26, 2024-09-16T00:33:08.777 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch": 92, 2024-09-16T00:33:08.777 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_epoch_barrier": 92, 2024-09-16T00:33:08.777 INFO:teuthology.orchestra.run.smithi038.stdout: "uptime": 171.624094864 2024-09-16T00:33:08.777 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-16T00:33:08.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.768+0000 7f198fff9640 1 -- 172.21.15.38:0/2128575666 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f19740a32d0 msgr2=0x7f19740a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:08.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.768+0000 7f198fff9640 1 --2- 172.21.15.38:0/2128575666 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f19740a32d0 0x7f19740a5790 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f1980020000 tx=0x7f198001d000 comp rx=0 tx=0).stop 2024-09-16T00:33:08.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.769+0000 7f198fff9640 1 -- 172.21.15.38:0/2128575666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19881027a0 msgr2=0x7f1988074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:08.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.769+0000 7f198fff9640 1 --2- 172.21.15.38:0/2128575666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19881027a0 0x7f1988074a30 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1978005ab0 tx=0x7f19780042f0 comp rx=0 tx=0).stop 2024-09-16T00:33:08.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.769+0000 7f198fff9640 1 -- 172.21.15.38:0/2128575666 shutdown_connections 2024-09-16T00:33:08.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.769+0000 7f198fff9640 1 --2- 172.21.15.38:0/2128575666 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f19740a32d0 0x7f19740a5790 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:08.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.769+0000 7f198fff9640 1 --2- 172.21.15.38:0/2128575666 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1988106f20 0x7f1988074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:08.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.769+0000 7f198fff9640 1 --2- 172.21.15.38:0/2128575666 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f19881027a0 0x7f1988074a30 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:08.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.769+0000 7f198fff9640 1 --2- 172.21.15.38:0/2128575666 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1988101df0 0x7f19880744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:08.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.769+0000 7f198fff9640 1 -- 172.21.15.38:0/2128575666 >> 172.21.15.38:0/2128575666 conn(0x7f19880fd8f0 msgr2=0x7f19880ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:08.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.769+0000 7f198fff9640 1 -- 172.21.15.38:0/2128575666 shutdown_connections 2024-09-16T00:33:08.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:08.769+0000 7f198fff9640 1 -- 172.21.15.38:0/2128575666 wait complete. 2024-09-16T00:33:09.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:09 smithi073 ceph-mon[35254]: pgmap v375: 129 pgs: 129 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-16T00:33:09.431 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:08 smithi038 ceph-mon[30945]: pgmap v375: 129 pgs: 129 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-16T00:33:09.462 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-09-16T00:33:09.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:09 smithi122 ceph-mon[35412]: pgmap v375: 129 pgs: 129 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 767 B/s wr, 5 op/s 2024-09-16T00:33:11.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:11 smithi073 ceph-mon[35254]: pgmap v376: 129 pgs: 129 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-16T00:33:11.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:11 smithi038 ceph-mon[30945]: pgmap v376: 129 pgs: 129 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-16T00:33:11.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:11 smithi122 ceph-mon[35412]: pgmap v376: 129 pgs: 129 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 853 B/s wr, 7 op/s 2024-09-16T00:33:13.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:13 smithi073 ceph-mon[35254]: pgmap v377: 129 pgs: 129 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-16T00:33:13.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:13 smithi038 ceph-mon[30945]: pgmap v377: 129 pgs: 129 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-16T00:33:13.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:13 smithi122 ceph-mon[35412]: pgmap v377: 129 pgs: 129 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-16T00:33:15.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:15 smithi073 ceph-mon[35254]: pgmap v378: 129 pgs: 129 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-16T00:33:15.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:15 smithi038 ceph-mon[30945]: pgmap v378: 129 pgs: 129 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-16T00:33:15.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:15 smithi122 ceph-mon[35412]: pgmap v378: 129 pgs: 129 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 853 B/s wr, 6 op/s 2024-09-16T00:33:17.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:17 smithi073 ceph-mon[35254]: pgmap v379: 129 pgs: 129 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-16T00:33:17.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:17 smithi038 ceph-mon[30945]: pgmap v379: 129 pgs: 129 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-16T00:33:17.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:17 smithi122 ceph-mon[35412]: pgmap v379: 129 pgs: 129 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 KiB/s rd, 767 B/s wr, 7 op/s 2024-09-16T00:33:19.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:19 smithi073 ceph-mon[35254]: pgmap v380: 129 pgs: 129 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-16T00:33:19.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:19 smithi038 ceph-mon[30945]: pgmap v380: 129 pgs: 129 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-16T00:33:19.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:19 smithi122 ceph-mon[35412]: pgmap v380: 129 pgs: 129 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-16T00:33:21.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:21 smithi073 ceph-mon[35254]: pgmap v381: 129 pgs: 129 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-16T00:33:21.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:21 smithi038 ceph-mon[30945]: pgmap v381: 129 pgs: 129 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-16T00:33:21.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:21 smithi122 ceph-mon[35412]: pgmap v381: 129 pgs: 129 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-16T00:33:23.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:23 smithi073 ceph-mon[35254]: pgmap v382: 129 pgs: 129 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T00:33:23.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:23 smithi038 ceph-mon[30945]: pgmap v382: 129 pgs: 129 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T00:33:23.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:23 smithi122 ceph-mon[35412]: pgmap v382: 129 pgs: 129 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T00:33:24.463 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.2 2024-09-16T00:33:24.463 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:33:24.726 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:25.144 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:25 smithi038 ceph-mon[30945]: pgmap v383: 129 pgs: 129 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-16T00:33:25.195 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:33:25.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.231+0000 7efd82101640 1 -- 172.21.15.38:0/2820085291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd7c109110 msgr2=0x7efd7c110c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:25.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.231+0000 7efd82101640 1 --2- 172.21.15.38:0/2820085291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd7c109110 0x7efd7c110c90 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7efd74009ca0 tx=0x7efd7402a8c0 comp rx=0 tx=0).stop 2024-09-16T00:33:25.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.231+0000 7efd82101640 1 -- 172.21.15.38:0/2820085291 shutdown_connections 2024-09-16T00:33:25.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.231+0000 7efd82101640 1 --2- 172.21.15.38:0/2820085291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd7c109110 0x7efd7c110c90 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:25.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.231+0000 7efd82101640 1 --2- 172.21.15.38:0/2820085291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd7c104990 0x7efd7c1089e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:25.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.231+0000 7efd82101640 1 --2- 172.21.15.38:0/2820085291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd7c103fe0 0x7efd7c1043c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:25.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.231+0000 7efd82101640 1 -- 172.21.15.38:0/2820085291 >> 172.21.15.38:0/2820085291 conn(0x7efd7c0ffa60 msgr2=0x7efd7c101e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:25.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.231+0000 7efd82101640 1 -- 172.21.15.38:0/2820085291 shutdown_connections 2024-09-16T00:33:25.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.231+0000 7efd82101640 1 -- 172.21.15.38:0/2820085291 wait complete. 2024-09-16T00:33:25.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.232+0000 7efd82101640 1 Processor -- start 2024-09-16T00:33:25.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.232+0000 7efd82101640 1 -- start start 2024-09-16T00:33:25.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.232+0000 7efd82101640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd7c103fe0 0x7efd7c19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:25.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.232+0000 7efd82101640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd7c104990 0x7efd7c19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:25.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.232+0000 7efd82101640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd7c109110 0x7efd7c19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:25.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.232+0000 7efd82101640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c114120 con 0x7efd7c109110 2024-09-16T00:33:25.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.232+0000 7efd82101640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c113fa0 con 0x7efd7c104990 2024-09-16T00:33:25.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.232+0000 7efd82101640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efd7c1142a0 con 0x7efd7c103fe0 2024-09-16T00:33:25.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.232+0000 7efd7bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd7c109110 0x7efd7c19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:25.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.232+0000 7efd7affd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd7c104990 0x7efd7c19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:25.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.232+0000 7efd7bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd7c109110 0x7efd7c19eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:45278/0 (socket says 172.21.15.38:45278) 2024-09-16T00:33:25.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.232+0000 7efd7bfff640 1 -- 172.21.15.38:0/2936631961 learned_addr learned my addr 172.21.15.38:0/2936631961 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:25.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.232+0000 7efd7affd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd7c104990 0x7efd7c19ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.38:40880/0 (socket says 172.21.15.38:40880) 2024-09-16T00:33:25.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.233+0000 7efd7b7fe640 1 --2- 172.21.15.38:0/2936631961 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd7c103fe0 0x7efd7c19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:25.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.233+0000 7efd7bfff640 1 -- 172.21.15.38:0/2936631961 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd7c103fe0 msgr2=0x7efd7c19a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:25.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.233+0000 7efd7bfff640 1 --2- 172.21.15.38:0/2936631961 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd7c103fe0 0x7efd7c19a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:25.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.233+0000 7efd7bfff640 1 -- 172.21.15.38:0/2936631961 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd7c104990 msgr2=0x7efd7c19ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:25.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.233+0000 7efd7bfff640 1 --2- 172.21.15.38:0/2936631961 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd7c104990 0x7efd7c19ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:25.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.233+0000 7efd7bfff640 1 -- 172.21.15.38:0/2936631961 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd7c19f5b0 con 0x7efd7c109110 2024-09-16T00:33:25.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.233+0000 7efd7bfff640 1 --2- 172.21.15.38:0/2936631961 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd7c109110 0x7efd7c19eed0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7efd74031a20 tx=0x7efd74031a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:25.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.233+0000 7efd78ff9640 1 -- 172.21.15.38:0/2936631961 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd74003c30 con 0x7efd7c109110 2024-09-16T00:33:25.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.233+0000 7efd82101640 1 -- 172.21.15.38:0/2936631961 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd7c19f840 con 0x7efd7c109110 2024-09-16T00:33:25.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.233+0000 7efd78ff9640 1 -- 172.21.15.38:0/2936631961 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efd74003dd0 con 0x7efd7c109110 2024-09-16T00:33:25.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.233+0000 7efd78ff9640 1 -- 172.21.15.38:0/2936631961 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd7402fcf0 con 0x7efd7c109110 2024-09-16T00:33:25.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.234+0000 7efd82101640 1 -- 172.21.15.38:0/2936631961 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd7c1a7180 con 0x7efd7c109110 2024-09-16T00:33:25.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.235+0000 7efd82101640 1 -- 172.21.15.38:0/2936631961 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd7c1043c0 con 0x7efd7c109110 2024-09-16T00:33:25.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.237+0000 7efd78ff9640 1 -- 172.21.15.38:0/2936631961 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7efd74004050 con 0x7efd7c109110 2024-09-16T00:33:25.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.238+0000 7efd78ff9640 1 --2- 172.21.15.38:0/2936631961 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efd640a32d0 0x7efd640a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:25.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.238+0000 7efd7b7fe640 1 --2- 172.21.15.38:0/2936631961 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efd640a32d0 0x7efd640a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:25.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.238+0000 7efd78ff9640 1 -- 172.21.15.38:0/2936631961 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7efd740f11f0 con 0x7efd7c109110 2024-09-16T00:33:25.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.238+0000 7efd7b7fe640 1 --2- 172.21.15.38:0/2936631961 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efd640a32d0 0x7efd640a5790 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7efd6c00bbe0 tx=0x7efd6c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:25.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.241+0000 7efd78ff9640 1 -- 172.21.15.38:0/2936631961 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efd740bdb70 con 0x7efd7c109110 2024-09-16T00:33:25.467 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:25 smithi122 ceph-mon[35412]: pgmap v383: 129 pgs: 129 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-16T00:33:25.471 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:25.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.489+0000 7efd82101640 1 -- 172.21.15.38:0/2936631961 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd7c10c380 con 0x7efd7c109110 2024-09-16T00:33:25.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.490+0000 7efd78ff9640 1 -- 172.21.15.38:0/2936631961 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7efd7402fe90 con 0x7efd7c109110 2024-09-16T00:33:25.490 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:33:25.490 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"btime":"2024-09-16T00:32:41:853514+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/367974527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":367974527},{"type":"v1","addr":"172.21.15.38:6835","nonce":367974527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/335077136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":335077136},{"type":"v1","addr":"172.21.15.38:6837","nonce":335077136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/3501211160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3501211160},{"type":"v1","addr":"172.21.15.38:6839","nonce":3501211160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6841/2623130915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":2623130915},{"type":"v1","addr":"172.21.15.38:6841","nonce":2623130915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/412277519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":412277519},{"type":"v1","addr":"172.21.15.122:6833","nonce":412277519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3987234757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3987234757},{"type":"v1","addr":"172.21.15.122:6835","nonce":3987234757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:30:26.764565+0000","modified":"2024-09-16T00:32:41.355828+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24440},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1210328177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1210328177},{"type":"v1","addr":"172.21.15.73:6833","nonce":1210328177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6837/1858299680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1858299680},{"type":"v1","addr":"172.21.15.122:6837","nonce":1858299680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1983975311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1983975311},{"type":"v1","addr":"172.21.15.122:6839","nonce":1983975311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.73:6835/2866363113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2866363113},{"type":"v1","addr":"172.21.15.73:6835","nonce":2866363113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.73:6837/2681081265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2681081265},{"type":"v1","addr":"172.21.15.73:6837","nonce":2681081265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.73:6839/2396224525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2396224525},{"type":"v1","addr":"172.21.15.73:6839","nonce":2396224525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24464,24440]},"id":1}]} 2024-09-16T00:33:25.491 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 26 2024-09-16T00:33:25.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.492+0000 7efd82101640 1 -- 172.21.15.38:0/2936631961 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efd640a32d0 msgr2=0x7efd640a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:25.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.492+0000 7efd82101640 1 --2- 172.21.15.38:0/2936631961 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efd640a32d0 0x7efd640a5790 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7efd6c00bbe0 tx=0x7efd6c00b390 comp rx=0 tx=0).stop 2024-09-16T00:33:25.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.492+0000 7efd82101640 1 -- 172.21.15.38:0/2936631961 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd7c109110 msgr2=0x7efd7c19eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:25.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.492+0000 7efd82101640 1 --2- 172.21.15.38:0/2936631961 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd7c109110 0x7efd7c19eed0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7efd74031a20 tx=0x7efd74031a50 comp rx=0 tx=0).stop 2024-09-16T00:33:25.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.492+0000 7efd82101640 1 -- 172.21.15.38:0/2936631961 shutdown_connections 2024-09-16T00:33:25.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.492+0000 7efd82101640 1 --2- 172.21.15.38:0/2936631961 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efd640a32d0 0x7efd640a5790 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:25.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.492+0000 7efd82101640 1 --2- 172.21.15.38:0/2936631961 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efd7c109110 0x7efd7c19eed0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:25.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.492+0000 7efd82101640 1 --2- 172.21.15.38:0/2936631961 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd7c104990 0x7efd7c19ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:25.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.492+0000 7efd82101640 1 --2- 172.21.15.38:0/2936631961 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efd7c103fe0 0x7efd7c19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:25.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.492+0000 7efd82101640 1 -- 172.21.15.38:0/2936631961 >> 172.21.15.38:0/2936631961 conn(0x7efd7c0ffa60 msgr2=0x7efd7c100d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:25.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.492+0000 7efd82101640 1 -- 172.21.15.38:0/2936631961 shutdown_connections 2024-09-16T00:33:25.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:25.492+0000 7efd82101640 1 -- 172.21.15.38:0/2936631961 wait complete. 2024-09-16T00:33:25.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:25 smithi073 ceph-mon[35254]: pgmap v383: 129 pgs: 129 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2024-09-16T00:33:26.214 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:26 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/2936631961' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:33:26.270 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db / --timeout 90 --expiration 150.2 --await-for 45.0' 2024-09-16T00:33:26.271 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db / --timeout 90 --expiration 150.2 --await-for 45.0 2024-09-16T00:33:26.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:26 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/2936631961' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:33:26.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:26 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/2936631961' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:33:26.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.815+0000 7f2069d09640 1 -- 172.21.15.38:0/4281721306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2064102ef0 msgr2=0x7f206410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:26.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.815+0000 7f2069d09640 1 --2- 172.21.15.38:0/4281721306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2064102ef0 0x7f206410da40 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f205c009ca0 tx=0x7f205c02a950 comp rx=0 tx=0).stop 2024-09-16T00:33:26.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.815+0000 7f2069d09640 1 -- 172.21.15.38:0/4281721306 shutdown_connections 2024-09-16T00:33:26.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.815+0000 7f2069d09640 1 --2- 172.21.15.38:0/4281721306 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2064102ef0 0x7f206410da40 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.815+0000 7f2069d09640 1 --2- 172.21.15.38:0/4281721306 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2064102570 0x7f20641029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.815+0000 7f2069d09640 1 --2- 172.21.15.38:0/4281721306 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2064100e20 0x7f2064101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.815+0000 7f2069d09640 1 -- 172.21.15.38:0/4281721306 >> 172.21.15.38:0/4281721306 conn(0x7f20640fc7d0 msgr2=0x7f20640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:26.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.815+0000 7f2069d09640 1 -- 172.21.15.38:0/4281721306 shutdown_connections 2024-09-16T00:33:26.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.815+0000 7f2069d09640 1 -- 172.21.15.38:0/4281721306 wait complete. 2024-09-16T00:33:26.831 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.816+0000 7f2069d09640 1 Processor -- start 2024-09-16T00:33:26.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.816+0000 7f2069d09640 1 -- start start 2024-09-16T00:33:26.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.816+0000 7f2069d09640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2064100e20 0x7f20641984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:26.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.816+0000 7f2069d09640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2064102570 0x7f2064198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:26.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.816+0000 7f2069d09640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2064102ef0 0x7f206419cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:26.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.816+0000 7f2069d09640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f2064110f80 con 0x7f2064102570 2024-09-16T00:33:26.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.816+0000 7f2069d09640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2064110e00 con 0x7f2064102ef0 2024-09-16T00:33:26.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.816+0000 7f2069d09640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2064111100 con 0x7f2064100e20 2024-09-16T00:33:26.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.816+0000 7f2062ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2064102570 0x7f2064198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:26.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.816+0000 7f2062ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2064102570 0x7f2064198a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34796/0 (socket says 172.21.15.38:34796) 2024-09-16T00:33:26.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.817+0000 7f2062ffd640 1 -- 172.21.15.38:0/372026647 learned_addr learned my addr 172.21.15.38:0/372026647 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:26.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.817+0000 7f20637fe640 1 --2- 172.21.15.38:0/372026647 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2064100e20 0x7f20641984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:26.832 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.817+0000 7f2063fff640 1 --2- 172.21.15.38:0/372026647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2064102ef0 0x7f206419cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:26.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.817+0000 7f2062ffd640 1 -- 172.21.15.38:0/372026647 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2064100e20 msgr2=0x7f20641984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:26.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.817+0000 7f2062ffd640 1 --2- 172.21.15.38:0/372026647 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2064100e20 0x7f20641984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.817+0000 7f2062ffd640 1 -- 172.21.15.38:0/372026647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2064102ef0 msgr2=0x7f206419cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:26.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.817+0000 7f2062ffd640 1 --2- 172.21.15.38:0/372026647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2064102ef0 0x7f206419cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.817+0000 7f2062ffd640 1 -- 172.21.15.38:0/372026647 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f206419d4a0 con 0x7f2064102570 2024-09-16T00:33:26.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.817+0000 7f20637fe640 1 --2- 172.21.15.38:0/372026647 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2064100e20 0x7f20641984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:26.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.817+0000 7f2062ffd640 1 --2- 172.21.15.38:0/372026647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2064102570 0x7f2064198a30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f205800d770 tx=0x7f205800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:26.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.817+0000 7f2060ff9640 1 -- 172.21.15.38:0/372026647 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2058017070 con 0x7f2064102570 2024-09-16T00:33:26.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.817+0000 7f2069d09640 1 -- 172.21.15.38:0/372026647 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f206419d790 con 0x7f2064102570 2024-09-16T00:33:26.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.817+0000 7f2060ff9640 1 -- 172.21.15.38:0/372026647 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2058005500 con 0x7f2064102570 2024-09-16T00:33:26.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.817+0000 7f2060ff9640 1 -- 172.21.15.38:0/372026647 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2058004e00 con 0x7f2064102570 2024-09-16T00:33:26.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.817+0000 7f2069d09640 1 -- 172.21.15.38:0/372026647 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20641a5070 con 0x7f2064102570 2024-09-16T00:33:26.833 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.820+0000 7f2060ff9640 1 -- 172.21.15.38:0/372026647 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f2058004fa0 con 0x7f2064102570 2024-09-16T00:33:26.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.820+0000 7f206c0e2740 1 Processor -- start 2024-09-16T00:33:26.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.820+0000 7f206c0e2740 1 -- start start 2024-09-16T00:33:26.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.820+0000 7f206c0e2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56344bd123c0 0x56344bd127a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:26.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.820+0000 7f206c0e2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56344bd12ce0 0x56344bd21550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:26.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.820+0000 7f206c0e2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56344bd21a90 0x56344bd23e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:26.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.820+0000 7f206c0e2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56344bc782e0 con 0x56344bd123c0 2024-09-16T00:33:26.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.820+0000 7f206c0e2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56344bc1fce0 con 0x56344bd21a90 2024-09-16T00:33:26.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.820+0000 7f206c0e2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56344bc742a0 con 0x56344bd12ce0 2024-09-16T00:33:26.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.821+0000 7f2063fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56344bd123c0 0x56344bd127a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:26.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.821+0000 7f2063fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56344bd123c0 0x56344bd127a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34806/0 (socket says 172.21.15.38:34806) 2024-09-16T00:33:26.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.821+0000 7f2063fff640 1 -- 172.21.15.38:0/4240587264 learned_addr learned my addr 172.21.15.38:0/4240587264 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:26.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.821+0000 7f20637fe640 1 --2- 172.21.15.38:0/4240587264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56344bd12ce0 0x56344bd21550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:26.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.821+0000 7f2062ffd640 1 --2- 172.21.15.38:0/4240587264 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56344bd21a90 0x56344bd23e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:26.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.821+0000 7f2060ff9640 1 --2- 172.21.15.38:0/372026647 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f204c0a32d0 0x7f204c0a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:26.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.821+0000 7f2063fff640 1 -- 172.21.15.38:0/4240587264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56344bd12ce0 msgr2=0x56344bd21550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:26.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.821+0000 7f2063fff640 1 --2- 172.21.15.38:0/4240587264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56344bd12ce0 0x56344bd21550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.821+0000 7f2063fff640 1 -- 172.21.15.38:0/4240587264 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56344bd21a90 msgr2=0x56344bd23e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:26.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.821+0000 7f2063fff640 1 --2- 172.21.15.38:0/4240587264 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56344bd21a90 0x56344bd23e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.821+0000 7f2063fff640 1 -- 172.21.15.38:0/4240587264 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56344bc04440 con 0x56344bd123c0 2024-09-16T00:33:26.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.821+0000 7f2060ff9640 1 -- 172.21.15.38:0/372026647 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f20580cf840 con 0x7f2064102570 2024-09-16T00:33:26.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.821+0000 7f2063fff640 1 --2- 172.21.15.38:0/4240587264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56344bd123c0 0x56344bd127a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f205c009ca0 tx=0x7f205c009820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddd6619cd3f2164 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:26.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.821+0000 7f2063fff640 1 --2- 172.21.15.38:0/372026647 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f204c0a32d0 0x7f204c0a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:26.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.822+0000 7f2049ffb640 1 -- 172.21.15.38:0/4240587264 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205c048070 con 0x56344bd123c0 2024-09-16T00:33:26.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.822+0000 7f2049ffb640 1 -- 172.21.15.38:0/4240587264 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f205c003970 con 0x56344bd123c0 2024-09-16T00:33:26.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.822+0000 7f2049ffb640 1 -- 172.21.15.38:0/4240587264 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205c034040 con 0x56344bd123c0 2024-09-16T00:33:26.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.822+0000 7f206c0e2740 1 -- 172.21.15.38:0/4240587264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56344bd123c0 msgr2=0x56344bd127a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:26.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.822+0000 7f2063fff640 1 --2- 172.21.15.38:0/372026647 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f204c0a32d0 0x7f204c0a5790 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f205c005fe0 tx=0x7f205c004990 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:26.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.822+0000 7f206c0e2740 1 --2- 172.21.15.38:0/4240587264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56344bd123c0 0x56344bd127a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f205c009ca0 tx=0x7f205c009820 comp rx=0 tx=0).stop 2024-09-16T00:33:26.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.823+0000 7f206c0e2740 1 -- 172.21.15.38:0/4240587264 shutdown_connections 2024-09-16T00:33:26.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.823+0000 7f206c0e2740 1 --2- 172.21.15.38:0/4240587264 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56344bd21a90 0x56344bd23e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.823+0000 7f206c0e2740 1 --2- 172.21.15.38:0/4240587264 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56344bd12ce0 0x56344bd21550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.823+0000 7f206c0e2740 1 --2- 172.21.15.38:0/4240587264 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56344bd123c0 0x56344bd127a0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.823+0000 7f206c0e2740 1 -- 172.21.15.38:0/4240587264 >> 172.21.15.38:0/4240587264 conn(0x56344bd0ea30 msgr2=0x56344bd0ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:26.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.823+0000 7f206c0e2740 1 -- 172.21.15.38:0/4240587264 shutdown_connections 2024-09-16T00:33:26.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.823+0000 7f206c0e2740 1 -- 172.21.15.38:0/4240587264 wait complete. 2024-09-16T00:33:26.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.823+0000 7f206c0e2740 1 -- start start 2024-09-16T00:33:26.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.823+0000 7f206c0e2740 1 Processor -- start 2024-09-16T00:33:26.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.823+0000 7f206c0e2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56344bd123c0 0x56344bd21400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:26.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.823+0000 7f206c0e2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56344bd12ce0 0x56344bd1e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:26.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.824+0000 7f206c0e2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56344bd21a90 0x56344bd1ec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:26.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.824+0000 7f206c0e2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56344bd245a0 con 0x56344bd12ce0 2024-09-16T00:33:26.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.824+0000 7f206c0e2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56344bc33050 con 0x56344bd21a90 2024-09-16T00:33:26.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.824+0000 7f206c0e2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56344bd24720 con 0x56344bd123c0 2024-09-16T00:33:26.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.824+0000 7f206c0e2740 10 client.0 _ll_register_callbacks cb 0x56344bc73160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:26.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.824+0000 7f2062ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56344bd12ce0 0x56344bd1e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:26.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.824+0000 7f2062ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56344bd12ce0 0x56344bd1e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34818/0 (socket says 172.21.15.38:34818) 2024-09-16T00:33:26.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.824+0000 7f2062ffd640 1 -- 172.21.15.38:0/4055422237 learned_addr learned my addr 172.21.15.38:0/4055422237 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:26.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.824+0000 7f2063fff640 1 --2- 172.21.15.38:0/4055422237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56344bd21a90 0x56344bd1ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:26.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.824+0000 7f20637fe640 1 --2- 172.21.15.38:0/4055422237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56344bd123c0 0x56344bd21400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:26.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.824+0000 7f2062ffd640 1 -- 172.21.15.38:0/4055422237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56344bd123c0 msgr2=0x56344bd21400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:26.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.824+0000 7f2062ffd640 1 --2- 172.21.15.38:0/4055422237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56344bd123c0 0x56344bd21400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.824+0000 7f2062ffd640 1 -- 172.21.15.38:0/4055422237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56344bd21a90 msgr2=0x56344bd1ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:26.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.824+0000 7f2062ffd640 1 --2- 172.21.15.38:0/4055422237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56344bd21a90 0x56344bd1ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.824+0000 7f2062ffd640 1 -- 172.21.15.38:0/4055422237 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56344b9e93c0 con 0x56344bd12ce0 2024-09-16T00:33:26.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.824+0000 7f2062ffd640 1 --2- 172.21.15.38:0/4055422237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56344bd12ce0 0x56344bd1e6b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f205809b910 tx=0x7f205809b2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aebab04e0a4624e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:26.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.824+0000 7f2042ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:33:26.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.824+0000 7f2042ffd640 1 -- 172.21.15.38:0/4055422237 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2058020070 con 0x56344bd12ce0 2024-09-16T00:33:26.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.824+0000 7f2042ffd640 1 -- 172.21.15.38:0/4055422237 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f20580cd0c0 con 0x56344bd12ce0 2024-09-16T00:33:26.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.824+0000 7f206c0e2740 1 -- 172.21.15.38:0/4055422237 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f205c003970 con 0x56344bd12ce0 2024-09-16T00:33:26.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.825+0000 7f2042ffd640 1 -- 172.21.15.38:0/4055422237 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f205809b980 con 0x56344bd12ce0 2024-09-16T00:33:26.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.825+0000 7f206c0e2740 10 client.15459 fetch_fsmap learned FSMap version 26 2024-09-16T00:33:26.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.825+0000 7f206c0e2740 1 -- 172.21.15.38:0/4055422237 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56344bd30000 con 0x56344bd12ce0 2024-09-16T00:33:26.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.825+0000 7f2042ffd640 1 -- 172.21.15.38:0/4055422237 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f205c003970 con 0x56344bd12ce0 2024-09-16T00:33:26.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.825+0000 7f2042ffd640 1 -- 172.21.15.38:0/4055422237 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f205809add0 con 0x56344bd12ce0 2024-09-16T00:33:26.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.825+0000 7f206c0e2740 10 client.15459 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:33:26.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.825+0000 7f206c0e2740 10 client.15459 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:33:26.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.825+0000 7f206c0e2740 20 client.15459 populate_metadata read hostname 'smithi038' 2024-09-16T00:33:26.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.825+0000 7f206c0e2740 1 --2- 172.21.15.38:0/4055422237 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x56344bd314c0 0x56344bd338b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:26.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.825+0000 7f206c0e2740 4 client.15459 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:33:26.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.825+0000 7f206c0e2740 1 -- 172.21.15.38:0/4055422237 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56344bc85850 con 0x56344bd314c0 2024-09-16T00:33:26.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.825+0000 7f20637fe640 1 --2- 172.21.15.38:0/4055422237 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x56344bd314c0 0x56344bd338b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:26.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.826+0000 7f20637fe640 1 --2- 172.21.15.38:0/4055422237 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x56344bd314c0 0x56344bd338b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=300f4c293865b372 server_cookie=16e8705110aa4390 in_seq=0 out_seq=0 2024-09-16T00:33:26.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.826+0000 7f2042ffd640 10 client.15459 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:33:26.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.826+0000 7f2042ffd640 1 -- 172.21.15.38:0/4055422237 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56344bc85850 con 0x56344bd314c0 2024-09-16T00:33:26.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.826+0000 7f2042ffd640 10 client.15459 handle_command_reply: tid=0 2024-09-16T00:33:26.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.826+0000 7f206c0e2740 1 client.15459 shutdown 2024-09-16T00:33:26.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.827+0000 7f206c0e2740 1 -- 172.21.15.38:0/4055422237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56344bd12ce0 msgr2=0x56344bd1e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:26.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.827+0000 7f206c0e2740 1 --2- 172.21.15.38:0/4055422237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56344bd12ce0 0x56344bd1e6b0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f205809b910 tx=0x7f205809b2e0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.827+0000 7f206c0e2740 1 -- 172.21.15.38:0/4055422237 shutdown_connections 2024-09-16T00:33:26.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.827+0000 7f206c0e2740 1 --2- 172.21.15.38:0/4055422237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56344bd21a90 0x56344bd1ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.827+0000 7f206c0e2740 1 --2- 172.21.15.38:0/4055422237 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x56344bd314c0 0x56344bd338b0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.827+0000 7f206c0e2740 1 --2- 172.21.15.38:0/4055422237 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56344bd12ce0 0x56344bd1e6b0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.827+0000 7f206c0e2740 1 --2- 172.21.15.38:0/4055422237 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56344bd123c0 0x56344bd21400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.827+0000 7f206c0e2740 1 -- 172.21.15.38:0/4055422237 >> 172.21.15.38:0/4055422237 conn(0x56344bd0ea30 msgr2=0x56344bd146c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:26.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.827+0000 7f2042ffd640 0 client.15459 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:33:26.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.827+0000 7f206c0e2740 1 -- 172.21.15.38:0/4055422237 shutdown_connections 2024-09-16T00:33:26.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.827+0000 7f206c0e2740 1 -- 172.21.15.38:0/4055422237 wait complete. 2024-09-16T00:33:26.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.827+0000 7f206c0e2740 20 client.15459 trim_cache size 0 max 16384 2024-09-16T00:33:26.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.846+0000 7f206c0e2740 1 Processor -- start 2024-09-16T00:33:26.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.846+0000 7f206c0e2740 1 -- start start 2024-09-16T00:33:26.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.846+0000 7f206c0e2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56344bd123c0 0x56344bd2d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:26.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.846+0000 7f206c0e2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56344bd12ce0 0x56344bd39180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:26.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.846+0000 7f206c0e2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56344bd21a90 0x56344bd396c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:26.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.846+0000 7f206c0e2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f20641af210 con 0x56344bd12ce0 2024-09-16T00:33:26.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.846+0000 7f206c0e2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f20641acae0 con 0x56344bd21a90 2024-09-16T00:33:26.847 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.846+0000 7f206c0e2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f20641aa930 con 0x56344bd123c0 2024-09-16T00:33:26.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.846+0000 7f2062ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56344bd12ce0 0x56344bd39180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:26.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.846+0000 7f2062ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56344bd12ce0 0x56344bd39180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34828/0 (socket says 172.21.15.38:34828) 2024-09-16T00:33:26.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.846+0000 7f2062ffd640 1 -- 172.21.15.38:0/6154941 learned_addr learned my addr 172.21.15.38:0/6154941 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:26.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.846+0000 7f2063fff640 1 --2- 172.21.15.38:0/6154941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56344bd21a90 0x56344bd396c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:26.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.846+0000 7f20637fe640 1 --2- 172.21.15.38:0/6154941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56344bd123c0 0x56344bd2d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:26.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.847+0000 7f2062ffd640 1 -- 172.21.15.38:0/6154941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56344bd123c0 msgr2=0x56344bd2d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:26.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.847+0000 7f2062ffd640 1 --2- 172.21.15.38:0/6154941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56344bd123c0 0x56344bd2d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.847+0000 7f2062ffd640 1 -- 172.21.15.38:0/6154941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56344bd21a90 msgr2=0x56344bd396c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:26.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.847+0000 7f2062ffd640 1 --2- 172.21.15.38:0/6154941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56344bd21a90 0x56344bd396c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.847+0000 7f2062ffd640 1 -- 172.21.15.38:0/6154941 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56344bd21770 con 0x56344bd12ce0 2024-09-16T00:33:26.848 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.847+0000 7f2062ffd640 1 --2- 172.21.15.38:0/6154941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56344bd12ce0 0x56344bd39180 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2058097de0 tx=0x7f2058099e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d3ff4ea2f05fe9 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:26.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.847+0000 7f20497fa640 1 -- 172.21.15.38:0/6154941 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2058020050 con 0x56344bd12ce0 2024-09-16T00:33:26.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.847+0000 7f20497fa640 1 -- 172.21.15.38:0/6154941 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20580046c0 con 0x56344bd12ce0 2024-09-16T00:33:26.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.847+0000 7f20497fa640 1 -- 172.21.15.38:0/6154941 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205809add0 con 0x56344bd12ce0 2024-09-16T00:33:26.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.848+0000 7f206c0e2740 1 -- 172.21.15.38:0/6154941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56344bd12ce0 msgr2=0x56344bd39180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:26.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.848+0000 7f206c0e2740 1 --2- 172.21.15.38:0/6154941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56344bd12ce0 0x56344bd39180 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2058097de0 tx=0x7f2058099e90 comp rx=0 tx=0).stop 2024-09-16T00:33:26.850 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.848+0000 7f206c0e2740 1 -- 172.21.15.38:0/6154941 shutdown_connections 2024-09-16T00:33:26.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.848+0000 7f206c0e2740 1 --2- 172.21.15.38:0/6154941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56344bd21a90 0x56344bd396c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.848+0000 7f206c0e2740 1 --2- 172.21.15.38:0/6154941 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56344bd12ce0 0x56344bd39180 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.848+0000 7f206c0e2740 1 --2- 172.21.15.38:0/6154941 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56344bd123c0 0x56344bd2d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.848+0000 7f206c0e2740 1 -- 172.21.15.38:0/6154941 >> 172.21.15.38:0/6154941 conn(0x56344bd0ea30 msgr2=0x56344bd19d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:26.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.848+0000 7f206c0e2740 1 -- 172.21.15.38:0/6154941 shutdown_connections 2024-09-16T00:33:26.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.848+0000 7f206c0e2740 1 -- 172.21.15.38:0/6154941 wait complete. 2024-09-16T00:33:26.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.849+0000 7f206c0e2740 1 -- start start 2024-09-16T00:33:26.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.849+0000 7f206c0e2740 1 Processor -- start 2024-09-16T00:33:26.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.849+0000 7f206c0e2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56344bd123c0 0x56344bd96e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:26.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.849+0000 7f206c0e2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56344bd12ce0 0x56344bd97360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:26.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.849+0000 7f206c0e2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56344bd21a90 0x56344bd978a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:26.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.849+0000 7f206c0e2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f20641aafa0 con 0x56344bd21a90 2024-09-16T00:33:26.851 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.849+0000 7f206c0e2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f20641ac410 con 0x56344bd12ce0 2024-09-16T00:33:26.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.849+0000 7f206c0e2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f20641ab670 con 0x56344bd123c0 2024-09-16T00:33:26.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.849+0000 7f206c0e2740 10 client.0 _ll_register_callbacks cb 0x7f20580056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:26.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.849+0000 7f2063fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56344bd21a90 0x56344bd978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:26.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.849+0000 7f2063fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56344bd21a90 0x56344bd978a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34844/0 (socket says 172.21.15.38:34844) 2024-09-16T00:33:26.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.849+0000 7f2063fff640 1 -- 172.21.15.38:0/2908825085 learned_addr learned my addr 172.21.15.38:0/2908825085 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:26.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.850+0000 7f20637fe640 1 --2- 172.21.15.38:0/2908825085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56344bd123c0 0x56344bd96e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:26.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.850+0000 7f2063fff640 1 -- 172.21.15.38:0/2908825085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56344bd123c0 msgr2=0x56344bd96e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:26.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.850+0000 7f2063fff640 1 --2- 172.21.15.38:0/2908825085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56344bd123c0 0x56344bd96e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.850+0000 7f2063fff640 1 -- 172.21.15.38:0/2908825085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56344bd12ce0 msgr2=0x56344bd97360 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:33:26.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.850+0000 7f2063fff640 1 --2- 172.21.15.38:0/2908825085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56344bd12ce0 0x56344bd97360 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.850+0000 7f2063fff640 1 -- 172.21.15.38:0/2908825085 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56344bd33c20 con 0x56344bd21a90 2024-09-16T00:33:26.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.850+0000 7f2063fff640 1 --2- 172.21.15.38:0/2908825085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56344bd21a90 0x56344bd978a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f205c009430 tx=0x7f205c046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=217584b44ee11da6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:26.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.850+0000 7f2042ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:33:26.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.850+0000 7f2042ffd640 1 -- 172.21.15.38:0/2908825085 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205c048040 con 0x56344bd21a90 2024-09-16T00:33:26.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.850+0000 7f2042ffd640 1 -- 172.21.15.38:0/2908825085 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f205c003b70 con 0x56344bd21a90 2024-09-16T00:33:26.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.850+0000 7f206c0e2740 1 -- 172.21.15.38:0/2908825085 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56344bc7fd60 con 0x56344bd21a90 2024-09-16T00:33:26.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.851+0000 7f2042ffd640 1 -- 172.21.15.38:0/2908825085 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f205c06a050 con 0x56344bd21a90 2024-09-16T00:33:26.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.851+0000 7f206c0e2740 10 client.15465 fetch_fsmap learned FSMap version 26 2024-09-16T00:33:26.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.851+0000 7f206c0e2740 1 -- 172.21.15.38:0/2908825085 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56344bd288c0 con 0x56344bd21a90 2024-09-16T00:33:26.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.851+0000 7f2042ffd640 1 -- 172.21.15.38:0/2908825085 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x56344bc7fd60 con 0x56344bd21a90 2024-09-16T00:33:26.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.851+0000 7f2042ffd640 1 -- 172.21.15.38:0/2908825085 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f205c059040 con 0x56344bd21a90 2024-09-16T00:33:26.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.851+0000 7f206c0e2740 10 client.15465 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:33:26.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.851+0000 7f206c0e2740 10 client.15465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:33:26.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.851+0000 7f206c0e2740 20 client.15465 populate_metadata read hostname 'smithi038' 2024-09-16T00:33:26.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.851+0000 7f206c0e2740 1 --2- 172.21.15.38:0/2908825085 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x56344bda0940 0x56344bda2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:26.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.851+0000 7f206c0e2740 4 client.15465 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:33:26.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.851+0000 7f206c0e2740 1 -- 172.21.15.38:0/2908825085 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56344bbfc210 con 0x56344bda0940 2024-09-16T00:33:26.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.852+0000 7f20637fe640 1 --2- 172.21.15.38:0/2908825085 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x56344bda0940 0x56344bda2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:26.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.852+0000 7f20637fe640 1 --2- 172.21.15.38:0/2908825085 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x56344bda0940 0x56344bda2d30 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b803af0b13bd188 server_cookie=7c212d5d5bcfbd19 in_seq=0 out_seq=0 2024-09-16T00:33:26.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.852+0000 7f2042ffd640 10 client.15465 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:33:26.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.852+0000 7f2042ffd640 1 -- 172.21.15.38:0/2908825085 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56344bbfc210 con 0x56344bda0940 2024-09-16T00:33:26.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.852+0000 7f2042ffd640 10 client.15465 handle_command_reply: tid=0 2024-09-16T00:33:26.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.852+0000 7f206c0e2740 1 client.15465 shutdown 2024-09-16T00:33:26.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.853+0000 7f206c0e2740 1 -- 172.21.15.38:0/2908825085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56344bd21a90 msgr2=0x56344bd978a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:26.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.853+0000 7f206c0e2740 1 --2- 172.21.15.38:0/2908825085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56344bd21a90 0x56344bd978a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f205c009430 tx=0x7f205c046000 comp rx=0 tx=0).stop 2024-09-16T00:33:26.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.853+0000 7f206c0e2740 1 -- 172.21.15.38:0/2908825085 shutdown_connections 2024-09-16T00:33:26.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.853+0000 7f206c0e2740 1 --2- 172.21.15.38:0/2908825085 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x56344bda0940 0x56344bda2d30 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.853+0000 7f206c0e2740 1 --2- 172.21.15.38:0/2908825085 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56344bd21a90 0x56344bd978a0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.853+0000 7f206c0e2740 1 --2- 172.21.15.38:0/2908825085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56344bd12ce0 0x56344bd97360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.853+0000 7f206c0e2740 1 --2- 172.21.15.38:0/2908825085 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56344bd123c0 0x56344bd96e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.853+0000 7f206c0e2740 1 -- 172.21.15.38:0/2908825085 >> 172.21.15.38:0/2908825085 conn(0x56344bd0ea30 msgr2=0x56344bd1ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:26.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.853+0000 7f2042ffd640 0 client.15465 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:33:26.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.853+0000 7f206c0e2740 1 -- 172.21.15.38:0/2908825085 shutdown_connections 2024-09-16T00:33:26.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.853+0000 7f206c0e2740 1 -- 172.21.15.38:0/2908825085 wait complete. 2024-09-16T00:33:26.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.853+0000 7f206c0e2740 20 client.15465 trim_cache size 0 max 16384 2024-09-16T00:33:26.857 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-16T00:33:26.857 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "no active scrubs running", 2024-09-16T00:33:26.857 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": {} 2024-09-16T00:33:26.857 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-16T00:33:26.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.854+0000 7f2069d09640 1 -- 172.21.15.38:0/372026647 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f204c0a32d0 msgr2=0x7f204c0a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:26.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.854+0000 7f2069d09640 1 --2- 172.21.15.38:0/372026647 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f204c0a32d0 0x7f204c0a5790 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f205c005fe0 tx=0x7f205c004990 comp rx=0 tx=0).stop 2024-09-16T00:33:26.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.854+0000 7f2069d09640 1 -- 172.21.15.38:0/372026647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2064102570 msgr2=0x7f2064198a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:26.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.854+0000 7f2069d09640 1 --2- 172.21.15.38:0/372026647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2064102570 0x7f2064198a30 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f205800d770 tx=0x7f205800dc30 comp rx=0 tx=0).stop 2024-09-16T00:33:26.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.854+0000 7f2069d09640 1 -- 172.21.15.38:0/372026647 shutdown_connections 2024-09-16T00:33:26.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.854+0000 7f2069d09640 1 --2- 172.21.15.38:0/372026647 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f204c0a32d0 0x7f204c0a5790 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.854+0000 7f2069d09640 1 --2- 172.21.15.38:0/372026647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2064102ef0 0x7f206419cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.854+0000 7f2069d09640 1 --2- 172.21.15.38:0/372026647 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2064102570 0x7f2064198a30 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.854+0000 7f2069d09640 1 --2- 172.21.15.38:0/372026647 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f2064100e20 0x7f20641984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:26.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.854+0000 7f2069d09640 1 -- 172.21.15.38:0/372026647 >> 172.21.15.38:0/372026647 conn(0x7f20640fc7d0 msgr2=0x7f206406a210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:26.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.855+0000 7f2069d09640 1 -- 172.21.15.38:0/372026647 shutdown_connections 2024-09-16T00:33:26.858 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:26.855+0000 7f2069d09640 1 -- 172.21.15.38:0/372026647 wait complete. 2024-09-16T00:33:27.105 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:27.214 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:27 smithi038 ceph-mon[30945]: pgmap v384: 129 pgs: 129 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:33:27.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:27 smithi122 ceph-mon[35412]: pgmap v384: 129 pgs: 129 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:33:27.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:27 smithi073 ceph-mon[35254]: pgmap v384: 129 pgs: 129 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:33:27.714 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:33:27.715 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:33:28.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.263+0000 7efe5a305640 1 -- 172.21.15.38:0/868339792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe54103740 msgr2=0x7efe54107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:28.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.263+0000 7efe5a305640 1 --2- 172.21.15.38:0/868339792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe54103740 0x7efe54107b80 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7efe3c009930 tx=0x7efe3c02a420 comp rx=0 tx=0).stop 2024-09-16T00:33:28.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.263+0000 7efe5a305640 1 -- 172.21.15.38:0/868339792 shutdown_connections 2024-09-16T00:33:28.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.263+0000 7efe5a305640 1 --2- 172.21.15.38:0/868339792 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe54108290 0x7efe5410ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:28.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.263+0000 7efe5a305640 1 --2- 172.21.15.38:0/868339792 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe54103740 0x7efe54107b80 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:28.276 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.263+0000 7efe5a305640 1 --2- 172.21.15.38:0/868339792 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe54102d90 0x7efe54103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:28.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.263+0000 7efe5a305640 1 -- 172.21.15.38:0/868339792 >> 172.21.15.38:0/868339792 conn(0x7efe540fc7d0 msgr2=0x7efe540febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:28.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.263+0000 7efe5a305640 1 -- 172.21.15.38:0/868339792 shutdown_connections 2024-09-16T00:33:28.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.263+0000 7efe5a305640 1 -- 172.21.15.38:0/868339792 wait complete. 2024-09-16T00:33:28.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.264+0000 7efe5a305640 1 Processor -- start 2024-09-16T00:33:28.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.264+0000 7efe5a305640 1 -- start start 2024-09-16T00:33:28.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.264+0000 7efe5a305640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe54102d90 0x7efe541009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:28.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.264+0000 7efe5a305640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe54103740 0x7efe54100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:28.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.264+0000 7efe5a305640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe54108290 0x7efe54101470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:28.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.264+0000 7efe5a305640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efe54112090 con 0x7efe54103740 2024-09-16T00:33:28.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.264+0000 7efe5a305640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efe54111f10 con 0x7efe54108290 2024-09-16T00:33:28.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.264+0000 7efe5a305640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efe54112210 con 0x7efe54102d90 2024-09-16T00:33:28.277 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.264+0000 7efe4bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe54103740 0x7efe54100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:28.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.264+0000 7efe4bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe54103740 0x7efe54100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34860/0 (socket says 172.21.15.38:34860) 2024-09-16T00:33:28.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.264+0000 7efe4bfff640 1 -- 172.21.15.38:0/3287586005 learned_addr learned my addr 172.21.15.38:0/3287586005 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:28.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.264+0000 7efe588a2640 1 --2- 172.21.15.38:0/3287586005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe54108290 0x7efe54101470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:28.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.264+0000 7efe53fff640 1 --2- 172.21.15.38:0/3287586005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe54102d90 0x7efe541009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:28.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.264+0000 7efe4bfff640 1 -- 172.21.15.38:0/3287586005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe54102d90 msgr2=0x7efe541009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:28.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.264+0000 7efe4bfff640 1 --2- 172.21.15.38:0/3287586005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe54102d90 0x7efe541009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:28.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.264+0000 7efe4bfff640 1 -- 172.21.15.38:0/3287586005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe54108290 msgr2=0x7efe54101470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:28.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.264+0000 7efe4bfff640 1 --2- 172.21.15.38:0/3287586005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe54108290 0x7efe54101470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:28.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.264+0000 7efe4bfff640 1 -- 172.21.15.38:0/3287586005 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe541a6a80 con 0x7efe54103740 2024-09-16T00:33:28.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.265+0000 7efe53fff640 1 --2- 172.21.15.38:0/3287586005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe54102d90 0x7efe541009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:28.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.265+0000 7efe4bfff640 1 --2- 172.21.15.38:0/3287586005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe54103740 0x7efe54100f30 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7efe3c005940 tx=0x7efe3c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:28.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.265+0000 7efe51ffb640 1 -- 172.21.15.38:0/3287586005 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3c03f030 con 0x7efe54103740 2024-09-16T00:33:28.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.265+0000 7efe5a305640 1 -- 172.21.15.38:0/3287586005 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe541a6d10 con 0x7efe54103740 2024-09-16T00:33:28.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.265+0000 7efe51ffb640 1 -- 172.21.15.38:0/3287586005 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe3c003ad0 con 0x7efe54103740 2024-09-16T00:33:28.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.265+0000 7efe51ffb640 1 -- 172.21.15.38:0/3287586005 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3c035070 con 0x7efe54103740 2024-09-16T00:33:28.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.265+0000 7efe5a305640 1 -- 172.21.15.38:0/3287586005 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe541a71e0 con 0x7efe54103740 2024-09-16T00:33:28.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.267+0000 7efe51ffb640 1 -- 172.21.15.38:0/3287586005 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7efe3c042030 con 0x7efe54103740 2024-09-16T00:33:28.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.268+0000 7efe5c6de740 1 Processor -- start 2024-09-16T00:33:28.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.268+0000 7efe5c6de740 1 -- start start 2024-09-16T00:33:28.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.268+0000 7efe5c6de740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561891164760 0x561891164b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:28.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.268+0000 7efe5c6de740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561891165080 0x56189116f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:28.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.268+0000 7efe51ffb640 1 --2- 172.21.15.38:0/3287586005 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efe400a3350 0x7efe400a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:28.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.268+0000 7efe5c6de740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56189116f940 0x561891171d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:28.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.268+0000 7efe5c6de740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5618910ca2a0 con 0x561891164760 2024-09-16T00:33:28.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.268+0000 7efe5c6de740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x561891071b10 con 0x561891165080 2024-09-16T00:33:28.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.268+0000 7efe5c6de740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5618910c6260 con 0x56189116f940 2024-09-16T00:33:28.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.269+0000 7efe4bfff640 1 --2- 172.21.15.38:0/3287586005 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efe400a3350 0x7efe400a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:28.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.269+0000 7efe588a2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561891164760 0x561891164b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:28.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.269+0000 7efe588a2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561891164760 0x561891164b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34868/0 (socket says 172.21.15.38:34868) 2024-09-16T00:33:28.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.269+0000 7efe588a2640 1 -- 172.21.15.38:0/1738368181 learned_addr learned my addr 172.21.15.38:0/1738368181 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:28.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.269+0000 7efe53fff640 1 --2- 172.21.15.38:0/1738368181 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561891165080 0x56189116f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:28.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.269+0000 7efe51ffb640 1 -- 172.21.15.38:0/3287586005 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7efe3c0f0400 con 0x7efe54103740 2024-09-16T00:33:28.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.269+0000 7efe588a2640 1 -- 172.21.15.38:0/1738368181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56189116f940 msgr2=0x561891171d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:28.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.269+0000 7efe588a2640 1 --2- 172.21.15.38:0/1738368181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56189116f940 0x561891171d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:28.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.269+0000 7efe588a2640 1 -- 172.21.15.38:0/1738368181 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561891165080 msgr2=0x56189116f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:28.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.269+0000 7efe588a2640 1 --2- 172.21.15.38:0/1738368181 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561891165080 0x56189116f400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:28.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.269+0000 7efe588a2640 1 -- 172.21.15.38:0/1738368181 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561890e3bb90 con 0x561891164760 2024-09-16T00:33:28.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.269+0000 7efe4bfff640 1 --2- 172.21.15.38:0/3287586005 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efe400a3350 0x7efe400a5810 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0bc4d0 tx=0x7efe3c02a420 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:28.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.269+0000 7efe588a2640 1 --2- 172.21.15.38:0/1738368181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561891164760 0x561891164b40 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7efe4c005f40 tx=0x7efe4c0108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af5158314da8dc2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:28.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.270+0000 7efe4a7fc640 1 -- 172.21.15.38:0/1738368181 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c011700 con 0x561891164760 2024-09-16T00:33:28.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.270+0000 7efe4a7fc640 1 -- 172.21.15.38:0/1738368181 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe4c011d80 con 0x561891164760 2024-09-16T00:33:28.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.270+0000 7efe4a7fc640 1 -- 172.21.15.38:0/1738368181 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c018380 con 0x561891164760 2024-09-16T00:33:28.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.270+0000 7efe5c6de740 1 -- 172.21.15.38:0/1738368181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561891164760 msgr2=0x561891164b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:28.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.270+0000 7efe5c6de740 1 --2- 172.21.15.38:0/1738368181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561891164760 0x561891164b40 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7efe4c005f40 tx=0x7efe4c0108b0 comp rx=0 tx=0).stop 2024-09-16T00:33:28.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.271+0000 7efe5c6de740 1 -- 172.21.15.38:0/1738368181 shutdown_connections 2024-09-16T00:33:28.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.271+0000 7efe5c6de740 1 --2- 172.21.15.38:0/1738368181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56189116f940 0x561891171d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:28.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.271+0000 7efe5c6de740 1 --2- 172.21.15.38:0/1738368181 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561891165080 0x56189116f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:28.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.271+0000 7efe5c6de740 1 --2- 172.21.15.38:0/1738368181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561891164760 0x561891164b40 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:28.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.271+0000 7efe5c6de740 1 -- 172.21.15.38:0/1738368181 >> 172.21.15.38:0/1738368181 conn(0x561891160cf0 msgr2=0x5618911610d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:28.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.271+0000 7efe5c6de740 1 -- 172.21.15.38:0/1738368181 shutdown_connections 2024-09-16T00:33:28.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.271+0000 7efe5c6de740 1 -- 172.21.15.38:0/1738368181 wait complete. 2024-09-16T00:33:28.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.271+0000 7efe5c6de740 1 -- start start 2024-09-16T00:33:28.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.271+0000 7efe5c6de740 1 Processor -- start 2024-09-16T00:33:28.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.272+0000 7efe5c6de740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561891164760 0x56189116c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:28.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.272+0000 7efe5c6de740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561891165080 0x56189116cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:28.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.272+0000 7efe5c6de740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56189116f940 0x56189116d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:28.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.272+0000 7efe5c6de740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x561891172510 con 0x561891165080 2024-09-16T00:33:28.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.272+0000 7efe5c6de740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5618910526e0 con 0x561891164760 2024-09-16T00:33:28.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.272+0000 7efe5c6de740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x561891172690 con 0x56189116f940 2024-09-16T00:33:28.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.272+0000 7efe5c6de740 10 client.0 _ll_register_callbacks cb 0x561891076a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:28.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.272+0000 7efe53fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561891165080 0x56189116cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:28.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.272+0000 7efe53fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561891165080 0x56189116cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34872/0 (socket says 172.21.15.38:34872) 2024-09-16T00:33:28.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.272+0000 7efe53fff640 1 -- 172.21.15.38:0/1974878206 learned_addr learned my addr 172.21.15.38:0/1974878206 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:28.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.272+0000 7efe588a2640 1 --2- 172.21.15.38:0/1974878206 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561891164760 0x56189116c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:28.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.272+0000 7efe588a2640 1 --2- 172.21.15.38:0/1974878206 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56189116f940 0x56189116d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:28.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.272+0000 7efe53fff640 1 -- 172.21.15.38:0/1974878206 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56189116f940 msgr2=0x56189116d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:28.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.272+0000 7efe53fff640 1 --2- 172.21.15.38:0/1974878206 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56189116f940 0x56189116d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:28.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.272+0000 7efe53fff640 1 -- 172.21.15.38:0/1974878206 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561891164760 msgr2=0x56189116c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:28.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.272+0000 7efe53fff640 1 --2- 172.21.15.38:0/1974878206 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561891164760 0x56189116c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:28.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.272+0000 7efe53fff640 1 -- 172.21.15.38:0/1974878206 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56189117ba30 con 0x561891165080 2024-09-16T00:33:28.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.272+0000 7efe53fff640 1 --2- 172.21.15.38:0/1974878206 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561891165080 0x56189116cd30 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7efe440137c0 tx=0x7efe44013c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd77fdf3aba71aaa server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:28.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.272+0000 7efe337fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:33:28.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.273+0000 7efe337fe640 1 -- 172.21.15.38:0/1974878206 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44011d50 con 0x561891165080 2024-09-16T00:33:28.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.273+0000 7efe337fe640 1 -- 172.21.15.38:0/1974878206 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7efe44004020 con 0x561891165080 2024-09-16T00:33:28.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.273+0000 7efe5c6de740 1 -- 172.21.15.38:0/1974878206 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe4c011d80 con 0x561891165080 2024-09-16T00:33:28.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.273+0000 7efe337fe640 1 -- 172.21.15.38:0/1974878206 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efe44026050 con 0x561891165080 2024-09-16T00:33:28.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.273+0000 7efe5c6de740 10 client.15477 fetch_fsmap learned FSMap version 26 2024-09-16T00:33:28.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.273+0000 7efe5c6de740 1 -- 172.21.15.38:0/1974878206 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56189117e420 con 0x561891165080 2024-09-16T00:33:28.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.273+0000 7efe337fe640 1 -- 172.21.15.38:0/1974878206 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe4c011d80 con 0x561891165080 2024-09-16T00:33:28.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.273+0000 7efe337fe640 1 -- 172.21.15.38:0/1974878206 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7efe4401a030 con 0x561891165080 2024-09-16T00:33:28.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.273+0000 7efe5c6de740 10 client.15477 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:33:28.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.273+0000 7efe5c6de740 10 client.15477 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-16T00:33:28.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.273+0000 7efe5c6de740 20 client.15477 populate_metadata read hostname 'smithi038' 2024-09-16T00:33:28.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.273+0000 7efe5c6de740 1 --2- 172.21.15.38:0/1974878206 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x56189117f8e0 0x561891181d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:28.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.273+0000 7efe5c6de740 4 client.15477 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:33:28.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.273+0000 7efe5c6de740 1 -- 172.21.15.38:0/1974878206 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5618910d79d0 con 0x56189117f8e0 2024-09-16T00:33:28.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.274+0000 7efe588a2640 1 --2- 172.21.15.38:0/1974878206 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x56189117f8e0 0x561891181d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:28.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.274+0000 7efe337fe640 10 client.15477 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:33:28.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.274+0000 7efe588a2640 1 --2- 172.21.15.38:0/1974878206 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x56189117f8e0 0x561891181d40 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd5bc1a9b0a3cf29 server_cookie=9006f77232a53c3b in_seq=0 out_seq=0 2024-09-16T00:33:28.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.275+0000 7efe337fe640 1 -- 172.21.15.38:0/1974878206 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5618910d79d0 con 0x56189117f8e0 2024-09-16T00:33:28.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.275+0000 7efe337fe640 10 client.15477 handle_command_reply: tid=0 2024-09-16T00:33:28.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.275+0000 7efe5c6de740 1 client.15477 shutdown 2024-09-16T00:33:28.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.275+0000 7efe5c6de740 1 -- 172.21.15.38:0/1974878206 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561891165080 msgr2=0x56189116cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:28.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.275+0000 7efe5c6de740 1 --2- 172.21.15.38:0/1974878206 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561891165080 0x56189116cd30 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7efe440137c0 tx=0x7efe44013c80 comp rx=0 tx=0).stop 2024-09-16T00:33:28.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.275+0000 7efe5c6de740 1 -- 172.21.15.38:0/1974878206 shutdown_connections 2024-09-16T00:33:28.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.275+0000 7efe5c6de740 1 --2- 172.21.15.38:0/1974878206 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56189116f940 0x56189116d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:28.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.275+0000 7efe5c6de740 1 --2- 172.21.15.38:0/1974878206 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x56189117f8e0 0x561891181d40 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:28.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.275+0000 7efe5c6de740 1 --2- 172.21.15.38:0/1974878206 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561891165080 0x56189116cd30 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:28.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.275+0000 7efe5c6de740 1 --2- 172.21.15.38:0/1974878206 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561891164760 0x56189116c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:28.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.275+0000 7efe337fe640 0 client.15477 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:33:28.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.275+0000 7efe5c6de740 1 -- 172.21.15.38:0/1974878206 >> 172.21.15.38:0/1974878206 conn(0x561891160cf0 msgr2=0x561891166a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:28.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.275+0000 7efe5c6de740 1 -- 172.21.15.38:0/1974878206 shutdown_connections 2024-09-16T00:33:28.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.276+0000 7efe5c6de740 1 -- 172.21.15.38:0/1974878206 wait complete. 2024-09-16T00:33:28.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.276+0000 7efe5c6de740 20 client.15477 trim_cache size 0 max 16384 2024-09-16T00:33:28.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.311+0000 7efe5c6de740 1 Processor -- start 2024-09-16T00:33:28.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.311+0000 7efe5c6de740 1 -- start start 2024-09-16T00:33:28.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.311+0000 7efe5c6de740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561891164760 0x56189116e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:28.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.311+0000 7efe5c6de740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561891165080 0x561891167210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:28.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.311+0000 7efe5c6de740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56189116f940 0x56189116b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:28.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.311+0000 7efe5c6de740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efe541b3750 con 0x561891164760 2024-09-16T00:33:28.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.311+0000 7efe5c6de740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efe541b1020 con 0x561891165080 2024-09-16T00:33:28.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.311+0000 7efe5c6de740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efe541aee70 con 0x56189116f940 2024-09-16T00:33:28.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.311+0000 7efe588a2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561891164760 0x56189116e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:28.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.311+0000 7efe588a2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561891164760 0x56189116e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34874/0 (socket says 172.21.15.38:34874) 2024-09-16T00:33:28.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.311+0000 7efe588a2640 1 -- 172.21.15.38:0/2974208760 learned_addr learned my addr 172.21.15.38:0/2974208760 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:28.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.311+0000 7efe53fff640 1 --2- 172.21.15.38:0/2974208760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561891165080 0x561891167210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:28.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.312+0000 7efe588a2640 1 --2- 172.21.15.38:0/2974208760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56189116f940 0x56189116b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:28.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.312+0000 7efe588a2640 1 -- 172.21.15.38:0/2974208760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56189116f940 msgr2=0x56189116b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:28.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.312+0000 7efe588a2640 1 --2- 172.21.15.38:0/2974208760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56189116f940 0x56189116b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:28.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.312+0000 7efe588a2640 1 -- 172.21.15.38:0/2974208760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561891165080 msgr2=0x561891167210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:28.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.312+0000 7efe588a2640 1 --2- 172.21.15.38:0/2974208760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561891165080 0x561891167210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:28.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.312+0000 7efe588a2640 1 -- 172.21.15.38:0/2974208760 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561891161da0 con 0x561891164760 2024-09-16T00:33:28.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.312+0000 7efe588a2640 1 --2- 172.21.15.38:0/2974208760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561891164760 0x56189116e6c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7efe4c010db0 tx=0x7efe4c009e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdd5d7ae4d3173db server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:28.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.312+0000 7efe49ffb640 1 -- 172.21.15.38:0/2974208760 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c019510 con 0x561891164760 2024-09-16T00:33:28.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.312+0000 7efe49ffb640 1 -- 172.21.15.38:0/2974208760 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efe4c020040 con 0x561891164760 2024-09-16T00:33:28.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.312+0000 7efe49ffb640 1 -- 172.21.15.38:0/2974208760 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c011760 con 0x561891164760 2024-09-16T00:33:28.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.313+0000 7efe5c6de740 1 -- 172.21.15.38:0/2974208760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561891164760 msgr2=0x56189116e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:28.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.313+0000 7efe5c6de740 1 --2- 172.21.15.38:0/2974208760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561891164760 0x56189116e6c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7efe4c010db0 tx=0x7efe4c009e80 comp rx=0 tx=0).stop 2024-09-16T00:33:28.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.313+0000 7efe5c6de740 1 -- 172.21.15.38:0/2974208760 shutdown_connections 2024-09-16T00:33:28.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.313+0000 7efe5c6de740 1 --2- 172.21.15.38:0/2974208760 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56189116f940 0x56189116b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:28.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.313+0000 7efe5c6de740 1 --2- 172.21.15.38:0/2974208760 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x561891165080 0x561891167210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:28.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.313+0000 7efe5c6de740 1 --2- 172.21.15.38:0/2974208760 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561891164760 0x56189116e6c0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:28.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.313+0000 7efe5c6de740 1 -- 172.21.15.38:0/2974208760 >> 172.21.15.38:0/2974208760 conn(0x561891160cf0 msgr2=0x56189115fe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:28.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.314+0000 7efe5c6de740 1 -- 172.21.15.38:0/2974208760 shutdown_connections 2024-09-16T00:33:28.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.314+0000 7efe5c6de740 1 -- 172.21.15.38:0/2974208760 wait complete. 2024-09-16T00:33:28.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.314+0000 7efe5c6de740 1 -- start start 2024-09-16T00:33:28.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.314+0000 7efe5c6de740 1 Processor -- start 2024-09-16T00:33:28.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.315+0000 7efe5c6de740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561891164760 0x561891177940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:28.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.315+0000 7efe5c6de740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561891165080 0x561891177e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:28.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.315+0000 7efe5c6de740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56189116f940 0x561891174910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:28.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.315+0000 7efe588a2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561891164760 0x561891177940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:28.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.315+0000 7efe588a2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561891164760 0x561891177940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34876/0 (socket says 172.21.15.38:34876) 2024-09-16T00:33:28.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.316+0000 7efe588a2640 1 -- 172.21.15.38:0/1855167345 learned_addr learned my addr 172.21.15.38:0/1855167345 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:28.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.316+0000 7efe53fff640 1 --2- 172.21.15.38:0/1855167345 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561891165080 0x561891177e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:28.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.316+0000 7efe5c6de740 1 -- 172.21.15.38:0/1855167345 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efe541af4e0 con 0x561891164760 2024-09-16T00:33:28.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.316+0000 7efe5c6de740 1 -- 172.21.15.38:0/1855167345 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efe541b0950 con 0x56189116f940 2024-09-16T00:33:28.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.316+0000 7efe5c6de740 1 -- 172.21.15.38:0/1855167345 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efe541afbb0 con 0x561891165080 2024-09-16T00:33:28.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.316+0000 7efe588a2640 1 -- 172.21.15.38:0/1855167345 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561891165080 msgr2=0x561891177e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:28.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.316+0000 7efe588a2640 1 --2- 172.21.15.38:0/1855167345 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561891165080 0x561891177e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:28.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.316+0000 7efe588a2640 1 -- 172.21.15.38:0/1855167345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56189116f940 msgr2=0x561891174910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:28.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.316+0000 7efe588a2640 1 --2- 172.21.15.38:0/1855167345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56189116f940 0x561891174910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:28.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.316+0000 7efe588a2640 1 -- 172.21.15.38:0/1855167345 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561891160580 con 0x561891164760 2024-09-16T00:33:28.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.317+0000 7efe588a2640 1 --2- 172.21.15.38:0/1855167345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561891164760 0x561891177940 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7efe4c0108f0 tx=0x7efe4c005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9fab990af71f4f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:28.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.317+0000 7efe337fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:33:28.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.317+0000 7efe5c6de740 10 client.0 _ll_register_callbacks cb 0x7efe44009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:28.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.317+0000 7efe5c6de740 1 -- 172.21.15.38:0/1855167345 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe4c020040 con 0x561891164760 2024-09-16T00:33:28.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.317+0000 7efe337fe640 1 -- 172.21.15.38:0/1855167345 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c005970 con 0x561891164760 2024-09-16T00:33:28.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.317+0000 7efe337fe640 1 -- 172.21.15.38:0/1855167345 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7efe4c019510 con 0x561891164760 2024-09-16T00:33:28.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.317+0000 7efe337fe640 1 -- 172.21.15.38:0/1855167345 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efe4c003970 con 0x561891164760 2024-09-16T00:33:28.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.318+0000 7efe5c6de740 10 client.15483 fetch_fsmap learned FSMap version 26 2024-09-16T00:33:28.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.318+0000 7efe5c6de740 1 -- 172.21.15.38:0/1855167345 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({fsmap=26}) -- 0x5618911f2ec0 con 0x561891164760 2024-09-16T00:33:28.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.318+0000 7efe337fe640 1 -- 172.21.15.38:0/1855167345 <== mon.0 v2:172.21.15.38:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7efe4c003b10 con 0x561891164760 2024-09-16T00:33:28.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.318+0000 7efe5c6de740 10 client.15483 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:33:28.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.318+0000 7efe5c6de740 10 client.15483 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-16T00:33:28.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.318+0000 7efe5c6de740 20 client.15483 populate_metadata read hostname 'smithi038' 2024-09-16T00:33:28.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.318+0000 7efe5c6de740 1 --2- 172.21.15.38:0/1855167345 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5618911f3410 0x5618911f5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:28.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.318+0000 7efe5c6de740 4 client.15483 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.2, "await_for": 45.0}] 2024-09-16T00:33:28.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.318+0000 7efe5c6de740 1 -- 172.21.15.38:0/1855167345 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 90.0, "expiration": 150.2, "await_for": 45.0}) -- 0x5618910d1d20 con 0x5618911f3410 2024-09-16T00:33:28.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.318+0000 7efe53fff640 1 --2- 172.21.15.38:0/1855167345 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5618911f3410 0x5618911f5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:28.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.319+0000 7efe53fff640 1 --2- 172.21.15.38:0/1855167345 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5618911f3410 0x5618911f5870 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af0af4cf34737193 server_cookie=e5fd1a9e7fcc8b5d in_seq=0 out_seq=0 2024-09-16T00:33:28.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.319+0000 7efe337fe640 10 client.15483 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:33:28.323 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:28.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.332+0000 7efe337fe640 1 -- 172.21.15.38:0/1855167345 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x5618910d1d20 con 0x5618911f3410 2024-09-16T00:33:28.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.332+0000 7efe337fe640 10 client.15483 handle_command_reply: tid=0 2024-09-16T00:33:28.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.332+0000 7efe5c6de740 1 client.15483 shutdown 2024-09-16T00:33:28.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.332+0000 7efe5c6de740 1 -- 172.21.15.38:0/1855167345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561891164760 msgr2=0x561891177940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:28.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.332+0000 7efe5c6de740 1 --2- 172.21.15.38:0/1855167345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561891164760 0x561891177940 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7efe4c0108f0 tx=0x7efe4c005c40 comp rx=0 tx=0).stop 2024-09-16T00:33:28.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.333+0000 7efe5c6de740 1 -- 172.21.15.38:0/1855167345 shutdown_connections 2024-09-16T00:33:28.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.333+0000 7efe5c6de740 1 --2- 172.21.15.38:0/1855167345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56189116f940 0x561891174910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:28.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.333+0000 7efe5c6de740 1 --2- 172.21.15.38:0/1855167345 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x561891165080 0x561891177e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:28.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.333+0000 7efe5c6de740 1 --2- 172.21.15.38:0/1855167345 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5618911f3410 0x5618911f5870 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:28.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.333+0000 7efe5c6de740 1 --2- 172.21.15.38:0/1855167345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x561891164760 0x561891177940 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:28.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.333+0000 7efe337fe640 0 client.15483 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:33:28.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.333+0000 7efe5c6de740 1 -- 172.21.15.38:0/1855167345 >> 172.21.15.38:0/1855167345 conn(0x561891160cf0 msgr2=0x56189116c610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:28.334 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.333+0000 7efe5c6de740 1 -- 172.21.15.38:0/1855167345 shutdown_connections 2024-09-16T00:33:28.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.333+0000 7efe5c6de740 1 -- 172.21.15.38:0/1855167345 wait complete. 2024-09-16T00:33:28.335 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.333+0000 7efe5c6de740 20 client.15483 trim_cache size 0 max 16384 2024-09-16T00:33:28.335 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-16T00:33:28.335 INFO:teuthology.orchestra.run.smithi038.stdout: "epoch": 26, 2024-09-16T00:33:28.335 INFO:teuthology.orchestra.run.smithi038.stdout: "leader": 24494, 2024-09-16T00:33:28.335 INFO:teuthology.orchestra.run.smithi038.stdout: "set_version": 2, 2024-09-16T00:33:28.335 INFO:teuthology.orchestra.run.smithi038.stdout: "sets": { 2024-09-16T00:33:28.335 INFO:teuthology.orchestra.run.smithi038.stdout: "6706a54b": { 2024-09-16T00:33:28.335 INFO:teuthology.orchestra.run.smithi038.stdout: "version": 2, 2024-09-16T00:33:28.335 INFO:teuthology.orchestra.run.smithi038.stdout: "age_ref": 0.0, 2024-09-16T00:33:28.336 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-16T00:33:28.336 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "QUIESCED", 2024-09-16T00:33:28.336 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-16T00:33:28.336 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:33:28.336 INFO:teuthology.orchestra.run.smithi038.stdout: "timeout": 90.0, 2024-09-16T00:33:28.336 INFO:teuthology.orchestra.run.smithi038.stdout: "expiration": 150.2, 2024-09-16T00:33:28.336 INFO:teuthology.orchestra.run.smithi038.stdout: "members": { 2024-09-16T00:33:28.336 INFO:teuthology.orchestra.run.smithi038.stdout: "file:/": { 2024-09-16T00:33:28.336 INFO:teuthology.orchestra.run.smithi038.stdout: "excluded": false, 2024-09-16T00:33:28.336 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-16T00:33:28.336 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "QUIESCED", 2024-09-16T00:33:28.336 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-16T00:33:28.336 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:33:28.337 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:33:28.337 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:33:28.337 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:33:28.337 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:33:28.337 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-16T00:33:28.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.334+0000 7efe5a305640 1 -- 172.21.15.38:0/3287586005 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efe400a3350 msgr2=0x7efe400a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:28.337 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.334+0000 7efe5a305640 1 --2- 172.21.15.38:0/3287586005 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efe400a3350 0x7efe400a5810 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7efe3c0bc4d0 tx=0x7efe3c02a420 comp rx=0 tx=0).stop 2024-09-16T00:33:28.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.334+0000 7efe5a305640 1 -- 172.21.15.38:0/3287586005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe54103740 msgr2=0x7efe54100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:28.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.334+0000 7efe5a305640 1 --2- 172.21.15.38:0/3287586005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe54103740 0x7efe54100f30 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7efe3c005940 tx=0x7efe3c004270 comp rx=0 tx=0).stop 2024-09-16T00:33:28.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.334+0000 7efe5a305640 1 -- 172.21.15.38:0/3287586005 shutdown_connections 2024-09-16T00:33:28.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.334+0000 7efe5a305640 1 --2- 172.21.15.38:0/3287586005 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efe400a3350 0x7efe400a5810 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:28.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.334+0000 7efe5a305640 1 --2- 172.21.15.38:0/3287586005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe54108290 0x7efe54101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:28.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.334+0000 7efe5a305640 1 --2- 172.21.15.38:0/3287586005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efe54103740 0x7efe54100f30 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:28.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.334+0000 7efe5a305640 1 --2- 172.21.15.38:0/3287586005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efe54102d90 0x7efe541009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:28.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.334+0000 7efe5a305640 1 -- 172.21.15.38:0/3287586005 >> 172.21.15.38:0/3287586005 conn(0x7efe540fc7d0 msgr2=0x7efe54105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:28.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.334+0000 7efe5a305640 1 -- 172.21.15.38:0/3287586005 shutdown_connections 2024-09-16T00:33:28.338 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:28.334+0000 7efe5a305640 1 -- 172.21.15.38:0/3287586005 wait complete. 2024-09-16T00:33:29.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.068+0000 7f57d6abd640 1 -- 172.21.15.38:0/3440973423 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57d0103760 msgr2=0x7f57d0107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:29.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.068+0000 7f57d6abd640 1 --2- 172.21.15.38:0/3440973423 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57d0103760 0x7f57d0107ba0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f57bc009930 tx=0x7f57bc02a420 comp rx=0 tx=0).stop 2024-09-16T00:33:29.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.069+0000 7f57d6abd640 1 -- 172.21.15.38:0/3440973423 shutdown_connections 2024-09-16T00:33:29.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.069+0000 7f57d6abd640 1 --2- 172.21.15.38:0/3440973423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57d01082e0 0x7f57d010ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:29.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.069+0000 7f57d6abd640 1 --2- 172.21.15.38:0/3440973423 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57d0103760 0x7f57d0107ba0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:29.082 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.069+0000 7f57d6abd640 1 --2- 172.21.15.38:0/3440973423 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57d0102db0 0x7f57d0103190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:29.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.069+0000 7f57d6abd640 1 -- 172.21.15.38:0/3440973423 >> 172.21.15.38:0/3440973423 conn(0x7f57d00fc7d0 msgr2=0x7f57d00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:29.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.069+0000 7f57d6abd640 1 -- 172.21.15.38:0/3440973423 shutdown_connections 2024-09-16T00:33:29.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.069+0000 7f57d6abd640 1 -- 172.21.15.38:0/3440973423 wait complete. 2024-09-16T00:33:29.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.069+0000 7f57d6abd640 1 Processor -- start 2024-09-16T00:33:29.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.069+0000 7f57d6abd640 1 -- start start 2024-09-16T00:33:29.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.069+0000 7f57d6abd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57d0102db0 0x7f57d01985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:29.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.069+0000 7f57d6abd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57d0103760 0x7f57d0198af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:29.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.069+0000 7f57d6abd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57d01082e0 0x7f57d019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:29.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.070+0000 7f57d6abd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f57d0112050 con 0x7f57d01082e0 2024-09-16T00:33:29.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.070+0000 7f57d6abd640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f57d0111ed0 con 0x7f57d0102db0 2024-09-16T00:33:29.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.070+0000 7f57d6abd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f57d01121d0 con 0x7f57d0103760 2024-09-16T00:33:29.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.070+0000 7f57d505a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57d01082e0 0x7f57d019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:29.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.070+0000 7f57d505a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57d01082e0 0x7f57d019ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34904/0 (socket says 172.21.15.38:34904) 2024-09-16T00:33:29.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.070+0000 7f57d505a640 1 -- 172.21.15.38:0/4140434524 learned_addr learned my addr 172.21.15.38:0/4140434524 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:29.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.070+0000 7f57d4859640 1 --2- 172.21.15.38:0/4140434524 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57d0102db0 0x7f57d01985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:29.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.070+0000 7f57cffff640 1 --2- 172.21.15.38:0/4140434524 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57d0103760 0x7f57d0198af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:29.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.070+0000 7f57d505a640 1 -- 172.21.15.38:0/4140434524 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57d0103760 msgr2=0x7f57d0198af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:29.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.070+0000 7f57d505a640 1 --2- 172.21.15.38:0/4140434524 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57d0103760 0x7f57d0198af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:29.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.070+0000 7f57d505a640 1 -- 172.21.15.38:0/4140434524 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57d0102db0 msgr2=0x7f57d01985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:29.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.070+0000 7f57d505a640 1 --2- 172.21.15.38:0/4140434524 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57d0102db0 0x7f57d01985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:29.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.070+0000 7f57d505a640 1 -- 172.21.15.38:0/4140434524 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d019d560 con 0x7f57d01082e0 2024-09-16T00:33:29.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.070+0000 7f57d505a640 1 --2- 172.21.15.38:0/4140434524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57d01082e0 0x7f57d019ce80 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f57c8009fd0 tx=0x7f57c800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:29.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.070+0000 7f57cdffb640 1 -- 172.21.15.38:0/4140434524 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c8017070 con 0x7f57d01082e0 2024-09-16T00:33:29.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.070+0000 7f57d6abd640 1 -- 172.21.15.38:0/4140434524 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57d019d850 con 0x7f57d01082e0 2024-09-16T00:33:29.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.070+0000 7f57cdffb640 1 -- 172.21.15.38:0/4140434524 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f57c8003e50 con 0x7f57d01082e0 2024-09-16T00:33:29.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.070+0000 7f57cdffb640 1 -- 172.21.15.38:0/4140434524 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c80106c0 con 0x7f57d01082e0 2024-09-16T00:33:29.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.071+0000 7f57d6abd640 1 -- 172.21.15.38:0/4140434524 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57d01a5130 con 0x7f57d01082e0 2024-09-16T00:33:29.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.072+0000 7f57d6abd640 1 -- 172.21.15.38:0/4140434524 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57d0103190 con 0x7f57d01082e0 2024-09-16T00:33:29.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.073+0000 7f57cdffb640 1 -- 172.21.15.38:0/4140434524 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f57c8003970 con 0x7f57d01082e0 2024-09-16T00:33:29.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.074+0000 7f57cdffb640 1 --2- 172.21.15.38:0/4140434524 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f57c00a32d0 0x7f57c00a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:29.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.074+0000 7f57d4859640 1 --2- 172.21.15.38:0/4140434524 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f57c00a32d0 0x7f57c00a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:29.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.074+0000 7f57cdffb640 1 -- 172.21.15.38:0/4140434524 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f57c80cfa80 con 0x7f57d01082e0 2024-09-16T00:33:29.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.075+0000 7f57d4859640 1 --2- 172.21.15.38:0/4140434524 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f57c00a32d0 0x7f57c00a5790 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f57c4005d00 tx=0x7f57c400b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:29.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.078+0000 7f57cdffb640 1 -- 172.21.15.38:0/4140434524 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f57c809bc70 con 0x7f57d01082e0 2024-09-16T00:33:29.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.313+0000 7f57d6abd640 1 -- 172.21.15.38:0/4140434524 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57d010a2c0 con 0x7f57d01082e0 2024-09-16T00:33:29.315 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:29 smithi038 ceph-mon[30945]: pgmap v385: 129 pgs: 129 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:33:29.315 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.314+0000 7f57cdffb640 1 -- 172.21.15.38:0/4140434524 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f57c809ff70 con 0x7f57d01082e0 2024-09-16T00:33:29.315 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:33:29.316 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"btime":"2024-09-16T00:32:41:853514+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/367974527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":367974527},{"type":"v1","addr":"172.21.15.38:6835","nonce":367974527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/335077136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":335077136},{"type":"v1","addr":"172.21.15.38:6837","nonce":335077136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/3501211160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3501211160},{"type":"v1","addr":"172.21.15.38:6839","nonce":3501211160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6841/2623130915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":2623130915},{"type":"v1","addr":"172.21.15.38:6841","nonce":2623130915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/412277519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":412277519},{"type":"v1","addr":"172.21.15.122:6833","nonce":412277519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3987234757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3987234757},{"type":"v1","addr":"172.21.15.122:6835","nonce":3987234757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:30:26.764565+0000","modified":"2024-09-16T00:32:41.355828+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24440},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1210328177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1210328177},{"type":"v1","addr":"172.21.15.73:6833","nonce":1210328177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6837/1858299680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1858299680},{"type":"v1","addr":"172.21.15.122:6837","nonce":1858299680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1983975311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1983975311},{"type":"v1","addr":"172.21.15.122:6839","nonce":1983975311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.73:6835/2866363113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2866363113},{"type":"v1","addr":"172.21.15.73:6835","nonce":2866363113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.73:6837/2681081265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2681081265},{"type":"v1","addr":"172.21.15.73:6837","nonce":2681081265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.73:6839/2396224525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2396224525},{"type":"v1","addr":"172.21.15.73:6839","nonce":2396224525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24464,24440]},"id":1}]} 2024-09-16T00:33:29.316 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 26 2024-09-16T00:33:29.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.317+0000 7f57d6abd640 1 -- 172.21.15.38:0/4140434524 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f57c00a32d0 msgr2=0x7f57c00a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:29.317 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.317+0000 7f57d6abd640 1 --2- 172.21.15.38:0/4140434524 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f57c00a32d0 0x7f57c00a5790 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f57c4005d00 tx=0x7f57c400b410 comp rx=0 tx=0).stop 2024-09-16T00:33:29.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.317+0000 7f57d6abd640 1 -- 172.21.15.38:0/4140434524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57d01082e0 msgr2=0x7f57d019ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:29.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.317+0000 7f57d6abd640 1 --2- 172.21.15.38:0/4140434524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57d01082e0 0x7f57d019ce80 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f57c8009fd0 tx=0x7f57c800b260 comp rx=0 tx=0).stop 2024-09-16T00:33:29.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.317+0000 7f57d6abd640 1 -- 172.21.15.38:0/4140434524 shutdown_connections 2024-09-16T00:33:29.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.317+0000 7f57d6abd640 1 --2- 172.21.15.38:0/4140434524 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f57c00a32d0 0x7f57c00a5790 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:29.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.317+0000 7f57d6abd640 1 --2- 172.21.15.38:0/4140434524 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f57d01082e0 0x7f57d019ce80 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:29.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.317+0000 7f57d6abd640 1 --2- 172.21.15.38:0/4140434524 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f57d0103760 0x7f57d0198af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:29.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.317+0000 7f57d6abd640 1 --2- 172.21.15.38:0/4140434524 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57d0102db0 0x7f57d01985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:29.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.317+0000 7f57d6abd640 1 -- 172.21.15.38:0/4140434524 >> 172.21.15.38:0/4140434524 conn(0x7f57d00fc7d0 msgr2=0x7f57d0105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:29.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.317+0000 7f57d6abd640 1 -- 172.21.15.38:0/4140434524 shutdown_connections 2024-09-16T00:33:29.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:29.317+0000 7f57d6abd640 1 -- 172.21.15.38:0/4140434524 wait complete. 2024-09-16T00:33:29.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:29 smithi122 ceph-mon[35412]: pgmap v385: 129 pgs: 129 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:33:29.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:29 smithi073 ceph-mon[35254]: pgmap v385: 129 pgs: 129 active+clean; 656 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:33:30.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:30 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/4140434524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:33:30.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:30 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/4140434524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:33:30.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:30 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/4140434524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:33:31.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:31 smithi038 ceph-mon[30945]: pgmap v386: 129 pgs: 129 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-16T00:33:31.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:31 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:33:31.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:31 smithi122 ceph-mon[35412]: pgmap v386: 129 pgs: 129 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-16T00:33:31.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:31 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:33:31.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:31 smithi073 ceph-mon[35254]: pgmap v386: 129 pgs: 129 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 597 B/s wr, 7 op/s 2024-09-16T00:33:31.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:31 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:33:32.720 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '6706a54b', quiesce took 8.3 seconds. Will release after: 21.9 2024-09-16T00:33:32.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:32 smithi038 ceph-mon[30945]: pgmap v387: 129 pgs: 129 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-16T00:33:32.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:32 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:33:32.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:32 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:33:33.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:32 smithi122 ceph-mon[35412]: pgmap v387: 129 pgs: 129 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-16T00:33:33.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:32 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:33:33.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:32 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:33:33.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:32 smithi073 ceph-mon[35254]: pgmap v387: 129 pgs: 129 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-16T00:33:33.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:32 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:33:33.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:32 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:33:34.214 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:33 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:34.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:33 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:33:34.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:33 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:33:34.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:33 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:34.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:33 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:33:34.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:33 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:33:34.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:33 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:33:34.320 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:33 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:33:34.320 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:33 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:33:34.911 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:33:35.214 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:34 smithi038 ceph-mon[30945]: pgmap v388: 129 pgs: 129 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-16T00:33:35.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:34 smithi122 ceph-mon[35412]: pgmap v388: 129 pgs: 129 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-16T00:33:35.219 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:35.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:34 smithi073 ceph-mon[35254]: pgmap v388: 129 pgs: 129 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-16T00:33:35.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.750+0000 7fa8ac4ff640 1 -- 172.21.15.38:0/2700268418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8a4108290 msgr2=0x7fa8a410ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:35.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.750+0000 7fa8ac4ff640 1 --2- 172.21.15.38:0/2700268418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8a4108290 0x7fa8a410ec30 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fa89c009cd0 tx=0x7fa89c02a8e0 comp rx=0 tx=0).stop 2024-09-16T00:33:35.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.750+0000 7fa8ac4ff640 1 -- 172.21.15.38:0/2700268418 shutdown_connections 2024-09-16T00:33:35.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.750+0000 7fa8ac4ff640 1 --2- 172.21.15.38:0/2700268418 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8a4108290 0x7fa8a410ec30 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:35.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.750+0000 7fa8ac4ff640 1 --2- 172.21.15.38:0/2700268418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8a4103740 0x7fa8a4107b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:35.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.750+0000 7fa8ac4ff640 1 --2- 172.21.15.38:0/2700268418 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8a4102d90 0x7fa8a4103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:35.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.750+0000 7fa8ac4ff640 1 -- 172.21.15.38:0/2700268418 >> 172.21.15.38:0/2700268418 conn(0x7fa8a40fc7d0 msgr2=0x7fa8a40febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:35.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.750+0000 7fa8ac4ff640 1 -- 172.21.15.38:0/2700268418 shutdown_connections 2024-09-16T00:33:35.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.750+0000 7fa8ac4ff640 1 -- 172.21.15.38:0/2700268418 wait complete. 2024-09-16T00:33:35.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.752+0000 7fa8ac4ff640 1 Processor -- start 2024-09-16T00:33:35.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.752+0000 7fa8ac4ff640 1 -- start start 2024-09-16T00:33:35.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.752+0000 7fa8ac4ff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8a4102d90 0x7fa8a4198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:35.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.752+0000 7fa8ac4ff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8a4103740 0x7fa8a4198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:35.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.752+0000 7fa8a9a9a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8a4103740 0x7fa8a4198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:35.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.752+0000 7fa8a9a9a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8a4103740 0x7fa8a4198b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34924/0 (socket says 172.21.15.38:34924) 2024-09-16T00:33:35.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.752+0000 7fa8ac4ff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8a4108290 0x7fa8a419cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:35.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.752+0000 7fa8a9a9a640 1 -- 172.21.15.38:0/1517681259 learned_addr learned my addr 172.21.15.38:0/1517681259 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:35.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.753+0000 7fa8ac4ff640 1 -- 172.21.15.38:0/1517681259 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a4112090 con 0x7fa8a4103740 2024-09-16T00:33:35.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.753+0000 7fa8ac4ff640 1 -- 172.21.15.38:0/1517681259 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a4111f10 con 0x7fa8a4102d90 2024-09-16T00:33:35.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.753+0000 7fa8ac4ff640 1 -- 172.21.15.38:0/1517681259 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa8a4112210 con 0x7fa8a4108290 2024-09-16T00:33:35.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.753+0000 7fa8aaa9c640 1 --2- 172.21.15.38:0/1517681259 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8a4108290 0x7fa8a419cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:35.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.753+0000 7fa8aa29b640 1 --2- 172.21.15.38:0/1517681259 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8a4102d90 0x7fa8a4198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:35.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.753+0000 7fa8a9a9a640 1 -- 172.21.15.38:0/1517681259 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8a4108290 msgr2=0x7fa8a419cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:35.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.753+0000 7fa8a9a9a640 1 --2- 172.21.15.38:0/1517681259 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8a4108290 0x7fa8a419cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:35.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.753+0000 7fa8a9a9a640 1 -- 172.21.15.38:0/1517681259 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8a4102d90 msgr2=0x7fa8a4198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:35.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.753+0000 7fa8a9a9a640 1 --2- 172.21.15.38:0/1517681259 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8a4102d90 0x7fa8a4198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:35.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.753+0000 7fa8a9a9a640 1 -- 172.21.15.38:0/1517681259 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8a419d5e0 con 0x7fa8a4103740 2024-09-16T00:33:35.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.753+0000 7fa8a9a9a640 1 --2- 172.21.15.38:0/1517681259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8a4103740 0x7fa8a4198b70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa89400e9c0 tx=0x7fa89400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:35.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.754+0000 7fa89b7fe640 1 -- 172.21.15.38:0/1517681259 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89400cd80 con 0x7fa8a4103740 2024-09-16T00:33:35.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.754+0000 7fa89b7fe640 1 -- 172.21.15.38:0/1517681259 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa894005500 con 0x7fa8a4103740 2024-09-16T00:33:35.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.754+0000 7fa8ac4ff640 1 -- 172.21.15.38:0/1517681259 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8a419d8d0 con 0x7fa8a4103740 2024-09-16T00:33:35.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.754+0000 7fa8ac4ff640 1 -- 172.21.15.38:0/1517681259 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8a41a51b0 con 0x7fa8a4103740 2024-09-16T00:33:35.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.754+0000 7fa89b7fe640 1 -- 172.21.15.38:0/1517681259 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa894010690 con 0x7fa8a4103740 2024-09-16T00:33:35.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.758+0000 7fa89b7fe640 1 -- 172.21.15.38:0/1517681259 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7fa894010830 con 0x7fa8a4103740 2024-09-16T00:33:35.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.758+0000 7fa8ae8d8740 1 Processor -- start 2024-09-16T00:33:35.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.758+0000 7fa8ae8d8740 1 -- start start 2024-09-16T00:33:35.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.759+0000 7fa8ae8d8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffcd21fb40 0x55ffcd21e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:35.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.759+0000 7fa8ae8d8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffcd21e5d0 0x55ffcd21ea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:35.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.759+0000 7fa8aaa9c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffcd21fb40 0x55ffcd21e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:35.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.759+0000 7fa8ae8d8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffcd229370 0x55ffcd22b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:35.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.759+0000 7fa8ae8d8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ffcd1842f0 con 0x55ffcd21fb40 2024-09-16T00:33:35.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.759+0000 7fa8ae8d8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ffcd12bcf0 con 0x55ffcd229370 2024-09-16T00:33:35.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.759+0000 7fa8aaa9c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffcd21fb40 0x55ffcd21e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34926/0 (socket says 172.21.15.38:34926) 2024-09-16T00:33:35.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.759+0000 7fa8ae8d8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ffcd1802b0 con 0x55ffcd21e5d0 2024-09-16T00:33:35.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.759+0000 7fa8aaa9c640 1 -- 172.21.15.38:0/1440856376 learned_addr learned my addr 172.21.15.38:0/1440856376 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:35.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.759+0000 7fa8aa29b640 1 --2- 172.21.15.38:0/1440856376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffcd21e5d0 0x55ffcd21ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:35.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.759+0000 7fa8aaa9c640 1 -- 172.21.15.38:0/1440856376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffcd21e5d0 msgr2=0x55ffcd21ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:35.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.759+0000 7fa8aaa9c640 1 --2- 172.21.15.38:0/1440856376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffcd21e5d0 0x55ffcd21ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:35.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.759+0000 7fa8aaa9c640 1 -- 172.21.15.38:0/1440856376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffcd229370 msgr2=0x55ffcd22b760 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:33:35.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.759+0000 7fa8aaa9c640 1 --2- 172.21.15.38:0/1440856376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffcd229370 0x55ffcd22b760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:35.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.759+0000 7fa8aaa9c640 1 -- 172.21.15.38:0/1440856376 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffcd1103a0 con 0x55ffcd21fb40 2024-09-16T00:33:35.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.760+0000 7fa8aaa9c640 1 --2- 172.21.15.38:0/1440856376 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffcd21fb40 0x55ffcd21e090 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa89c009cd0 tx=0x7fa89c009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f85b8900a1bba72 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:35.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.760+0000 7fa89b7fe640 1 --2- 172.21.15.38:0/1517681259 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fa8900a3420 0x7fa8900a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:35.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.760+0000 7fa8aa29b640 1 --2- 172.21.15.38:0/1517681259 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fa8900a3420 0x7fa8900a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:35.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.760+0000 7fa898ff9640 1 -- 172.21.15.38:0/1440856376 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89c048070 con 0x55ffcd21fb40 2024-09-16T00:33:35.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.760+0000 7fa89b7fe640 1 -- 172.21.15.38:0/1517681259 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7fa8940cffe0 con 0x7fa8a4103740 2024-09-16T00:33:35.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.760+0000 7fa898ff9640 1 -- 172.21.15.38:0/1440856376 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa89c036070 con 0x55ffcd21fb40 2024-09-16T00:33:35.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.760+0000 7fa898ff9640 1 -- 172.21.15.38:0/1440856376 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89c004440 con 0x55ffcd21fb40 2024-09-16T00:33:35.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.761+0000 7fa8ae8d8740 1 -- 172.21.15.38:0/1440856376 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffcd21fb40 msgr2=0x55ffcd21e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:35.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.761+0000 7fa8aa29b640 1 --2- 172.21.15.38:0/1517681259 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fa8900a3420 0x7fa8900a58e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x55ffcd21bac0 tx=0x7fa8a0005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:35.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.761+0000 7fa8ae8d8740 1 --2- 172.21.15.38:0/1440856376 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffcd21fb40 0x55ffcd21e090 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa89c009cd0 tx=0x7fa89c009a20 comp rx=0 tx=0).stop 2024-09-16T00:33:35.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.761+0000 7fa8ae8d8740 1 -- 172.21.15.38:0/1440856376 shutdown_connections 2024-09-16T00:33:35.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.761+0000 7fa8ae8d8740 1 --2- 172.21.15.38:0/1440856376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffcd229370 0x55ffcd22b760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:35.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.761+0000 7fa8ae8d8740 1 --2- 172.21.15.38:0/1440856376 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffcd21e5d0 0x55ffcd21ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:35.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.761+0000 7fa8ae8d8740 1 --2- 172.21.15.38:0/1440856376 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffcd21fb40 0x55ffcd21e090 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:35.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.761+0000 7fa8ae8d8740 1 -- 172.21.15.38:0/1440856376 >> 172.21.15.38:0/1440856376 conn(0x55ffcd1e7310 msgr2=0x55ffcd21ac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:35.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.762+0000 7fa8ae8d8740 1 -- 172.21.15.38:0/1440856376 shutdown_connections 2024-09-16T00:33:35.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.762+0000 7fa8ae8d8740 1 -- 172.21.15.38:0/1440856376 wait complete. 2024-09-16T00:33:35.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.762+0000 7fa8ae8d8740 1 -- start start 2024-09-16T00:33:35.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.762+0000 7fa8ae8d8740 1 Processor -- start 2024-09-16T00:33:35.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.763+0000 7fa8ae8d8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffcd21e5d0 0x55ffcd226100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:35.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.763+0000 7fa8ae8d8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffcd21fb40 0x55ffcd226640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:35.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.763+0000 7fa8ae8d8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffcd229370 0x55ffcd226b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:35.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.763+0000 7fa8aa29b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffcd21e5d0 0x55ffcd226100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:35.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.763+0000 7fa8aaa9c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffcd229370 0x55ffcd226b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:35.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.763+0000 7fa8a9a9a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffcd21fb40 0x55ffcd226640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:35.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.763+0000 7fa8aaa9c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffcd229370 0x55ffcd226b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34936/0 (socket says 172.21.15.38:34936) 2024-09-16T00:33:35.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.763+0000 7fa8a9a9a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffcd21fb40 0x55ffcd226640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.38:51640/0 (socket says 172.21.15.38:51640) 2024-09-16T00:33:35.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.763+0000 7fa8aa29b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffcd21e5d0 0x55ffcd226100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.38:57060/0 (socket says 172.21.15.38:57060) 2024-09-16T00:33:35.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.763+0000 7fa8aaa9c640 1 -- 172.21.15.38:0/2056690618 learned_addr learned my addr 172.21.15.38:0/2056690618 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:35.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.763+0000 7fa8ae8d8740 1 -- 172.21.15.38:0/2056690618 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ffcd230420 con 0x55ffcd229370 2024-09-16T00:33:35.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.763+0000 7fa8ae8d8740 1 -- 172.21.15.38:0/2056690618 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ffcd13f060 con 0x55ffcd21fb40 2024-09-16T00:33:35.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.763+0000 7fa8ae8d8740 1 -- 172.21.15.38:0/2056690618 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ffcd2305a0 con 0x55ffcd21e5d0 2024-09-16T00:33:35.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.763+0000 7fa8a9a9a640 1 -- 172.21.15.38:0/2056690618 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffcd21e5d0 msgr2=0x55ffcd226100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:35.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.763+0000 7fa8a9a9a640 1 --2- 172.21.15.38:0/2056690618 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffcd21e5d0 0x55ffcd226100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:35.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.763+0000 7fa8a9a9a640 1 -- 172.21.15.38:0/2056690618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffcd229370 msgr2=0x55ffcd226b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:35.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.763+0000 7fa8a9a9a640 1 --2- 172.21.15.38:0/2056690618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffcd229370 0x55ffcd226b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:35.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.763+0000 7fa8a9a9a640 1 -- 172.21.15.38:0/2056690618 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffccef53c0 con 0x55ffcd21fb40 2024-09-16T00:33:35.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.764+0000 7fa8a9a9a640 1 --2- 172.21.15.38:0/2056690618 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffcd21fb40 0x55ffcd226640 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa8940cd5b0 tx=0x7fa89409c7c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=48b367ad51ead02d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:35.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.764+0000 7fa88dffb640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-09-16T00:33:35.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.764+0000 7fa8ae8d8740 10 client.0 _ll_register_callbacks cb 0x55ffcce928c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:35.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.764+0000 7fa8ae8d8740 1 -- 172.21.15.38:0/2056690618 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa89c036070 con 0x55ffcd21fb40 2024-09-16T00:33:35.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.764+0000 7fa8aa29b640 1 --2- 172.21.15.38:0/2056690618 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffcd21e5d0 0x55ffcd226100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:33:35.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.764+0000 7fa8aaa9c640 1 --2- 172.21.15.38:0/2056690618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffcd229370 0x55ffcd226b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:33:35.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.764+0000 7fa88dffb640 1 -- 172.21.15.38:0/2056690618 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8940b5060 con 0x55ffcd21fb40 2024-09-16T00:33:35.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.764+0000 7fa88dffb640 1 -- 172.21.15.38:0/2056690618 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7fa8940cd800 con 0x55ffcd21fb40 2024-09-16T00:33:35.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.764+0000 7fa88dffb640 1 -- 172.21.15.38:0/2056690618 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa8940a0470 con 0x55ffcd21fb40 2024-09-16T00:33:35.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.764+0000 7fa8ae8d8740 10 client.25133 fetch_fsmap learned FSMap version 26 2024-09-16T00:33:35.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.764+0000 7fa8ae8d8740 1 -- 172.21.15.38:0/2056690618 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55ffcd22e2f0 con 0x55ffcd21fb40 2024-09-16T00:33:35.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.765+0000 7fa88dffb640 1 -- 172.21.15.38:0/2056690618 <== mon.2 v2:172.21.15.73:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa8940b0170 con 0x55ffcd21fb40 2024-09-16T00:33:35.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.765+0000 7fa8ae8d8740 10 client.25133 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:33:35.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.765+0000 7fa8ae8d8740 10 client.25133 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-16T00:33:35.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.765+0000 7fa8ae8d8740 20 client.25133 populate_metadata read hostname 'smithi038' 2024-09-16T00:33:35.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.765+0000 7fa8ae8d8740 1 --2- 172.21.15.38:0/2056690618 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x55ffcd22f640 0x55ffcd22fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:35.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.765+0000 7fa8ae8d8740 4 client.25133 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:33:35.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.765+0000 7fa8ae8d8740 1 -- 172.21.15.38:0/2056690618 --> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ffcd191860 con 0x55ffcd22f640 2024-09-16T00:33:35.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.765+0000 7fa8aa29b640 1 --2- 172.21.15.38:0/2056690618 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x55ffcd22f640 0x55ffcd22fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:35.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.766+0000 7fa8aa29b640 1 --2- 172.21.15.38:0/2056690618 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x55ffcd22f640 0x55ffcd22fa50 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7059c93cebf01e1f server_cookie=81962105c84dac8e in_seq=0 out_seq=0 2024-09-16T00:33:35.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.766+0000 7fa88dffb640 10 client.25133 ms_handle_connect on v2:172.21.15.73:6834/2866363113 2024-09-16T00:33:35.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.766+0000 7fa88dffb640 1 -- 172.21.15.38:0/2056690618 <== mds.2 v2:172.21.15.73:6834/2866363113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ffcd191860 con 0x55ffcd22f640 2024-09-16T00:33:35.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.766+0000 7fa88dffb640 10 client.25133 handle_command_reply: tid=0 2024-09-16T00:33:35.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.766+0000 7fa8ae8d8740 1 client.25133 shutdown 2024-09-16T00:33:35.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.767+0000 7fa8ae8d8740 1 -- 172.21.15.38:0/2056690618 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffcd21fb40 msgr2=0x55ffcd226640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:35.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.767+0000 7fa8ae8d8740 1 --2- 172.21.15.38:0/2056690618 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffcd21fb40 0x55ffcd226640 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa8940cd5b0 tx=0x7fa89409c7c0 comp rx=0 tx=0).stop 2024-09-16T00:33:35.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.767+0000 7fa8ae8d8740 1 -- 172.21.15.38:0/2056690618 shutdown_connections 2024-09-16T00:33:35.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.767+0000 7fa8ae8d8740 1 --2- 172.21.15.38:0/2056690618 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x55ffcd22f640 0x55ffcd22fa50 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:35.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.767+0000 7fa8ae8d8740 1 --2- 172.21.15.38:0/2056690618 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffcd229370 0x55ffcd226b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:35.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.767+0000 7fa8ae8d8740 1 --2- 172.21.15.38:0/2056690618 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffcd21fb40 0x55ffcd226640 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:35.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.767+0000 7fa88dffb640 0 client.25133 ms_handle_reset on v2:172.21.15.73:6834/2866363113 2024-09-16T00:33:35.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.767+0000 7fa8ae8d8740 1 --2- 172.21.15.38:0/2056690618 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffcd21e5d0 0x55ffcd226100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:35.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.767+0000 7fa8ae8d8740 1 -- 172.21.15.38:0/2056690618 >> 172.21.15.38:0/2056690618 conn(0x55ffcd1e7310 msgr2=0x55ffcd220930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:35.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.767+0000 7fa8ae8d8740 1 -- 172.21.15.38:0/2056690618 shutdown_connections 2024-09-16T00:33:35.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.767+0000 7fa8ae8d8740 1 -- 172.21.15.38:0/2056690618 wait complete. 2024-09-16T00:33:35.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.767+0000 7fa8ae8d8740 20 client.25133 trim_cache size 0 max 16384 2024-09-16T00:33:35.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.792+0000 7fa8ae8d8740 1 Processor -- start 2024-09-16T00:33:35.792 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.792+0000 7fa8ae8d8740 1 -- start start 2024-09-16T00:33:35.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.792+0000 7fa8ae8d8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffcd21e5d0 0x55ffcd2269d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:35.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.792+0000 7fa8ae8d8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffcd21fb40 0x55ffcd240b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:35.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.792+0000 7fa8ae8d8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffcd229370 0x55ffcd241060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:35.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.793+0000 7fa8ae8d8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa8880123b0 con 0x55ffcd21e5d0 2024-09-16T00:33:35.793 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.793+0000 7fa8ae8d8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa88800fc80 con 0x55ffcd229370 2024-09-16T00:33:35.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.793+0000 7fa8ae8d8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa88800dad0 con 0x55ffcd21fb40 2024-09-16T00:33:35.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.793+0000 7fa8aa29b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffcd21e5d0 0x55ffcd2269d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:35.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.793+0000 7fa8aa29b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffcd21e5d0 0x55ffcd2269d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34942/0 (socket says 172.21.15.38:34942) 2024-09-16T00:33:35.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.793+0000 7fa8aa29b640 1 -- 172.21.15.38:0/3346169001 learned_addr learned my addr 172.21.15.38:0/3346169001 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:35.794 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.793+0000 7fa8aaa9c640 1 --2- 172.21.15.38:0/3346169001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffcd229370 0x55ffcd241060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:35.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.793+0000 7fa8aa29b640 1 -- 172.21.15.38:0/3346169001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffcd21fb40 msgr2=0x55ffcd240b20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:33:35.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.793+0000 7fa8aa29b640 1 --2- 172.21.15.38:0/3346169001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffcd21fb40 0x55ffcd240b20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:35.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.793+0000 7fa8aa29b640 1 -- 172.21.15.38:0/3346169001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffcd229370 msgr2=0x55ffcd241060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:35.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.793+0000 7fa8aa29b640 1 --2- 172.21.15.38:0/3346169001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffcd229370 0x55ffcd241060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:35.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.793+0000 7fa8aa29b640 1 -- 172.21.15.38:0/3346169001 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffcd197e70 con 0x55ffcd21e5d0 2024-09-16T00:33:35.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.793+0000 7fa8aa29b640 1 --2- 172.21.15.38:0/3346169001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffcd21e5d0 0x55ffcd2269d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0005d00 tx=0x7fa8a0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd9bf4da1d636658 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:35.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.794+0000 7fa88ffff640 1 -- 172.21.15.38:0/3346169001 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a0004020 con 0x55ffcd21e5d0 2024-09-16T00:33:35.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.794+0000 7fa88ffff640 1 -- 172.21.15.38:0/3346169001 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa8a00041c0 con 0x55ffcd21e5d0 2024-09-16T00:33:35.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.794+0000 7fa88ffff640 1 -- 172.21.15.38:0/3346169001 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8a002a7a0 con 0x55ffcd21e5d0 2024-09-16T00:33:35.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.795+0000 7fa8ae8d8740 1 -- 172.21.15.38:0/3346169001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffcd21e5d0 msgr2=0x55ffcd2269d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:35.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.795+0000 7fa8ae8d8740 1 --2- 172.21.15.38:0/3346169001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffcd21e5d0 0x55ffcd2269d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa8a0005d00 tx=0x7fa8a0017000 comp rx=0 tx=0).stop 2024-09-16T00:33:35.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.795+0000 7fa8ae8d8740 1 -- 172.21.15.38:0/3346169001 shutdown_connections 2024-09-16T00:33:35.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.795+0000 7fa8ae8d8740 1 --2- 172.21.15.38:0/3346169001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffcd229370 0x55ffcd241060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:35.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.795+0000 7fa8ae8d8740 1 --2- 172.21.15.38:0/3346169001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffcd21fb40 0x55ffcd240b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:35.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.795+0000 7fa8ae8d8740 1 --2- 172.21.15.38:0/3346169001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffcd21e5d0 0x55ffcd2269d0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:35.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.795+0000 7fa8ae8d8740 1 -- 172.21.15.38:0/3346169001 >> 172.21.15.38:0/3346169001 conn(0x55ffcd1e7310 msgr2=0x55ffcd220930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:35.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.795+0000 7fa8ae8d8740 1 -- 172.21.15.38:0/3346169001 shutdown_connections 2024-09-16T00:33:35.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.795+0000 7fa8ae8d8740 1 -- 172.21.15.38:0/3346169001 wait complete. 2024-09-16T00:33:35.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.795+0000 7fa8ae8d8740 1 -- start start 2024-09-16T00:33:35.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.796+0000 7fa8ae8d8740 1 Processor -- start 2024-09-16T00:33:35.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.796+0000 7fa8ae8d8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffcd21e5d0 0x55ffcd235f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:35.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.797+0000 7fa8aa29b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffcd21e5d0 0x55ffcd235f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:35.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.797+0000 7fa8aa29b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffcd21e5d0 0x55ffcd235f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:34958/0 (socket says 172.21.15.38:34958) 2024-09-16T00:33:35.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.797+0000 7fa8aa29b640 1 -- 172.21.15.38:0/3882605173 learned_addr learned my addr 172.21.15.38:0/3882605173 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:35.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.797+0000 7fa8ae8d8740 1 --2- 172.21.15.38:0/3882605173 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffcd21fb40 0x55ffcd2312c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:35.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.797+0000 7fa8ae8d8740 1 --2- 172.21.15.38:0/3882605173 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffcd229370 0x55ffcd231800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:35.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.797+0000 7fa8ae8d8740 1 -- 172.21.15.38:0/3882605173 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa88800e140 con 0x55ffcd21e5d0 2024-09-16T00:33:35.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.797+0000 7fa8ae8d8740 1 -- 172.21.15.38:0/3882605173 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa88800f5b0 con 0x55ffcd21fb40 2024-09-16T00:33:35.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.797+0000 7fa8ae8d8740 1 -- 172.21.15.38:0/3882605173 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa88800e810 con 0x55ffcd229370 2024-09-16T00:33:35.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.797+0000 7fa8a9a9a640 1 --2- 172.21.15.38:0/3882605173 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffcd21fb40 0x55ffcd2312c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:35.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.797+0000 7fa8aaa9c640 1 --2- 172.21.15.38:0/3882605173 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffcd229370 0x55ffcd231800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:35.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.797+0000 7fa8a9a9a640 1 -- 172.21.15.38:0/3882605173 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffcd229370 msgr2=0x55ffcd231800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:35.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.797+0000 7fa8a9a9a640 1 --2- 172.21.15.38:0/3882605173 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffcd229370 0x55ffcd231800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:35.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.797+0000 7fa8a9a9a640 1 -- 172.21.15.38:0/3882605173 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffcd21e5d0 msgr2=0x55ffcd235f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:35.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.797+0000 7fa8a9a9a640 1 --2- 172.21.15.38:0/3882605173 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffcd21e5d0 0x55ffcd235f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:35.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.797+0000 7fa8a9a9a640 1 -- 172.21.15.38:0/3882605173 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffcd232030 con 0x55ffcd21fb40 2024-09-16T00:33:35.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.797+0000 7fa8ae8d8740 10 client.0 _ll_register_callbacks cb 0x7fa8940056b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:35.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.797+0000 7fa8ae8d8740 1 -- 172.21.15.38:0/3882605173 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa8a00041c0 con 0x55ffcd21fb40 2024-09-16T00:33:35.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.797+0000 7fa8aaa9c640 1 --2- 172.21.15.38:0/3882605173 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffcd229370 0x55ffcd231800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:33:35.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.797+0000 7fa8aa29b640 1 --2- 172.21.15.38:0/3882605173 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffcd21e5d0 0x55ffcd235f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:33:35.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.798+0000 7fa8a9a9a640 1 --2- 172.21.15.38:0/3882605173 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffcd21fb40 0x55ffcd2312c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x55ffcd227660 tx=0x7fa8940cf070 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4a4de51156e7ce6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:35.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.798+0000 7fa88dffb640 10 client.25139 ms_handle_connect on v2:172.21.15.73:3300/0 2024-09-16T00:33:35.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.798+0000 7fa88dffb640 1 -- 172.21.15.38:0/3882605173 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa89409be90 con 0x55ffcd21fb40 2024-09-16T00:33:35.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.798+0000 7fa88dffb640 1 -- 172.21.15.38:0/3882605173 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7fa894098730 con 0x55ffcd21fb40 2024-09-16T00:33:35.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.798+0000 7fa88dffb640 1 -- 172.21.15.38:0/3882605173 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fa8940a4ab0 con 0x55ffcd21fb40 2024-09-16T00:33:35.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.798+0000 7fa8ae8d8740 10 client.25139 fetch_fsmap learned FSMap version 26 2024-09-16T00:33:35.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.798+0000 7fa8ae8d8740 1 -- 172.21.15.38:0/3882605173 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55ffcd2a39a0 con 0x55ffcd21fb40 2024-09-16T00:33:35.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.799+0000 7fa88dffb640 1 -- 172.21.15.38:0/3882605173 <== mon.2 v2:172.21.15.73:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fa89409c320 con 0x55ffcd21fb40 2024-09-16T00:33:35.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.799+0000 7fa8ae8d8740 10 client.25139 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:33:35.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.799+0000 7fa8ae8d8740 10 client.25139 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-16T00:33:35.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.799+0000 7fa8ae8d8740 20 client.25139 populate_metadata read hostname 'smithi038' 2024-09-16T00:33:35.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.799+0000 7fa8ae8d8740 1 --2- 172.21.15.38:0/3882605173 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x55ffcd22f640 0x55ffcd2a5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:35.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.799+0000 7fa8ae8d8740 4 client.25139 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:33:35.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.799+0000 7fa8ae8d8740 1 -- 172.21.15.38:0/3882605173 --> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ffcd18bd70 con 0x55ffcd22f640 2024-09-16T00:33:35.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.799+0000 7fa8aa29b640 1 --2- 172.21.15.38:0/3882605173 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x55ffcd22f640 0x55ffcd2a5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:35.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.799+0000 7fa8aa29b640 1 --2- 172.21.15.38:0/3882605173 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x55ffcd22f640 0x55ffcd2a5fc0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ad180bc4d6fbf4b0 server_cookie=15cd990882bdedf9 in_seq=0 out_seq=0 2024-09-16T00:33:35.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.799+0000 7fa88dffb640 10 client.25139 ms_handle_connect on v2:172.21.15.73:6834/2866363113 2024-09-16T00:33:35.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.800+0000 7fa88dffb640 1 -- 172.21.15.38:0/3882605173 <== mds.2 v2:172.21.15.73:6834/2866363113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ffcd18bd70 con 0x55ffcd22f640 2024-09-16T00:33:35.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.800+0000 7fa88dffb640 10 client.25139 handle_command_reply: tid=0 2024-09-16T00:33:35.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.800+0000 7fa8ae8d8740 1 client.25139 shutdown 2024-09-16T00:33:35.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.800+0000 7fa8ae8d8740 1 -- 172.21.15.38:0/3882605173 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffcd21fb40 msgr2=0x55ffcd2312c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:35.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.800+0000 7fa8ae8d8740 1 --2- 172.21.15.38:0/3882605173 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffcd21fb40 0x55ffcd2312c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x55ffcd227660 tx=0x7fa8940cf070 comp rx=0 tx=0).stop 2024-09-16T00:33:35.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.800+0000 7fa8ae8d8740 1 -- 172.21.15.38:0/3882605173 shutdown_connections 2024-09-16T00:33:35.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.800+0000 7fa8ae8d8740 1 --2- 172.21.15.38:0/3882605173 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x55ffcd22f640 0x55ffcd2a5fc0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:35.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.800+0000 7fa8ae8d8740 1 --2- 172.21.15.38:0/3882605173 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ffcd229370 0x55ffcd231800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:35.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.800+0000 7fa8ae8d8740 1 --2- 172.21.15.38:0/3882605173 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ffcd21fb40 0x55ffcd2312c0 secure :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0x55ffcd227660 tx=0x7fa8940cf070 comp rx=0 tx=0).stop 2024-09-16T00:33:35.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.800+0000 7fa8ae8d8740 1 --2- 172.21.15.38:0/3882605173 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ffcd21e5d0 0x55ffcd235f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:35.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.800+0000 7fa8ae8d8740 1 -- 172.21.15.38:0/3882605173 >> 172.21.15.38:0/3882605173 conn(0x55ffcd1e7310 msgr2=0x55ffcd219310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:35.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.800+0000 7fa88dffb640 0 client.25139 ms_handle_reset on v2:172.21.15.73:6834/2866363113 2024-09-16T00:33:35.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.800+0000 7fa8ae8d8740 1 -- 172.21.15.38:0/3882605173 shutdown_connections 2024-09-16T00:33:35.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.800+0000 7fa8ae8d8740 1 -- 172.21.15.38:0/3882605173 wait complete. 2024-09-16T00:33:35.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.800+0000 7fa8ae8d8740 20 client.25139 trim_cache size 0 max 16384 2024-09-16T00:33:35.805 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-09-16T00:33:35.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.801+0000 7fa8ac4ff640 1 -- 172.21.15.38:0/1517681259 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fa8900a3420 msgr2=0x7fa8900a58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:35.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.801+0000 7fa8ac4ff640 1 --2- 172.21.15.38:0/1517681259 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fa8900a3420 0x7fa8900a58e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x55ffcd21bac0 tx=0x7fa8a0005f90 comp rx=0 tx=0).stop 2024-09-16T00:33:35.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.801+0000 7fa8ac4ff640 1 -- 172.21.15.38:0/1517681259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8a4103740 msgr2=0x7fa8a4198b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:35.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.801+0000 7fa8ac4ff640 1 --2- 172.21.15.38:0/1517681259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8a4103740 0x7fa8a4198b70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fa89400e9c0 tx=0x7fa89400ee80 comp rx=0 tx=0).stop 2024-09-16T00:33:35.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.801+0000 7fa8ac4ff640 1 -- 172.21.15.38:0/1517681259 shutdown_connections 2024-09-16T00:33:35.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.801+0000 7fa8ac4ff640 1 --2- 172.21.15.38:0/1517681259 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fa8900a3420 0x7fa8900a58e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:35.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.801+0000 7fa8ac4ff640 1 --2- 172.21.15.38:0/1517681259 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa8a4108290 0x7fa8a419cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:35.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.801+0000 7fa8ac4ff640 1 --2- 172.21.15.38:0/1517681259 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa8a4103740 0x7fa8a4198b70 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:35.805 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.801+0000 7fa8ac4ff640 1 --2- 172.21.15.38:0/1517681259 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8a4102d90 0x7fa8a4198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:35.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.801+0000 7fa8ac4ff640 1 -- 172.21.15.38:0/1517681259 >> 172.21.15.38:0/1517681259 conn(0x7fa8a40fc7d0 msgr2=0x7fa8a4105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:35.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.801+0000 7fa8ac4ff640 1 -- 172.21.15.38:0/1517681259 shutdown_connections 2024-09-16T00:33:35.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:35.801+0000 7fa8ac4ff640 1 -- 172.21.15.38:0/1517681259 wait complete. 2024-09-16T00:33:36.631 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:33:36.895 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:36 smithi038 ceph-mon[30945]: pgmap v389: 129 pgs: 129 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-16T00:33:36.948 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:37.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:36 smithi122 ceph-mon[35412]: pgmap v389: 129 pgs: 129 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-16T00:33:37.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:36 smithi073 ceph-mon[35254]: pgmap v389: 129 pgs: 129 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 6 op/s 2024-09-16T00:33:37.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.451+0000 7fd762181640 1 -- 172.21.15.38:0/155361504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd75c102db0 msgr2=0x7fd75c103190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:37.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.451+0000 7fd762181640 1 --2- 172.21.15.38:0/155361504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd75c102db0 0x7fd75c103190 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fd74c009930 tx=0x7fd74c02a420 comp rx=0 tx=0).stop 2024-09-16T00:33:37.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.452+0000 7fd762181640 1 -- 172.21.15.38:0/155361504 shutdown_connections 2024-09-16T00:33:37.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.452+0000 7fd762181640 1 --2- 172.21.15.38:0/155361504 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd75c1082e0 0x7fd75c10ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:37.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.452+0000 7fd762181640 1 --2- 172.21.15.38:0/155361504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd75c103760 0x7fd75c107ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:37.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.452+0000 7fd762181640 1 --2- 172.21.15.38:0/155361504 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd75c102db0 0x7fd75c103190 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:37.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.452+0000 7fd762181640 1 -- 172.21.15.38:0/155361504 >> 172.21.15.38:0/155361504 conn(0x7fd75c0fc7d0 msgr2=0x7fd75c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:37.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.452+0000 7fd762181640 1 -- 172.21.15.38:0/155361504 shutdown_connections 2024-09-16T00:33:37.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.452+0000 7fd762181640 1 -- 172.21.15.38:0/155361504 wait complete. 2024-09-16T00:33:37.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.452+0000 7fd762181640 1 Processor -- start 2024-09-16T00:33:37.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.453+0000 7fd762181640 1 -- start start 2024-09-16T00:33:37.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.453+0000 7fd762181640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd75c102db0 0x7fd75c1a0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:37.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.453+0000 7fd762181640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd75c103760 0x7fd75c1a0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:37.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.453+0000 7fd762181640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd75c1082e0 0x7fd75c19a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:37.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.453+0000 7fd762181640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c112090 con 0x7fd75c1082e0 2024-09-16T00:33:37.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.453+0000 7fd762181640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c111f10 con 0x7fd75c103760 2024-09-16T00:33:37.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.453+0000 7fd762181640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c112210 con 0x7fd75c102db0 2024-09-16T00:33:37.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.453+0000 7fd75bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd75c1082e0 0x7fd75c19a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:37.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.453+0000 7fd75bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd75c1082e0 0x7fd75c19a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55310/0 (socket says 172.21.15.38:55310) 2024-09-16T00:33:37.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.453+0000 7fd75bfff640 1 -- 172.21.15.38:0/3143391549 learned_addr learned my addr 172.21.15.38:0/3143391549 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:37.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.453+0000 7fd75b7fe640 1 --2- 172.21.15.38:0/3143391549 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd75c102db0 0x7fd75c1a0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:37.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.453+0000 7fd75affd640 1 --2- 172.21.15.38:0/3143391549 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd75c103760 0x7fd75c1a0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:37.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.454+0000 7fd75bfff640 1 -- 172.21.15.38:0/3143391549 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd75c102db0 msgr2=0x7fd75c1a0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:37.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.454+0000 7fd75bfff640 1 --2- 172.21.15.38:0/3143391549 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd75c102db0 0x7fd75c1a0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:37.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.454+0000 7fd75bfff640 1 -- 172.21.15.38:0/3143391549 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd75c103760 msgr2=0x7fd75c1a0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:37.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.454+0000 7fd75bfff640 1 --2- 172.21.15.38:0/3143391549 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd75c103760 0x7fd75c1a0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:37.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.454+0000 7fd75bfff640 1 -- 172.21.15.38:0/3143391549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd75c19ad00 con 0x7fd75c1082e0 2024-09-16T00:33:37.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.454+0000 7fd75affd640 1 --2- 172.21.15.38:0/3143391549 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd75c103760 0x7fd75c1a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:37.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.454+0000 7fd75bfff640 1 --2- 172.21.15.38:0/3143391549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd75c1082e0 0x7fd75c19a5d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fd75400eaa0 tx=0x7fd75400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:37.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.454+0000 7fd758ff9640 1 -- 172.21.15.38:0/3143391549 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd75400cdb0 con 0x7fd75c1082e0 2024-09-16T00:33:37.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.454+0000 7fd762181640 1 -- 172.21.15.38:0/3143391549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd75c19aff0 con 0x7fd75c1082e0 2024-09-16T00:33:37.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.454+0000 7fd758ff9640 1 -- 172.21.15.38:0/3143391549 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd754005020 con 0x7fd75c1082e0 2024-09-16T00:33:37.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.454+0000 7fd758ff9640 1 -- 172.21.15.38:0/3143391549 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7540116b0 con 0x7fd75c1082e0 2024-09-16T00:33:37.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.454+0000 7fd762181640 1 -- 172.21.15.38:0/3143391549 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd75c1a72e0 con 0x7fd75c1082e0 2024-09-16T00:33:37.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.458+0000 7fd76455a740 1 Processor -- start 2024-09-16T00:33:37.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.458+0000 7fd758ff9640 1 -- 172.21.15.38:0/3143391549 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7fd754022060 con 0x7fd75c1082e0 2024-09-16T00:33:37.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.459+0000 7fd76455a740 1 -- start start 2024-09-16T00:33:37.483 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.459+0000 7fd76455a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef2e1953f0 0x55ef2e193a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:37.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.459+0000 7fd76455a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef2e193fb0 0x55ef2e1943f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:37.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.459+0000 7fd76455a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef2e19ecf0 0x55ef2e1a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:37.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.459+0000 7fd76455a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ef2e0f12f0 con 0x55ef2e1953f0 2024-09-16T00:33:37.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.459+0000 7fd76455a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ef2e098cf0 con 0x55ef2e193fb0 2024-09-16T00:33:37.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.459+0000 7fd76455a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ef2e0ed2b0 con 0x55ef2e19ecf0 2024-09-16T00:33:37.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.459+0000 7fd75affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef2e1953f0 0x55ef2e193a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:37.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.459+0000 7fd75affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef2e1953f0 0x55ef2e193a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55320/0 (socket says 172.21.15.38:55320) 2024-09-16T00:33:37.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.459+0000 7fd75affd640 1 -- 172.21.15.38:0/249871208 learned_addr learned my addr 172.21.15.38:0/249871208 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:37.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.460+0000 7fd758ff9640 1 --2- 172.21.15.38:0/3143391549 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd7440a3350 0x7fd7440a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:37.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.460+0000 7fd75b7fe640 1 --2- 172.21.15.38:0/249871208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef2e193fb0 0x55ef2e1943f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:37.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.460+0000 7fd758ff9640 1 -- 172.21.15.38:0/3143391549 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7fd754020070 con 0x7fd75c1082e0 2024-09-16T00:33:37.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.460+0000 7fd75affd640 1 -- 172.21.15.38:0/249871208 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef2e19ecf0 msgr2=0x55ef2e1a10e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:33:37.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.460+0000 7fd75affd640 1 --2- 172.21.15.38:0/249871208 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef2e19ecf0 0x55ef2e1a10e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:37.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.460+0000 7fd75affd640 1 -- 172.21.15.38:0/249871208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef2e193fb0 msgr2=0x55ef2e1943f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:37.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.460+0000 7fd75affd640 1 --2- 172.21.15.38:0/249871208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef2e193fb0 0x55ef2e1943f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:37.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.460+0000 7fd75affd640 1 -- 172.21.15.38:0/249871208 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef2e07d3a0 con 0x55ef2e1953f0 2024-09-16T00:33:37.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.460+0000 7fd75bfff640 1 --2- 172.21.15.38:0/3143391549 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd7440a3350 0x7fd7440a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:37.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.460+0000 7fd75affd640 1 --2- 172.21.15.38:0/249871208 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef2e1953f0 0x55ef2e193a70 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd750005fd0 tx=0x7fd75000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e908984ac4e9df server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:37.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.461+0000 7fd762181640 1 -- 172.21.15.38:0/249871208 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd750015070 con 0x55ef2e1953f0 2024-09-16T00:33:37.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.461+0000 7fd75bfff640 1 --2- 172.21.15.38:0/3143391549 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd7440a3350 0x7fd7440a5810 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd7540982c0 tx=0x7fd754098150 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:37.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.461+0000 7fd762181640 1 -- 172.21.15.38:0/249871208 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd75001a940 con 0x55ef2e1953f0 2024-09-16T00:33:37.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.461+0000 7fd762181640 1 -- 172.21.15.38:0/249871208 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7500119a0 con 0x55ef2e1953f0 2024-09-16T00:33:37.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.461+0000 7fd76455a740 1 -- 172.21.15.38:0/249871208 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef2e1953f0 msgr2=0x55ef2e193a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:37.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.461+0000 7fd76455a740 1 --2- 172.21.15.38:0/249871208 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef2e1953f0 0x55ef2e193a70 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fd750005fd0 tx=0x7fd75000f730 comp rx=0 tx=0).stop 2024-09-16T00:33:37.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.462+0000 7fd76455a740 1 -- 172.21.15.38:0/249871208 shutdown_connections 2024-09-16T00:33:37.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.462+0000 7fd76455a740 1 --2- 172.21.15.38:0/249871208 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef2e19ecf0 0x55ef2e1a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:37.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.462+0000 7fd76455a740 1 --2- 172.21.15.38:0/249871208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef2e193fb0 0x55ef2e1943f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:37.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.462+0000 7fd76455a740 1 --2- 172.21.15.38:0/249871208 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef2e1953f0 0x55ef2e193a70 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:37.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.462+0000 7fd76455a740 1 -- 172.21.15.38:0/249871208 >> 172.21.15.38:0/249871208 conn(0x55ef2e154310 msgr2=0x55ef2e190730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:37.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.462+0000 7fd76455a740 1 -- 172.21.15.38:0/249871208 shutdown_connections 2024-09-16T00:33:37.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.462+0000 7fd76455a740 1 -- 172.21.15.38:0/249871208 wait complete. 2024-09-16T00:33:37.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.462+0000 7fd76455a740 1 -- start start 2024-09-16T00:33:37.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.463+0000 7fd76455a740 1 Processor -- start 2024-09-16T00:33:37.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.463+0000 7fd76455a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef2e193fb0 0x55ef2e19e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:37.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.464+0000 7fd76455a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef2e1953f0 0x55ef2e19b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:37.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.464+0000 7fd76455a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef2e19ecf0 0x55ef2e19bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:37.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.464+0000 7fd76455a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55ef2e1a18c0 con 0x55ef2e193fb0 2024-09-16T00:33:37.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.464+0000 7fd76455a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55ef2e0ac060 con 0x55ef2e1953f0 2024-09-16T00:33:37.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.464+0000 7fd76455a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55ef2e1a1a40 con 0x55ef2e19ecf0 2024-09-16T00:33:37.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.464+0000 7fd75affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef2e193fb0 0x55ef2e19e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:37.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.464+0000 7fd76455a740 10 client.0 _ll_register_callbacks cb 0x55ef2ddff8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:37.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.464+0000 7fd75affd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef2e193fb0 0x55ef2e19e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55330/0 (socket says 172.21.15.38:55330) 2024-09-16T00:33:37.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.464+0000 7fd75affd640 1 -- 172.21.15.38:0/2350356933 learned_addr learned my addr 172.21.15.38:0/2350356933 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:37.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.464+0000 7fd75b7fe640 1 --2- 172.21.15.38:0/2350356933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef2e1953f0 0x55ef2e19b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:37.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.464+0000 7fd75b7fe640 1 -- 172.21.15.38:0/2350356933 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef2e19ecf0 msgr2=0x55ef2e19bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:37.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.464+0000 7fd75b7fe640 1 --2- 172.21.15.38:0/2350356933 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef2e19ecf0 0x55ef2e19bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:37.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.464+0000 7fd75b7fe640 1 -- 172.21.15.38:0/2350356933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef2e193fb0 msgr2=0x55ef2e19e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:37.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.464+0000 7fd75b7fe640 1 --2- 172.21.15.38:0/2350356933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef2e193fb0 0x55ef2e19e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:37.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.464+0000 7fd75b7fe640 1 -- 172.21.15.38:0/2350356933 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef2de623c0 con 0x55ef2e1953f0 2024-09-16T00:33:37.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.464+0000 7fd75b7fe640 1 --2- 172.21.15.38:0/2350356933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef2e1953f0 0x55ef2e19b980 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd74c005940 tx=0x7fd74c0043b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8605466a70912f4d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:37.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.464+0000 7fd73affd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-09-16T00:33:37.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.465+0000 7fd73affd640 1 -- 172.21.15.38:0/2350356933 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c035070 con 0x55ef2e1953f0 2024-09-16T00:33:37.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.465+0000 7fd73affd640 1 -- 172.21.15.38:0/2350356933 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7fd74c003a00 con 0x55ef2e1953f0 2024-09-16T00:33:37.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.465+0000 7fd76455a740 1 -- 172.21.15.38:0/2350356933 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd75001a940 con 0x55ef2e1953f0 2024-09-16T00:33:37.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.465+0000 7fd73affd640 1 -- 172.21.15.38:0/2350356933 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd74c04d050 con 0x55ef2e1953f0 2024-09-16T00:33:37.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.465+0000 7fd76455a740 10 client.25151 fetch_fsmap learned FSMap version 26 2024-09-16T00:33:37.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.465+0000 7fd76455a740 1 -- 172.21.15.38:0/2350356933 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ef2e1ad310 con 0x55ef2e1953f0 2024-09-16T00:33:37.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.465+0000 7fd73affd640 1 -- 172.21.15.38:0/2350356933 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd75001a940 con 0x55ef2e1953f0 2024-09-16T00:33:37.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.466+0000 7fd73affd640 1 -- 172.21.15.38:0/2350356933 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd74c03b430 con 0x55ef2e1953f0 2024-09-16T00:33:37.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.466+0000 7fd76455a740 10 client.25151 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:33:37.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.466+0000 7fd76455a740 10 client.25151 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:33:37.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.466+0000 7fd76455a740 20 client.25151 populate_metadata read hostname 'smithi038' 2024-09-16T00:33:37.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.466+0000 7fd76455a740 1 --2- 172.21.15.38:0/2350356933 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x55ef2e1ae820 0x55ef2e1b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:37.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.466+0000 7fd76455a740 4 client.25151 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:33:37.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.466+0000 7fd76455a740 1 -- 172.21.15.38:0/2350356933 --> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef2e0fe860 con 0x55ef2e1ae820 2024-09-16T00:33:37.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.466+0000 7fd75affd640 1 --2- 172.21.15.38:0/2350356933 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x55ef2e1ae820 0x55ef2e1b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:37.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.467+0000 7fd73affd640 10 client.25151 ms_handle_connect on v2:172.21.15.73:6836/2681081265 2024-09-16T00:33:37.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.467+0000 7fd75affd640 1 --2- 172.21.15.38:0/2350356933 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x55ef2e1ae820 0x55ef2e1b0ce0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=235990f08d0f9e94 server_cookie=82ea3c03b46df8b in_seq=0 out_seq=0 2024-09-16T00:33:37.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.467+0000 7fd73affd640 1 -- 172.21.15.38:0/2350356933 <== mds.1 v2:172.21.15.73:6836/2681081265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55ef2e0fe860 con 0x55ef2e1ae820 2024-09-16T00:33:37.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.467+0000 7fd73affd640 10 client.25151 handle_command_reply: tid=0 2024-09-16T00:33:37.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.467+0000 7fd76455a740 1 client.25151 shutdown 2024-09-16T00:33:37.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.468+0000 7fd76455a740 1 -- 172.21.15.38:0/2350356933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef2e1953f0 msgr2=0x55ef2e19b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:37.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.468+0000 7fd76455a740 1 --2- 172.21.15.38:0/2350356933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef2e1953f0 0x55ef2e19b980 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd74c005940 tx=0x7fd74c0043b0 comp rx=0 tx=0).stop 2024-09-16T00:33:37.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.468+0000 7fd76455a740 1 -- 172.21.15.38:0/2350356933 shutdown_connections 2024-09-16T00:33:37.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.468+0000 7fd76455a740 1 --2- 172.21.15.38:0/2350356933 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef2e19ecf0 0x55ef2e19bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:37.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.468+0000 7fd76455a740 1 --2- 172.21.15.38:0/2350356933 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef2e1953f0 0x55ef2e19b980 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:37.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.468+0000 7fd76455a740 1 --2- 172.21.15.38:0/2350356933 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x55ef2e1ae820 0x55ef2e1b0ce0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:37.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.468+0000 7fd76455a740 1 --2- 172.21.15.38:0/2350356933 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef2e193fb0 0x55ef2e19e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:37.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.468+0000 7fd76455a740 1 -- 172.21.15.38:0/2350356933 >> 172.21.15.38:0/2350356933 conn(0x55ef2e154310 msgr2=0x55ef2e1871a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:37.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.468+0000 7fd73affd640 0 client.25151 ms_handle_reset on v2:172.21.15.73:6836/2681081265 2024-09-16T00:33:37.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.468+0000 7fd76455a740 1 -- 172.21.15.38:0/2350356933 shutdown_connections 2024-09-16T00:33:37.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.468+0000 7fd76455a740 1 -- 172.21.15.38:0/2350356933 wait complete. 2024-09-16T00:33:37.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.468+0000 7fd76455a740 20 client.25151 trim_cache size 0 max 16384 2024-09-16T00:33:37.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.485+0000 7fd76455a740 1 Processor -- start 2024-09-16T00:33:37.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.485+0000 7fd76455a740 1 -- start start 2024-09-16T00:33:37.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.485+0000 7fd76455a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef2e154310 0x55ef2e190100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:37.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.485+0000 7fd76455a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef2e193fb0 0x55ef2e1aab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:37.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.485+0000 7fd76455a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef2e1953f0 0x55ef2e1b65b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:37.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.485+0000 7fd76455a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c1af360 con 0x55ef2e193fb0 2024-09-16T00:33:37.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.485+0000 7fd76455a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c1acc30 con 0x55ef2e1953f0 2024-09-16T00:33:37.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.485+0000 7fd76455a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c1aab00 con 0x55ef2e154310 2024-09-16T00:33:37.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.485+0000 7fd75b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef2e193fb0 0x55ef2e1aab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:37.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.485+0000 7fd75b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef2e193fb0 0x55ef2e1aab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55342/0 (socket says 172.21.15.38:55342) 2024-09-16T00:33:37.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.485+0000 7fd75b7fe640 1 -- 172.21.15.38:0/164801988 learned_addr learned my addr 172.21.15.38:0/164801988 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:37.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.485+0000 7fd75affd640 1 --2- 172.21.15.38:0/164801988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef2e1953f0 0x55ef2e1b65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:37.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.486+0000 7fd75affd640 1 --2- 172.21.15.38:0/164801988 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef2e154310 0x55ef2e190100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:37.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.486+0000 7fd75b7fe640 1 -- 172.21.15.38:0/164801988 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef2e154310 msgr2=0x55ef2e190100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:37.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.486+0000 7fd75b7fe640 1 --2- 172.21.15.38:0/164801988 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef2e154310 0x55ef2e190100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:37.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.486+0000 7fd75b7fe640 1 -- 172.21.15.38:0/164801988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef2e1953f0 msgr2=0x55ef2e1b65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:37.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.486+0000 7fd75b7fe640 1 --2- 172.21.15.38:0/164801988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef2e1953f0 0x55ef2e1b65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:37.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.486+0000 7fd75b7fe640 1 -- 172.21.15.38:0/164801988 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef2e190c70 con 0x55ef2e193fb0 2024-09-16T00:33:37.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.486+0000 7fd75affd640 1 --2- 172.21.15.38:0/164801988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef2e1953f0 0x55ef2e1b65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:33:37.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.486+0000 7fd75b7fe640 1 --2- 172.21.15.38:0/164801988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef2e193fb0 0x55ef2e1aab20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd74c0096c0 tx=0x7fd74c005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e0506a67fdce923 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:37.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.486+0000 7fd7417fa640 1 -- 172.21.15.38:0/164801988 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c04a070 con 0x55ef2e193fb0 2024-09-16T00:33:37.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.486+0000 7fd7417fa640 1 -- 172.21.15.38:0/164801988 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd74c03bc40 con 0x55ef2e193fb0 2024-09-16T00:33:37.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.486+0000 7fd7417fa640 1 -- 172.21.15.38:0/164801988 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c035040 con 0x55ef2e193fb0 2024-09-16T00:33:37.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.487+0000 7fd76455a740 1 -- 172.21.15.38:0/164801988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef2e193fb0 msgr2=0x55ef2e1aab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:37.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.487+0000 7fd76455a740 1 --2- 172.21.15.38:0/164801988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef2e193fb0 0x55ef2e1aab20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd74c0096c0 tx=0x7fd74c005780 comp rx=0 tx=0).stop 2024-09-16T00:33:37.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.488+0000 7fd76455a740 1 -- 172.21.15.38:0/164801988 shutdown_connections 2024-09-16T00:33:37.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.488+0000 7fd76455a740 1 --2- 172.21.15.38:0/164801988 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef2e1953f0 0x55ef2e1b65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:37.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.488+0000 7fd76455a740 1 --2- 172.21.15.38:0/164801988 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef2e193fb0 0x55ef2e1aab20 secure :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fd74c0096c0 tx=0x7fd74c005780 comp rx=0 tx=0).stop 2024-09-16T00:33:37.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.488+0000 7fd76455a740 1 --2- 172.21.15.38:0/164801988 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef2e154310 0x55ef2e190100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:37.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.488+0000 7fd76455a740 1 -- 172.21.15.38:0/164801988 >> 172.21.15.38:0/164801988 conn(0x55ef2e197370 msgr2=0x55ef2e1871a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:37.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.488+0000 7fd76455a740 1 -- 172.21.15.38:0/164801988 shutdown_connections 2024-09-16T00:33:37.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.488+0000 7fd76455a740 1 -- 172.21.15.38:0/164801988 wait complete. 2024-09-16T00:33:37.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.488+0000 7fd76455a740 1 -- start start 2024-09-16T00:33:37.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.489+0000 7fd76455a740 1 Processor -- start 2024-09-16T00:33:37.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.490+0000 7fd76455a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef2e154310 0x55ef2e1a2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:37.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.490+0000 7fd76455a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef2e1953f0 0x55ef2e1a3100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:37.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.490+0000 7fd75b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef2e1953f0 0x55ef2e1a3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:37.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.490+0000 7fd75b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef2e1953f0 0x55ef2e1a3100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55346/0 (socket says 172.21.15.38:55346) 2024-09-16T00:33:37.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.490+0000 7fd75affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef2e154310 0x55ef2e1a2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:37.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.490+0000 7fd75affd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef2e154310 0x55ef2e1a2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.38:37398/0 (socket says 172.21.15.38:37398) 2024-09-16T00:33:37.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.490+0000 7fd75b7fe640 1 -- 172.21.15.38:0/946340434 learned_addr learned my addr 172.21.15.38:0/946340434 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:37.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.490+0000 7fd76455a740 1 --2- 172.21.15.38:0/946340434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef2e19ecf0 0x55ef2e1a3640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:37.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.491+0000 7fd76455a740 1 -- 172.21.15.38:0/946340434 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c1ab150 con 0x55ef2e1953f0 2024-09-16T00:33:37.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.491+0000 7fd76455a740 1 -- 172.21.15.38:0/946340434 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c1ac560 con 0x55ef2e19ecf0 2024-09-16T00:33:37.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.491+0000 7fd76455a740 1 -- 172.21.15.38:0/946340434 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd75c1ab7c0 con 0x55ef2e154310 2024-09-16T00:33:37.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.491+0000 7fd76455a740 10 client.0 _ll_register_callbacks cb 0x7fd74c009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:37.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.491+0000 7fd75affd640 1 --2- 172.21.15.38:0/946340434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef2e19ecf0 0x55ef2e1a3640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:37.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.491+0000 7fd75b7fe640 1 -- 172.21.15.38:0/946340434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef2e154310 msgr2=0x55ef2e1a2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:37.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.491+0000 7fd75b7fe640 1 --2- 172.21.15.38:0/946340434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef2e154310 0x55ef2e1a2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:37.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.491+0000 7fd75b7fe640 1 -- 172.21.15.38:0/946340434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef2e19ecf0 msgr2=0x55ef2e1a3640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:37.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.491+0000 7fd75b7fe640 1 --2- 172.21.15.38:0/946340434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef2e19ecf0 0x55ef2e1a3640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:37.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.491+0000 7fd75b7fe640 1 -- 172.21.15.38:0/946340434 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef2e195040 con 0x55ef2e1953f0 2024-09-16T00:33:37.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.491+0000 7fd75affd640 1 --2- 172.21.15.38:0/946340434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef2e19ecf0 0x55ef2e1a3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:33:37.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.492+0000 7fd75b7fe640 1 --2- 172.21.15.38:0/946340434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef2e1953f0 0x55ef2e1a3100 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd74c02a940 tx=0x7fd74c02a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44abcbb142e5c133 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:37.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.492+0000 7fd73affd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:33:37.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.492+0000 7fd73affd640 1 -- 172.21.15.38:0/946340434 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd74c035070 con 0x55ef2e1953f0 2024-09-16T00:33:37.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.492+0000 7fd73affd640 1 -- 172.21.15.38:0/946340434 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7fd74c03ba40 con 0x55ef2e1953f0 2024-09-16T00:33:37.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.492+0000 7fd76455a740 1 -- 172.21.15.38:0/946340434 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd74c03bc40 con 0x55ef2e1953f0 2024-09-16T00:33:37.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.493+0000 7fd73affd640 1 -- 172.21.15.38:0/946340434 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd74c04a040 con 0x55ef2e1953f0 2024-09-16T00:33:37.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.493+0000 7fd76455a740 10 client.15522 fetch_fsmap learned FSMap version 26 2024-09-16T00:33:37.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.493+0000 7fd76455a740 1 -- 172.21.15.38:0/946340434 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55ef2e219900 con 0x55ef2e1953f0 2024-09-16T00:33:37.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.493+0000 7fd73affd640 1 -- 172.21.15.38:0/946340434 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd74c03bc40 con 0x55ef2e1953f0 2024-09-16T00:33:37.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.493+0000 7fd73affd640 1 -- 172.21.15.38:0/946340434 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd74c031a20 con 0x55ef2e1953f0 2024-09-16T00:33:37.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.493+0000 7fd76455a740 10 client.15522 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:33:37.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.493+0000 7fd76455a740 10 client.15522 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:33:37.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.493+0000 7fd76455a740 20 client.15522 populate_metadata read hostname 'smithi038' 2024-09-16T00:33:37.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.493+0000 7fd76455a740 1 --2- 172.21.15.38:0/946340434 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x55ef2e219eb0 0x55ef2e21c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:37.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.493+0000 7fd76455a740 4 client.15522 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:33:37.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.493+0000 7fd76455a740 1 -- 172.21.15.38:0/946340434 --> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ef2e0f8d70 con 0x55ef2e219eb0 2024-09-16T00:33:37.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.494+0000 7fd75affd640 1 --2- 172.21.15.38:0/946340434 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x55ef2e219eb0 0x55ef2e21c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:37.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.494+0000 7fd73affd640 10 client.15522 ms_handle_connect on v2:172.21.15.73:6836/2681081265 2024-09-16T00:33:37.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.494+0000 7fd75affd640 1 --2- 172.21.15.38:0/946340434 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x55ef2e219eb0 0x55ef2e21c2d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c50c3853b65e5d3f server_cookie=ca585cb134fcbe35 in_seq=0 out_seq=0 2024-09-16T00:33:37.500 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.494+0000 7fd73affd640 1 -- 172.21.15.38:0/946340434 <== mds.1 v2:172.21.15.73:6836/2681081265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ef2e0f8d70 con 0x55ef2e219eb0 2024-09-16T00:33:37.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.494+0000 7fd73affd640 10 client.15522 handle_command_reply: tid=0 2024-09-16T00:33:37.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.494+0000 7fd76455a740 1 client.15522 shutdown 2024-09-16T00:33:37.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.495+0000 7fd76455a740 1 -- 172.21.15.38:0/946340434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef2e1953f0 msgr2=0x55ef2e1a3100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:37.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.495+0000 7fd76455a740 1 --2- 172.21.15.38:0/946340434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef2e1953f0 0x55ef2e1a3100 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fd74c02a940 tx=0x7fd74c02a420 comp rx=0 tx=0).stop 2024-09-16T00:33:37.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.495+0000 7fd76455a740 1 -- 172.21.15.38:0/946340434 shutdown_connections 2024-09-16T00:33:37.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.495+0000 7fd76455a740 1 --2- 172.21.15.38:0/946340434 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ef2e19ecf0 0x55ef2e1a3640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:37.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.495+0000 7fd76455a740 1 --2- 172.21.15.38:0/946340434 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x55ef2e219eb0 0x55ef2e21c2d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:37.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.495+0000 7fd73affd640 0 client.15522 ms_handle_reset on v2:172.21.15.73:6836/2681081265 2024-09-16T00:33:37.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.495+0000 7fd76455a740 1 --2- 172.21.15.38:0/946340434 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55ef2e1953f0 0x55ef2e1a3100 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:37.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.495+0000 7fd76455a740 1 --2- 172.21.15.38:0/946340434 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55ef2e154310 0x55ef2e1a2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:37.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.495+0000 7fd76455a740 1 -- 172.21.15.38:0/946340434 >> 172.21.15.38:0/946340434 conn(0x55ef2e197370 msgr2=0x55ef2e19a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:37.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.496+0000 7fd76455a740 1 -- 172.21.15.38:0/946340434 shutdown_connections 2024-09-16T00:33:37.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.496+0000 7fd76455a740 1 -- 172.21.15.38:0/946340434 wait complete. 2024-09-16T00:33:37.501 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.496+0000 7fd76455a740 20 client.15522 trim_cache size 0 max 16384 2024-09-16T00:33:37.502 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-09-16T00:33:37.502 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.497+0000 7fd7427fc640 1 -- 172.21.15.38:0/3143391549 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd7440a3350 msgr2=0x7fd7440a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:37.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.497+0000 7fd7427fc640 1 --2- 172.21.15.38:0/3143391549 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd7440a3350 0x7fd7440a5810 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd7540982c0 tx=0x7fd754098150 comp rx=0 tx=0).stop 2024-09-16T00:33:37.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.497+0000 7fd7427fc640 1 -- 172.21.15.38:0/3143391549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd75c1082e0 msgr2=0x7fd75c19a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:37.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.497+0000 7fd7427fc640 1 --2- 172.21.15.38:0/3143391549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd75c1082e0 0x7fd75c19a5d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fd75400eaa0 tx=0x7fd75400ef60 comp rx=0 tx=0).stop 2024-09-16T00:33:37.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.497+0000 7fd7427fc640 1 -- 172.21.15.38:0/3143391549 shutdown_connections 2024-09-16T00:33:37.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.497+0000 7fd7427fc640 1 --2- 172.21.15.38:0/3143391549 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd7440a3350 0x7fd7440a5810 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:37.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.497+0000 7fd7427fc640 1 --2- 172.21.15.38:0/3143391549 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd75c1082e0 0x7fd75c19a5d0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:37.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.497+0000 7fd7427fc640 1 --2- 172.21.15.38:0/3143391549 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd75c103760 0x7fd75c1a0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:37.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.497+0000 7fd7427fc640 1 --2- 172.21.15.38:0/3143391549 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd75c102db0 0x7fd75c1a0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:37.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.497+0000 7fd7427fc640 1 -- 172.21.15.38:0/3143391549 >> 172.21.15.38:0/3143391549 conn(0x7fd75c0fc7d0 msgr2=0x7fd75c10d020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:37.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.497+0000 7fd7427fc640 1 -- 172.21.15.38:0/3143391549 shutdown_connections 2024-09-16T00:33:37.503 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:37.497+0000 7fd7427fc640 1 -- 172.21.15.38:0/3143391549 wait complete. 2024-09-16T00:33:38.348 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:33:38.665 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:39.115 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:38 smithi038 ceph-mon[30945]: pgmap v390: 129 pgs: 129 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-16T00:33:39.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:38 smithi122 ceph-mon[35412]: pgmap v390: 129 pgs: 129 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-16T00:33:39.240 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.239+0000 7f32be5fa640 1 -- 172.21.15.38:0/3347627690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32b8101360 msgr2=0x7f32b810df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:39.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.239+0000 7f32be5fa640 1 --2- 172.21.15.38:0/3347627690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32b8101360 0x7f32b810df90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f32ac009900 tx=0x7f32ac02a410 comp rx=0 tx=0).stop 2024-09-16T00:33:39.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.239+0000 7f32be5fa640 1 -- 172.21.15.38:0/3347627690 shutdown_connections 2024-09-16T00:33:39.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.239+0000 7f32be5fa640 1 --2- 172.21.15.38:0/3347627690 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32b810e4d0 0x7f32b81108e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:39.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.239+0000 7f32be5fa640 1 --2- 172.21.15.38:0/3347627690 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32b8101360 0x7f32b810df90 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:39.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.239+0000 7f32be5fa640 1 --2- 172.21.15.38:0/3347627690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32b81009b0 0x7f32b8100d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:39.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.239+0000 7f32be5fa640 1 -- 172.21.15.38:0/3347627690 >> 172.21.15.38:0/3347627690 conn(0x7f32b80fc7d0 msgr2=0x7f32b80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:39.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.239+0000 7f32be5fa640 1 -- 172.21.15.38:0/3347627690 shutdown_connections 2024-09-16T00:33:39.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.239+0000 7f32be5fa640 1 -- 172.21.15.38:0/3347627690 wait complete. 2024-09-16T00:33:39.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.240+0000 7f32be5fa640 1 Processor -- start 2024-09-16T00:33:39.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.240+0000 7f32be5fa640 1 -- start start 2024-09-16T00:33:39.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.240+0000 7f32be5fa640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32b81009b0 0x7f32b81a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:39.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.240+0000 7f32be5fa640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32b810e4d0 0x7f32b81a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:39.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.240+0000 7f32be5fa640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32b819a760 0x7f32b819abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:39.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.240+0000 7f32be5fa640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f32b8114220 con 0x7f32b810e4d0 2024-09-16T00:33:39.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.240+0000 7f32be5fa640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f32b81140a0 con 0x7f32b81009b0 2024-09-16T00:33:39.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.240+0000 7f32be5fa640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f32b81143a0 con 0x7f32b819a760 2024-09-16T00:33:39.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.240+0000 7f32b77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32b810e4d0 0x7f32b81a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:39.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.240+0000 7f32b77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32b810e4d0 0x7f32b81a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55364/0 (socket says 172.21.15.38:55364) 2024-09-16T00:33:39.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.240+0000 7f32b77fe640 1 -- 172.21.15.38:0/2104117944 learned_addr learned my addr 172.21.15.38:0/2104117944 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:39.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.241+0000 7f32b77fe640 1 -- 172.21.15.38:0/2104117944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32b819a760 msgr2=0x7f32b819abe0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T00:33:39.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.241+0000 7f32b77fe640 1 --2- 172.21.15.38:0/2104117944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32b819a760 0x7f32b819abe0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:39.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.241+0000 7f32b77fe640 1 -- 172.21.15.38:0/2104117944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32b81009b0 msgr2=0x7f32b81a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:39.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.241+0000 7f32b7fff640 1 --2- 172.21.15.38:0/2104117944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32b81009b0 0x7f32b81a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:39.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.241+0000 7f32b77fe640 1 --2- 172.21.15.38:0/2104117944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32b81009b0 0x7f32b81a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:39.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.241+0000 7f32b77fe640 1 -- 172.21.15.38:0/2104117944 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32b819b310 con 0x7f32b810e4d0 2024-09-16T00:33:39.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.241+0000 7f32b77fe640 1 --2- 172.21.15.38:0/2104117944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32b810e4d0 0x7f32b81a0bd0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f32ac005940 tx=0x7f32ac004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:39.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.241+0000 7f32b57fa640 1 -- 172.21.15.38:0/2104117944 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ac03f030 con 0x7f32b810e4d0 2024-09-16T00:33:39.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.241+0000 7f32b57fa640 1 -- 172.21.15.38:0/2104117944 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32ac004440 con 0x7f32b810e4d0 2024-09-16T00:33:39.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.241+0000 7f32be5fa640 1 -- 172.21.15.38:0/2104117944 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32b819b5a0 con 0x7f32b810e4d0 2024-09-16T00:33:39.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.242+0000 7f32b57fa640 1 -- 172.21.15.38:0/2104117944 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ac035070 con 0x7f32b810e4d0 2024-09-16T00:33:39.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.242+0000 7f32be5fa640 1 -- 172.21.15.38:0/2104117944 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32b81a7610 con 0x7f32b810e4d0 2024-09-16T00:33:39.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.245+0000 7f32b57fa640 1 -- 172.21.15.38:0/2104117944 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f32ac042030 con 0x7f32b810e4d0 2024-09-16T00:33:39.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.246+0000 7f32c09d3740 1 Processor -- start 2024-09-16T00:33:39.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.246+0000 7f32c09d3740 1 -- start start 2024-09-16T00:33:39.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.246+0000 7f32c09d3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c78f8e350 0x560c78f8c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:39.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.246+0000 7f32c09d3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c78f8cde0 0x560c78f8d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:39.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.246+0000 7f32c09d3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560c78f97b80 0x560c78f99f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:39.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.246+0000 7f32b57fa640 1 --2- 172.21.15.38:0/2104117944 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f32a00a3350 0x7f32a00a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:39.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.246+0000 7f32c09d3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560c78eea2f0 con 0x560c78f8cde0 2024-09-16T00:33:39.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.246+0000 7f32c09d3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560c78e91cf0 con 0x560c78f8e350 2024-09-16T00:33:39.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.246+0000 7f32c09d3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560c78ee62b0 con 0x560c78f97b80 2024-09-16T00:33:39.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.246+0000 7f32bcb97640 1 --2- 172.21.15.38:0/2104117944 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f32a00a3350 0x7f32a00a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:39.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.247+0000 7f32bcb97640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c78f8e350 0x560c78f8c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:39.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.247+0000 7f32bcb97640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c78f8e350 0x560c78f8c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.38:50148/0 (socket says 172.21.15.38:50148) 2024-09-16T00:33:39.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.247+0000 7f32bcb97640 1 -- 172.21.15.38:0/3016315124 learned_addr learned my addr 172.21.15.38:0/3016315124 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:39.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.247+0000 7f32b57fa640 1 -- 172.21.15.38:0/2104117944 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f32ac0effd0 con 0x7f32b810e4d0 2024-09-16T00:33:39.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.247+0000 7f32bcb97640 1 -- 172.21.15.38:0/3016315124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560c78f97b80 msgr2=0x560c78f99f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:39.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.247+0000 7f32b77fe640 1 --2- 172.21.15.38:0/3016315124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560c78f97b80 0x560c78f99f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:39.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.247+0000 7f32bcb97640 1 --2- 172.21.15.38:0/3016315124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560c78f97b80 0x560c78f99f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:39.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.247+0000 7f32bcb97640 1 -- 172.21.15.38:0/3016315124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c78f8cde0 msgr2=0x560c78f8d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:39.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.247+0000 7f32b7fff640 1 --2- 172.21.15.38:0/3016315124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c78f8cde0 0x560c78f8d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:39.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.247+0000 7f32bcb97640 1 --2- 172.21.15.38:0/3016315124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c78f8cde0 0x560c78f8d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:39.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.247+0000 7f32bcb97640 1 -- 172.21.15.38:0/3016315124 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c78e763a0 con 0x560c78f8e350 2024-09-16T00:33:39.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.247+0000 7f32b7fff640 1 --2- 172.21.15.38:0/3016315124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c78f8cde0 0x560c78f8d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:33:39.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.248+0000 7f32bcb97640 1 --2- 172.21.15.38:0/2104117944 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f32a00a3350 0x7f32a00a5810 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f32b0009fd0 tx=0x7f32b0009e00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:39.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.248+0000 7f32bcb97640 1 --2- 172.21.15.38:0/3016315124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c78f8e350 0x560c78f8c8a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f32b0010c70 tx=0x7f32b001a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=146dc0179105e3e6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:39.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.248+0000 7f329e7fc640 1 -- 172.21.15.38:0/3016315124 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b0021070 con 0x560c78f8e350 2024-09-16T00:33:39.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.248+0000 7f329e7fc640 1 -- 172.21.15.38:0/3016315124 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32b0010d60 con 0x560c78f8e350 2024-09-16T00:33:39.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.248+0000 7f329e7fc640 1 -- 172.21.15.38:0/3016315124 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32b0005440 con 0x560c78f8e350 2024-09-16T00:33:39.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.249+0000 7f32c09d3740 1 -- 172.21.15.38:0/3016315124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c78f8e350 msgr2=0x560c78f8c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:39.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.249+0000 7f32c09d3740 1 --2- 172.21.15.38:0/3016315124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c78f8e350 0x560c78f8c8a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f32b0010c70 tx=0x7f32b001a000 comp rx=0 tx=0).stop 2024-09-16T00:33:39.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.249+0000 7f32c09d3740 1 -- 172.21.15.38:0/3016315124 shutdown_connections 2024-09-16T00:33:39.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.249+0000 7f32c09d3740 1 --2- 172.21.15.38:0/3016315124 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560c78f97b80 0x560c78f99f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:39.252 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.249+0000 7f32c09d3740 1 --2- 172.21.15.38:0/3016315124 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c78f8cde0 0x560c78f8d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:39.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.249+0000 7f32c09d3740 1 --2- 172.21.15.38:0/3016315124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c78f8e350 0x560c78f8c8a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:39.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.249+0000 7f32c09d3740 1 -- 172.21.15.38:0/3016315124 >> 172.21.15.38:0/3016315124 conn(0x560c78f4d310 msgr2=0x560c78f89470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:39.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.249+0000 7f32c09d3740 1 -- 172.21.15.38:0/3016315124 shutdown_connections 2024-09-16T00:33:39.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.249+0000 7f32c09d3740 1 -- 172.21.15.38:0/3016315124 wait complete. 2024-09-16T00:33:39.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.250+0000 7f32c09d3740 1 -- start start 2024-09-16T00:33:39.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.250+0000 7f32c09d3740 1 Processor -- start 2024-09-16T00:33:39.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.250+0000 7f32c09d3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560c78f8cde0 0x560c78f97580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:39.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.250+0000 7f32c09d3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c78f8e350 0x560c78f94830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:39.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.251+0000 7f32b7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560c78f8cde0 0x560c78f97580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:39.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.251+0000 7f32b77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c78f8e350 0x560c78f94830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:39.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.251+0000 7f32b7fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560c78f8cde0 0x560c78f97580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.38:37432/0 (socket says 172.21.15.38:37432) 2024-09-16T00:33:39.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.251+0000 7f32b77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c78f8e350 0x560c78f94830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55374/0 (socket says 172.21.15.38:55374) 2024-09-16T00:33:39.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.251+0000 7f32b7fff640 1 -- 172.21.15.38:0/1193396146 learned_addr learned my addr 172.21.15.38:0/1193396146 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:39.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.251+0000 7f32c09d3740 1 --2- 172.21.15.38:0/1193396146 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c78f97b80 0x560c78f94e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:39.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.251+0000 7f32c09d3740 1 -- 172.21.15.38:0/1193396146 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x560c78f9a820 con 0x560c78f8e350 2024-09-16T00:33:39.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.251+0000 7f32c09d3740 1 -- 172.21.15.38:0/1193396146 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x560c78ea5060 con 0x560c78f97b80 2024-09-16T00:33:39.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.251+0000 7f32c09d3740 1 -- 172.21.15.38:0/1193396146 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560c78f9a9a0 con 0x560c78f8cde0 2024-09-16T00:33:39.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.251+0000 7f32bcb97640 1 --2- 172.21.15.38:0/1193396146 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c78f97b80 0x560c78f94e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:39.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.251+0000 7f32b7fff640 1 -- 172.21.15.38:0/1193396146 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c78f97b80 msgr2=0x560c78f94e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:39.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.251+0000 7f32b7fff640 1 --2- 172.21.15.38:0/1193396146 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c78f97b80 0x560c78f94e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:39.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.251+0000 7f32b7fff640 1 -- 172.21.15.38:0/1193396146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c78f8e350 msgr2=0x560c78f94830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:39.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.251+0000 7f32b7fff640 1 --2- 172.21.15.38:0/1193396146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c78f8e350 0x560c78f94830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:39.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.251+0000 7f32b7fff640 1 -- 172.21.15.38:0/1193396146 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c78c5b3c0 con 0x560c78f8cde0 2024-09-16T00:33:39.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.251+0000 7f32bcb97640 1 --2- 172.21.15.38:0/1193396146 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c78f97b80 0x560c78f94e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:39.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.251+0000 7f32b77fe640 1 --2- 172.21.15.38:0/1193396146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c78f8e350 0x560c78f94830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:33:39.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.251+0000 7f32c09d3740 10 client.0 _ll_register_callbacks cb 0x560c78bf88c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:39.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.251+0000 7f32c09d3740 1 -- 172.21.15.38:0/1193396146 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32b0010d60 con 0x560c78f8cde0 2024-09-16T00:33:39.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.252+0000 7f32b7fff640 1 --2- 172.21.15.38:0/1193396146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560c78f8cde0 0x560c78f97580 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f32a8011690 tx=0x7f32a8011b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fc897b6e59eb0e9e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:39.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.252+0000 7f32977fe640 10 client.25108 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-16T00:33:39.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.252+0000 7f32977fe640 1 -- 172.21.15.38:0/1193396146 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a8004220 con 0x560c78f8cde0 2024-09-16T00:33:39.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.252+0000 7f32977fe640 1 -- 172.21.15.38:0/1193396146 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f32a80043c0 con 0x560c78f8cde0 2024-09-16T00:33:39.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.252+0000 7f32977fe640 1 -- 172.21.15.38:0/1193396146 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f32a800f940 con 0x560c78f8cde0 2024-09-16T00:33:39.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.252+0000 7f32c09d3740 10 client.25108 fetch_fsmap learned FSMap version 26 2024-09-16T00:33:39.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.252+0000 7f32c09d3740 1 -- 172.21.15.38:0/1193396146 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({fsmap=26}) -- 0x560c78fa6190 con 0x560c78f8cde0 2024-09-16T00:33:39.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.253+0000 7f32977fe640 1 -- 172.21.15.38:0/1193396146 <== mon.1 v2:172.21.15.122:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f32a800fae0 con 0x560c78f8cde0 2024-09-16T00:33:39.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.253+0000 7f32c09d3740 10 client.25108 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:33:39.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.253+0000 7f32c09d3740 10 client.25108 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:33:39.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.253+0000 7f32c09d3740 20 client.25108 populate_metadata read hostname 'smithi038' 2024-09-16T00:33:39.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.253+0000 7f32c09d3740 1 --2- 172.21.15.38:0/1193396146 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x560c78fa7590 0x560c78fa9a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:39.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.253+0000 7f32c09d3740 4 client.25108 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:33:39.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.253+0000 7f32c09d3740 1 -- 172.21.15.38:0/1193396146 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c78ef7860 con 0x560c78fa7590 2024-09-16T00:33:39.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.253+0000 7f32b77fe640 1 --2- 172.21.15.38:0/1193396146 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x560c78fa7590 0x560c78fa9a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:39.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.254+0000 7f32977fe640 10 client.25108 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:33:39.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.253+0000 7f32b77fe640 1 --2- 172.21.15.38:0/1193396146 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x560c78fa7590 0x560c78fa9a20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a75e177c0739f6b6 server_cookie=1c7e8e0cff4615a3 in_seq=0 out_seq=0 2024-09-16T00:33:39.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.254+0000 7f32977fe640 1 -- 172.21.15.38:0/1193396146 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x560c78ef7860 con 0x560c78fa7590 2024-09-16T00:33:39.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.254+0000 7f32977fe640 10 client.25108 handle_command_reply: tid=0 2024-09-16T00:33:39.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.254+0000 7f32c09d3740 1 client.25108 shutdown 2024-09-16T00:33:39.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.254+0000 7f32c09d3740 1 -- 172.21.15.38:0/1193396146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560c78f8cde0 msgr2=0x560c78f97580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:39.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.254+0000 7f32c09d3740 1 --2- 172.21.15.38:0/1193396146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560c78f8cde0 0x560c78f97580 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f32a8011690 tx=0x7f32a8011b50 comp rx=0 tx=0).stop 2024-09-16T00:33:39.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.255+0000 7f32c09d3740 1 -- 172.21.15.38:0/1193396146 shutdown_connections 2024-09-16T00:33:39.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.255+0000 7f32c09d3740 1 --2- 172.21.15.38:0/1193396146 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c78f97b80 0x560c78f94e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:39.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.255+0000 7f32c09d3740 1 --2- 172.21.15.38:0/1193396146 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x560c78fa7590 0x560c78fa9a20 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:39.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.255+0000 7f32977fe640 0 client.25108 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:33:39.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.255+0000 7f32c09d3740 1 --2- 172.21.15.38:0/1193396146 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c78f8e350 0x560c78f94830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:39.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.255+0000 7f32c09d3740 1 --2- 172.21.15.38:0/1193396146 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560c78f8cde0 0x560c78f97580 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:39.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.255+0000 7f32c09d3740 1 -- 172.21.15.38:0/1193396146 >> 172.21.15.38:0/1193396146 conn(0x560c78f4d310 msgr2=0x560c78f8f140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:39.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.255+0000 7f32c09d3740 1 -- 172.21.15.38:0/1193396146 shutdown_connections 2024-09-16T00:33:39.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.255+0000 7f32c09d3740 1 -- 172.21.15.38:0/1193396146 wait complete. 2024-09-16T00:33:39.259 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.255+0000 7f32c09d3740 20 client.25108 trim_cache size 0 max 16384 2024-09-16T00:33:39.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.280+0000 7f32c09d3740 1 Processor -- start 2024-09-16T00:33:39.282 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.281+0000 7f32c09d3740 1 -- start start 2024-09-16T00:33:39.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.281+0000 7f32c09d3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c78f8cde0 0x560c78fa3b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:39.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.281+0000 7f32c09d3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560c78f8e350 0x560c78f8f930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:39.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.281+0000 7f32c09d3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c78f97b80 0x560c78f8fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:39.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.281+0000 7f32c09d3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f32b81af890 con 0x560c78f8cde0 2024-09-16T00:33:39.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.281+0000 7f32c09d3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f32b81ad160 con 0x560c78f97b80 2024-09-16T00:33:39.283 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.281+0000 7f32c09d3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f32b81aafb0 con 0x560c78f8e350 2024-09-16T00:33:39.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.281+0000 7f32b7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c78f8cde0 0x560c78fa3b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:39.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.281+0000 7f32b7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c78f8cde0 0x560c78fa3b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55378/0 (socket says 172.21.15.38:55378) 2024-09-16T00:33:39.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.281+0000 7f32b7fff640 1 -- 172.21.15.38:0/151528772 learned_addr learned my addr 172.21.15.38:0/151528772 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:39.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.281+0000 7f32b77fe640 1 --2- 172.21.15.38:0/151528772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560c78f8e350 0x560c78f8f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:39.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.282+0000 7f32bcb97640 1 --2- 172.21.15.38:0/151528772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c78f97b80 0x560c78f8fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:39.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.282+0000 7f32b7fff640 1 -- 172.21.15.38:0/151528772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560c78f8e350 msgr2=0x560c78f8f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:39.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.282+0000 7f32b7fff640 1 --2- 172.21.15.38:0/151528772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560c78f8e350 0x560c78f8f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:39.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.282+0000 7f32b7fff640 1 -- 172.21.15.38:0/151528772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c78f97b80 msgr2=0x560c78f8fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:39.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.282+0000 7f32b7fff640 1 --2- 172.21.15.38:0/151528772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c78f97b80 0x560c78f8fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:39.284 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.282+0000 7f32b7fff640 1 -- 172.21.15.38:0/151528772 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c78f8a170 con 0x560c78f8cde0 2024-09-16T00:33:39.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.282+0000 7f32b7fff640 1 --2- 172.21.15.38:0/151528772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c78f8cde0 0x560c78fa3b10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f32a8011fd0 tx=0x7f32a8014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37f3d1b770cc423c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:39.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.283+0000 7f329dffb640 1 -- 172.21.15.38:0/151528772 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a800f730 con 0x560c78f8cde0 2024-09-16T00:33:39.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.283+0000 7f329dffb640 1 -- 172.21.15.38:0/151528772 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32b0010d60 con 0x560c78f8cde0 2024-09-16T00:33:39.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.283+0000 7f329dffb640 1 -- 172.21.15.38:0/151528772 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a801a440 con 0x560c78f8cde0 2024-09-16T00:33:39.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.284+0000 7f32c09d3740 1 -- 172.21.15.38:0/151528772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c78f8cde0 msgr2=0x560c78fa3b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:39.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.284+0000 7f32c09d3740 1 --2- 172.21.15.38:0/151528772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c78f8cde0 0x560c78fa3b10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f32a8011fd0 tx=0x7f32a8014000 comp rx=0 tx=0).stop 2024-09-16T00:33:39.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.284+0000 7f32c09d3740 1 -- 172.21.15.38:0/151528772 shutdown_connections 2024-09-16T00:33:39.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.284+0000 7f32c09d3740 1 --2- 172.21.15.38:0/151528772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c78f97b80 0x560c78f8fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:39.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.284+0000 7f32c09d3740 1 --2- 172.21.15.38:0/151528772 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560c78f8e350 0x560c78f8f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:39.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.284+0000 7f32c09d3740 1 --2- 172.21.15.38:0/151528772 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c78f8cde0 0x560c78fa3b10 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:39.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.284+0000 7f32c09d3740 1 -- 172.21.15.38:0/151528772 >> 172.21.15.38:0/151528772 conn(0x560c78f4d310 msgr2=0x560c78f7fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:39.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.284+0000 7f32c09d3740 1 -- 172.21.15.38:0/151528772 shutdown_connections 2024-09-16T00:33:39.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.284+0000 7f32c09d3740 1 -- 172.21.15.38:0/151528772 wait complete. 2024-09-16T00:33:39.287 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.284+0000 7f32c09d3740 1 -- start start 2024-09-16T00:33:39.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.285+0000 7f32c09d3740 1 Processor -- start 2024-09-16T00:33:39.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.285+0000 7f32c09d3740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560c78f8e350 0x560c78f9b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:39.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.285+0000 7f32c09d3740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c78f97b80 0x560c78f9be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:39.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.285+0000 7f32c09d3740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c78fa7590 0x560c78f9c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:39.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.285+0000 7f32c09d3740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f32b81ab620 con 0x560c78f97b80 2024-09-16T00:33:39.288 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.285+0000 7f32c09d3740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f32b81aca90 con 0x560c78fa7590 2024-09-16T00:33:39.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.285+0000 7f32c09d3740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f32b81abcf0 con 0x560c78f8e350 2024-09-16T00:33:39.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.285+0000 7f32c09d3740 10 client.0 _ll_register_callbacks cb 0x7f32a8009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:39.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.285+0000 7f32b77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c78f97b80 0x560c78f9be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:39.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.285+0000 7f32b77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c78f97b80 0x560c78f9be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55388/0 (socket says 172.21.15.38:55388) 2024-09-16T00:33:39.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.285+0000 7f32b77fe640 1 -- 172.21.15.38:0/1741289523 learned_addr learned my addr 172.21.15.38:0/1741289523 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:39.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.285+0000 7f32b7fff640 1 --2- 172.21.15.38:0/1741289523 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560c78f8e350 0x560c78f9b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:39.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.286+0000 7f32b77fe640 1 -- 172.21.15.38:0/1741289523 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560c78f8e350 msgr2=0x560c78f9b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:39.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.286+0000 7f32b77fe640 1 --2- 172.21.15.38:0/1741289523 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560c78f8e350 0x560c78f9b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:39.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.286+0000 7f32b77fe640 1 -- 172.21.15.38:0/1741289523 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c78fa7590 msgr2=0x560c78f9c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:39.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.286+0000 7f32b77fe640 1 --2- 172.21.15.38:0/1741289523 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c78fa7590 0x560c78f9c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:39.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.286+0000 7f32b77fe640 1 -- 172.21.15.38:0/1741289523 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c79011260 con 0x560c78f97b80 2024-09-16T00:33:39.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.286+0000 7f32b77fe640 1 --2- 172.21.15.38:0/1741289523 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c78f97b80 0x560c78f9be70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f32ac0b8510 tx=0x7f32ac0ed5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5421d8b523aea037 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:39.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.286+0000 7f32977fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:33:39.289 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.286+0000 7f32977fe640 1 -- 172.21.15.38:0/1741289523 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32ac049040 con 0x560c78f97b80 2024-09-16T00:33:39.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.286+0000 7f32977fe640 1 -- 172.21.15.38:0/1741289523 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f32ac0b8720 con 0x560c78f97b80 2024-09-16T00:33:39.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.286+0000 7f32c09d3740 1 -- 172.21.15.38:0/1741289523 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32b0010d60 con 0x560c78f97b80 2024-09-16T00:33:39.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.287+0000 7f32977fe640 1 -- 172.21.15.38:0/1741289523 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f32ac0c00c0 con 0x560c78f97b80 2024-09-16T00:33:39.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.287+0000 7f32c09d3740 10 client.15537 fetch_fsmap learned FSMap version 26 2024-09-16T00:33:39.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.287+0000 7f32c09d3740 1 -- 172.21.15.38:0/1741289523 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x560c79012500 con 0x560c78f97b80 2024-09-16T00:33:39.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.287+0000 7f32977fe640 1 -- 172.21.15.38:0/1741289523 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f32b0010d60 con 0x560c78f97b80 2024-09-16T00:33:39.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.287+0000 7f32977fe640 1 -- 172.21.15.38:0/1741289523 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f32ac04c060 con 0x560c78f97b80 2024-09-16T00:33:39.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.287+0000 7f32c09d3740 10 client.15537 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:33:39.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.287+0000 7f32c09d3740 10 client.15537 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:33:39.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.287+0000 7f32c09d3740 20 client.15537 populate_metadata read hostname 'smithi038' 2024-09-16T00:33:39.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.287+0000 7f32c09d3740 1 --2- 172.21.15.38:0/1741289523 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x560c79012c20 0x560c79015010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:39.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.287+0000 7f32c09d3740 4 client.15537 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:33:39.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.287+0000 7f32c09d3740 1 -- 172.21.15.38:0/1741289523 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560c78ef1d70 con 0x560c79012c20 2024-09-16T00:33:39.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.288+0000 7f32b7fff640 1 --2- 172.21.15.38:0/1741289523 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x560c79012c20 0x560c79015010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:39.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.288+0000 7f32b7fff640 1 --2- 172.21.15.38:0/1741289523 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x560c79012c20 0x560c79015010 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91500105f1a8be55 server_cookie=9fe5c9604e7b06f4 in_seq=0 out_seq=0 2024-09-16T00:33:39.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.288+0000 7f32977fe640 10 client.15537 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:33:39.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.288+0000 7f32977fe640 1 -- 172.21.15.38:0/1741289523 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560c78ef1d70 con 0x560c79012c20 2024-09-16T00:33:39.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.288+0000 7f32977fe640 10 client.15537 handle_command_reply: tid=0 2024-09-16T00:33:39.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.288+0000 7f32c09d3740 1 client.15537 shutdown 2024-09-16T00:33:39.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.289+0000 7f32c09d3740 1 -- 172.21.15.38:0/1741289523 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c78f97b80 msgr2=0x560c78f9be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:39.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.289+0000 7f32c09d3740 1 --2- 172.21.15.38:0/1741289523 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c78f97b80 0x560c78f9be70 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f32ac0b8510 tx=0x7f32ac0ed5d0 comp rx=0 tx=0).stop 2024-09-16T00:33:39.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.289+0000 7f32c09d3740 1 -- 172.21.15.38:0/1741289523 shutdown_connections 2024-09-16T00:33:39.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.289+0000 7f32c09d3740 1 --2- 172.21.15.38:0/1741289523 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560c78fa7590 0x560c78f9c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:39.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.289+0000 7f32c09d3740 1 --2- 172.21.15.38:0/1741289523 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x560c79012c20 0x560c79015010 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:39.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.289+0000 7f32c09d3740 1 --2- 172.21.15.38:0/1741289523 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560c78f97b80 0x560c78f9be70 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:39.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.289+0000 7f32c09d3740 1 --2- 172.21.15.38:0/1741289523 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560c78f8e350 0x560c78f9b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:39.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.289+0000 7f32c09d3740 1 -- 172.21.15.38:0/1741289523 >> 172.21.15.38:0/1741289523 conn(0x560c78f4d310 msgr2=0x560c78f940c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:39.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.289+0000 7f32977fe640 0 client.15537 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:33:39.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.289+0000 7f32c09d3740 1 -- 172.21.15.38:0/1741289523 shutdown_connections 2024-09-16T00:33:39.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.289+0000 7f32c09d3740 1 -- 172.21.15.38:0/1741289523 wait complete. 2024-09-16T00:33:39.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.289+0000 7f32c09d3740 20 client.15537 trim_cache size 0 max 16384 2024-09-16T00:33:39.293 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-09-16T00:33:39.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.290+0000 7f32be5fa640 1 -- 172.21.15.38:0/2104117944 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f32a00a3350 msgr2=0x7f32a00a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:39.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.290+0000 7f32be5fa640 1 --2- 172.21.15.38:0/2104117944 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f32a00a3350 0x7f32a00a5810 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f32b0009fd0 tx=0x7f32b0009e00 comp rx=0 tx=0).stop 2024-09-16T00:33:39.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.290+0000 7f32be5fa640 1 -- 172.21.15.38:0/2104117944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32b810e4d0 msgr2=0x7f32b81a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:39.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.290+0000 7f32be5fa640 1 --2- 172.21.15.38:0/2104117944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32b810e4d0 0x7f32b81a0bd0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f32ac005940 tx=0x7f32ac004040 comp rx=0 tx=0).stop 2024-09-16T00:33:39.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.290+0000 7f32be5fa640 1 -- 172.21.15.38:0/2104117944 shutdown_connections 2024-09-16T00:33:39.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.290+0000 7f32be5fa640 1 --2- 172.21.15.38:0/2104117944 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f32a00a3350 0x7f32a00a5810 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:39.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.290+0000 7f32be5fa640 1 --2- 172.21.15.38:0/2104117944 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f32b819a760 0x7f32b819abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:39.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.290+0000 7f32be5fa640 1 --2- 172.21.15.38:0/2104117944 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f32b810e4d0 0x7f32b81a0bd0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:39.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.290+0000 7f32be5fa640 1 --2- 172.21.15.38:0/2104117944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32b81009b0 0x7f32b81a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:39.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.290+0000 7f32be5fa640 1 -- 172.21.15.38:0/2104117944 >> 172.21.15.38:0/2104117944 conn(0x7f32b80fc7d0 msgr2=0x7f32b80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:39.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.291+0000 7f32be5fa640 1 -- 172.21.15.38:0/2104117944 shutdown_connections 2024-09-16T00:33:39.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:39.291+0000 7f32be5fa640 1 -- 172.21.15.38:0/2104117944 wait complete. 2024-09-16T00:33:39.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:38 smithi073 ceph-mon[35254]: pgmap v390: 129 pgs: 129 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-16T00:33:41.122 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:33:41.122 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:33:41.214 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:40 smithi038 ceph-mon[30945]: pgmap v391: 129 pgs: 129 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-16T00:33:41.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:40 smithi122 ceph-mon[35412]: pgmap v391: 129 pgs: 129 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-16T00:33:41.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:40 smithi073 ceph-mon[35254]: pgmap v391: 129 pgs: 129 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 682 B/s wr, 7 op/s 2024-09-16T00:33:41.432 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:41.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.946+0000 7efdd475b640 1 -- 172.21.15.38:0/2190566244 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdcc102d30 msgr2=0x7efdcc103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:41.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.946+0000 7efdd475b640 1 --2- 172.21.15.38:0/2190566244 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdcc102d30 0x7efdcc103110 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7efdc8009930 tx=0x7efdc802a420 comp rx=0 tx=0).stop 2024-09-16T00:33:41.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.946+0000 7efdd475b640 1 -- 172.21.15.38:0/2190566244 shutdown_connections 2024-09-16T00:33:41.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.946+0000 7efdd475b640 1 --2- 172.21.15.38:0/2190566244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efdcc1082a0 0x7efdcc10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:41.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.946+0000 7efdd475b640 1 --2- 172.21.15.38:0/2190566244 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efdcc1036e0 0x7efdcc107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:41.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.946+0000 7efdd475b640 1 --2- 172.21.15.38:0/2190566244 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdcc102d30 0x7efdcc103110 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:41.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.946+0000 7efdd475b640 1 -- 172.21.15.38:0/2190566244 >> 172.21.15.38:0/2190566244 conn(0x7efdcc0fc7d0 msgr2=0x7efdcc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:41.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.946+0000 7efdd475b640 1 -- 172.21.15.38:0/2190566244 shutdown_connections 2024-09-16T00:33:41.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.946+0000 7efdd475b640 1 -- 172.21.15.38:0/2190566244 wait complete. 2024-09-16T00:33:41.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.947+0000 7efdd475b640 1 Processor -- start 2024-09-16T00:33:41.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.947+0000 7efdd475b640 1 -- start start 2024-09-16T00:33:41.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.947+0000 7efdd475b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efdcc102d30 0x7efdcc100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:41.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.947+0000 7efdd475b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efdcc1036e0 0x7efdcc100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:41.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.947+0000 7efdd475b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdcc1082a0 0x7efdcc101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:41.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.947+0000 7efdd475b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efdcc1120a0 con 0x7efdcc1082a0 2024-09-16T00:33:41.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.947+0000 7efdd475b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efdcc111f20 con 0x7efdcc102d30 2024-09-16T00:33:41.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.947+0000 7efdd475b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efdcc112220 con 0x7efdcc1036e0 2024-09-16T00:33:41.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.948+0000 7efdd2cf8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdcc1082a0 0x7efdcc101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:41.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.948+0000 7efdd24f7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efdcc102d30 0x7efdcc100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:41.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.948+0000 7efdd1cf6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efdcc1036e0 0x7efdcc100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:41.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.948+0000 7efdd2cf8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdcc1082a0 0x7efdcc101410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55408/0 (socket says 172.21.15.38:55408) 2024-09-16T00:33:41.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.948+0000 7efdd24f7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efdcc102d30 0x7efdcc100990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.38:50176/0 (socket says 172.21.15.38:50176) 2024-09-16T00:33:41.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.948+0000 7efdd2cf8640 1 -- 172.21.15.38:0/2739169570 learned_addr learned my addr 172.21.15.38:0/2739169570 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:41.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.948+0000 7efdd1cf6640 1 -- 172.21.15.38:0/2739169570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efdcc102d30 msgr2=0x7efdcc100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:41.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.948+0000 7efdd1cf6640 1 --2- 172.21.15.38:0/2739169570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efdcc102d30 0x7efdcc100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:41.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.948+0000 7efdd1cf6640 1 -- 172.21.15.38:0/2739169570 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdcc1082a0 msgr2=0x7efdcc101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:41.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.948+0000 7efdd1cf6640 1 --2- 172.21.15.38:0/2739169570 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdcc1082a0 0x7efdcc101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:41.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.948+0000 7efdd1cf6640 1 -- 172.21.15.38:0/2739169570 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efdcc1a6a80 con 0x7efdcc1036e0 2024-09-16T00:33:41.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.948+0000 7efdd1cf6640 1 --2- 172.21.15.38:0/2739169570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efdcc1036e0 0x7efdcc100ed0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efdcc0697d0 tx=0x7efdbc00ee50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:41.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.948+0000 7efdc37fe640 1 -- 172.21.15.38:0/2739169570 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdbc00cc40 con 0x7efdcc1036e0 2024-09-16T00:33:41.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.949+0000 7efdc37fe640 1 -- 172.21.15.38:0/2739169570 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdbc00cde0 con 0x7efdcc1036e0 2024-09-16T00:33:41.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.949+0000 7efdd475b640 1 -- 172.21.15.38:0/2739169570 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efdcc1a6d70 con 0x7efdcc1036e0 2024-09-16T00:33:41.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.949+0000 7efdc37fe640 1 -- 172.21.15.38:0/2739169570 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdbc018980 con 0x7efdcc1036e0 2024-09-16T00:33:41.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.949+0000 7efdd475b640 1 -- 172.21.15.38:0/2739169570 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efdcc1a7250 con 0x7efdcc1036e0 2024-09-16T00:33:41.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.951+0000 7efdc37fe640 1 -- 172.21.15.38:0/2739169570 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7efdbc018b20 con 0x7efdcc1036e0 2024-09-16T00:33:41.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.951+0000 7efdd6b34740 1 Processor -- start 2024-09-16T00:33:41.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.952+0000 7efdd6b34740 1 -- start start 2024-09-16T00:33:41.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.952+0000 7efdd6b34740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564467797b60 0x564467797f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:41.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.952+0000 7efdd6b34740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564467798480 0x5644677a2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:41.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.952+0000 7efdd6b34740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644677a2d40 0x5644677a5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:41.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.952+0000 7efdd6b34740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5644676fd280 con 0x564467798480 2024-09-16T00:33:41.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.952+0000 7efdd6b34740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5644676a4b10 con 0x5644677a2d40 2024-09-16T00:33:41.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.952+0000 7efdd6b34740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5644676f9240 con 0x564467797b60 2024-09-16T00:33:41.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.952+0000 7efdd24f7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564467798480 0x5644677a2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:41.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.952+0000 7efdd24f7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564467798480 0x5644677a2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55410/0 (socket says 172.21.15.38:55410) 2024-09-16T00:33:41.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.952+0000 7efdd24f7640 1 -- 172.21.15.38:0/1927812226 learned_addr learned my addr 172.21.15.38:0/1927812226 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:41.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.952+0000 7efdd2cf8640 1 --2- 172.21.15.38:0/1927812226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564467797b60 0x564467797f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:41.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.952+0000 7efdc37fe640 1 --2- 172.21.15.38:0/2739169570 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efdb80a3350 0x7efdb80a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:41.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.953+0000 7efdd24f7640 1 -- 172.21.15.38:0/1927812226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564467797b60 msgr2=0x564467797f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:41.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.953+0000 7efdd24f7640 1 --2- 172.21.15.38:0/1927812226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564467797b60 0x564467797f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:41.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.953+0000 7efdd24f7640 1 -- 172.21.15.38:0/1927812226 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644677a2d40 msgr2=0x5644677a5130 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:33:41.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.953+0000 7efdd24f7640 1 --2- 172.21.15.38:0/1927812226 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644677a2d40 0x5644677a5130 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:41.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.953+0000 7efdd24f7640 1 -- 172.21.15.38:0/1927812226 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644676b11f0 con 0x564467798480 2024-09-16T00:33:41.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.953+0000 7efdd2cf8640 1 --2- 172.21.15.38:0/1927812226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564467797b60 0x564467797f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:33:41.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.953+0000 7efdc37fe640 1 -- 172.21.15.38:0/2739169570 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7efdbc0d00f0 con 0x7efdcc1036e0 2024-09-16T00:33:41.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.953+0000 7efdd24f7640 1 --2- 172.21.15.38:0/1927812226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564467798480 0x5644677a2800 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7efdc802a940 tx=0x7efdc8005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4f68db4b9cc0042 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:41.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.953+0000 7efdd2cf8640 1 --2- 172.21.15.38:0/2739169570 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efdb80a3350 0x7efdb80a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:41.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.954+0000 7efdc0ff9640 1 -- 172.21.15.38:0/1927812226 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc8040070 con 0x564467798480 2024-09-16T00:33:41.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.954+0000 7efdd2cf8640 1 --2- 172.21.15.38:0/2739169570 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efdb80a3350 0x7efdb80a5810 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7efdc40049d0 tx=0x7efdc4004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:41.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.954+0000 7efdc0ff9640 1 -- 172.21.15.38:0/1927812226 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdc8031870 con 0x564467798480 2024-09-16T00:33:41.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.954+0000 7efdc0ff9640 1 -- 172.21.15.38:0/1927812226 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdc8035040 con 0x564467798480 2024-09-16T00:33:41.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.954+0000 7efdd6b34740 1 -- 172.21.15.38:0/1927812226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564467798480 msgr2=0x5644677a2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:41.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.954+0000 7efdd6b34740 1 --2- 172.21.15.38:0/1927812226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564467798480 0x5644677a2800 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7efdc802a940 tx=0x7efdc8005bd0 comp rx=0 tx=0).stop 2024-09-16T00:33:41.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.955+0000 7efdd6b34740 1 -- 172.21.15.38:0/1927812226 shutdown_connections 2024-09-16T00:33:41.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.955+0000 7efdd6b34740 1 --2- 172.21.15.38:0/1927812226 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644677a2d40 0x5644677a5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:41.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.955+0000 7efdd6b34740 1 --2- 172.21.15.38:0/1927812226 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564467798480 0x5644677a2800 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:41.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.955+0000 7efdd6b34740 1 --2- 172.21.15.38:0/1927812226 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564467797b60 0x564467797f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:41.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.955+0000 7efdd6b34740 1 -- 172.21.15.38:0/1927812226 >> 172.21.15.38:0/1927812226 conn(0x5644677940f0 msgr2=0x5644677944d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:41.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.955+0000 7efdd6b34740 1 -- 172.21.15.38:0/1927812226 shutdown_connections 2024-09-16T00:33:41.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.955+0000 7efdd6b34740 1 -- 172.21.15.38:0/1927812226 wait complete. 2024-09-16T00:33:41.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.955+0000 7efdd6b34740 1 -- start start 2024-09-16T00:33:41.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.955+0000 7efdd6b34740 1 Processor -- start 2024-09-16T00:33:41.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.956+0000 7efdd6b34740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564467797b60 0x56446779fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:41.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.956+0000 7efdd6b34740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564467798480 0x5644677a0220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:41.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.956+0000 7efdd6b34740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644677a2d40 0x5644677a0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:41.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.956+0000 7efdd6b34740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5644677a5910 con 0x564467798480 2024-09-16T00:33:41.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.956+0000 7efdd6b34740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5644676b7f10 con 0x5644677a2d40 2024-09-16T00:33:41.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.956+0000 7efdd6b34740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5644677a5a90 con 0x564467797b60 2024-09-16T00:33:41.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.956+0000 7efdd6b34740 10 client.0 _ll_register_callbacks cb 0x564467680a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:41.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.956+0000 7efdd1cf6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564467798480 0x5644677a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:41.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.956+0000 7efdd24f7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564467797b60 0x56446779fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:41.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.956+0000 7efdd1cf6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564467798480 0x5644677a0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55420/0 (socket says 172.21.15.38:55420) 2024-09-16T00:33:41.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.956+0000 7efdd1cf6640 1 -- 172.21.15.38:0/3536999129 learned_addr learned my addr 172.21.15.38:0/3536999129 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:41.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.956+0000 7efdd24f7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564467797b60 0x56446779fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.38:37476/0 (socket says 172.21.15.38:37476) 2024-09-16T00:33:41.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.956+0000 7efdd2cf8640 1 --2- 172.21.15.38:0/3536999129 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644677a2d40 0x5644677a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:41.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.956+0000 7efdd1cf6640 1 -- 172.21.15.38:0/3536999129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564467797b60 msgr2=0x56446779fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:41.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.956+0000 7efdd1cf6640 1 --2- 172.21.15.38:0/3536999129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564467797b60 0x56446779fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:41.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.956+0000 7efdd1cf6640 1 -- 172.21.15.38:0/3536999129 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644677a2d40 msgr2=0x5644677a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:41.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.956+0000 7efdd1cf6640 1 --2- 172.21.15.38:0/3536999129 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644677a2d40 0x5644677a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:41.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.956+0000 7efdd1cf6640 1 -- 172.21.15.38:0/3536999129 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56446746e6a0 con 0x564467798480 2024-09-16T00:33:41.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.956+0000 7efdd1cf6640 1 --2- 172.21.15.38:0/3536999129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564467798480 0x5644677a0220 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7efdbc098c50 tx=0x7efdbc09bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbc1a6ef5e699d83 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:41.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.956+0000 7efdb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:33:41.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.957+0000 7efdb5ffb640 1 -- 172.21.15.38:0/3536999129 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdbc0bc0e0 con 0x564467798480 2024-09-16T00:33:41.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.957+0000 7efdb5ffb640 1 -- 172.21.15.38:0/3536999129 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7efdbc0c94d0 con 0x564467798480 2024-09-16T00:33:41.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.957+0000 7efdd6b34740 1 -- 172.21.15.38:0/3536999129 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdc8031870 con 0x564467798480 2024-09-16T00:33:41.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.957+0000 7efdb5ffb640 1 -- 172.21.15.38:0/3536999129 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efdbc0c96d0 con 0x564467798480 2024-09-16T00:33:41.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.957+0000 7efdd6b34740 10 client.15549 fetch_fsmap learned FSMap version 26 2024-09-16T00:33:41.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.957+0000 7efdd6b34740 1 -- 172.21.15.38:0/3536999129 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5644677b1540 con 0x564467798480 2024-09-16T00:33:41.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.957+0000 7efdb5ffb640 1 -- 172.21.15.38:0/3536999129 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdc8031870 con 0x564467798480 2024-09-16T00:33:41.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.958+0000 7efdb5ffb640 1 -- 172.21.15.38:0/3536999129 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7efdbc0cd1b0 con 0x564467798480 2024-09-16T00:33:41.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.958+0000 7efdd6b34740 10 client.15549 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:33:41.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.958+0000 7efdd6b34740 10 client.15549 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:33:41.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.958+0000 7efdd6b34740 20 client.15549 populate_metadata read hostname 'smithi038' 2024-09-16T00:33:41.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.958+0000 7efdd6b34740 1 --2- 172.21.15.38:0/3536999129 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5644677b2950 0x5644677b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:41.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.958+0000 7efdd6b34740 4 client.15549 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:33:41.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.958+0000 7efdd6b34740 1 -- 172.21.15.38:0/3536999129 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56446770a7f0 con 0x5644677b2950 2024-09-16T00:33:41.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.958+0000 7efdd24f7640 1 --2- 172.21.15.38:0/3536999129 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5644677b2950 0x5644677b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:41.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.959+0000 7efdb5ffb640 10 client.15549 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:33:41.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.959+0000 7efdd24f7640 1 --2- 172.21.15.38:0/3536999129 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5644677b2950 0x5644677b4d40 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae4c73db01654e48 server_cookie=1c64af40bac94a91 in_seq=0 out_seq=0 2024-09-16T00:33:41.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.959+0000 7efdb5ffb640 1 -- 172.21.15.38:0/3536999129 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56446770a7f0 con 0x5644677b2950 2024-09-16T00:33:41.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.959+0000 7efdb5ffb640 10 client.15549 handle_command_reply: tid=0 2024-09-16T00:33:41.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.960+0000 7efdd6b34740 1 client.15549 shutdown 2024-09-16T00:33:41.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.960+0000 7efdd6b34740 1 -- 172.21.15.38:0/3536999129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564467798480 msgr2=0x5644677a0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:41.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.960+0000 7efdd6b34740 1 --2- 172.21.15.38:0/3536999129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564467798480 0x5644677a0220 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7efdbc098c50 tx=0x7efdbc09bcd0 comp rx=0 tx=0).stop 2024-09-16T00:33:41.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.960+0000 7efdd6b34740 1 -- 172.21.15.38:0/3536999129 shutdown_connections 2024-09-16T00:33:41.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.960+0000 7efdd6b34740 1 --2- 172.21.15.38:0/3536999129 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644677a2d40 0x5644677a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:41.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.960+0000 7efdd6b34740 1 --2- 172.21.15.38:0/3536999129 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5644677b2950 0x5644677b4d40 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:41.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.960+0000 7efdd6b34740 1 --2- 172.21.15.38:0/3536999129 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564467798480 0x5644677a0220 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:41.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.960+0000 7efdb5ffb640 0 client.15549 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:33:41.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.960+0000 7efdd6b34740 1 --2- 172.21.15.38:0/3536999129 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564467797b60 0x56446779fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:41.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.960+0000 7efdd6b34740 1 -- 172.21.15.38:0/3536999129 >> 172.21.15.38:0/3536999129 conn(0x5644677940f0 msgr2=0x564467799dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:41.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.960+0000 7efdd6b34740 1 -- 172.21.15.38:0/3536999129 shutdown_connections 2024-09-16T00:33:41.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.960+0000 7efdd6b34740 1 -- 172.21.15.38:0/3536999129 wait complete. 2024-09-16T00:33:41.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.960+0000 7efdd6b34740 20 client.15549 trim_cache size 0 max 16384 2024-09-16T00:33:41.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.978+0000 7efdd6b34740 1 Processor -- start 2024-09-16T00:33:41.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.978+0000 7efdd6b34740 1 -- start start 2024-09-16T00:33:41.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.978+0000 7efdd6b34740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564467797b60 0x564467799cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:41.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.978+0000 7efdd6b34740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564467798480 0x5644677ba5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:41.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.978+0000 7efdd6b34740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644677a2d40 0x5644677bab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:41.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.978+0000 7efdd6b34740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efdcc1b3780 con 0x564467798480 2024-09-16T00:33:41.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.978+0000 7efdd6b34740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efdcc1b1050 con 0x564467797b60 2024-09-16T00:33:41.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.978+0000 7efdd6b34740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efdcc1aeea0 con 0x5644677a2d40 2024-09-16T00:33:41.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.978+0000 7efdd1cf6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564467798480 0x5644677ba5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:41.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.978+0000 7efdd1cf6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564467798480 0x5644677ba5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55428/0 (socket says 172.21.15.38:55428) 2024-09-16T00:33:41.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.978+0000 7efdd1cf6640 1 -- 172.21.15.38:0/2062413774 learned_addr learned my addr 172.21.15.38:0/2062413774 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:41.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.979+0000 7efdd24f7640 1 --2- 172.21.15.38:0/2062413774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564467797b60 0x564467799cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:41.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.979+0000 7efdd1cf6640 1 -- 172.21.15.38:0/2062413774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644677a2d40 msgr2=0x5644677bab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:41.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.979+0000 7efdd2cf8640 1 --2- 172.21.15.38:0/2062413774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644677a2d40 0x5644677bab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:41.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.979+0000 7efdd1cf6640 1 --2- 172.21.15.38:0/2062413774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644677a2d40 0x5644677bab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:41.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.979+0000 7efdd1cf6640 1 -- 172.21.15.38:0/2062413774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564467797b60 msgr2=0x564467799cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:41.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.979+0000 7efdd1cf6640 1 --2- 172.21.15.38:0/2062413774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564467797b60 0x564467799cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:41.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.979+0000 7efdd1cf6640 1 -- 172.21.15.38:0/2062413774 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644677a25b0 con 0x564467798480 2024-09-16T00:33:41.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.979+0000 7efdd2cf8640 1 --2- 172.21.15.38:0/2062413774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644677a2d40 0x5644677bab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:33:41.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.979+0000 7efdd24f7640 1 --2- 172.21.15.38:0/2062413774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564467797b60 0x564467799cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:41.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.979+0000 7efdd1cf6640 1 --2- 172.21.15.38:0/2062413774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564467798480 0x5644677ba5f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7efdbc000c00 tx=0x7efdbc0a4fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c4733bdfc1083c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:41.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.979+0000 7efdb7fff640 1 -- 172.21.15.38:0/2062413774 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdbc098520 con 0x564467798480 2024-09-16T00:33:41.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.979+0000 7efdb7fff640 1 -- 172.21.15.38:0/2062413774 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdbc0c9480 con 0x564467798480 2024-09-16T00:33:41.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.979+0000 7efdb7fff640 1 -- 172.21.15.38:0/2062413774 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdbc0a0120 con 0x564467798480 2024-09-16T00:33:41.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.980+0000 7efdd6b34740 1 -- 172.21.15.38:0/2062413774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564467798480 msgr2=0x5644677ba5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:41.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.980+0000 7efdd6b34740 1 --2- 172.21.15.38:0/2062413774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564467798480 0x5644677ba5f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7efdbc000c00 tx=0x7efdbc0a4fc0 comp rx=0 tx=0).stop 2024-09-16T00:33:41.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.980+0000 7efdd6b34740 1 -- 172.21.15.38:0/2062413774 shutdown_connections 2024-09-16T00:33:41.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.980+0000 7efdd6b34740 1 --2- 172.21.15.38:0/2062413774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5644677a2d40 0x5644677bab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:41.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.980+0000 7efdd6b34740 1 --2- 172.21.15.38:0/2062413774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564467798480 0x5644677ba5f0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:41.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.980+0000 7efdd6b34740 1 --2- 172.21.15.38:0/2062413774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564467797b60 0x564467799cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:41.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.980+0000 7efdd6b34740 1 -- 172.21.15.38:0/2062413774 >> 172.21.15.38:0/2062413774 conn(0x5644677940f0 msgr2=0x5644677995b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:41.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.980+0000 7efdd6b34740 1 -- 172.21.15.38:0/2062413774 shutdown_connections 2024-09-16T00:33:41.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.980+0000 7efdd6b34740 1 -- 172.21.15.38:0/2062413774 wait complete. 2024-09-16T00:33:41.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.981+0000 7efdd6b34740 1 -- start start 2024-09-16T00:33:41.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.981+0000 7efdd6b34740 1 Processor -- start 2024-09-16T00:33:41.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.981+0000 7efdd6b34740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564467797b60 0x5644677a6a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:41.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.981+0000 7efdd6b34740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564467798480 0x5644677a6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:41.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.981+0000 7efdd6b34740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644677a2d40 0x5644677a74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:41.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.981+0000 7efdd6b34740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efdcc1af510 con 0x564467798480 2024-09-16T00:33:41.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.981+0000 7efdd6b34740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efdcc1b0980 con 0x5644677a2d40 2024-09-16T00:33:41.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.981+0000 7efdd6b34740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efdcc1afbe0 con 0x564467797b60 2024-09-16T00:33:41.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.981+0000 7efdd6b34740 10 client.0 _ll_register_callbacks cb 0x7efdbc010ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:41.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.981+0000 7efdd1cf6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564467798480 0x5644677a6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:41.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.981+0000 7efdd1cf6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564467798480 0x5644677a6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55430/0 (socket says 172.21.15.38:55430) 2024-09-16T00:33:41.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.982+0000 7efdd1cf6640 1 -- 172.21.15.38:0/1963543751 learned_addr learned my addr 172.21.15.38:0/1963543751 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:41.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.982+0000 7efdd2cf8640 1 --2- 172.21.15.38:0/1963543751 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644677a2d40 0x5644677a74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:41.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.982+0000 7efdd24f7640 1 --2- 172.21.15.38:0/1963543751 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564467797b60 0x5644677a6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:41.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.982+0000 7efdd1cf6640 1 -- 172.21.15.38:0/1963543751 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564467797b60 msgr2=0x5644677a6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:41.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.982+0000 7efdd1cf6640 1 --2- 172.21.15.38:0/1963543751 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564467797b60 0x5644677a6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:41.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.982+0000 7efdd1cf6640 1 -- 172.21.15.38:0/1963543751 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644677a2d40 msgr2=0x5644677a74e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:41.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.982+0000 7efdd1cf6640 1 --2- 172.21.15.38:0/1963543751 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644677a2d40 0x5644677a74e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:41.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.982+0000 7efdd1cf6640 1 -- 172.21.15.38:0/1963543751 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56446781c5f0 con 0x564467798480 2024-09-16T00:33:41.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.982+0000 7efdd1cf6640 1 --2- 172.21.15.38:0/1963543751 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564467798480 0x5644677a6fa0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7efdbc09bf80 tx=0x7efdbc018420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adff6c9700696851 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:41.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.982+0000 7efdb5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:33:41.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.982+0000 7efdb5ffb640 1 -- 172.21.15.38:0/1963543751 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efdbc0a0580 con 0x564467798480 2024-09-16T00:33:41.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.983+0000 7efdb5ffb640 1 -- 172.21.15.38:0/1963543751 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7efdbc0b50f0 con 0x564467798480 2024-09-16T00:33:41.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.983+0000 7efdd6b34740 1 -- 172.21.15.38:0/1963543751 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efdbc0c9480 con 0x564467798480 2024-09-16T00:33:41.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.983+0000 7efdb5ffb640 1 -- 172.21.15.38:0/1963543751 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efdbc0a0120 con 0x564467798480 2024-09-16T00:33:41.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.983+0000 7efdd6b34740 10 client.15555 fetch_fsmap learned FSMap version 26 2024-09-16T00:33:41.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.983+0000 7efdd6b34740 1 -- 172.21.15.38:0/1963543751 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56446781d860 con 0x564467798480 2024-09-16T00:33:41.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.983+0000 7efdb5ffb640 1 -- 172.21.15.38:0/1963543751 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efdbc0c9480 con 0x564467798480 2024-09-16T00:33:41.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.983+0000 7efdb5ffb640 1 -- 172.21.15.38:0/1963543751 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7efdbc09b370 con 0x564467798480 2024-09-16T00:33:41.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.983+0000 7efdd6b34740 10 client.15555 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:33:41.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.983+0000 7efdd6b34740 10 client.15555 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:33:41.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.983+0000 7efdd6b34740 20 client.15555 populate_metadata read hostname 'smithi038' 2024-09-16T00:33:41.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.983+0000 7efdd6b34740 1 --2- 172.21.15.38:0/1963543751 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5644677b2950 0x56446781fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:41.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.983+0000 7efdd6b34740 4 client.15555 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:33:41.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.983+0000 7efdd6b34740 1 -- 172.21.15.38:0/1963543751 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564467704d00 con 0x5644677b2950 2024-09-16T00:33:41.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.984+0000 7efdd24f7640 1 --2- 172.21.15.38:0/1963543751 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5644677b2950 0x56446781fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:41.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.984+0000 7efdb5ffb640 10 client.15555 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:33:41.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.984+0000 7efdd24f7640 1 --2- 172.21.15.38:0/1963543751 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5644677b2950 0x56446781fd50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14c47f9a8e19491d server_cookie=e45f5239caa4ed82 in_seq=0 out_seq=0 2024-09-16T00:33:41.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.985+0000 7efdb5ffb640 1 -- 172.21.15.38:0/1963543751 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564467704d00 con 0x5644677b2950 2024-09-16T00:33:41.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.986+0000 7efdb5ffb640 10 client.15555 handle_command_reply: tid=0 2024-09-16T00:33:41.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.986+0000 7efdd6b34740 1 client.15555 shutdown 2024-09-16T00:33:41.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.986+0000 7efdd6b34740 1 -- 172.21.15.38:0/1963543751 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564467798480 msgr2=0x5644677a6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:41.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.986+0000 7efdd6b34740 1 --2- 172.21.15.38:0/1963543751 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564467798480 0x5644677a6fa0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7efdbc09bf80 tx=0x7efdbc018420 comp rx=0 tx=0).stop 2024-09-16T00:33:41.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.986+0000 7efdd6b34740 1 -- 172.21.15.38:0/1963543751 shutdown_connections 2024-09-16T00:33:41.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.986+0000 7efdd6b34740 1 --2- 172.21.15.38:0/1963543751 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5644677a2d40 0x5644677a74e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:41.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.986+0000 7efdd6b34740 1 --2- 172.21.15.38:0/1963543751 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5644677b2950 0x56446781fd50 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:41.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.986+0000 7efdd6b34740 1 --2- 172.21.15.38:0/1963543751 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564467798480 0x5644677a6fa0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:41.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.986+0000 7efdb5ffb640 0 client.15555 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:33:41.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.986+0000 7efdd6b34740 1 --2- 172.21.15.38:0/1963543751 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564467797b60 0x5644677a6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:41.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.986+0000 7efdd6b34740 1 -- 172.21.15.38:0/1963543751 >> 172.21.15.38:0/1963543751 conn(0x5644677940f0 msgr2=0x564467799dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:41.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.986+0000 7efdd6b34740 1 -- 172.21.15.38:0/1963543751 shutdown_connections 2024-09-16T00:33:41.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.986+0000 7efdd6b34740 1 -- 172.21.15.38:0/1963543751 wait complete. 2024-09-16T00:33:41.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.987+0000 7efdd6b34740 20 client.15555 trim_cache size 0 max 16384 2024-09-16T00:33:41.990 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-16T00:33:41.991 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0, 2024-09-16T00:33:41.991 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_tag": "0e5118de-a712-4932-baa0-b41ab2415f76", 2024-09-16T00:33:41.991 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": "asynchronous" 2024-09-16T00:33:41.991 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-16T00:33:41.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.987+0000 7efdd475b640 1 -- 172.21.15.38:0/2739169570 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efdb80a3350 msgr2=0x7efdb80a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:41.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.987+0000 7efdd475b640 1 --2- 172.21.15.38:0/2739169570 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efdb80a3350 0x7efdb80a5810 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7efdc40049d0 tx=0x7efdc4004a40 comp rx=0 tx=0).stop 2024-09-16T00:33:41.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.987+0000 7efdd475b640 1 -- 172.21.15.38:0/2739169570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efdcc1036e0 msgr2=0x7efdcc100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:41.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.987+0000 7efdd475b640 1 --2- 172.21.15.38:0/2739169570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efdcc1036e0 0x7efdcc100ed0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7efdcc0697d0 tx=0x7efdbc00ee50 comp rx=0 tx=0).stop 2024-09-16T00:33:41.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.987+0000 7efdd475b640 1 -- 172.21.15.38:0/2739169570 shutdown_connections 2024-09-16T00:33:41.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.987+0000 7efdd475b640 1 --2- 172.21.15.38:0/2739169570 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efdb80a3350 0x7efdb80a5810 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:41.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.987+0000 7efdd475b640 1 --2- 172.21.15.38:0/2739169570 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efdcc1082a0 0x7efdcc101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:41.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.987+0000 7efdd475b640 1 --2- 172.21.15.38:0/2739169570 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efdcc1036e0 0x7efdcc100ed0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:41.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.987+0000 7efdd475b640 1 --2- 172.21.15.38:0/2739169570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efdcc102d30 0x7efdcc100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:41.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.987+0000 7efdd475b640 1 -- 172.21.15.38:0/2739169570 >> 172.21.15.38:0/2739169570 conn(0x7efdcc0fc7d0 msgr2=0x7efdcc105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:41.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.988+0000 7efdd475b640 1 -- 172.21.15.38:0/2739169570 shutdown_connections 2024-09-16T00:33:41.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:41.988+0000 7efdd475b640 1 -- 172.21.15.38:0/2739169570 wait complete. 2024-09-16T00:33:42.869 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:33:43.177 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:43.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:42 smithi038 ceph-mon[30945]: pgmap v392: 129 pgs: 129 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-16T00:33:43.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:42 smithi038 ceph-mon[30945]: scrub queued for path: / 2024-09-16T00:33:43.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:42 smithi038 ceph-mon[30945]: scrub summary: idle+waiting paths [/] 2024-09-16T00:33:43.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:42 smithi038 ceph-mon[30945]: scrub summary: active paths [/] 2024-09-16T00:33:43.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:42 smithi038 ceph-mon[30945]: scrub summary: idle+waiting paths [/] 2024-09-16T00:33:43.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:42 smithi038 ceph-mon[30945]: scrub summary: idle 2024-09-16T00:33:43.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:42 smithi122 ceph-mon[35412]: pgmap v392: 129 pgs: 129 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-16T00:33:43.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:42 smithi122 ceph-mon[35412]: scrub queued for path: / 2024-09-16T00:33:43.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:42 smithi122 ceph-mon[35412]: scrub summary: idle+waiting paths [/] 2024-09-16T00:33:43.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:42 smithi122 ceph-mon[35412]: scrub summary: active paths [/] 2024-09-16T00:33:43.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:42 smithi122 ceph-mon[35412]: scrub summary: idle+waiting paths [/] 2024-09-16T00:33:43.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:42 smithi122 ceph-mon[35412]: scrub summary: idle 2024-09-16T00:33:43.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:42 smithi073 ceph-mon[35254]: pgmap v392: 129 pgs: 129 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-16T00:33:43.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:42 smithi073 ceph-mon[35254]: scrub queued for path: / 2024-09-16T00:33:43.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:42 smithi073 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-09-16T00:33:43.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:42 smithi073 ceph-mon[35254]: scrub summary: active paths [/] 2024-09-16T00:33:43.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:42 smithi073 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-09-16T00:33:43.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:42 smithi073 ceph-mon[35254]: scrub summary: idle 2024-09-16T00:33:43.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.742+0000 7f649b3c6640 1 -- 172.21.15.38:0/4053027250 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6494102d30 msgr2=0x7f6494103110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:43.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.742+0000 7f649b3c6640 1 --2- 172.21.15.38:0/4053027250 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6494102d30 0x7f6494103110 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f6490009930 tx=0x7f649002a420 comp rx=0 tx=0).stop 2024-09-16T00:33:43.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.742+0000 7f649b3c6640 1 -- 172.21.15.38:0/4053027250 shutdown_connections 2024-09-16T00:33:43.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.743+0000 7f649b3c6640 1 --2- 172.21.15.38:0/4053027250 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64941082a0 0x7f649410ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:43.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.743+0000 7f649b3c6640 1 --2- 172.21.15.38:0/4053027250 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64941036e0 0x7f6494107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:43.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.743+0000 7f649b3c6640 1 --2- 172.21.15.38:0/4053027250 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6494102d30 0x7f6494103110 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:43.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.743+0000 7f649b3c6640 1 -- 172.21.15.38:0/4053027250 >> 172.21.15.38:0/4053027250 conn(0x7f64940fc7d0 msgr2=0x7f64940febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:43.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.743+0000 7f649b3c6640 1 -- 172.21.15.38:0/4053027250 shutdown_connections 2024-09-16T00:33:43.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.743+0000 7f649b3c6640 1 -- 172.21.15.38:0/4053027250 wait complete. 2024-09-16T00:33:43.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.743+0000 7f649b3c6640 1 Processor -- start 2024-09-16T00:33:43.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.744+0000 7f649b3c6640 1 -- start start 2024-09-16T00:33:43.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.744+0000 7f649b3c6640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6494102d30 0x7f64941985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:43.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.744+0000 7f649b3c6640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64941036e0 0x7f6494198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:43.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.744+0000 7f649b3c6640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64941082a0 0x7f649419ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:43.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.744+0000 7f649b3c6640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f64941120a0 con 0x7f6494102d30 2024-09-16T00:33:43.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.744+0000 7f649b3c6640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f6494111f20 con 0x7f64941082a0 2024-09-16T00:33:43.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.744+0000 7f649b3c6640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f6494112220 con 0x7f64941036e0 2024-09-16T00:33:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.744+0000 7f6498961640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64941036e0 0x7f6494198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.744+0000 7f6499162640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6494102d30 0x7f64941985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.744+0000 7f6498961640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64941036e0 0x7f6494198b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.38:37510/0 (socket says 172.21.15.38:37510) 2024-09-16T00:33:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.744+0000 7f6498961640 1 -- 172.21.15.38:0/3843654156 learned_addr learned my addr 172.21.15.38:0/3843654156 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.744+0000 7f6499963640 1 --2- 172.21.15.38:0/3843654156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64941082a0 0x7f649419ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.745+0000 7f6498961640 1 -- 172.21.15.38:0/3843654156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64941082a0 msgr2=0x7f649419ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.745+0000 7f6498961640 1 --2- 172.21.15.38:0/3843654156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64941082a0 0x7f649419ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.745+0000 7f6498961640 1 -- 172.21.15.38:0/3843654156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6494102d30 msgr2=0x7f64941985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.745+0000 7f6498961640 1 --2- 172.21.15.38:0/3843654156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6494102d30 0x7f64941985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.745+0000 7f6498961640 1 -- 172.21.15.38:0/3843654156 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f649419d5b0 con 0x7f64941036e0 2024-09-16T00:33:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.745+0000 7f6499162640 1 --2- 172.21.15.38:0/3843654156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6494102d30 0x7f64941985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:33:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.745+0000 7f6498961640 1 --2- 172.21.15.38:0/3843654156 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64941036e0 0x7f6494198b20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f648400e9c0 tx=0x7f648400ee80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.746+0000 7f648a7fc640 1 -- 172.21.15.38:0/3843654156 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f648400ccc0 con 0x7f64941036e0 2024-09-16T00:33:43.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.746+0000 7f649b3c6640 1 -- 172.21.15.38:0/3843654156 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f649419d8a0 con 0x7f64941036e0 2024-09-16T00:33:43.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.746+0000 7f649b3c6640 1 -- 172.21.15.38:0/3843654156 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64941008c0 con 0x7f64941036e0 2024-09-16T00:33:43.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.746+0000 7f648a7fc640 1 -- 172.21.15.38:0/3843654156 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6484005500 con 0x7f64941036e0 2024-09-16T00:33:43.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.746+0000 7f648a7fc640 1 -- 172.21.15.38:0/3843654156 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6484010690 con 0x7f64941036e0 2024-09-16T00:33:43.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.749+0000 7f649d79f740 1 Processor -- start 2024-09-16T00:33:43.750 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.749+0000 7f649d79f740 1 -- start start 2024-09-16T00:33:43.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.749+0000 7f649d79f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563debe5d4a0 0x563debe5d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:43.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.749+0000 7f649d79f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563debe5ddc0 0x563debe68140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:43.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.749+0000 7f649d79f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563debe68680 0x563debe6aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:43.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.749+0000 7f649d79f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563debdc32e0 con 0x563debe5d4a0 2024-09-16T00:33:43.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.749+0000 7f649d79f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563debd6ace0 con 0x563debe68680 2024-09-16T00:33:43.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.749+0000 7f649d79f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563debdbf2a0 con 0x563debe5ddc0 2024-09-16T00:33:43.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.750+0000 7f6499963640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563debe5d4a0 0x563debe5d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:43.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.750+0000 7f6499963640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563debe5d4a0 0x563debe5d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55458/0 (socket says 172.21.15.38:55458) 2024-09-16T00:33:43.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.750+0000 7f6499963640 1 -- 172.21.15.38:0/1099388174 learned_addr learned my addr 172.21.15.38:0/1099388174 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:43.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.750+0000 7f6499162640 1 --2- 172.21.15.38:0/1099388174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563debe5ddc0 0x563debe68140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:43.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.750+0000 7f6499162640 1 -- 172.21.15.38:0/1099388174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563debe68680 msgr2=0x563debe6aa70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:33:43.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.750+0000 7f6499162640 1 --2- 172.21.15.38:0/1099388174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563debe68680 0x563debe6aa70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:43.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.750+0000 7f6499162640 1 -- 172.21.15.38:0/1099388174 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563debe5d4a0 msgr2=0x563debe5d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:43.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.750+0000 7f6499162640 1 --2- 172.21.15.38:0/1099388174 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563debe5d4a0 0x563debe5d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:43.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.750+0000 7f6499162640 1 -- 172.21.15.38:0/1099388174 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563debd4f440 con 0x563debe5ddc0 2024-09-16T00:33:43.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.750+0000 7f6499963640 1 --2- 172.21.15.38:0/1099388174 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563debe5d4a0 0x563debe5d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:33:43.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.750+0000 7f648a7fc640 1 -- 172.21.15.38:0/3843654156 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f6484004e80 con 0x7f64941036e0 2024-09-16T00:33:43.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.750+0000 7f6499162640 1 --2- 172.21.15.38:0/1099388174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563debe5ddc0 0x563debe68140 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6490031470 tx=0x7f6490031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2027662074a68987 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:43.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.751+0000 7f647f7fe640 1 -- 172.21.15.38:0/1099388174 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6490035050 con 0x563debe5ddc0 2024-09-16T00:33:43.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.751+0000 7f647f7fe640 1 -- 172.21.15.38:0/1099388174 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6490005500 con 0x563debe5ddc0 2024-09-16T00:33:43.753 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.751+0000 7f647f7fe640 1 -- 172.21.15.38:0/1099388174 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6490004770 con 0x563debe5ddc0 2024-09-16T00:33:43.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.751+0000 7f648a7fc640 1 --2- 172.21.15.38:0/3843654156 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f64800a33d0 0x7f64800a5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:43.756 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.752+0000 7f649d79f740 1 -- 172.21.15.38:0/1099388174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563debe5ddc0 msgr2=0x563debe68140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:43.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.752+0000 7f649d79f740 1 --2- 172.21.15.38:0/1099388174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563debe5ddc0 0x563debe68140 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6490031470 tx=0x7f6490031400 comp rx=0 tx=0).stop 2024-09-16T00:33:43.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.752+0000 7f648a7fc640 1 -- 172.21.15.38:0/3843654156 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f64840d00f0 con 0x7f64941036e0 2024-09-16T00:33:43.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.752+0000 7f649d79f740 1 -- 172.21.15.38:0/1099388174 shutdown_connections 2024-09-16T00:33:43.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.752+0000 7f649d79f740 1 --2- 172.21.15.38:0/1099388174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563debe68680 0x563debe6aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:43.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.752+0000 7f649d79f740 1 --2- 172.21.15.38:0/1099388174 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563debe5ddc0 0x563debe68140 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:43.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.752+0000 7f649d79f740 1 --2- 172.21.15.38:0/1099388174 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563debe5d4a0 0x563debe5d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:43.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.752+0000 7f649d79f740 1 -- 172.21.15.38:0/1099388174 >> 172.21.15.38:0/1099388174 conn(0x563debe59a30 msgr2=0x563debe59e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:43.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.752+0000 7f6499963640 1 --2- 172.21.15.38:0/3843654156 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f64800a33d0 0x7f64800a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:43.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.752+0000 7f649d79f740 1 -- 172.21.15.38:0/1099388174 shutdown_connections 2024-09-16T00:33:43.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.752+0000 7f649d79f740 1 -- 172.21.15.38:0/1099388174 wait complete. 2024-09-16T00:33:43.757 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.752+0000 7f6499963640 1 --2- 172.21.15.38:0/3843654156 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f64800a33d0 0x7f64800a5890 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f648c005c40 tx=0x7f648c009760 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:43.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.753+0000 7f649d79f740 1 -- start start 2024-09-16T00:33:43.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.753+0000 7f649d79f740 1 Processor -- start 2024-09-16T00:33:43.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.753+0000 7f649d79f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563debe5d4a0 0x563debe68030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:43.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.753+0000 7f649d79f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563debe5ddc0 0x563debe652e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:43.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.753+0000 7f649d79f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563debe68680 0x563debe65880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:43.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.753+0000 7f649d79f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x563debe6b250 con 0x563debe5ddc0 2024-09-16T00:33:43.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.753+0000 7f649d79f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x563debd7e050 con 0x563debe68680 2024-09-16T00:33:43.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.753+0000 7f649d79f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x563debe6b3d0 con 0x563debe5d4a0 2024-09-16T00:33:43.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.753+0000 7f649d79f740 10 client.0 _ll_register_callbacks cb 0x563debdbe160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:43.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.754+0000 7f6498961640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563debe5ddc0 0x563debe652e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:43.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.754+0000 7f6498961640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563debe5ddc0 0x563debe652e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55470/0 (socket says 172.21.15.38:55470) 2024-09-16T00:33:43.758 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.754+0000 7f6498961640 1 -- 172.21.15.38:0/3457040793 learned_addr learned my addr 172.21.15.38:0/3457040793 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:43.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.754+0000 7f6499162640 1 --2- 172.21.15.38:0/3457040793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563debe5d4a0 0x563debe68030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:43.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.754+0000 7f6499963640 1 --2- 172.21.15.38:0/3457040793 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563debe68680 0x563debe65880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:43.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.754+0000 7f6499162640 1 -- 172.21.15.38:0/3457040793 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563debe68680 msgr2=0x563debe65880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:43.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.754+0000 7f6499162640 1 --2- 172.21.15.38:0/3457040793 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563debe68680 0x563debe65880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:43.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.754+0000 7f6499162640 1 -- 172.21.15.38:0/3457040793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563debe5ddc0 msgr2=0x563debe652e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:43.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.754+0000 7f6499162640 1 --2- 172.21.15.38:0/3457040793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563debe5ddc0 0x563debe652e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:43.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.754+0000 7f6499162640 1 -- 172.21.15.38:0/3457040793 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563debb343c0 con 0x563debe5d4a0 2024-09-16T00:33:43.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.754+0000 7f6498961640 1 --2- 172.21.15.38:0/3457040793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563debe5ddc0 0x563debe652e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:33:43.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.754+0000 7f6499162640 1 --2- 172.21.15.38:0/3457040793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563debe5d4a0 0x563debe68030 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6490000c00 tx=0x7f6490032e00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3903ae7b5d88471 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:43.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.755+0000 7f647cff9640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-16T00:33:43.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.755+0000 7f647cff9640 1 -- 172.21.15.38:0/3457040793 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6490050020 con 0x563debe5d4a0 2024-09-16T00:33:43.759 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.755+0000 7f647cff9640 1 -- 172.21.15.38:0/3457040793 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f649003aa80 con 0x563debe5d4a0 2024-09-16T00:33:43.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.755+0000 7f649d79f740 1 -- 172.21.15.38:0/3457040793 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6490005500 con 0x563debe5d4a0 2024-09-16T00:33:43.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.755+0000 7f647cff9640 1 -- 172.21.15.38:0/3457040793 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f649005b050 con 0x563debe5d4a0 2024-09-16T00:33:43.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.755+0000 7f649d79f740 10 client.25138 fetch_fsmap learned FSMap version 26 2024-09-16T00:33:43.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.755+0000 7f649d79f740 1 -- 172.21.15.38:0/3457040793 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563debe6b250 con 0x563debe5d4a0 2024-09-16T00:33:43.760 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.755+0000 7f647cff9640 1 -- 172.21.15.38:0/3457040793 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f6490009520 con 0x563debe5d4a0 2024-09-16T00:33:43.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.756+0000 7f647cff9640 1 -- 172.21.15.38:0/3457040793 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f649004c4b0 con 0x563debe5d4a0 2024-09-16T00:33:43.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.756+0000 7f649d79f740 10 client.25138 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:33:43.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.756+0000 7f649d79f740 10 client.25138 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:33:43.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.756+0000 7f649d79f740 20 client.25138 populate_metadata read hostname 'smithi038' 2024-09-16T00:33:43.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.756+0000 7f649d79f740 1 --2- 172.21.15.38:0/3457040793 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x563debe77e40 0x563debe7a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:43.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.756+0000 7f649d79f740 4 client.25138 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:33:43.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.756+0000 7f649d79f740 1 -- 172.21.15.38:0/3457040793 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6490005500 con 0x563debe77e40 2024-09-16T00:33:43.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.756+0000 7f6498961640 1 --2- 172.21.15.38:0/3457040793 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x563debe77e40 0x563debe7a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:43.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.757+0000 7f647cff9640 10 client.25138 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:33:43.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.757+0000 7f6498961640 1 --2- 172.21.15.38:0/3457040793 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x563debe77e40 0x563debe7a230 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91a412d20f1147a3 server_cookie=b7a73dbcaaf25935 in_seq=0 out_seq=0 2024-09-16T00:33:43.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.757+0000 7f647cff9640 1 -- 172.21.15.38:0/3457040793 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f6490005500 con 0x563debe77e40 2024-09-16T00:33:43.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.757+0000 7f647cff9640 10 client.25138 handle_command_reply: tid=0 2024-09-16T00:33:43.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.757+0000 7f649d79f740 1 client.25138 shutdown 2024-09-16T00:33:43.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.758+0000 7f649d79f740 1 -- 172.21.15.38:0/3457040793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563debe5d4a0 msgr2=0x563debe68030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:43.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.758+0000 7f649d79f740 1 --2- 172.21.15.38:0/3457040793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563debe5d4a0 0x563debe68030 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6490000c00 tx=0x7f6490032e00 comp rx=0 tx=0).stop 2024-09-16T00:33:43.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.758+0000 7f649d79f740 1 -- 172.21.15.38:0/3457040793 shutdown_connections 2024-09-16T00:33:43.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.758+0000 7f649d79f740 1 --2- 172.21.15.38:0/3457040793 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563debe68680 0x563debe65880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:43.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.758+0000 7f649d79f740 1 --2- 172.21.15.38:0/3457040793 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x563debe77e40 0x563debe7a230 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:43.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.758+0000 7f649d79f740 1 --2- 172.21.15.38:0/3457040793 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563debe5ddc0 0x563debe652e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:43.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.758+0000 7f649d79f740 1 --2- 172.21.15.38:0/3457040793 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563debe5d4a0 0x563debe68030 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:43.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.758+0000 7f649d79f740 1 -- 172.21.15.38:0/3457040793 >> 172.21.15.38:0/3457040793 conn(0x563debe59a30 msgr2=0x563debe5f760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:43.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.758+0000 7f647cff9640 0 client.25138 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:33:43.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.758+0000 7f649d79f740 1 -- 172.21.15.38:0/3457040793 shutdown_connections 2024-09-16T00:33:43.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.758+0000 7f649d79f740 1 -- 172.21.15.38:0/3457040793 wait complete. 2024-09-16T00:33:43.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.758+0000 7f649d79f740 20 client.25138 trim_cache size 0 max 16384 2024-09-16T00:33:43.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.793+0000 7f649d79f740 1 Processor -- start 2024-09-16T00:33:43.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.793+0000 7f649d79f740 1 -- start start 2024-09-16T00:33:43.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.793+0000 7f649d79f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563debe5d4a0 0x563debe63bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:43.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.793+0000 7f649d79f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563debe5ddc0 0x563debe7f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:43.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.793+0000 7f649d79f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563debe68680 0x563debe7fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:43.795 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.793+0000 7f649d79f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f6478012460 con 0x563debe68680 2024-09-16T00:33:43.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.793+0000 7f649d79f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f647800fd30 con 0x563debe5d4a0 2024-09-16T00:33:43.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.793+0000 7f649d79f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f647800db80 con 0x563debe5ddc0 2024-09-16T00:33:43.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.793+0000 7f6499963640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563debe68680 0x563debe7fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:43.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.793+0000 7f6499162640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563debe5d4a0 0x563debe63bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:43.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.793+0000 7f6499963640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563debe68680 0x563debe7fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55486/0 (socket says 172.21.15.38:55486) 2024-09-16T00:33:43.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.793+0000 7f6499963640 1 -- 172.21.15.38:0/2558287810 learned_addr learned my addr 172.21.15.38:0/2558287810 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:43.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.793+0000 7f6498961640 1 --2- 172.21.15.38:0/2558287810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563debe5ddc0 0x563debe7f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:43.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.794+0000 7f6498961640 1 -- 172.21.15.38:0/2558287810 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563debe5d4a0 msgr2=0x563debe63bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:43.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.794+0000 7f6498961640 1 --2- 172.21.15.38:0/2558287810 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563debe5d4a0 0x563debe63bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:43.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.794+0000 7f6498961640 1 -- 172.21.15.38:0/2558287810 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563debe68680 msgr2=0x563debe7fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:43.796 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.794+0000 7f6498961640 1 --2- 172.21.15.38:0/2558287810 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563debe68680 0x563debe7fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:43.797 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.794+0000 7f6498961640 1 -- 172.21.15.38:0/2558287810 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563debe66750 con 0x563debe5ddc0 2024-09-16T00:33:43.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.794+0000 7f6498961640 1 --2- 172.21.15.38:0/2558287810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563debe5ddc0 0x563debe7f890 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f648400c6a0 tx=0x7f64840cf040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab1d3685695acfcd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:43.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.794+0000 7f647effd640 1 -- 172.21.15.38:0/2558287810 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64840a0580 con 0x563debe5ddc0 2024-09-16T00:33:43.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.794+0000 7f647effd640 1 -- 172.21.15.38:0/2558287810 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64840a4e40 con 0x563debe5ddc0 2024-09-16T00:33:43.800 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.794+0000 7f647effd640 1 -- 172.21.15.38:0/2558287810 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6484005170 con 0x563debe5ddc0 2024-09-16T00:33:43.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.795+0000 7f649d79f740 1 -- 172.21.15.38:0/2558287810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563debe5ddc0 msgr2=0x563debe7f890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:43.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.795+0000 7f649d79f740 1 --2- 172.21.15.38:0/2558287810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563debe5ddc0 0x563debe7f890 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f648400c6a0 tx=0x7f64840cf040 comp rx=0 tx=0).stop 2024-09-16T00:33:43.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.795+0000 7f649d79f740 1 -- 172.21.15.38:0/2558287810 shutdown_connections 2024-09-16T00:33:43.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.795+0000 7f649d79f740 1 --2- 172.21.15.38:0/2558287810 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563debe68680 0x563debe7fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:43.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.795+0000 7f649d79f740 1 --2- 172.21.15.38:0/2558287810 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563debe5ddc0 0x563debe7f890 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:43.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.795+0000 7f649d79f740 1 --2- 172.21.15.38:0/2558287810 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563debe5d4a0 0x563debe63bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:43.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.795+0000 7f649d79f740 1 -- 172.21.15.38:0/2558287810 >> 172.21.15.38:0/2558287810 conn(0x563debe59a30 msgr2=0x563debe58b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:43.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.795+0000 7f649d79f740 1 -- 172.21.15.38:0/2558287810 shutdown_connections 2024-09-16T00:33:43.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.795+0000 7f649d79f740 1 -- 172.21.15.38:0/2558287810 wait complete. 2024-09-16T00:33:43.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.796+0000 7f649d79f740 1 -- start start 2024-09-16T00:33:43.801 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.796+0000 7f649d79f740 1 Processor -- start 2024-09-16T00:33:43.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.796+0000 7f649d79f740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563debe5d4a0 0x563debe70f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:43.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.796+0000 7f649d79f740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563debe5ddc0 0x563debe6e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:43.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.796+0000 7f649d79f740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563debe68680 0x563debe6e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:43.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.797+0000 7f649d79f740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f647800e1f0 con 0x563debe68680 2024-09-16T00:33:43.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.797+0000 7f649d79f740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f647800f660 con 0x563debe5d4a0 2024-09-16T00:33:43.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.797+0000 7f649d79f740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f647800e8c0 con 0x563debe5ddc0 2024-09-16T00:33:43.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.797+0000 7f6499963640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563debe68680 0x563debe6e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:43.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.797+0000 7f6499963640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563debe68680 0x563debe6e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:55498/0 (socket says 172.21.15.38:55498) 2024-09-16T00:33:43.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.797+0000 7f6499963640 1 -- 172.21.15.38:0/830265983 learned_addr learned my addr 172.21.15.38:0/830265983 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:43.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.797+0000 7f649d79f740 10 client.0 _ll_register_callbacks cb 0x7f648c009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:43.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.797+0000 7f6499162640 1 --2- 172.21.15.38:0/830265983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563debe5d4a0 0x563debe70f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:43.802 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.797+0000 7f6498961640 1 --2- 172.21.15.38:0/830265983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563debe5ddc0 0x563debe6e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:43.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.797+0000 7f6499162640 1 -- 172.21.15.38:0/830265983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563debe5ddc0 msgr2=0x563debe6e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:43.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.797+0000 7f6499162640 1 --2- 172.21.15.38:0/830265983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563debe5ddc0 0x563debe6e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:43.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.797+0000 7f6499162640 1 -- 172.21.15.38:0/830265983 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563debe68680 msgr2=0x563debe6e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:43.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.797+0000 7f6499162640 1 --2- 172.21.15.38:0/830265983 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563debe68680 0x563debe6e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:43.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.797+0000 7f6499162640 1 -- 172.21.15.38:0/830265983 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563debe6f0a0 con 0x563debe5d4a0 2024-09-16T00:33:43.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.797+0000 7f6499162640 1 --2- 172.21.15.38:0/830265983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563debe5d4a0 0x563debe70f10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6490005c40 tx=0x7f649004c630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e98ecd71166bf75f server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:43.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.797+0000 7f647cff9640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-09-16T00:33:43.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.798+0000 7f647cff9640 1 -- 172.21.15.38:0/830265983 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6490050050 con 0x563debe5d4a0 2024-09-16T00:33:43.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.798+0000 7f647cff9640 1 -- 172.21.15.38:0/830265983 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f6490031cc0 con 0x563debe5d4a0 2024-09-16T00:33:43.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.798+0000 7f649d79f740 1 -- 172.21.15.38:0/830265983 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64840a4e40 con 0x563debe5d4a0 2024-09-16T00:33:43.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.798+0000 7f647cff9640 1 -- 172.21.15.38:0/830265983 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f6490035040 con 0x563debe5d4a0 2024-09-16T00:33:43.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.798+0000 7f649d79f740 10 client.25196 fetch_fsmap learned FSMap version 26 2024-09-16T00:33:43.803 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.798+0000 7f649d79f740 1 -- 172.21.15.38:0/830265983 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x563debee29b0 con 0x563debe5d4a0 2024-09-16T00:33:43.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.798+0000 7f647cff9640 1 -- 172.21.15.38:0/830265983 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f64840a4e40 con 0x563debe5d4a0 2024-09-16T00:33:43.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.799+0000 7f647cff9640 1 -- 172.21.15.38:0/830265983 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f649003ab80 con 0x563debe5d4a0 2024-09-16T00:33:43.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.799+0000 7f649d79f740 10 client.25196 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:33:43.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.799+0000 7f649d79f740 10 client.25196 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:33:43.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.799+0000 7f649d79f740 20 client.25196 populate_metadata read hostname 'smithi038' 2024-09-16T00:33:43.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.799+0000 7f649d79f740 1 --2- 172.21.15.38:0/830265983 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x563debee2e60 0x563debee5280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:43.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.799+0000 7f649d79f740 4 client.25196 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:33:43.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.799+0000 7f649d79f740 1 -- 172.21.15.38:0/830265983 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563debdd0850 con 0x563debee2e60 2024-09-16T00:33:43.804 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.799+0000 7f6498961640 1 --2- 172.21.15.38:0/830265983 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x563debee2e60 0x563debee5280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:43.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.799+0000 7f6498961640 1 --2- 172.21.15.38:0/830265983 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x563debee2e60 0x563debee5280 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6729f93b4cacee1e server_cookie=59fcfc0278178320 in_seq=0 out_seq=0 2024-09-16T00:33:43.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.799+0000 7f647cff9640 10 client.25196 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:33:43.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.800+0000 7f647cff9640 1 -- 172.21.15.38:0/830265983 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563debdd0850 con 0x563debee2e60 2024-09-16T00:33:43.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.800+0000 7f647cff9640 10 client.25196 handle_command_reply: tid=0 2024-09-16T00:33:43.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.800+0000 7f649d79f740 1 client.25196 shutdown 2024-09-16T00:33:43.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.800+0000 7f649d79f740 1 -- 172.21.15.38:0/830265983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563debe5d4a0 msgr2=0x563debe70f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:43.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.800+0000 7f649d79f740 1 --2- 172.21.15.38:0/830265983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563debe5d4a0 0x563debe70f10 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6490005c40 tx=0x7f649004c630 comp rx=0 tx=0).stop 2024-09-16T00:33:43.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.800+0000 7f649d79f740 1 -- 172.21.15.38:0/830265983 shutdown_connections 2024-09-16T00:33:43.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.800+0000 7f649d79f740 1 --2- 172.21.15.38:0/830265983 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x563debee2e60 0x563debee5280 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:43.806 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.800+0000 7f647cff9640 0 client.25196 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:33:43.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.800+0000 7f649d79f740 1 --2- 172.21.15.38:0/830265983 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x563debe68680 0x563debe6e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:43.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.800+0000 7f649d79f740 1 --2- 172.21.15.38:0/830265983 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x563debe5ddc0 0x563debe6e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:43.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.800+0000 7f649d79f740 1 --2- 172.21.15.38:0/830265983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x563debe5d4a0 0x563debe70f10 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:43.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.800+0000 7f649d79f740 1 -- 172.21.15.38:0/830265983 >> 172.21.15.38:0/830265983 conn(0x563debe59a30 msgr2=0x563debe66b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:43.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.800+0000 7f649d79f740 1 -- 172.21.15.38:0/830265983 shutdown_connections 2024-09-16T00:33:43.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.800+0000 7f649d79f740 1 -- 172.21.15.38:0/830265983 wait complete. 2024-09-16T00:33:43.807 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.801+0000 7f649d79f740 20 client.25196 trim_cache size 0 max 16384 2024-09-16T00:33:43.807 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-16T00:33:43.807 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "no active scrubs running", 2024-09-16T00:33:43.808 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": {} 2024-09-16T00:33:43.808 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-16T00:33:43.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.801+0000 7f649b3c6640 1 -- 172.21.15.38:0/3843654156 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f64800a33d0 msgr2=0x7f64800a5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:43.808 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.801+0000 7f649b3c6640 1 --2- 172.21.15.38:0/3843654156 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f64800a33d0 0x7f64800a5890 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f648c005c40 tx=0x7f648c009760 comp rx=0 tx=0).stop 2024-09-16T00:33:43.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.801+0000 7f649b3c6640 1 -- 172.21.15.38:0/3843654156 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64941036e0 msgr2=0x7f6494198b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:43.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.801+0000 7f649b3c6640 1 --2- 172.21.15.38:0/3843654156 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64941036e0 0x7f6494198b20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f648400e9c0 tx=0x7f648400ee80 comp rx=0 tx=0).stop 2024-09-16T00:33:43.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.801+0000 7f649b3c6640 1 -- 172.21.15.38:0/3843654156 shutdown_connections 2024-09-16T00:33:43.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.801+0000 7f649b3c6640 1 --2- 172.21.15.38:0/3843654156 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f64800a33d0 0x7f64800a5890 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:43.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.801+0000 7f649b3c6640 1 --2- 172.21.15.38:0/3843654156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64941082a0 0x7f649419ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:43.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.801+0000 7f649b3c6640 1 --2- 172.21.15.38:0/3843654156 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f64941036e0 0x7f6494198b20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:43.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.801+0000 7f649b3c6640 1 --2- 172.21.15.38:0/3843654156 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f6494102d30 0x7f64941985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:43.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.801+0000 7f649b3c6640 1 -- 172.21.15.38:0/3843654156 >> 172.21.15.38:0/3843654156 conn(0x7f64940fc7d0 msgr2=0x7f6494105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:43.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.801+0000 7f649b3c6640 1 -- 172.21.15.38:0/3843654156 shutdown_connections 2024-09-16T00:33:43.809 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:43.802+0000 7f649b3c6640 1 -- 172.21.15.38:0/3843654156 wait complete. 2024-09-16T00:33:44.642 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:33:44.642 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:33:44.921 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:44 smithi038 ceph-mon[30945]: pgmap v393: 129 pgs: 129 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-16T00:33:44.987 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:45.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:44 smithi122 ceph-mon[35412]: pgmap v393: 129 pgs: 129 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-16T00:33:45.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:44 smithi073 ceph-mon[35254]: pgmap v393: 129 pgs: 129 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-16T00:33:45.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.526+0000 7f8cdd7a3640 1 -- 172.21.15.38:0/3494684424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cd8108150 msgr2=0x7f8cd810ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:45.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.526+0000 7f8cdd7a3640 1 --2- 172.21.15.38:0/3494684424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cd8108150 0x7f8cd810ea80 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8cd0009ca0 tx=0x7f8cd002a8c0 comp rx=0 tx=0).stop 2024-09-16T00:33:45.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.527+0000 7f8cdd7a3640 1 -- 172.21.15.38:0/3494684424 shutdown_connections 2024-09-16T00:33:45.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.527+0000 7f8cdd7a3640 1 --2- 172.21.15.38:0/3494684424 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cd8108150 0x7f8cd810ea80 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:45.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.527+0000 7f8cdd7a3640 1 --2- 172.21.15.38:0/3494684424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8cd81012d0 0x7f8cd8107a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:45.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.527+0000 7f8cdd7a3640 1 --2- 172.21.15.38:0/3494684424 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cd8100920 0x7f8cd8100d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:45.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.527+0000 7f8cdd7a3640 1 -- 172.21.15.38:0/3494684424 >> 172.21.15.38:0/3494684424 conn(0x7f8cd8075f70 msgr2=0x7f8cd8076380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:45.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.527+0000 7f8cdd7a3640 1 -- 172.21.15.38:0/3494684424 shutdown_connections 2024-09-16T00:33:45.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.527+0000 7f8cdd7a3640 1 -- 172.21.15.38:0/3494684424 wait complete. 2024-09-16T00:33:45.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.527+0000 7f8cdd7a3640 1 Processor -- start 2024-09-16T00:33:45.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.528+0000 7f8cdd7a3640 1 -- start start 2024-09-16T00:33:45.529 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.528+0000 7f8cdd7a3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8cd8100920 0x7f8cd81983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:45.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.528+0000 7f8cdd7a3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cd81012d0 0x7f8cd8198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:45.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.528+0000 7f8cdd7a3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cd8108150 0x7f8cd819ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:45.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.528+0000 7f8cd77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cd8108150 0x7f8cd819ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:45.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.528+0000 7f8cd6ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8cd8100920 0x7f8cd81983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:45.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.528+0000 7f8cd6ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8cd8100920 0x7f8cd81983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.38:50274/0 (socket says 172.21.15.38:50274) 2024-09-16T00:33:45.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.528+0000 7f8cd6ffd640 1 -- 172.21.15.38:0/4079506283 learned_addr learned my addr 172.21.15.38:0/4079506283 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:45.530 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.529+0000 7f8cd67fc640 1 --2- 172.21.15.38:0/4079506283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cd81012d0 0x7f8cd8198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:45.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.529+0000 7f8cdd7a3640 1 -- 172.21.15.38:0/4079506283 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd8111f60 con 0x7f8cd8108150 2024-09-16T00:33:45.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.529+0000 7f8cdd7a3640 1 -- 172.21.15.38:0/4079506283 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd8111de0 con 0x7f8cd8100920 2024-09-16T00:33:45.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.529+0000 7f8cdd7a3640 1 -- 172.21.15.38:0/4079506283 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8cd81120e0 con 0x7f8cd81012d0 2024-09-16T00:33:45.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.529+0000 7f8cd77fe640 1 -- 172.21.15.38:0/4079506283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cd81012d0 msgr2=0x7f8cd8198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:45.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.529+0000 7f8cd77fe640 1 --2- 172.21.15.38:0/4079506283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cd81012d0 0x7f8cd8198920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:45.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.529+0000 7f8cd77fe640 1 -- 172.21.15.38:0/4079506283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8cd8100920 msgr2=0x7f8cd81983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:45.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.529+0000 7f8cd77fe640 1 --2- 172.21.15.38:0/4079506283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8cd8100920 0x7f8cd81983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:45.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.529+0000 7f8cd77fe640 1 -- 172.21.15.38:0/4079506283 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cd819d390 con 0x7f8cd8108150 2024-09-16T00:33:45.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.529+0000 7f8cd77fe640 1 --2- 172.21.15.38:0/4079506283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cd8108150 0x7f8cd819ccb0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8cd002ade0 tx=0x7f8cd002f3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:45.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.530+0000 7f8cbffff640 1 -- 172.21.15.38:0/4079506283 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd0003b70 con 0x7f8cd8108150 2024-09-16T00:33:45.531 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.530+0000 7f8cbffff640 1 -- 172.21.15.38:0/4079506283 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8cd0003d10 con 0x7f8cd8108150 2024-09-16T00:33:45.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.530+0000 7f8cdd7a3640 1 -- 172.21.15.38:0/4079506283 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cd819d620 con 0x7f8cd8108150 2024-09-16T00:33:45.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.530+0000 7f8cbffff640 1 -- 172.21.15.38:0/4079506283 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cd002f610 con 0x7f8cd8108150 2024-09-16T00:33:45.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.530+0000 7f8cdd7a3640 1 -- 172.21.15.38:0/4079506283 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cd819daa0 con 0x7f8cd8108150 2024-09-16T00:33:45.532 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.532+0000 7f8cdd7a3640 1 -- 172.21.15.38:0/4079506283 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cd8100d00 con 0x7f8cd8108150 2024-09-16T00:33:45.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.533+0000 7f8cbffff640 1 -- 172.21.15.38:0/4079506283 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f8cd0004050 con 0x7f8cd8108150 2024-09-16T00:33:45.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.535+0000 7f8cbffff640 1 --2- 172.21.15.38:0/4079506283 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f8cc00a3350 0x7f8cc00a5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:45.535 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.535+0000 7f8cbffff640 1 -- 172.21.15.38:0/4079506283 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f8cd00f5220 con 0x7f8cd8108150 2024-09-16T00:33:45.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.535+0000 7f8cd6ffd640 1 --2- 172.21.15.38:0/4079506283 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f8cc00a3350 0x7f8cc00a5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:45.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.536+0000 7f8cd6ffd640 1 --2- 172.21.15.38:0/4079506283 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f8cc00a3350 0x7f8cc00a5810 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8009d20 tx=0x7f8cc8005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:45.543 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.543+0000 7f8cbffff640 1 -- 172.21.15.38:0/4079506283 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8cd00c1c90 con 0x7f8cd8108150 2024-09-16T00:33:45.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.890+0000 7f8cdd7a3640 1 -- 172.21.15.38:0/4079506283 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8cd802d7c0 con 0x7f8cd8108150 2024-09-16T00:33:45.894 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.891+0000 7f8cbffff640 1 -- 172.21.15.38:0/4079506283 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f8cd00c1590 con 0x7f8cd8108150 2024-09-16T00:33:45.894 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 26 2024-09-16T00:33:45.895 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:33:45.895 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"btime":"2024-09-16T00:32:41:853514+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/367974527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":367974527},{"type":"v1","addr":"172.21.15.38:6835","nonce":367974527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/335077136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":335077136},{"type":"v1","addr":"172.21.15.38:6837","nonce":335077136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/3501211160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3501211160},{"type":"v1","addr":"172.21.15.38:6839","nonce":3501211160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6841/2623130915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":2623130915},{"type":"v1","addr":"172.21.15.38:6841","nonce":2623130915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/412277519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":412277519},{"type":"v1","addr":"172.21.15.122:6833","nonce":412277519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3987234757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3987234757},{"type":"v1","addr":"172.21.15.122:6835","nonce":3987234757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:30:26.764565+0000","modified":"2024-09-16T00:32:41.355828+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24440},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1210328177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1210328177},{"type":"v1","addr":"172.21.15.73:6833","nonce":1210328177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6837/1858299680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1858299680},{"type":"v1","addr":"172.21.15.122:6837","nonce":1858299680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1983975311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1983975311},{"type":"v1","addr":"172.21.15.122:6839","nonce":1983975311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.73:6835/2866363113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2866363113},{"type":"v1","addr":"172.21.15.73:6835","nonce":2866363113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.73:6837/2681081265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2681081265},{"type":"v1","addr":"172.21.15.73:6837","nonce":2681081265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.73:6839/2396224525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2396224525},{"type":"v1","addr":"172.21.15.73:6839","nonce":2396224525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24464,24440]},"id":1}]} 2024-09-16T00:33:45.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.893+0000 7f8cdd7a3640 1 -- 172.21.15.38:0/4079506283 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f8cc00a3350 msgr2=0x7f8cc00a5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:45.895 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.893+0000 7f8cdd7a3640 1 --2- 172.21.15.38:0/4079506283 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f8cc00a3350 0x7f8cc00a5810 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8cc8009d20 tx=0x7f8cc8005f90 comp rx=0 tx=0).stop 2024-09-16T00:33:45.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.893+0000 7f8cdd7a3640 1 -- 172.21.15.38:0/4079506283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cd8108150 msgr2=0x7f8cd819ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:45.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.893+0000 7f8cdd7a3640 1 --2- 172.21.15.38:0/4079506283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cd8108150 0x7f8cd819ccb0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8cd002ade0 tx=0x7f8cd002f3e0 comp rx=0 tx=0).stop 2024-09-16T00:33:45.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.893+0000 7f8cdd7a3640 1 -- 172.21.15.38:0/4079506283 shutdown_connections 2024-09-16T00:33:45.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.893+0000 7f8cdd7a3640 1 --2- 172.21.15.38:0/4079506283 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f8cc00a3350 0x7f8cc00a5810 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:45.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.893+0000 7f8cdd7a3640 1 --2- 172.21.15.38:0/4079506283 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8cd8108150 0x7f8cd819ccb0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:45.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.893+0000 7f8cdd7a3640 1 --2- 172.21.15.38:0/4079506283 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8cd81012d0 0x7f8cd8198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:45.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.893+0000 7f8cdd7a3640 1 --2- 172.21.15.38:0/4079506283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8cd8100920 0x7f8cd81983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:45.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.893+0000 7f8cdd7a3640 1 -- 172.21.15.38:0/4079506283 >> 172.21.15.38:0/4079506283 conn(0x7f8cd8075f70 msgr2=0x7f8cd80ff860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:45.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.894+0000 7f8cdd7a3640 1 -- 172.21.15.38:0/4079506283 shutdown_connections 2024-09-16T00:33:45.896 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:45.894+0000 7f8cdd7a3640 1 -- 172.21.15.38:0/4079506283 wait complete. 2024-09-16T00:33:46.215 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:45 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/4079506283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:33:46.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:45 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/4079506283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:33:46.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:45 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/4079506283' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:33:46.576 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:33:46.863 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:46.936 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:46 smithi038 ceph-mon[30945]: pgmap v394: 129 pgs: 129 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-16T00:33:47.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:46 smithi122 ceph-mon[35412]: pgmap v394: 129 pgs: 129 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-16T00:33:47.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:46 smithi073 ceph-mon[35254]: pgmap v394: 129 pgs: 129 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-16T00:33:47.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.463+0000 7efcf5910640 1 -- 172.21.15.38:0/4240717668 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcf0106f40 msgr2=0x7efcf010eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:47.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.463+0000 7efcf5910640 1 --2- 172.21.15.38:0/4240717668 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcf0106f40 0x7efcf010eac0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7efce8009ca0 tx=0x7efce802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:33:47.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.464+0000 7efcf5910640 1 -- 172.21.15.38:0/4240717668 shutdown_connections 2024-09-16T00:33:47.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.464+0000 7efcf5910640 1 --2- 172.21.15.38:0/4240717668 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcf0106f40 0x7efcf010eac0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:47.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.464+0000 7efcf5910640 1 --2- 172.21.15.38:0/4240717668 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efcf01027c0 0x7efcf0106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:47.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.464+0000 7efcf5910640 1 --2- 172.21.15.38:0/4240717668 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efcf0101e10 0x7efcf01021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:47.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.464+0000 7efcf5910640 1 -- 172.21.15.38:0/4240717668 >> 172.21.15.38:0/4240717668 conn(0x7efcf00fd8d0 msgr2=0x7efcf00ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:47.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.464+0000 7efcf5910640 1 -- 172.21.15.38:0/4240717668 shutdown_connections 2024-09-16T00:33:47.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.464+0000 7efcf5910640 1 -- 172.21.15.38:0/4240717668 wait complete. 2024-09-16T00:33:47.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.465+0000 7efcf5910640 1 Processor -- start 2024-09-16T00:33:47.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.465+0000 7efcf5910640 1 -- start start 2024-09-16T00:33:47.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.465+0000 7efcf5910640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efcf0101e10 0x7efcf00744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:47.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.465+0000 7efcf5910640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efcf01027c0 0x7efcf0074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:47.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.465+0000 7efcf5910640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcf0106f40 0x7efcf0074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:47.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.466+0000 7efcef7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcf0106f40 0x7efcf0074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:47.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.466+0000 7efcef7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcf0106f40 0x7efcf0074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54008/0 (socket says 172.21.15.38:54008) 2024-09-16T00:33:47.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.466+0000 7efceeffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efcf0101e10 0x7efcf00744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:47.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.466+0000 7efcef7fe640 1 -- 172.21.15.38:0/1061431731 learned_addr learned my addr 172.21.15.38:0/1061431731 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:47.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.466+0000 7efcf5910640 1 -- 172.21.15.38:0/1061431731 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efcf0111ff0 con 0x7efcf0106f40 2024-09-16T00:33:47.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.466+0000 7efcf5910640 1 -- 172.21.15.38:0/1061431731 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efcf0111e70 con 0x7efcf01027c0 2024-09-16T00:33:47.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.466+0000 7efcf5910640 1 -- 172.21.15.38:0/1061431731 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efcf0112170 con 0x7efcf0101e10 2024-09-16T00:33:47.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.466+0000 7efceeffd640 1 -- 172.21.15.38:0/1061431731 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efcf01027c0 msgr2=0x7efcf0074a30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T00:33:47.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.466+0000 7efcee7fc640 1 --2- 172.21.15.38:0/1061431731 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efcf01027c0 0x7efcf0074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:47.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.466+0000 7efceeffd640 1 --2- 172.21.15.38:0/1061431731 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efcf01027c0 0x7efcf0074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:47.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.466+0000 7efceeffd640 1 -- 172.21.15.38:0/1061431731 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcf0106f40 msgr2=0x7efcf0074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:47.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.466+0000 7efceeffd640 1 --2- 172.21.15.38:0/1061431731 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcf0106f40 0x7efcf0074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:47.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.466+0000 7efceeffd640 1 -- 172.21.15.38:0/1061431731 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcf01a66e0 con 0x7efcf0101e10 2024-09-16T00:33:47.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.466+0000 7efcef7fe640 1 --2- 172.21.15.38:0/1061431731 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcf0106f40 0x7efcf0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:33:47.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.467+0000 7efceeffd640 1 --2- 172.21.15.38:0/1061431731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efcf0101e10 0x7efcf00744f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efce000d720 tx=0x7efce000dbe0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:47.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.467+0000 7efcf490e640 1 -- 172.21.15.38:0/1061431731 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce0017070 con 0x7efcf0101e10 2024-09-16T00:33:47.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.467+0000 7efcf5910640 1 -- 172.21.15.38:0/1061431731 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcf01a69d0 con 0x7efcf0101e10 2024-09-16T00:33:47.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.467+0000 7efcf490e640 1 -- 172.21.15.38:0/1061431731 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efce000de30 con 0x7efcf0101e10 2024-09-16T00:33:47.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.468+0000 7efcf490e640 1 -- 172.21.15.38:0/1061431731 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce00047a0 con 0x7efcf0101e10 2024-09-16T00:33:47.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.468+0000 7efcf5910640 1 -- 172.21.15.38:0/1061431731 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcf01a6f10 con 0x7efcf0101e10 2024-09-16T00:33:47.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.471+0000 7efcf490e640 1 -- 172.21.15.38:0/1061431731 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7efce0004220 con 0x7efcf0101e10 2024-09-16T00:33:47.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.472+0000 7efcf7ce9740 1 Processor -- start 2024-09-16T00:33:47.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.472+0000 7efcf7ce9740 1 -- start start 2024-09-16T00:33:47.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.472+0000 7efcf490e640 1 --2- 172.21.15.38:0/1061431731 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efcd80a31b0 0x7efcd80a5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:47.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.472+0000 7efcef7fe640 1 --2- 172.21.15.38:0/1061431731 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efcd80a31b0 0x7efcd80a5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:47.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.473+0000 7efcf7ce9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56094171eb40 0x56094171d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:47.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.473+0000 7efcf7ce9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094171d5d0 0x56094171da10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:47.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.473+0000 7efcf7ce9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560941730b80 0x560941732f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:47.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.473+0000 7efcf7ce9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5609416832f0 con 0x560941730b80 2024-09-16T00:33:47.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.473+0000 7efcf7ce9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56094162acf0 con 0x56094171d5d0 2024-09-16T00:33:47.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.473+0000 7efcf7ce9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56094167f2b0 con 0x56094171eb40 2024-09-16T00:33:47.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.473+0000 7efcef7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560941730b80 0x560941732f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:47.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.473+0000 7efcee7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094171d5d0 0x56094171da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:47.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.473+0000 7efcee7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094171d5d0 0x56094171da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.38:60380/0 (socket says 172.21.15.38:60380) 2024-09-16T00:33:47.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.473+0000 7efcef7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560941730b80 0x560941732f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54024/0 (socket says 172.21.15.38:54024) 2024-09-16T00:33:47.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.473+0000 7efcee7fc640 1 -- 172.21.15.38:0/744126067 learned_addr learned my addr 172.21.15.38:0/744126067 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:47.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.473+0000 7efcf490e640 1 -- 172.21.15.38:0/1061431731 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7efce00cf760 con 0x7efcf0101e10 2024-09-16T00:33:47.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.473+0000 7efcef7fe640 1 -- 172.21.15.38:0/744126067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56094171eb40 msgr2=0x56094171d090 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:33:47.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.473+0000 7efcef7fe640 1 --2- 172.21.15.38:0/744126067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56094171eb40 0x56094171d090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:47.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.473+0000 7efcef7fe640 1 -- 172.21.15.38:0/744126067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094171d5d0 msgr2=0x56094171da10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:47.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.473+0000 7efcef7fe640 1 --2- 172.21.15.38:0/744126067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094171d5d0 0x56094171da10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:47.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.473+0000 7efcef7fe640 1 -- 172.21.15.38:0/744126067 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56094160f3a0 con 0x560941730b80 2024-09-16T00:33:47.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.474+0000 7efcef7fe640 1 --2- 172.21.15.38:0/744126067 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560941730b80 0x560941732f70 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7efce8041000 tx=0x7efce8040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7156b1742007239 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:47.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.474+0000 7efcef7fe640 1 --2- 172.21.15.38:0/1061431731 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efcd80a31b0 0x7efcd80a5670 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7efce8033040 tx=0x7efce8032fb0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:47.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.474+0000 7efcd5ffb640 1 -- 172.21.15.38:0/744126067 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce8050070 con 0x560941730b80 2024-09-16T00:33:47.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.474+0000 7efcd5ffb640 1 -- 172.21.15.38:0/744126067 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efce8036040 con 0x560941730b80 2024-09-16T00:33:47.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.476+0000 7efcf7ce9740 1 -- 172.21.15.38:0/744126067 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560941730b80 msgr2=0x560941732f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:47.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.476+0000 7efcd5ffb640 1 -- 172.21.15.38:0/744126067 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce802fa70 con 0x560941730b80 2024-09-16T00:33:47.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.476+0000 7efcf7ce9740 1 --2- 172.21.15.38:0/744126067 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560941730b80 0x560941732f70 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7efce8041000 tx=0x7efce8040000 comp rx=0 tx=0).stop 2024-09-16T00:33:47.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.476+0000 7efcf7ce9740 1 -- 172.21.15.38:0/744126067 shutdown_connections 2024-09-16T00:33:47.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.476+0000 7efcf7ce9740 1 --2- 172.21.15.38:0/744126067 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560941730b80 0x560941732f70 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:47.484 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.476+0000 7efcf7ce9740 1 --2- 172.21.15.38:0/744126067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094171d5d0 0x56094171da10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:47.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.476+0000 7efcf7ce9740 1 --2- 172.21.15.38:0/744126067 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56094171eb40 0x56094171d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:47.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.476+0000 7efcf7ce9740 1 -- 172.21.15.38:0/744126067 >> 172.21.15.38:0/744126067 conn(0x5609416e6310 msgr2=0x560941719c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:47.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.476+0000 7efcf7ce9740 1 -- 172.21.15.38:0/744126067 shutdown_connections 2024-09-16T00:33:47.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.477+0000 7efcf7ce9740 1 -- 172.21.15.38:0/744126067 wait complete. 2024-09-16T00:33:47.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.477+0000 7efcf7ce9740 1 -- start start 2024-09-16T00:33:47.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.478+0000 7efcf7ce9740 1 Processor -- start 2024-09-16T00:33:47.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.478+0000 7efcf7ce9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094171d5d0 0x560941732c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:47.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.478+0000 7efcf7ce9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56094171eb40 0x56094172ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:47.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.478+0000 7efcf7ce9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560941730440 0x5609417308c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:47.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.478+0000 7efceeffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094171d5d0 0x560941732c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:47.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.479+0000 7efceeffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094171d5d0 0x560941732c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.38:60382/0 (socket says 172.21.15.38:60382) 2024-09-16T00:33:47.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.479+0000 7efcef7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560941730440 0x5609417308c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:47.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.479+0000 7efcef7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560941730440 0x5609417308c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54028/0 (socket says 172.21.15.38:54028) 2024-09-16T00:33:47.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.479+0000 7efceeffd640 1 -- 172.21.15.38:0/162405503 learned_addr learned my addr 172.21.15.38:0/162405503 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:47.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.479+0000 7efcf7ce9740 1 -- 172.21.15.38:0/162405503 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5609417336f0 con 0x560941730440 2024-09-16T00:33:47.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.479+0000 7efcf7ce9740 1 -- 172.21.15.38:0/162405503 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56094163e060 con 0x56094171d5d0 2024-09-16T00:33:47.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.479+0000 7efcf7ce9740 1 -- 172.21.15.38:0/162405503 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x560941733870 con 0x56094171eb40 2024-09-16T00:33:47.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.479+0000 7efcef7fe640 1 -- 172.21.15.38:0/162405503 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56094171eb40 msgr2=0x56094172ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:47.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.479+0000 7efcef7fe640 1 --2- 172.21.15.38:0/162405503 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56094171eb40 0x56094172ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:47.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.479+0000 7efcef7fe640 1 -- 172.21.15.38:0/162405503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094171d5d0 msgr2=0x560941732c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:47.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.479+0000 7efcef7fe640 1 --2- 172.21.15.38:0/162405503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094171d5d0 0x560941732c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:47.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.479+0000 7efcef7fe640 1 -- 172.21.15.38:0/162405503 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5609413f43c0 con 0x560941730440 2024-09-16T00:33:47.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.480+0000 7efcef7fe640 1 --2- 172.21.15.38:0/162405503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560941730440 0x5609417308c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7efce8041000 tx=0x7efce8030000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a989518c4694e59 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:47.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.480+0000 7efcceffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:33:47.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.480+0000 7efcceffd640 1 -- 172.21.15.38:0/162405503 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce805b060 con 0x560941730440 2024-09-16T00:33:47.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.480+0000 7efcf7ce9740 10 client.0 _ll_register_callbacks cb 0x5609413918c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:47.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.480+0000 7efcceffd640 1 -- 172.21.15.38:0/162405503 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7efce8004790 con 0x560941730440 2024-09-16T00:33:47.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.480+0000 7efcf7ce9740 1 -- 172.21.15.38:0/162405503 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efce8036040 con 0x560941730440 2024-09-16T00:33:47.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.480+0000 7efcceffd640 1 -- 172.21.15.38:0/162405503 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efce8062050 con 0x560941730440 2024-09-16T00:33:47.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.480+0000 7efcf7ce9740 10 client.15591 fetch_fsmap learned FSMap version 26 2024-09-16T00:33:47.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.480+0000 7efcf7ce9740 1 -- 172.21.15.38:0/162405503 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({fsmap=26}) -- 0x56094173ecf0 con 0x560941730440 2024-09-16T00:33:47.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.481+0000 7efcceffd640 1 -- 172.21.15.38:0/162405503 <== mon.0 v2:172.21.15.38:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7efce8044040 con 0x560941730440 2024-09-16T00:33:47.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.481+0000 7efcf7ce9740 10 client.15591 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:33:47.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.481+0000 7efcf7ce9740 10 client.15591 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-16T00:33:47.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.481+0000 7efcf7ce9740 20 client.15591 populate_metadata read hostname 'smithi038' 2024-09-16T00:33:47.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.481+0000 7efcf7ce9740 1 --2- 172.21.15.38:0/162405503 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x5609417401a0 0x560941742640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:47.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.481+0000 7efcf7ce9740 4 client.15591 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:33:47.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.481+0000 7efcf7ce9740 1 -- 172.21.15.38:0/162405503 --> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7efce8036040 con 0x5609417401a0 2024-09-16T00:33:47.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.481+0000 7efceeffd640 1 --2- 172.21.15.38:0/162405503 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x5609417401a0 0x560941742640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:47.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.482+0000 7efcceffd640 10 client.15591 ms_handle_connect on v2:172.21.15.73:6834/2866363113 2024-09-16T00:33:47.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.482+0000 7efceeffd640 1 --2- 172.21.15.38:0/162405503 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x5609417401a0 0x560941742640 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d73f2b188b536bdc server_cookie=b40140c4ac840c21 in_seq=0 out_seq=0 2024-09-16T00:33:47.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.483+0000 7efcceffd640 1 -- 172.21.15.38:0/162405503 <== mds.2 v2:172.21.15.73:6834/2866363113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7efce8036040 con 0x5609417401a0 2024-09-16T00:33:47.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.483+0000 7efcceffd640 10 client.15591 handle_command_reply: tid=0 2024-09-16T00:33:47.488 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.483+0000 7efcf7ce9740 1 client.15591 shutdown 2024-09-16T00:33:47.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.483+0000 7efcf7ce9740 1 -- 172.21.15.38:0/162405503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560941730440 msgr2=0x5609417308c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:47.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.483+0000 7efcf7ce9740 1 --2- 172.21.15.38:0/162405503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560941730440 0x5609417308c0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7efce8041000 tx=0x7efce8030000 comp rx=0 tx=0).stop 2024-09-16T00:33:47.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.483+0000 7efcf7ce9740 1 -- 172.21.15.38:0/162405503 shutdown_connections 2024-09-16T00:33:47.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.483+0000 7efcf7ce9740 1 --2- 172.21.15.38:0/162405503 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x5609417401a0 0x560941742640 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:47.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.483+0000 7efcf7ce9740 1 --2- 172.21.15.38:0/162405503 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x560941730440 0x5609417308c0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:47.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.483+0000 7efcf7ce9740 1 --2- 172.21.15.38:0/162405503 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56094171eb40 0x56094172ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:47.489 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.483+0000 7efcf7ce9740 1 --2- 172.21.15.38:0/162405503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094171d5d0 0x560941732c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:47.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.483+0000 7efcceffd640 0 client.15591 ms_handle_reset on v2:172.21.15.73:6834/2866363113 2024-09-16T00:33:47.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.483+0000 7efcf7ce9740 1 -- 172.21.15.38:0/162405503 >> 172.21.15.38:0/162405503 conn(0x5609416e6310 msgr2=0x56094171f960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:47.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.483+0000 7efcf7ce9740 1 -- 172.21.15.38:0/162405503 shutdown_connections 2024-09-16T00:33:47.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.483+0000 7efcf7ce9740 1 -- 172.21.15.38:0/162405503 wait complete. 2024-09-16T00:33:47.490 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.483+0000 7efcf7ce9740 20 client.15591 trim_cache size 0 max 16384 2024-09-16T00:33:47.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.510+0000 7efcf7ce9740 1 Processor -- start 2024-09-16T00:33:47.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.510+0000 7efcf7ce9740 1 -- start start 2024-09-16T00:33:47.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.511+0000 7efcf7ce9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094171d5d0 0x560941719d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:47.511 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.511+0000 7efcf7ce9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56094171eb40 0x560941718bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:47.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.511+0000 7efcee7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56094171eb40 0x560941718bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:47.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.511+0000 7efcee7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56094171eb40 0x560941718bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54044/0 (socket says 172.21.15.38:54044) 2024-09-16T00:33:47.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.511+0000 7efcee7fc640 1 -- 172.21.15.38:0/1808367906 learned_addr learned my addr 172.21.15.38:0/1808367906 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:47.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.511+0000 7efcf7ce9740 1 --2- 172.21.15.38:0/1808367906 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560941730440 0x560941747f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:47.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.511+0000 7efcf7ce9740 1 -- 172.21.15.38:0/1808367906 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efcf01aef80 con 0x56094171eb40 2024-09-16T00:33:47.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.511+0000 7efcf7ce9740 1 -- 172.21.15.38:0/1808367906 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efcf01ac850 con 0x56094171d5d0 2024-09-16T00:33:47.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.511+0000 7efcf7ce9740 1 -- 172.21.15.38:0/1808367906 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efcf01aa6e0 con 0x560941730440 2024-09-16T00:33:47.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.512+0000 7efcee7fc640 1 -- 172.21.15.38:0/1808367906 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560941730440 msgr2=0x560941747f50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:33:47.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.512+0000 7efcee7fc640 1 --2- 172.21.15.38:0/1808367906 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560941730440 0x560941747f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:47.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.512+0000 7efcee7fc640 1 -- 172.21.15.38:0/1808367906 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094171d5d0 msgr2=0x560941719d70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:33:47.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.512+0000 7efcee7fc640 1 --2- 172.21.15.38:0/1808367906 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094171d5d0 0x560941719d70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:47.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.512+0000 7efcee7fc640 1 -- 172.21.15.38:0/1808367906 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56094172e450 con 0x56094171eb40 2024-09-16T00:33:47.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.512+0000 7efcee7fc640 1 --2- 172.21.15.38:0/1808367906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56094171eb40 0x560941718bf0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7efce4009ac0 tx=0x7efce4009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f89f12cf44a827b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:47.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.513+0000 7efcd57fa640 1 -- 172.21.15.38:0/1808367906 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce4011d20 con 0x56094171eb40 2024-09-16T00:33:47.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.513+0000 7efcd57fa640 1 -- 172.21.15.38:0/1808367906 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efce4005500 con 0x56094171eb40 2024-09-16T00:33:47.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.513+0000 7efcd57fa640 1 -- 172.21.15.38:0/1808367906 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce40126b0 con 0x56094171eb40 2024-09-16T00:33:47.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.515+0000 7efcf7ce9740 1 -- 172.21.15.38:0/1808367906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56094171eb40 msgr2=0x560941718bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:47.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.515+0000 7efcf7ce9740 1 --2- 172.21.15.38:0/1808367906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56094171eb40 0x560941718bf0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7efce4009ac0 tx=0x7efce4009f80 comp rx=0 tx=0).stop 2024-09-16T00:33:47.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.515+0000 7efcf7ce9740 1 -- 172.21.15.38:0/1808367906 shutdown_connections 2024-09-16T00:33:47.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.515+0000 7efcf7ce9740 1 --2- 172.21.15.38:0/1808367906 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x560941730440 0x560941747f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:47.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.515+0000 7efcf7ce9740 1 --2- 172.21.15.38:0/1808367906 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56094171eb40 0x560941718bf0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:47.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.515+0000 7efcf7ce9740 1 --2- 172.21.15.38:0/1808367906 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094171d5d0 0x560941719d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:47.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.515+0000 7efcf7ce9740 1 -- 172.21.15.38:0/1808367906 >> 172.21.15.38:0/1808367906 conn(0x5609416e6310 msgr2=0x5609417184e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:47.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.515+0000 7efcf7ce9740 1 -- 172.21.15.38:0/1808367906 shutdown_connections 2024-09-16T00:33:47.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.516+0000 7efcf7ce9740 1 -- 172.21.15.38:0/1808367906 wait complete. 2024-09-16T00:33:47.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.516+0000 7efcf7ce9740 1 -- start start 2024-09-16T00:33:47.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.517+0000 7efcf7ce9740 1 Processor -- start 2024-09-16T00:33:47.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.517+0000 7efcf7ce9740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56094171d5d0 0x560941734bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:47.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.518+0000 7efcf7ce9740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560941730440 0x560941735100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:47.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.518+0000 7efcf7ce9740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5609417401a0 0x5609417a9f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:47.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.518+0000 7efcf7ce9740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efcf01aad30 con 0x56094171d5d0 2024-09-16T00:33:47.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.518+0000 7efceeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56094171d5d0 0x560941734bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:47.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.518+0000 7efcf7ce9740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efcf01ac180 con 0x560941730440 2024-09-16T00:33:47.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.518+0000 7efcf7ce9740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efcf01ab3e0 con 0x5609417401a0 2024-09-16T00:33:47.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.518+0000 7efcf7ce9740 10 client.0 _ll_register_callbacks cb 0x7efce804e070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:47.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.518+0000 7efceeffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56094171d5d0 0x560941734bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54046/0 (socket says 172.21.15.38:54046) 2024-09-16T00:33:47.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.518+0000 7efceeffd640 1 -- 172.21.15.38:0/1560481090 learned_addr learned my addr 172.21.15.38:0/1560481090 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:47.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.518+0000 7efcef7fe640 1 --2- 172.21.15.38:0/1560481090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5609417401a0 0x5609417a9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:47.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.518+0000 7efcee7fc640 1 --2- 172.21.15.38:0/1560481090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560941730440 0x560941735100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:47.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.519+0000 7efcef7fe640 1 -- 172.21.15.38:0/1560481090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560941730440 msgr2=0x560941735100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:47.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.519+0000 7efcef7fe640 1 --2- 172.21.15.38:0/1560481090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560941730440 0x560941735100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:47.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.519+0000 7efcef7fe640 1 -- 172.21.15.38:0/1560481090 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56094171d5d0 msgr2=0x560941734bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:47.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.519+0000 7efcef7fe640 1 --2- 172.21.15.38:0/1560481090 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56094171d5d0 0x560941734bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:47.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.519+0000 7efcef7fe640 1 -- 172.21.15.38:0/1560481090 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560941735990 con 0x5609417401a0 2024-09-16T00:33:47.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.519+0000 7efcee7fc640 1 --2- 172.21.15.38:0/1560481090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560941730440 0x560941735100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:47.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.519+0000 7efceeffd640 1 --2- 172.21.15.38:0/1560481090 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56094171d5d0 0x560941734bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:33:47.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.519+0000 7efcef7fe640 1 --2- 172.21.15.38:0/1560481090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5609417401a0 0x5609417a9f20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efce8009a60 tx=0x7efce8039000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=34b36b458bc24bf2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:47.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.519+0000 7efcceffd640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-16T00:33:47.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.520+0000 7efcceffd640 1 -- 172.21.15.38:0/1560481090 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efce805b040 con 0x5609417401a0 2024-09-16T00:33:47.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.520+0000 7efcceffd640 1 -- 172.21.15.38:0/1560481090 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7efce8004630 con 0x5609417401a0 2024-09-16T00:33:47.521 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.520+0000 7efcf7ce9740 1 -- 172.21.15.38:0/1560481090 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efce4005500 con 0x5609417401a0 2024-09-16T00:33:47.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.520+0000 7efcceffd640 1 -- 172.21.15.38:0/1560481090 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7efce8050070 con 0x5609417401a0 2024-09-16T00:33:47.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.520+0000 7efcf7ce9740 10 client.25159 fetch_fsmap learned FSMap version 26 2024-09-16T00:33:47.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.520+0000 7efcf7ce9740 1 -- 172.21.15.38:0/1560481090 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5609417ab530 con 0x5609417401a0 2024-09-16T00:33:47.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.520+0000 7efcceffd640 1 -- 172.21.15.38:0/1560481090 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efce804d020 con 0x5609417401a0 2024-09-16T00:33:47.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.521+0000 7efcceffd640 1 -- 172.21.15.38:0/1560481090 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7efce8044040 con 0x5609417401a0 2024-09-16T00:33:47.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.521+0000 7efcf7ce9740 10 client.25159 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:33:47.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.521+0000 7efcf7ce9740 10 client.25159 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-16T00:33:47.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.521+0000 7efcf7ce9740 20 client.25159 populate_metadata read hostname 'smithi038' 2024-09-16T00:33:47.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.521+0000 7efcf7ce9740 1 --2- 172.21.15.38:0/1560481090 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x5609417abbf0 0x5609417adfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:47.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.521+0000 7efcf7ce9740 4 client.25159 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:33:47.522 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.521+0000 7efcf7ce9740 1 -- 172.21.15.38:0/1560481090 --> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7efce4005500 con 0x5609417abbf0 2024-09-16T00:33:47.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.522+0000 7efceeffd640 1 --2- 172.21.15.38:0/1560481090 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x5609417abbf0 0x5609417adfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:47.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.523+0000 7efceeffd640 1 --2- 172.21.15.38:0/1560481090 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x5609417abbf0 0x5609417adfe0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5942f81e9254b679 server_cookie=33ed5d67f969f0a0 in_seq=0 out_seq=0 2024-09-16T00:33:47.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.523+0000 7efcceffd640 10 client.25159 ms_handle_connect on v2:172.21.15.73:6834/2866363113 2024-09-16T00:33:47.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.523+0000 7efcceffd640 1 -- 172.21.15.38:0/1560481090 <== mds.2 v2:172.21.15.73:6834/2866363113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7efce4005500 con 0x5609417abbf0 2024-09-16T00:33:47.523 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.523+0000 7efcceffd640 10 client.25159 handle_command_reply: tid=0 2024-09-16T00:33:47.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.523+0000 7efcf7ce9740 1 client.25159 shutdown 2024-09-16T00:33:47.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.524+0000 7efcf7ce9740 1 -- 172.21.15.38:0/1560481090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5609417401a0 msgr2=0x5609417a9f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:47.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.524+0000 7efcf7ce9740 1 --2- 172.21.15.38:0/1560481090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5609417401a0 0x5609417a9f20 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7efce8009a60 tx=0x7efce8039000 comp rx=0 tx=0).stop 2024-09-16T00:33:47.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.524+0000 7efcf7ce9740 1 -- 172.21.15.38:0/1560481090 shutdown_connections 2024-09-16T00:33:47.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.524+0000 7efcf7ce9740 1 --2- 172.21.15.38:0/1560481090 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x5609417abbf0 0x5609417adfe0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:47.524 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.524+0000 7efcceffd640 0 client.25159 ms_handle_reset on v2:172.21.15.73:6834/2866363113 2024-09-16T00:33:47.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.524+0000 7efcf7ce9740 1 --2- 172.21.15.38:0/1560481090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5609417401a0 0x5609417a9f20 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:47.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.524+0000 7efcf7ce9740 1 --2- 172.21.15.38:0/1560481090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560941730440 0x560941735100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:47.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.524+0000 7efcf7ce9740 1 --2- 172.21.15.38:0/1560481090 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56094171d5d0 0x560941734bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:47.525 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.524+0000 7efcf7ce9740 1 -- 172.21.15.38:0/1560481090 >> 172.21.15.38:0/1560481090 conn(0x5609416e6310 msgr2=0x560941718d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:47.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.525+0000 7efcf7ce9740 1 -- 172.21.15.38:0/1560481090 shutdown_connections 2024-09-16T00:33:47.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.525+0000 7efcf7ce9740 1 -- 172.21.15.38:0/1560481090 wait complete. 2024-09-16T00:33:47.526 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.525+0000 7efcf7ce9740 20 client.25159 trim_cache size 0 max 16384 2024-09-16T00:33:47.526 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-09-16T00:33:47.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.526+0000 7efcf5910640 1 -- 172.21.15.38:0/1061431731 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efcd80a31b0 msgr2=0x7efcd80a5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:47.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.526+0000 7efcf5910640 1 --2- 172.21.15.38:0/1061431731 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efcd80a31b0 0x7efcd80a5670 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7efce8033040 tx=0x7efce8032fb0 comp rx=0 tx=0).stop 2024-09-16T00:33:47.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.526+0000 7efcf5910640 1 -- 172.21.15.38:0/1061431731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efcf0101e10 msgr2=0x7efcf00744f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:47.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.526+0000 7efcf5910640 1 --2- 172.21.15.38:0/1061431731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efcf0101e10 0x7efcf00744f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7efce000d720 tx=0x7efce000dbe0 comp rx=0 tx=0).stop 2024-09-16T00:33:47.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.526+0000 7efcf5910640 1 -- 172.21.15.38:0/1061431731 shutdown_connections 2024-09-16T00:33:47.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.526+0000 7efcf5910640 1 --2- 172.21.15.38:0/1061431731 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efcd80a31b0 0x7efcd80a5670 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:47.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.526+0000 7efcf5910640 1 --2- 172.21.15.38:0/1061431731 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efcf0106f40 0x7efcf0074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:47.527 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.526+0000 7efcf5910640 1 --2- 172.21.15.38:0/1061431731 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efcf01027c0 0x7efcf0074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:47.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.526+0000 7efcf5910640 1 --2- 172.21.15.38:0/1061431731 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efcf0101e10 0x7efcf00744f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:47.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.526+0000 7efcf5910640 1 -- 172.21.15.38:0/1061431731 >> 172.21.15.38:0/1061431731 conn(0x7efcf00fd8d0 msgr2=0x7efcf00fe170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:47.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.527+0000 7efcf5910640 1 -- 172.21.15.38:0/1061431731 shutdown_connections 2024-09-16T00:33:47.528 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:47.527+0000 7efcf5910640 1 -- 172.21.15.38:0/1061431731 wait complete. 2024-09-16T00:33:48.208 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:33:48.481 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:49.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.038+0000 7ffaf1a27640 1 -- 172.21.15.38:0/57414787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffaec1036e0 msgr2=0x7ffaec107b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:49.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.038+0000 7ffaf1a27640 1 --2- 172.21.15.38:0/57414787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffaec1036e0 0x7ffaec107b90 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7ffae0009930 tx=0x7ffae002a420 comp rx=0 tx=0).stop 2024-09-16T00:33:49.041 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.038+0000 7ffaf1a27640 1 -- 172.21.15.38:0/57414787 shutdown_connections 2024-09-16T00:33:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.038+0000 7ffaf1a27640 1 --2- 172.21.15.38:0/57414787 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffaec1082a0 0x7ffaec10ec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.038+0000 7ffaf1a27640 1 --2- 172.21.15.38:0/57414787 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffaec1036e0 0x7ffaec107b90 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.038+0000 7ffaf1a27640 1 --2- 172.21.15.38:0/57414787 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffaec102d30 0x7ffaec103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.038+0000 7ffaf1a27640 1 -- 172.21.15.38:0/57414787 >> 172.21.15.38:0/57414787 conn(0x7ffaec0fc7d0 msgr2=0x7ffaec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.038+0000 7ffaf1a27640 1 -- 172.21.15.38:0/57414787 shutdown_connections 2024-09-16T00:33:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.038+0000 7ffaf1a27640 1 -- 172.21.15.38:0/57414787 wait complete. 2024-09-16T00:33:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.039+0000 7ffaf1a27640 1 Processor -- start 2024-09-16T00:33:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.039+0000 7ffaf1a27640 1 -- start start 2024-09-16T00:33:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.039+0000 7ffaf1a27640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffaec102d30 0x7ffaec198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.039+0000 7ffaf1a27640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffaec1036e0 0x7ffaec198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.039+0000 7ffaf1a27640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffaec1082a0 0x7ffaec19ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.039+0000 7ffaf1a27640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec112090 con 0x7ffaec1082a0 2024-09-16T00:33:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.039+0000 7ffaf1a27640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec111f10 con 0x7ffaec1036e0 2024-09-16T00:33:49.042 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.039+0000 7ffaf1a27640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec112210 con 0x7ffaec102d30 2024-09-16T00:33:49.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.040+0000 7ffaeb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffaec1082a0 0x7ffaec19ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:49.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.040+0000 7ffaeb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffaec1082a0 0x7ffaec19ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54058/0 (socket says 172.21.15.38:54058) 2024-09-16T00:33:49.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.040+0000 7ffaeb7fe640 1 -- 172.21.15.38:0/3928706031 learned_addr learned my addr 172.21.15.38:0/3928706031 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:49.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.040+0000 7ffaea7fc640 1 --2- 172.21.15.38:0/3928706031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffaec1036e0 0x7ffaec198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:49.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.040+0000 7ffaeaffd640 1 --2- 172.21.15.38:0/3928706031 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffaec102d30 0x7ffaec198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:49.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.040+0000 7ffaeb7fe640 1 -- 172.21.15.38:0/3928706031 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffaec102d30 msgr2=0x7ffaec198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:49.043 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.040+0000 7ffaeb7fe640 1 --2- 172.21.15.38:0/3928706031 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffaec102d30 0x7ffaec198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:49.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.040+0000 7ffaeb7fe640 1 -- 172.21.15.38:0/3928706031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffaec1036e0 msgr2=0x7ffaec198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:49.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.040+0000 7ffaeb7fe640 1 --2- 172.21.15.38:0/3928706031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffaec1036e0 0x7ffaec198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:49.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.040+0000 7ffaeb7fe640 1 -- 172.21.15.38:0/3928706031 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaec19d540 con 0x7ffaec1082a0 2024-09-16T00:33:49.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.040+0000 7ffaeb7fe640 1 --2- 172.21.15.38:0/3928706031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffaec1082a0 0x7ffaec19ce60 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ffae400cce0 tx=0x7ffae400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:49.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.041+0000 7ffaeaffd640 1 --2- 172.21.15.38:0/3928706031 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffaec102d30 0x7ffaec198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:33:49.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.041+0000 7ffaf0a25640 1 -- 172.21.15.38:0/3928706031 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae400f450 con 0x7ffaec1082a0 2024-09-16T00:33:49.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.041+0000 7ffaf0a25640 1 -- 172.21.15.38:0/3928706031 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffae400fad0 con 0x7ffaec1082a0 2024-09-16T00:33:49.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.041+0000 7ffaf0a25640 1 -- 172.21.15.38:0/3928706031 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae4018c10 con 0x7ffaec1082a0 2024-09-16T00:33:49.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.041+0000 7ffaf1a27640 1 -- 172.21.15.38:0/3928706031 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaec19d830 con 0x7ffaec1082a0 2024-09-16T00:33:49.044 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.041+0000 7ffaf1a27640 1 -- 172.21.15.38:0/3928706031 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaec19dc90 con 0x7ffaec1082a0 2024-09-16T00:33:49.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.045+0000 7ffaf0a25640 1 -- 172.21.15.38:0/3928706031 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7ffae400fc80 con 0x7ffaec1082a0 2024-09-16T00:33:49.045 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.045+0000 7ffaf0a25640 1 --2- 172.21.15.38:0/3928706031 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ffad40a3420 0x7ffad40a58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:49.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.045+0000 7ffaf3e00740 1 Processor -- start 2024-09-16T00:33:49.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.045+0000 7ffaf0a25640 1 -- 172.21.15.38:0/3928706031 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7ffae40d00d0 con 0x7ffaec1082a0 2024-09-16T00:33:49.046 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.046+0000 7ffaeaffd640 1 --2- 172.21.15.38:0/3928706031 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ffad40a3420 0x7ffad40a58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:49.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.046+0000 7ffaf3e00740 1 -- start start 2024-09-16T00:33:49.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.046+0000 7ffaf3e00740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593d8d50f50 0x5593d8d4f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:49.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.046+0000 7ffaf3e00740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593d8d4f9e0 0x5593d8d4fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:49.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.046+0000 7ffaf3e00740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593d8d49660 0x5593d8d49af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:49.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.046+0000 7ffaf3e00740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5593d8cb12f0 con 0x5593d8d4f9e0 2024-09-16T00:33:49.047 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.046+0000 7ffaf3e00740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5593d8c58cf0 con 0x5593d8d49660 2024-09-16T00:33:49.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.046+0000 7ffaf3e00740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5593d8cad2b0 con 0x5593d8d50f50 2024-09-16T00:33:49.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.046+0000 7ffaea7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593d8d4f9e0 0x5593d8d4fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:49.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.046+0000 7ffaea7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593d8d4f9e0 0x5593d8d4fe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54066/0 (socket says 172.21.15.38:54066) 2024-09-16T00:33:49.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.046+0000 7ffaea7fc640 1 -- 172.21.15.38:0/991889354 learned_addr learned my addr 172.21.15.38:0/991889354 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:49.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.046+0000 7ffaeb7fe640 1 --2- 172.21.15.38:0/991889354 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593d8d49660 0x5593d8d49af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:49.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.046+0000 7ffaeaffd640 1 --2- 172.21.15.38:0/3928706031 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ffad40a3420 0x7ffad40a58e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ffadc005d00 tx=0x7ffadc00b410 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:49.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.046+0000 7ffaeaffd640 1 --2- 172.21.15.38:0/991889354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593d8d50f50 0x5593d8d4f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:49.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.046+0000 7ffaea7fc640 1 -- 172.21.15.38:0/991889354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593d8d50f50 msgr2=0x5593d8d4f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:49.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.046+0000 7ffaea7fc640 1 --2- 172.21.15.38:0/991889354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593d8d50f50 0x5593d8d4f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:49.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.046+0000 7ffaea7fc640 1 -- 172.21.15.38:0/991889354 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593d8d49660 msgr2=0x5593d8d49af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:49.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.046+0000 7ffaea7fc640 1 --2- 172.21.15.38:0/991889354 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593d8d49660 0x5593d8d49af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:49.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.047+0000 7ffaea7fc640 1 -- 172.21.15.38:0/991889354 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593d8c3d3a0 con 0x5593d8d4f9e0 2024-09-16T00:33:49.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.047+0000 7ffaeb7fe640 1 --2- 172.21.15.38:0/991889354 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593d8d49660 0x5593d8d49af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:49.048 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.047+0000 7ffaea7fc640 1 --2- 172.21.15.38:0/991889354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593d8d4f9e0 0x5593d8d4fe20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ffae0009900 tx=0x7ffae0004170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=982e419aa8f3af73 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:49.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.047+0000 7ffad1ffb640 1 -- 172.21.15.38:0/991889354 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae0040070 con 0x5593d8d4f9e0 2024-09-16T00:33:49.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.047+0000 7ffad1ffb640 1 -- 172.21.15.38:0/991889354 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffae0005500 con 0x5593d8d4f9e0 2024-09-16T00:33:49.049 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.047+0000 7ffad1ffb640 1 -- 172.21.15.38:0/991889354 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae0035040 con 0x5593d8d4f9e0 2024-09-16T00:33:49.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.048+0000 7ffaf3e00740 1 -- 172.21.15.38:0/991889354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593d8d4f9e0 msgr2=0x5593d8d4fe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:49.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.048+0000 7ffaf3e00740 1 --2- 172.21.15.38:0/991889354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593d8d4f9e0 0x5593d8d4fe20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7ffae0009900 tx=0x7ffae0004170 comp rx=0 tx=0).stop 2024-09-16T00:33:49.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.048+0000 7ffaf3e00740 1 -- 172.21.15.38:0/991889354 shutdown_connections 2024-09-16T00:33:49.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.048+0000 7ffaf3e00740 1 --2- 172.21.15.38:0/991889354 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593d8d49660 0x5593d8d49af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:49.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.048+0000 7ffaf3e00740 1 --2- 172.21.15.38:0/991889354 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593d8d4f9e0 0x5593d8d4fe20 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:49.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.048+0000 7ffaf3e00740 1 --2- 172.21.15.38:0/991889354 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593d8d50f50 0x5593d8d4f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:49.051 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.048+0000 7ffaf3e00740 1 -- 172.21.15.38:0/991889354 >> 172.21.15.38:0/991889354 conn(0x5593d8d14310 msgr2=0x5593d8d4c070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:49.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.048+0000 7ffaf3e00740 1 -- 172.21.15.38:0/991889354 shutdown_connections 2024-09-16T00:33:49.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.048+0000 7ffaf3e00740 1 -- 172.21.15.38:0/991889354 wait complete. 2024-09-16T00:33:49.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.049+0000 7ffaf3e00740 1 -- start start 2024-09-16T00:33:49.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.049+0000 7ffaf3e00740 1 Processor -- start 2024-09-16T00:33:49.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.049+0000 7ffaf3e00740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593d8d49660 0x5593d8d57e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:49.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.049+0000 7ffaf3e00740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593d8d4f9e0 0x5593d8d56190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:49.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.049+0000 7ffaf3e00740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593d8d50f50 0x5593d8d566d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:49.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.049+0000 7ffaf3e00740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5593d8d4a320 con 0x5593d8d50f50 2024-09-16T00:33:49.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.049+0000 7ffaf3e00740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5593d8c6c060 con 0x5593d8d4f9e0 2024-09-16T00:33:49.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.049+0000 7ffaf3e00740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5593d8d4a4a0 con 0x5593d8d49660 2024-09-16T00:33:49.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.049+0000 7ffaf3e00740 10 client.0 _ll_register_callbacks cb 0x5593d89bf8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:49.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.050+0000 7ffaea7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593d8d4f9e0 0x5593d8d56190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:49.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.050+0000 7ffaea7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593d8d4f9e0 0x5593d8d56190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.38:60428/0 (socket says 172.21.15.38:60428) 2024-09-16T00:33:49.052 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.050+0000 7ffaea7fc640 1 -- 172.21.15.38:0/1238912400 learned_addr learned my addr 172.21.15.38:0/1238912400 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:49.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.050+0000 7ffaeb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593d8d50f50 0x5593d8d566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:49.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.050+0000 7ffaeaffd640 1 --2- 172.21.15.38:0/1238912400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593d8d49660 0x5593d8d57e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:49.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.050+0000 7ffaea7fc640 1 -- 172.21.15.38:0/1238912400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593d8d49660 msgr2=0x5593d8d57e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:49.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.050+0000 7ffaea7fc640 1 --2- 172.21.15.38:0/1238912400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593d8d49660 0x5593d8d57e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:49.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.050+0000 7ffaea7fc640 1 -- 172.21.15.38:0/1238912400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593d8d50f50 msgr2=0x5593d8d566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:49.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.050+0000 7ffaea7fc640 1 --2- 172.21.15.38:0/1238912400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593d8d50f50 0x5593d8d566d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:49.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.050+0000 7ffaea7fc640 1 -- 172.21.15.38:0/1238912400 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593d8a223c0 con 0x5593d8d4f9e0 2024-09-16T00:33:49.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.050+0000 7ffaeb7fe640 1 --2- 172.21.15.38:0/1238912400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593d8d50f50 0x5593d8d566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:33:49.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.050+0000 7ffaea7fc640 1 --2- 172.21.15.38:0/1238912400 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593d8d4f9e0 0x5593d8d56190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ffae0006010 tx=0x7ffae0005250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=490d817407271910 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:49.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.050+0000 7ffacaffd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-09-16T00:33:49.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.050+0000 7ffacaffd640 1 -- 172.21.15.38:0/1238912400 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae004d060 con 0x5593d8d4f9e0 2024-09-16T00:33:49.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.051+0000 7ffacaffd640 1 -- 172.21.15.38:0/1238912400 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7ffae0004bb0 con 0x5593d8d4f9e0 2024-09-16T00:33:49.053 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.051+0000 7ffaf3e00740 1 -- 172.21.15.38:0/1238912400 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffae0005500 con 0x5593d8d4f9e0 2024-09-16T00:33:49.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.051+0000 7ffacaffd640 1 -- 172.21.15.38:0/1238912400 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ffae0054050 con 0x5593d8d4f9e0 2024-09-16T00:33:49.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.051+0000 7ffaf3e00740 10 client.25226 fetch_fsmap learned FSMap version 26 2024-09-16T00:33:49.054 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.051+0000 7ffaf3e00740 1 -- 172.21.15.38:0/1238912400 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5593d8d6d320 con 0x5593d8d4f9e0 2024-09-16T00:33:49.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.051+0000 7ffacaffd640 1 -- 172.21.15.38:0/1238912400 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffae0005500 con 0x5593d8d4f9e0 2024-09-16T00:33:49.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.051+0000 7ffacaffd640 1 -- 172.21.15.38:0/1238912400 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ffae00036f0 con 0x5593d8d4f9e0 2024-09-16T00:33:49.055 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.052+0000 7ffaf3e00740 10 client.25226 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:33:49.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.052+0000 7ffaf3e00740 10 client.25226 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:33:49.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.052+0000 7ffaf3e00740 20 client.25226 populate_metadata read hostname 'smithi038' 2024-09-16T00:33:49.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.052+0000 7ffaf3e00740 1 --2- 172.21.15.38:0/1238912400 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x5593d8d6e670 0x5593d8d70a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:49.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.052+0000 7ffaf3e00740 4 client.25226 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:33:49.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.052+0000 7ffaf3e00740 1 -- 172.21.15.38:0/1238912400 --> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593d8cbe860 con 0x5593d8d6e670 2024-09-16T00:33:49.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.052+0000 7ffaeaffd640 1 --2- 172.21.15.38:0/1238912400 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x5593d8d6e670 0x5593d8d70a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:49.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.052+0000 7ffacaffd640 10 client.25226 ms_handle_connect on v2:172.21.15.73:6836/2681081265 2024-09-16T00:33:49.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.053+0000 7ffaeaffd640 1 --2- 172.21.15.38:0/1238912400 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x5593d8d6e670 0x5593d8d70a60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=56bd0b644f7b181f server_cookie=5b0e1b2909fb38d9 in_seq=0 out_seq=0 2024-09-16T00:33:49.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.053+0000 7ffacaffd640 1 -- 172.21.15.38:0/1238912400 <== mds.1 v2:172.21.15.73:6836/2681081265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5593d8cbe860 con 0x5593d8d6e670 2024-09-16T00:33:49.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.053+0000 7ffacaffd640 10 client.25226 handle_command_reply: tid=0 2024-09-16T00:33:49.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.053+0000 7ffaf3e00740 1 client.25226 shutdown 2024-09-16T00:33:49.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.054+0000 7ffaf3e00740 1 -- 172.21.15.38:0/1238912400 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593d8d4f9e0 msgr2=0x5593d8d56190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:49.056 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.054+0000 7ffaf3e00740 1 --2- 172.21.15.38:0/1238912400 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593d8d4f9e0 0x5593d8d56190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ffae0006010 tx=0x7ffae0005250 comp rx=0 tx=0).stop 2024-09-16T00:33:49.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.054+0000 7ffaf3e00740 1 -- 172.21.15.38:0/1238912400 shutdown_connections 2024-09-16T00:33:49.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.054+0000 7ffaf3e00740 1 --2- 172.21.15.38:0/1238912400 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x5593d8d6e670 0x5593d8d70a60 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:49.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.054+0000 7ffacaffd640 0 client.25226 ms_handle_reset on v2:172.21.15.73:6836/2681081265 2024-09-16T00:33:49.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.054+0000 7ffaf3e00740 1 --2- 172.21.15.38:0/1238912400 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593d8d50f50 0x5593d8d566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:49.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.054+0000 7ffaf3e00740 1 --2- 172.21.15.38:0/1238912400 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593d8d4f9e0 0x5593d8d56190 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:49.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.054+0000 7ffaf3e00740 1 --2- 172.21.15.38:0/1238912400 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593d8d49660 0x5593d8d57e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:49.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.054+0000 7ffaf3e00740 1 -- 172.21.15.38:0/1238912400 >> 172.21.15.38:0/1238912400 conn(0x5593d8d14310 msgr2=0x5593d8d51d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:49.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.054+0000 7ffaf3e00740 1 -- 172.21.15.38:0/1238912400 shutdown_connections 2024-09-16T00:33:49.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.054+0000 7ffaf3e00740 1 -- 172.21.15.38:0/1238912400 wait complete. 2024-09-16T00:33:49.057 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.054+0000 7ffaf3e00740 20 client.25226 trim_cache size 0 max 16384 2024-09-16T00:33:49.077 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.077+0000 7ffaf3e00740 1 Processor -- start 2024-09-16T00:33:49.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.077+0000 7ffaf3e00740 1 -- start start 2024-09-16T00:33:49.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.077+0000 7ffaf3e00740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593d8d49660 0x5593d8d4c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:49.078 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.077+0000 7ffaf3e00740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593d8d4f9e0 0x5593d8d53730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:49.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.078+0000 7ffaf3e00740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593d8d50f50 0x5593d8d53c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:49.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.078+0000 7ffaf3e00740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec1af1f0 con 0x5593d8d49660 2024-09-16T00:33:49.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.078+0000 7ffaf3e00740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec1acac0 con 0x5593d8d50f50 2024-09-16T00:33:49.083 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.078+0000 7ffaf3e00740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec1aa910 con 0x5593d8d4f9e0 2024-09-16T00:33:49.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.078+0000 7ffaeaffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593d8d49660 0x5593d8d4c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:49.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.078+0000 7ffaeaffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593d8d49660 0x5593d8d4c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54080/0 (socket says 172.21.15.38:54080) 2024-09-16T00:33:49.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.078+0000 7ffaeaffd640 1 -- 172.21.15.38:0/3425123408 learned_addr learned my addr 172.21.15.38:0/3425123408 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:49.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.078+0000 7ffaea7fc640 1 --2- 172.21.15.38:0/3425123408 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593d8d4f9e0 0x5593d8d53730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:49.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.078+0000 7ffaeb7fe640 1 --2- 172.21.15.38:0/3425123408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593d8d50f50 0x5593d8d53c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:49.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.078+0000 7ffaeaffd640 1 -- 172.21.15.38:0/3425123408 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593d8d4f9e0 msgr2=0x5593d8d53730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:49.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.078+0000 7ffaeaffd640 1 --2- 172.21.15.38:0/3425123408 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593d8d4f9e0 0x5593d8d53730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:49.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.078+0000 7ffaeaffd640 1 -- 172.21.15.38:0/3425123408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593d8d50f50 msgr2=0x5593d8d53c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:49.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.078+0000 7ffaeaffd640 1 --2- 172.21.15.38:0/3425123408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593d8d50f50 0x5593d8d53c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:49.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.078+0000 7ffaeaffd640 1 -- 172.21.15.38:0/3425123408 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5593d8cc4e70 con 0x5593d8d49660 2024-09-16T00:33:49.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.078+0000 7ffaeaffd640 1 --2- 172.21.15.38:0/3425123408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593d8d49660 0x5593d8d4c220 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x5593d8d4a4f0 tx=0x7ffadc013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d18aa2d18367ac7c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:49.084 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.079+0000 7ffad17fa640 1 -- 172.21.15.38:0/3425123408 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffadc004020 con 0x5593d8d49660 2024-09-16T00:33:49.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.079+0000 7ffad17fa640 1 -- 172.21.15.38:0/3425123408 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffadc0041c0 con 0x5593d8d49660 2024-09-16T00:33:49.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.079+0000 7ffad17fa640 1 -- 172.21.15.38:0/3425123408 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffadc01d670 con 0x5593d8d49660 2024-09-16T00:33:49.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.079+0000 7ffaf3e00740 1 -- 172.21.15.38:0/3425123408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593d8d49660 msgr2=0x5593d8d4c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:49.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.079+0000 7ffaf3e00740 1 --2- 172.21.15.38:0/3425123408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593d8d49660 0x5593d8d4c220 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x5593d8d4a4f0 tx=0x7ffadc013000 comp rx=0 tx=0).stop 2024-09-16T00:33:49.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.080+0000 7ffaf3e00740 1 -- 172.21.15.38:0/3425123408 shutdown_connections 2024-09-16T00:33:49.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.080+0000 7ffaf3e00740 1 --2- 172.21.15.38:0/3425123408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593d8d50f50 0x5593d8d53c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:49.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.080+0000 7ffaf3e00740 1 --2- 172.21.15.38:0/3425123408 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593d8d4f9e0 0x5593d8d53730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:49.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.080+0000 7ffaf3e00740 1 --2- 172.21.15.38:0/3425123408 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593d8d49660 0x5593d8d4c220 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:49.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.080+0000 7ffaf3e00740 1 -- 172.21.15.38:0/3425123408 >> 172.21.15.38:0/3425123408 conn(0x5593d8d14310 msgr2=0x5593d8d4bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:49.085 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.080+0000 7ffaf3e00740 1 -- 172.21.15.38:0/3425123408 shutdown_connections 2024-09-16T00:33:49.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.080+0000 7ffaf3e00740 1 -- 172.21.15.38:0/3425123408 wait complete. 2024-09-16T00:33:49.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.080+0000 7ffaf3e00740 1 -- start start 2024-09-16T00:33:49.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.080+0000 7ffaf3e00740 1 Processor -- start 2024-09-16T00:33:49.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.081+0000 7ffaf3e00740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593d8d49660 0x5593d8d73220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:49.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.081+0000 7ffaf3e00740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593d8d4f9e0 0x5593d8d73760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:49.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.081+0000 7ffaf3e00740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593d8d50f50 0x5593d8dd0ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:49.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.081+0000 7ffaf3e00740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec1aaf80 con 0x5593d8d50f50 2024-09-16T00:33:49.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.081+0000 7ffaf3e00740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec1ac3f0 con 0x5593d8d49660 2024-09-16T00:33:49.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.081+0000 7ffaf3e00740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ffaec1ab650 con 0x5593d8d4f9e0 2024-09-16T00:33:49.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.081+0000 7ffaf3e00740 10 client.0 _ll_register_callbacks cb 0x7ffae0004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:49.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.081+0000 7ffaeb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593d8d50f50 0x5593d8dd0ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:49.086 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.081+0000 7ffaeb7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593d8d50f50 0x5593d8dd0ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54090/0 (socket says 172.21.15.38:54090) 2024-09-16T00:33:49.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.081+0000 7ffaeb7fe640 1 -- 172.21.15.38:0/2083817190 learned_addr learned my addr 172.21.15.38:0/2083817190 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:49.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.081+0000 7ffaeaffd640 1 --2- 172.21.15.38:0/2083817190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593d8d49660 0x5593d8d73220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:49.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.081+0000 7ffaea7fc640 1 --2- 172.21.15.38:0/2083817190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593d8d4f9e0 0x5593d8d73760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:49.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.081+0000 7ffaeb7fe640 1 -- 172.21.15.38:0/2083817190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593d8d4f9e0 msgr2=0x5593d8d73760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:49.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.081+0000 7ffaeb7fe640 1 --2- 172.21.15.38:0/2083817190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593d8d4f9e0 0x5593d8d73760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:49.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.081+0000 7ffaeb7fe640 1 -- 172.21.15.38:0/2083817190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593d8d49660 msgr2=0x5593d8d73220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:49.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.081+0000 7ffaeb7fe640 1 --2- 172.21.15.38:0/2083817190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593d8d49660 0x5593d8d73220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:49.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.081+0000 7ffaeb7fe640 1 -- 172.21.15.38:0/2083817190 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593d8dd1690 con 0x5593d8d50f50 2024-09-16T00:33:49.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.081+0000 7ffaeb7fe640 1 --2- 172.21.15.38:0/2083817190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593d8d50f50 0x5593d8dd0ff0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ffae4098670 tx=0x7ffae409be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89340caccb8a47a4 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:49.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.081+0000 7ffacaffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:33:49.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.082+0000 7ffacaffd640 1 -- 172.21.15.38:0/2083817190 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffae40a05c0 con 0x5593d8d50f50 2024-09-16T00:33:49.087 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.082+0000 7ffacaffd640 1 -- 172.21.15.38:0/2083817190 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7ffae409a510 con 0x5593d8d50f50 2024-09-16T00:33:49.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.082+0000 7ffaf3e00740 1 -- 172.21.15.38:0/2083817190 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffadc0041c0 con 0x5593d8d50f50 2024-09-16T00:33:49.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.082+0000 7ffacaffd640 1 -- 172.21.15.38:0/2083817190 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7ffae40ad050 con 0x5593d8d50f50 2024-09-16T00:33:49.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.082+0000 7ffaf3e00740 10 client.15615 fetch_fsmap learned FSMap version 26 2024-09-16T00:33:49.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.082+0000 7ffaf3e00740 1 -- 172.21.15.38:0/2083817190 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5593d8dd8b10 con 0x5593d8d50f50 2024-09-16T00:33:49.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.082+0000 7ffacaffd640 1 -- 172.21.15.38:0/2083817190 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ffadc0041c0 con 0x5593d8d50f50 2024-09-16T00:33:49.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.082+0000 7ffacaffd640 1 -- 172.21.15.38:0/2083817190 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7ffae4020030 con 0x5593d8d50f50 2024-09-16T00:33:49.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.083+0000 7ffaf3e00740 10 client.15615 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:33:49.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.083+0000 7ffaf3e00740 10 client.15615 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:33:49.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.083+0000 7ffaf3e00740 20 client.15615 populate_metadata read hostname 'smithi038' 2024-09-16T00:33:49.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.083+0000 7ffaf3e00740 1 --2- 172.21.15.38:0/2083817190 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x5593d8dd9230 0x5593d8ddb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:49.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.083+0000 7ffaf3e00740 4 client.15615 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:33:49.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.083+0000 7ffaf3e00740 1 -- 172.21.15.38:0/2083817190 --> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5593d8cb8d70 con 0x5593d8dd9230 2024-09-16T00:33:49.089 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.083+0000 7ffaeaffd640 1 --2- 172.21.15.38:0/2083817190 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x5593d8dd9230 0x5593d8ddb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:49.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.083+0000 7ffaeaffd640 1 --2- 172.21.15.38:0/2083817190 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x5593d8dd9230 0x5593d8ddb620 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=77e26cd705273e5d server_cookie=b9c028ee8680ac2a in_seq=0 out_seq=0 2024-09-16T00:33:49.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.083+0000 7ffacaffd640 10 client.15615 ms_handle_connect on v2:172.21.15.73:6836/2681081265 2024-09-16T00:33:49.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.084+0000 7ffacaffd640 1 -- 172.21.15.38:0/2083817190 <== mds.1 v2:172.21.15.73:6836/2681081265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5593d8cb8d70 con 0x5593d8dd9230 2024-09-16T00:33:49.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.084+0000 7ffacaffd640 10 client.15615 handle_command_reply: tid=0 2024-09-16T00:33:49.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.084+0000 7ffaf3e00740 1 client.15615 shutdown 2024-09-16T00:33:49.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.084+0000 7ffaf3e00740 1 -- 172.21.15.38:0/2083817190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593d8d50f50 msgr2=0x5593d8dd0ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:49.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.084+0000 7ffaf3e00740 1 --2- 172.21.15.38:0/2083817190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593d8d50f50 0x5593d8dd0ff0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ffae4098670 tx=0x7ffae409be70 comp rx=0 tx=0).stop 2024-09-16T00:33:49.090 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.084+0000 7ffaf3e00740 1 -- 172.21.15.38:0/2083817190 shutdown_connections 2024-09-16T00:33:49.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.084+0000 7ffaf3e00740 1 --2- 172.21.15.38:0/2083817190 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x5593d8dd9230 0x5593d8ddb620 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:49.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.084+0000 7ffaf3e00740 1 --2- 172.21.15.38:0/2083817190 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5593d8d50f50 0x5593d8dd0ff0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:49.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.084+0000 7ffaf3e00740 1 --2- 172.21.15.38:0/2083817190 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5593d8d4f9e0 0x5593d8d73760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:49.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.084+0000 7ffaf3e00740 1 --2- 172.21.15.38:0/2083817190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5593d8d49660 0x5593d8d73220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:49.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.084+0000 7ffaf3e00740 1 -- 172.21.15.38:0/2083817190 >> 172.21.15.38:0/2083817190 conn(0x5593d8d14310 msgr2=0x5593d8d534b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:49.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.084+0000 7ffacaffd640 0 client.15615 ms_handle_reset on v2:172.21.15.73:6836/2681081265 2024-09-16T00:33:49.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.084+0000 7ffaf3e00740 1 -- 172.21.15.38:0/2083817190 shutdown_connections 2024-09-16T00:33:49.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.084+0000 7ffaf3e00740 1 -- 172.21.15.38:0/2083817190 wait complete. 2024-09-16T00:33:49.091 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.084+0000 7ffaf3e00740 20 client.15615 trim_cache size 0 max 16384 2024-09-16T00:33:49.092 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-09-16T00:33:49.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.085+0000 7ffaf1a27640 1 -- 172.21.15.38:0/3928706031 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ffad40a3420 msgr2=0x7ffad40a58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:49.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.085+0000 7ffaf1a27640 1 --2- 172.21.15.38:0/3928706031 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ffad40a3420 0x7ffad40a58e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7ffadc005d00 tx=0x7ffadc00b410 comp rx=0 tx=0).stop 2024-09-16T00:33:49.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.085+0000 7ffaf1a27640 1 -- 172.21.15.38:0/3928706031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffaec1082a0 msgr2=0x7ffaec19ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:49.092 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.085+0000 7ffaf1a27640 1 --2- 172.21.15.38:0/3928706031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffaec1082a0 0x7ffaec19ce60 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7ffae400cce0 tx=0x7ffae400a620 comp rx=0 tx=0).stop 2024-09-16T00:33:49.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.085+0000 7ffaf1a27640 1 -- 172.21.15.38:0/3928706031 shutdown_connections 2024-09-16T00:33:49.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.085+0000 7ffaf1a27640 1 --2- 172.21.15.38:0/3928706031 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ffad40a3420 0x7ffad40a58e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:49.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.085+0000 7ffaf1a27640 1 --2- 172.21.15.38:0/3928706031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ffaec1082a0 0x7ffaec19ce60 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:49.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.085+0000 7ffaf1a27640 1 --2- 172.21.15.38:0/3928706031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffaec1036e0 0x7ffaec198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:49.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.085+0000 7ffaf1a27640 1 --2- 172.21.15.38:0/3928706031 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ffaec102d30 0x7ffaec198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:49.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.085+0000 7ffaf1a27640 1 -- 172.21.15.38:0/3928706031 >> 172.21.15.38:0/3928706031 conn(0x7ffaec0fc7d0 msgr2=0x7ffaec106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:49.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.085+0000 7ffaf1a27640 1 -- 172.21.15.38:0/3928706031 shutdown_connections 2024-09-16T00:33:49.093 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:49.085+0000 7ffaf1a27640 1 -- 172.21.15.38:0/3928706031 wait complete. 2024-09-16T00:33:49.214 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:48 smithi038 ceph-mon[30945]: pgmap v395: 129 pgs: 129 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-16T00:33:49.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:48 smithi122 ceph-mon[35412]: pgmap v395: 129 pgs: 129 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-16T00:33:49.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:48 smithi073 ceph-mon[35254]: pgmap v395: 129 pgs: 129 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-16T00:33:49.796 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:33:50.068 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:50.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.689+0000 7f5093a89640 1 -- 172.21.15.38:0/102261337 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f508c106f40 msgr2=0x7f508c10eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:50.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.689+0000 7f5093a89640 1 --2- 172.21.15.38:0/102261337 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f508c106f40 0x7f508c10eac0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f5084009ca0 tx=0x7f508402a8c0 comp rx=0 tx=0).stop 2024-09-16T00:33:50.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.689+0000 7f5093a89640 1 -- 172.21.15.38:0/102261337 shutdown_connections 2024-09-16T00:33:50.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.689+0000 7f5093a89640 1 --2- 172.21.15.38:0/102261337 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f508c106f40 0x7f508c10eac0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:50.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.689+0000 7f5093a89640 1 --2- 172.21.15.38:0/102261337 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f508c1027c0 0x7f508c106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:50.691 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.689+0000 7f5093a89640 1 --2- 172.21.15.38:0/102261337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f508c101e10 0x7f508c1021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:50.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.689+0000 7f5093a89640 1 -- 172.21.15.38:0/102261337 >> 172.21.15.38:0/102261337 conn(0x7f508c0fd8d0 msgr2=0x7f508c0ffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:50.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.689+0000 7f5093a89640 1 -- 172.21.15.38:0/102261337 shutdown_connections 2024-09-16T00:33:50.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.689+0000 7f5093a89640 1 -- 172.21.15.38:0/102261337 wait complete. 2024-09-16T00:33:50.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.690+0000 7f5093a89640 1 Processor -- start 2024-09-16T00:33:50.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.690+0000 7f5093a89640 1 -- start start 2024-09-16T00:33:50.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.691+0000 7f5093a89640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f508c101e10 0x7f508c1a4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:50.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.691+0000 7f5093a89640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f508c1027c0 0x7f508c1a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:50.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.691+0000 7f5091825640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f508c101e10 0x7f508c1a4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:50.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.691+0000 7f5093a89640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f508c106f40 0x7f508c10db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:50.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.691+0000 7f5093a89640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f508c111ff0 con 0x7f508c101e10 2024-09-16T00:33:50.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.691+0000 7f5093a89640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f508c111e70 con 0x7f508c1027c0 2024-09-16T00:33:50.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.691+0000 7f5093a89640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f508c112170 con 0x7f508c106f40 2024-09-16T00:33:50.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.691+0000 7f5091825640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f508c101e10 0x7f508c1a4780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54108/0 (socket says 172.21.15.38:54108) 2024-09-16T00:33:50.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.691+0000 7f5091825640 1 -- 172.21.15.38:0/1436904257 learned_addr learned my addr 172.21.15.38:0/1436904257 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:50.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.691+0000 7f5091024640 1 --2- 172.21.15.38:0/1436904257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f508c1027c0 0x7f508c1a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:50.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.691+0000 7f5092026640 1 --2- 172.21.15.38:0/1436904257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f508c106f40 0x7f508c10db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:50.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.692+0000 7f5091024640 1 -- 172.21.15.38:0/1436904257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f508c106f40 msgr2=0x7f508c10db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:50.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.692+0000 7f5091024640 1 --2- 172.21.15.38:0/1436904257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f508c106f40 0x7f508c10db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:50.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.692+0000 7f5091024640 1 -- 172.21.15.38:0/1436904257 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f508c101e10 msgr2=0x7f508c1a4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:50.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.692+0000 7f5091024640 1 --2- 172.21.15.38:0/1436904257 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f508c101e10 0x7f508c1a4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:50.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.692+0000 7f5091024640 1 -- 172.21.15.38:0/1436904257 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f508c10e320 con 0x7f508c1027c0 2024-09-16T00:33:50.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.692+0000 7f5091024640 1 --2- 172.21.15.38:0/1436904257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f508c1027c0 0x7f508c1a4cc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f507c005fd0 tx=0x7f507c00dde0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:50.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.692+0000 7f5091825640 1 --2- 172.21.15.38:0/1436904257 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f508c101e10 0x7f508c1a4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:33:50.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.692+0000 7f5082ffd640 1 -- 172.21.15.38:0/1436904257 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c009c30 con 0x7f508c1027c0 2024-09-16T00:33:50.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.692+0000 7f5082ffd640 1 -- 172.21.15.38:0/1436904257 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f507c009dd0 con 0x7f508c1027c0 2024-09-16T00:33:50.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.692+0000 7f5093a89640 1 -- 172.21.15.38:0/1436904257 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f508c10e610 con 0x7f508c1027c0 2024-09-16T00:33:50.694 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.693+0000 7f5082ffd640 1 -- 172.21.15.38:0/1436904257 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c010460 con 0x7f508c1027c0 2024-09-16T00:33:50.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.693+0000 7f5093a89640 1 -- 172.21.15.38:0/1436904257 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f508c1ab5a0 con 0x7f508c1027c0 2024-09-16T00:33:50.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.696+0000 7f5095e62740 1 Processor -- start 2024-09-16T00:33:50.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.697+0000 7f5095e62740 1 -- start start 2024-09-16T00:33:50.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.697+0000 7f5095e62740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d46df6bb40 0x55d46df6a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:50.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.697+0000 7f5095e62740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d46df6a5d0 0x55d46df6aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:50.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.697+0000 7f5095e62740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d46df75370 0x55d46df77760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:50.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.697+0000 7f5082ffd640 1 -- 172.21.15.38:0/1436904257 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 19) ==== 138406+0+0 (secure 0 0 0) 0x7f507c003f60 con 0x7f508c1027c0 2024-09-16T00:33:50.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.697+0000 7f5095e62740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d46ded02f0 con 0x55d46df6bb40 2024-09-16T00:33:50.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.697+0000 7f5095e62740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d46de77cf0 con 0x55d46df75370 2024-09-16T00:33:50.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.697+0000 7f5095e62740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d46decc2b0 con 0x55d46df6a5d0 2024-09-16T00:33:50.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.697+0000 7f5091825640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d46df6a5d0 0x55d46df6aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:50.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.697+0000 7f5092026640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d46df6bb40 0x55d46df6a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:50.698 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.697+0000 7f5091024640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d46df75370 0x55d46df77760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:50.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.697+0000 7f5091024640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d46df75370 0x55d46df77760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.38:60490/0 (socket says 172.21.15.38:60490) 2024-09-16T00:33:50.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.697+0000 7f5091024640 1 -- 172.21.15.38:0/709875280 learned_addr learned my addr 172.21.15.38:0/709875280 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:50.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.698+0000 7f5091024640 1 -- 172.21.15.38:0/709875280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d46df6a5d0 msgr2=0x55d46df6aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:50.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.698+0000 7f5091024640 1 --2- 172.21.15.38:0/709875280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d46df6a5d0 0x55d46df6aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:50.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.698+0000 7f5091024640 1 -- 172.21.15.38:0/709875280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d46df6bb40 msgr2=0x55d46df6a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:50.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.698+0000 7f5091024640 1 --2- 172.21.15.38:0/709875280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d46df6bb40 0x55d46df6a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:50.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.698+0000 7f5091024640 1 -- 172.21.15.38:0/709875280 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d46de5c3a0 con 0x55d46df75370 2024-09-16T00:33:50.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.698+0000 7f5092026640 1 --2- 172.21.15.38:0/709875280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d46df6bb40 0x55d46df6a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:33:50.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.699+0000 7f5082ffd640 1 --2- 172.21.15.38:0/1436904257 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f50780a33d0 0x7f50780a5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:50.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.699+0000 7f5082ffd640 1 -- 172.21.15.38:0/1436904257 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f507c0cf7b0 con 0x7f508c1027c0 2024-09-16T00:33:50.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.699+0000 7f5091024640 1 --2- 172.21.15.38:0/709875280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d46df75370 0x55d46df77760 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f507c0d5bf0 tx=0x7f507c0d7670 comp rx=0 tx=0).ready entity=mon.2 client_cookie=165e4fb53a0d4390 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:50.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.699+0000 7f5092026640 1 --2- 172.21.15.38:0/1436904257 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f50780a33d0 0x7f50780a5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:50.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.699+0000 7f5077fff640 1 -- 172.21.15.38:0/709875280 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c097b30 con 0x55d46df75370 2024-09-16T00:33:50.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.699+0000 7f5077fff640 1 -- 172.21.15.38:0/709875280 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f507c09bf90 con 0x55d46df75370 2024-09-16T00:33:50.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.699+0000 7f5077fff640 1 -- 172.21.15.38:0/709875280 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c09b1f0 con 0x55d46df75370 2024-09-16T00:33:50.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.700+0000 7f5095e62740 1 -- 172.21.15.38:0/709875280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d46df75370 msgr2=0x55d46df77760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:50.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.700+0000 7f5095e62740 1 --2- 172.21.15.38:0/709875280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d46df75370 0x55d46df77760 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f507c0d5bf0 tx=0x7f507c0d7670 comp rx=0 tx=0).stop 2024-09-16T00:33:50.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.700+0000 7f5092026640 1 --2- 172.21.15.38:0/1436904257 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f50780a33d0 0x7f50780a5890 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5084006010 tx=0x7f5084005d80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:50.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.700+0000 7f5095e62740 1 -- 172.21.15.38:0/709875280 shutdown_connections 2024-09-16T00:33:50.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.700+0000 7f5095e62740 1 --2- 172.21.15.38:0/709875280 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d46df75370 0x55d46df77760 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:50.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.700+0000 7f5095e62740 1 --2- 172.21.15.38:0/709875280 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d46df6a5d0 0x55d46df6aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:50.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.700+0000 7f5095e62740 1 --2- 172.21.15.38:0/709875280 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d46df6bb40 0x55d46df6a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:50.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.700+0000 7f5095e62740 1 -- 172.21.15.38:0/709875280 >> 172.21.15.38:0/709875280 conn(0x55d46df33310 msgr2=0x55d46df66c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:50.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.701+0000 7f5095e62740 1 -- 172.21.15.38:0/709875280 shutdown_connections 2024-09-16T00:33:50.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.701+0000 7f5095e62740 1 -- 172.21.15.38:0/709875280 wait complete. 2024-09-16T00:33:50.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.701+0000 7f5095e62740 1 -- start start 2024-09-16T00:33:50.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.701+0000 7f5095e62740 1 Processor -- start 2024-09-16T00:33:50.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.702+0000 7f5095e62740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d46df6a5d0 0x55d46df74d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:50.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.702+0000 7f5095e62740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d46df6bb40 0x55d46df71fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:50.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.702+0000 7f5095e62740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d46df75370 0x55d46df725d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:50.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.702+0000 7f5095e62740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55d46df78070 con 0x55d46df6bb40 2024-09-16T00:33:50.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.702+0000 7f5095e62740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55d46de8b060 con 0x55d46df6a5d0 2024-09-16T00:33:50.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.702+0000 7f5095e62740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55d46df781f0 con 0x55d46df75370 2024-09-16T00:33:50.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.702+0000 7f5095e62740 10 client.0 _ll_register_callbacks cb 0x55d46dbde8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:50.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.702+0000 7f5092026640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d46df75370 0x55d46df725d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:50.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.702+0000 7f5092026640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d46df75370 0x55d46df725d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.38:60210/0 (socket says 172.21.15.38:60210) 2024-09-16T00:33:50.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.702+0000 7f5092026640 1 -- 172.21.15.38:0/4095431931 learned_addr learned my addr 172.21.15.38:0/4095431931 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:50.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.703+0000 7f5091825640 1 --2- 172.21.15.38:0/4095431931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d46df6a5d0 0x55d46df74d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:50.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.703+0000 7f5091024640 1 --2- 172.21.15.38:0/4095431931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d46df6bb40 0x55d46df71fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:50.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.703+0000 7f5092026640 1 -- 172.21.15.38:0/4095431931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d46df6a5d0 msgr2=0x55d46df74d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:50.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.703+0000 7f5092026640 1 --2- 172.21.15.38:0/4095431931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d46df6a5d0 0x55d46df74d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:50.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.703+0000 7f5092026640 1 -- 172.21.15.38:0/4095431931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d46df6bb40 msgr2=0x55d46df71fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:50.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.703+0000 7f5092026640 1 --2- 172.21.15.38:0/4095431931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d46df6bb40 0x55d46df71fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:50.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.703+0000 7f5092026640 1 -- 172.21.15.38:0/4095431931 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d46dc413c0 con 0x55d46df75370 2024-09-16T00:33:50.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.703+0000 7f5091825640 1 --2- 172.21.15.38:0/4095431931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d46df6a5d0 0x55d46df74d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:33:50.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.703+0000 7f5092026640 1 --2- 172.21.15.38:0/4095431931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d46df75370 0x55d46df725d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5084035000 tx=0x7f508402a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54ffe66f84e50884 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:50.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.703+0000 7f50757fa640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-16T00:33:50.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.704+0000 7f50757fa640 1 -- 172.21.15.38:0/4095431931 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5084031d80 con 0x55d46df75370 2024-09-16T00:33:50.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.704+0000 7f5095e62740 1 -- 172.21.15.38:0/4095431931 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f507c09bf90 con 0x55d46df75370 2024-09-16T00:33:50.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.704+0000 7f50757fa640 1 -- 172.21.15.38:0/4095431931 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f5084003fe0 con 0x55d46df75370 2024-09-16T00:33:50.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.704+0000 7f50757fa640 1 -- 172.21.15.38:0/4095431931 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f50840041e0 con 0x55d46df75370 2024-09-16T00:33:50.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.704+0000 7f5095e62740 10 client.25180 fetch_fsmap learned FSMap version 26 2024-09-16T00:33:50.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.704+0000 7f5095e62740 1 -- 172.21.15.38:0/4095431931 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55d46df83b00 con 0x55d46df75370 2024-09-16T00:33:50.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.704+0000 7f50757fa640 1 -- 172.21.15.38:0/4095431931 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f507c09bf90 con 0x55d46df75370 2024-09-16T00:33:50.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.705+0000 7f50757fa640 1 -- 172.21.15.38:0/4095431931 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5084051070 con 0x55d46df75370 2024-09-16T00:33:50.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.705+0000 7f5095e62740 10 client.25180 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:33:50.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.705+0000 7f5095e62740 10 client.25180 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:33:50.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.705+0000 7f5095e62740 20 client.25180 populate_metadata read hostname 'smithi038' 2024-09-16T00:33:50.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.705+0000 7f5095e62740 1 --2- 172.21.15.38:0/4095431931 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55d46df85270 0x55d46df87680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:50.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.705+0000 7f5095e62740 4 client.25180 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:33:50.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.705+0000 7f5095e62740 1 -- 172.21.15.38:0/4095431931 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d46dedd860 con 0x55d46df85270 2024-09-16T00:33:50.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.706+0000 7f5091825640 1 --2- 172.21.15.38:0/4095431931 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55d46df85270 0x55d46df87680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:50.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.706+0000 7f5091825640 1 --2- 172.21.15.38:0/4095431931 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55d46df85270 0x55d46df87680 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4e8f55b20e7d7c6 server_cookie=1bfb0408899fe24f in_seq=0 out_seq=0 2024-09-16T00:33:50.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.706+0000 7f50757fa640 10 client.25180 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:33:50.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.707+0000 7f50757fa640 1 -- 172.21.15.38:0/4095431931 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55d46dedd860 con 0x55d46df85270 2024-09-16T00:33:50.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.707+0000 7f50757fa640 10 client.25180 handle_command_reply: tid=0 2024-09-16T00:33:50.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.707+0000 7f5095e62740 1 client.25180 shutdown 2024-09-16T00:33:50.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.707+0000 7f5095e62740 1 -- 172.21.15.38:0/4095431931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d46df75370 msgr2=0x55d46df725d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:50.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.707+0000 7f5095e62740 1 --2- 172.21.15.38:0/4095431931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d46df75370 0x55d46df725d0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5084035000 tx=0x7f508402a8c0 comp rx=0 tx=0).stop 2024-09-16T00:33:50.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.707+0000 7f5095e62740 1 -- 172.21.15.38:0/4095431931 shutdown_connections 2024-09-16T00:33:50.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.707+0000 7f5095e62740 1 --2- 172.21.15.38:0/4095431931 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d46df75370 0x55d46df725d0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:50.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.707+0000 7f5095e62740 1 --2- 172.21.15.38:0/4095431931 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55d46df85270 0x55d46df87680 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:50.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.707+0000 7f5095e62740 1 --2- 172.21.15.38:0/4095431931 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d46df6bb40 0x55d46df71fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:50.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.707+0000 7f5095e62740 1 --2- 172.21.15.38:0/4095431931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d46df6a5d0 0x55d46df74d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:50.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.707+0000 7f5095e62740 1 -- 172.21.15.38:0/4095431931 >> 172.21.15.38:0/4095431931 conn(0x55d46df33310 msgr2=0x55d46df6c930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:50.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.707+0000 7f50757fa640 0 client.25180 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:33:50.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.707+0000 7f5095e62740 1 -- 172.21.15.38:0/4095431931 shutdown_connections 2024-09-16T00:33:50.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.707+0000 7f5095e62740 1 -- 172.21.15.38:0/4095431931 wait complete. 2024-09-16T00:33:50.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.707+0000 7f5095e62740 20 client.25180 trim_cache size 0 max 16384 2024-09-16T00:33:50.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.733+0000 7f5095e62740 1 Processor -- start 2024-09-16T00:33:50.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.733+0000 7f5095e62740 1 -- start start 2024-09-16T00:33:50.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.733+0000 7f5095e62740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d46df6a5d0 0x55d46df70f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:50.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.733+0000 7f5095e62740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d46df6bb40 0x55d46df81340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:50.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.733+0000 7f5095e62740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d46df75370 0x55d46df73f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:50.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.733+0000 7f5095e62740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f508c1b7a40 con 0x55d46df6bb40 2024-09-16T00:33:50.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.734+0000 7f5095e62740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f508c1b5310 con 0x55d46df6a5d0 2024-09-16T00:33:50.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.734+0000 7f5095e62740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f508c1b3210 con 0x55d46df75370 2024-09-16T00:33:50.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.734+0000 7f5091024640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d46df6bb40 0x55d46df81340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:50.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.734+0000 7f5091024640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d46df6bb40 0x55d46df81340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54122/0 (socket says 172.21.15.38:54122) 2024-09-16T00:33:50.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.734+0000 7f5091024640 1 -- 172.21.15.38:0/2770584464 learned_addr learned my addr 172.21.15.38:0/2770584464 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:50.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.734+0000 7f5091825640 1 --2- 172.21.15.38:0/2770584464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d46df6a5d0 0x55d46df70f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:50.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.734+0000 7f5092026640 1 --2- 172.21.15.38:0/2770584464 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d46df75370 0x55d46df73f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:50.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.734+0000 7f5091024640 1 -- 172.21.15.38:0/2770584464 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d46df75370 msgr2=0x55d46df73f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:50.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.734+0000 7f5091024640 1 --2- 172.21.15.38:0/2770584464 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d46df75370 0x55d46df73f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:50.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.734+0000 7f5091024640 1 -- 172.21.15.38:0/2770584464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d46df6a5d0 msgr2=0x55d46df70f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:50.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.734+0000 7f5091024640 1 --2- 172.21.15.38:0/2770584464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d46df6a5d0 0x55d46df70f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:50.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.734+0000 7f5091024640 1 -- 172.21.15.38:0/2770584464 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d46df70530 con 0x55d46df6bb40 2024-09-16T00:33:50.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.734+0000 7f5091825640 1 --2- 172.21.15.38:0/2770584464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d46df6a5d0 0x55d46df70f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:33:50.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.734+0000 7f5091024640 1 --2- 172.21.15.38:0/2770584464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d46df6bb40 0x55d46df81340 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f507c009610 tx=0x7f507c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4b9164033e6dad6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:50.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.735+0000 7f50777fe640 1 -- 172.21.15.38:0/2770584464 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c09b3d0 con 0x55d46df6bb40 2024-09-16T00:33:50.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.735+0000 7f50777fe640 1 -- 172.21.15.38:0/2770584464 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f507c0d5380 con 0x55d46df6bb40 2024-09-16T00:33:50.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.735+0000 7f50777fe640 1 -- 172.21.15.38:0/2770584464 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f507c09b660 con 0x55d46df6bb40 2024-09-16T00:33:50.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.736+0000 7f5095e62740 1 -- 172.21.15.38:0/2770584464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d46df6bb40 msgr2=0x55d46df81340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:50.739 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.736+0000 7f5095e62740 1 --2- 172.21.15.38:0/2770584464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d46df6bb40 0x55d46df81340 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f507c009610 tx=0x7f507c000bc0 comp rx=0 tx=0).stop 2024-09-16T00:33:50.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.736+0000 7f5095e62740 1 -- 172.21.15.38:0/2770584464 shutdown_connections 2024-09-16T00:33:50.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.736+0000 7f5095e62740 1 --2- 172.21.15.38:0/2770584464 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d46df75370 0x55d46df73f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:50.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.736+0000 7f5095e62740 1 --2- 172.21.15.38:0/2770584464 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d46df6bb40 0x55d46df81340 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:50.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.736+0000 7f5095e62740 1 --2- 172.21.15.38:0/2770584464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d46df6a5d0 0x55d46df70f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:50.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.736+0000 7f5095e62740 1 -- 172.21.15.38:0/2770584464 >> 172.21.15.38:0/2770584464 conn(0x55d46df33310 msgr2=0x55d46df73840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:50.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.736+0000 7f5095e62740 1 -- 172.21.15.38:0/2770584464 shutdown_connections 2024-09-16T00:33:50.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.736+0000 7f5095e62740 1 -- 172.21.15.38:0/2770584464 wait complete. 2024-09-16T00:33:50.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.736+0000 7f5095e62740 1 -- start start 2024-09-16T00:33:50.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.736+0000 7f5095e62740 1 Processor -- start 2024-09-16T00:33:50.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.737+0000 7f5095e62740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d46df6a5d0 0x55d46df78470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:50.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.737+0000 7f5091825640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d46df6a5d0 0x55d46df78470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:50.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.737+0000 7f5091825640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d46df6a5d0 0x55d46df78470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54124/0 (socket says 172.21.15.38:54124) 2024-09-16T00:33:50.740 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.737+0000 7f5091825640 1 -- 172.21.15.38:0/2695507749 learned_addr learned my addr 172.21.15.38:0/2695507749 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:50.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.737+0000 7f5095e62740 1 --2- 172.21.15.38:0/2695507749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d46df6bb40 0x55d46df789b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:50.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.737+0000 7f5095e62740 1 --2- 172.21.15.38:0/2695507749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d46df75370 0x55d46df78ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:50.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.737+0000 7f5095e62740 1 -- 172.21.15.38:0/2695507749 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f508c1b37d0 con 0x55d46df6a5d0 2024-09-16T00:33:50.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.737+0000 7f5095e62740 1 -- 172.21.15.38:0/2695507749 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f508c1b4c40 con 0x55d46df6bb40 2024-09-16T00:33:50.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.737+0000 7f5095e62740 1 -- 172.21.15.38:0/2695507749 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f508c1b3ea0 con 0x55d46df75370 2024-09-16T00:33:50.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.738+0000 7f5092026640 1 --2- 172.21.15.38:0/2695507749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d46df75370 0x55d46df78ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:50.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.738+0000 7f5091024640 1 --2- 172.21.15.38:0/2695507749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d46df6bb40 0x55d46df789b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:50.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.738+0000 7f5091825640 1 -- 172.21.15.38:0/2695507749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d46df75370 msgr2=0x55d46df78ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:50.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.738+0000 7f5091825640 1 --2- 172.21.15.38:0/2695507749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d46df75370 0x55d46df78ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:50.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.738+0000 7f5091825640 1 -- 172.21.15.38:0/2695507749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d46df6bb40 msgr2=0x55d46df789b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:50.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.738+0000 7f5091825640 1 --2- 172.21.15.38:0/2695507749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d46df6bb40 0x55d46df789b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:50.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.738+0000 7f5091825640 1 -- 172.21.15.38:0/2695507749 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d46dfee1a0 con 0x55d46df6a5d0 2024-09-16T00:33:50.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.738+0000 7f5095e62740 10 client.0 _ll_register_callbacks cb 0x7f508402f430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:50.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.738+0000 7f5095e62740 1 -- 172.21.15.38:0/2695507749 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f507c0d5380 con 0x55d46df6a5d0 2024-09-16T00:33:50.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.738+0000 7f5091825640 1 --2- 172.21.15.38:0/2695507749 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d46df6a5d0 0x55d46df78470 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x55d46df67ac0 tx=0x7f5088003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7de777f0574a8669 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:50.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.738+0000 7f50757fa640 10 client.15633 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:33:50.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.739+0000 7f50757fa640 1 -- 172.21.15.38:0/2695507749 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5088012b20 con 0x55d46df6a5d0 2024-09-16T00:33:50.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.739+0000 7f50757fa640 1 -- 172.21.15.38:0/2695507749 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f5088012cc0 con 0x55d46df6a5d0 2024-09-16T00:33:50.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.739+0000 7f50757fa640 1 -- 172.21.15.38:0/2695507749 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5088005020 con 0x55d46df6a5d0 2024-09-16T00:33:50.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.739+0000 7f5095e62740 10 client.15633 fetch_fsmap learned FSMap version 26 2024-09-16T00:33:50.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.739+0000 7f5095e62740 1 -- 172.21.15.38:0/2695507749 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({fsmap=26}) -- 0x55d46dfef190 con 0x55d46df6a5d0 2024-09-16T00:33:50.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.739+0000 7f50757fa640 1 -- 172.21.15.38:0/2695507749 <== mon.0 v2:172.21.15.38:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f50880051c0 con 0x55d46df6a5d0 2024-09-16T00:33:50.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.739+0000 7f5095e62740 10 client.15633 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:33:50.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.739+0000 7f5095e62740 10 client.15633 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:33:50.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.739+0000 7f5095e62740 20 client.15633 populate_metadata read hostname 'smithi038' 2024-09-16T00:33:50.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.739+0000 7f5095e62740 1 --2- 172.21.15.38:0/2695507749 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55d46dfef850 0x55d46dff1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:50.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.739+0000 7f5095e62740 4 client.15633 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:33:50.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.739+0000 7f5095e62740 1 -- 172.21.15.38:0/2695507749 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d46ded7d70 con 0x55d46dfef850 2024-09-16T00:33:50.744 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.740+0000 7f5091024640 1 --2- 172.21.15.38:0/2695507749 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55d46dfef850 0x55d46dff1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:50.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.740+0000 7f5091024640 1 --2- 172.21.15.38:0/2695507749 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55d46dfef850 0x55d46dff1d10 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=845237dc525c4e9 server_cookie=f60fe148c0e348a3 in_seq=0 out_seq=0 2024-09-16T00:33:50.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.740+0000 7f50757fa640 10 client.15633 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:33:50.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.740+0000 7f50757fa640 1 -- 172.21.15.38:0/2695507749 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d46ded7d70 con 0x55d46dfef850 2024-09-16T00:33:50.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.740+0000 7f50757fa640 10 client.15633 handle_command_reply: tid=0 2024-09-16T00:33:50.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.740+0000 7f5095e62740 1 client.15633 shutdown 2024-09-16T00:33:50.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.740+0000 7f5095e62740 1 -- 172.21.15.38:0/2695507749 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d46df6a5d0 msgr2=0x55d46df78470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:50.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.740+0000 7f5095e62740 1 --2- 172.21.15.38:0/2695507749 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d46df6a5d0 0x55d46df78470 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x55d46df67ac0 tx=0x7f5088003d00 comp rx=0 tx=0).stop 2024-09-16T00:33:50.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.741+0000 7f5095e62740 1 -- 172.21.15.38:0/2695507749 shutdown_connections 2024-09-16T00:33:50.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.741+0000 7f5095e62740 1 --2- 172.21.15.38:0/2695507749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55d46df75370 0x55d46df78ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:50.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.741+0000 7f5095e62740 1 --2- 172.21.15.38:0/2695507749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55d46df6bb40 0x55d46df789b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:50.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.741+0000 7f5095e62740 1 --2- 172.21.15.38:0/2695507749 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55d46dfef850 0x55d46dff1d10 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:50.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.741+0000 7f5095e62740 1 --2- 172.21.15.38:0/2695507749 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55d46df6a5d0 0x55d46df78470 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:50.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.741+0000 7f5095e62740 1 -- 172.21.15.38:0/2695507749 >> 172.21.15.38:0/2695507749 conn(0x55d46df33310 msgr2=0x55d46df74060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:50.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.741+0000 7f50757fa640 0 client.15633 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:33:50.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.741+0000 7f5095e62740 1 -- 172.21.15.38:0/2695507749 shutdown_connections 2024-09-16T00:33:50.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.741+0000 7f5095e62740 1 -- 172.21.15.38:0/2695507749 wait complete. 2024-09-16T00:33:50.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.741+0000 7f5095e62740 20 client.15633 trim_cache size 0 max 16384 2024-09-16T00:33:50.747 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-09-16T00:33:50.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.742+0000 7f5093a89640 1 -- 172.21.15.38:0/1436904257 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f50780a33d0 msgr2=0x7f50780a5890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:50.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.742+0000 7f5093a89640 1 --2- 172.21.15.38:0/1436904257 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f50780a33d0 0x7f50780a5890 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f5084006010 tx=0x7f5084005d80 comp rx=0 tx=0).stop 2024-09-16T00:33:50.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.742+0000 7f5093a89640 1 -- 172.21.15.38:0/1436904257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f508c1027c0 msgr2=0x7f508c1a4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:50.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.742+0000 7f5093a89640 1 --2- 172.21.15.38:0/1436904257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f508c1027c0 0x7f508c1a4cc0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f507c005fd0 tx=0x7f507c00dde0 comp rx=0 tx=0).stop 2024-09-16T00:33:50.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.742+0000 7f5093a89640 1 -- 172.21.15.38:0/1436904257 shutdown_connections 2024-09-16T00:33:50.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.742+0000 7f5093a89640 1 --2- 172.21.15.38:0/1436904257 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f50780a33d0 0x7f50780a5890 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:50.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.742+0000 7f5093a89640 1 --2- 172.21.15.38:0/1436904257 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f508c106f40 0x7f508c10db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:50.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.742+0000 7f5093a89640 1 --2- 172.21.15.38:0/1436904257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f508c1027c0 0x7f508c1a4cc0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:50.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.742+0000 7f5093a89640 1 --2- 172.21.15.38:0/1436904257 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f508c101e10 0x7f508c1a4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:50.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.742+0000 7f5093a89640 1 -- 172.21.15.38:0/1436904257 >> 172.21.15.38:0/1436904257 conn(0x7f508c0fd8d0 msgr2=0x7f508c0ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:50.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.743+0000 7f5093a89640 1 -- 172.21.15.38:0/1436904257 shutdown_connections 2024-09-16T00:33:50.749 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:50.743+0000 7f5093a89640 1 -- 172.21.15.38:0/1436904257 wait complete. 2024-09-16T00:33:50.944 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:50 smithi038 ceph-mon[30945]: pgmap v396: 129 pgs: 129 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-16T00:33:51.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:50 smithi122 ceph-mon[35412]: pgmap v396: 129 pgs: 129 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-16T00:33:51.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:50 smithi073 ceph-mon[35254]: pgmap v396: 129 pgs: 129 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 85 B/s wr, 7 op/s 2024-09-16T00:33:52.519 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:33:52.519 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:33:52.898 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:52.951 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:52 smithi038 ceph-mon[30945]: pgmap v397: 129 pgs: 129 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-09-16T00:33:52.952 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:52 smithi038 ceph-mon[30945]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-16T00:33:53.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:52 smithi122 ceph-mon[35412]: pgmap v397: 129 pgs: 129 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-09-16T00:33:53.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:52 smithi122 ceph-mon[35412]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-16T00:33:53.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:52 smithi073 ceph-mon[35254]: pgmap v397: 129 pgs: 129 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 5 op/s 2024-09-16T00:33:53.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:52 smithi073 ceph-mon[35254]: mgrmap e20: x(active, since 10m), standbys: z, y 2024-09-16T00:33:53.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.455+0000 7f7e6b065640 1 -- 172.21.15.38:0/1084452574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e64100e20 msgr2=0x7f7e64101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:53.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.455+0000 7f7e6b065640 1 --2- 172.21.15.38:0/1084452574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e64100e20 0x7f7e64101200 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f7e58009900 tx=0x7f7e5802a480 comp rx=0 tx=0).stop 2024-09-16T00:33:53.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.456+0000 7f7e6b065640 1 -- 172.21.15.38:0/1084452574 shutdown_connections 2024-09-16T00:33:53.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.456+0000 7f7e6b065640 1 --2- 172.21.15.38:0/1084452574 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e64102ef0 0x7f7e6410da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:53.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.456+0000 7f7e6b065640 1 --2- 172.21.15.38:0/1084452574 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e64102570 0x7f7e641029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:53.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.456+0000 7f7e6b065640 1 --2- 172.21.15.38:0/1084452574 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e64100e20 0x7f7e64101200 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:53.457 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.456+0000 7f7e6b065640 1 -- 172.21.15.38:0/1084452574 >> 172.21.15.38:0/1084452574 conn(0x7f7e640fc7d0 msgr2=0x7f7e640febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:53.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.456+0000 7f7e6b065640 1 -- 172.21.15.38:0/1084452574 shutdown_connections 2024-09-16T00:33:53.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.456+0000 7f7e6b065640 1 -- 172.21.15.38:0/1084452574 wait complete. 2024-09-16T00:33:53.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.457+0000 7f7e6b065640 1 Processor -- start 2024-09-16T00:33:53.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.457+0000 7f7e6b065640 1 -- start start 2024-09-16T00:33:53.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.457+0000 7f7e6b065640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e64100e20 0x7f7e641984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:53.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.457+0000 7f7e6b065640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e64102570 0x7f7e64198a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:53.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.457+0000 7f7e63fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e64102570 0x7f7e64198a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:53.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.457+0000 7f7e68e01640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e64100e20 0x7f7e641984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:53.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.457+0000 7f7e63fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e64102570 0x7f7e64198a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54152/0 (socket says 172.21.15.38:54152) 2024-09-16T00:33:53.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.457+0000 7f7e63fff640 1 -- 172.21.15.38:0/2156135019 learned_addr learned my addr 172.21.15.38:0/2156135019 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:53.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.457+0000 7f7e6b065640 1 --2- 172.21.15.38:0/2156135019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e64102ef0 0x7f7e6419cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:53.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.457+0000 7f7e6b065640 1 -- 172.21.15.38:0/2156135019 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7e64110f80 con 0x7f7e64102570 2024-09-16T00:33:53.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.457+0000 7f7e6b065640 1 -- 172.21.15.38:0/2156135019 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7e64110e00 con 0x7f7e64102ef0 2024-09-16T00:33:53.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.457+0000 7f7e6b065640 1 -- 172.21.15.38:0/2156135019 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7e64111100 con 0x7f7e64100e20 2024-09-16T00:33:53.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.458+0000 7f7e69602640 1 --2- 172.21.15.38:0/2156135019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e64102ef0 0x7f7e6419cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:53.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.458+0000 7f7e63fff640 1 -- 172.21.15.38:0/2156135019 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e64100e20 msgr2=0x7f7e641984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:53.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.458+0000 7f7e63fff640 1 --2- 172.21.15.38:0/2156135019 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e64100e20 0x7f7e641984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:53.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.458+0000 7f7e63fff640 1 -- 172.21.15.38:0/2156135019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e64102ef0 msgr2=0x7f7e6419cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:53.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.458+0000 7f7e63fff640 1 --2- 172.21.15.38:0/2156135019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e64102ef0 0x7f7e6419cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:53.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.458+0000 7f7e63fff640 1 -- 172.21.15.38:0/2156135019 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e6419d480 con 0x7f7e64102570 2024-09-16T00:33:53.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.458+0000 7f7e63fff640 1 --2- 172.21.15.38:0/2156135019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e64102570 0x7f7e64198a10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000ed60 tx=0x7f7e5000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:53.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.459+0000 7f7e61ffb640 1 -- 172.21.15.38:0/2156135019 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e50009e30 con 0x7f7e64102570 2024-09-16T00:33:53.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.459+0000 7f7e6b065640 1 -- 172.21.15.38:0/2156135019 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e6419d770 con 0x7f7e64102570 2024-09-16T00:33:53.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.459+0000 7f7e61ffb640 1 -- 172.21.15.38:0/2156135019 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e50005500 con 0x7f7e64102570 2024-09-16T00:33:53.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.459+0000 7f7e61ffb640 1 -- 172.21.15.38:0/2156135019 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e50010690 con 0x7f7e64102570 2024-09-16T00:33:53.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.459+0000 7f7e6b065640 1 -- 172.21.15.38:0/2156135019 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e640fcbb0 con 0x7f7e64102570 2024-09-16T00:33:53.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.462+0000 7f7e6d43e740 1 Processor -- start 2024-09-16T00:33:53.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.462+0000 7f7e6d43e740 1 -- start start 2024-09-16T00:33:53.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.462+0000 7f7e6d43e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b9b547a80 0x556b9b547e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:53.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.462+0000 7f7e6d43e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b9b5483a0 0x556b9b55b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:53.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.462+0000 7f7e6d43e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9b55b550 0x556b9b55d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:53.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.462+0000 7f7e6d43e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556b9b4ad280 con 0x556b9b547a80 2024-09-16T00:33:53.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.462+0000 7f7e6d43e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556b9b454b10 con 0x556b9b55b550 2024-09-16T00:33:53.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.463+0000 7f7e6d43e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556b9b4a9240 con 0x556b9b5483a0 2024-09-16T00:33:53.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.463+0000 7f7e69602640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b9b547a80 0x556b9b547e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:53.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.463+0000 7f7e69602640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b9b547a80 0x556b9b547e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54168/0 (socket says 172.21.15.38:54168) 2024-09-16T00:33:53.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.463+0000 7f7e69602640 1 -- 172.21.15.38:0/3631125127 learned_addr learned my addr 172.21.15.38:0/3631125127 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:53.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.463+0000 7f7e68e01640 1 --2- 172.21.15.38:0/3631125127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b9b5483a0 0x556b9b55b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:53.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.463+0000 7f7e69602640 1 -- 172.21.15.38:0/3631125127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b9b5483a0 msgr2=0x556b9b55b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:53.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.463+0000 7f7e69602640 1 --2- 172.21.15.38:0/3631125127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b9b5483a0 0x556b9b55b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:53.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.463+0000 7f7e69602640 1 -- 172.21.15.38:0/3631125127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9b55b550 msgr2=0x556b9b55d940 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-16T00:33:53.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.463+0000 7f7e69602640 1 --2- 172.21.15.38:0/3631125127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9b55b550 0x556b9b55d940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:53.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.463+0000 7f7e69602640 1 -- 172.21.15.38:0/3631125127 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b9b4611f0 con 0x556b9b547a80 2024-09-16T00:33:53.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.464+0000 7f7e69602640 1 --2- 172.21.15.38:0/3631125127 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b9b547a80 0x556b9b547e60 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c009fd0 tx=0x7f7e5c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b40ce826a6899481 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:53.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.464+0000 7f7e61ffb640 1 -- 172.21.15.38:0/2156135019 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f7e50004f80 con 0x7f7e64102570 2024-09-16T00:33:53.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.464+0000 7f7e4effd640 1 -- 172.21.15.38:0/3631125127 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c004d20 con 0x556b9b547a80 2024-09-16T00:33:53.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.464+0000 7f7e4effd640 1 -- 172.21.15.38:0/3631125127 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e5c004050 con 0x556b9b547a80 2024-09-16T00:33:53.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.464+0000 7f7e4effd640 1 -- 172.21.15.38:0/3631125127 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5c005280 con 0x556b9b547a80 2024-09-16T00:33:53.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.465+0000 7f7e6d43e740 1 -- 172.21.15.38:0/3631125127 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b9b547a80 msgr2=0x556b9b547e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:53.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.465+0000 7f7e6d43e740 1 --2- 172.21.15.38:0/3631125127 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b9b547a80 0x556b9b547e60 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c009fd0 tx=0x7f7e5c0118d0 comp rx=0 tx=0).stop 2024-09-16T00:33:53.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.465+0000 7f7e6d43e740 1 -- 172.21.15.38:0/3631125127 shutdown_connections 2024-09-16T00:33:53.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.465+0000 7f7e6d43e740 1 --2- 172.21.15.38:0/3631125127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9b55b550 0x556b9b55d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:53.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.465+0000 7f7e6d43e740 1 --2- 172.21.15.38:0/3631125127 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b9b5483a0 0x556b9b55b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:53.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.465+0000 7f7e6d43e740 1 --2- 172.21.15.38:0/3631125127 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b9b547a80 0x556b9b547e60 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:53.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.465+0000 7f7e6d43e740 1 -- 172.21.15.38:0/3631125127 >> 172.21.15.38:0/3631125127 conn(0x556b9b5440f0 msgr2=0x556b9b5444d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:53.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.465+0000 7f7e61ffb640 1 --2- 172.21.15.38:0/2156135019 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f7e540a3340 0x7f7e540a5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:53.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.465+0000 7f7e6d43e740 1 -- 172.21.15.38:0/3631125127 shutdown_connections 2024-09-16T00:33:53.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.466+0000 7f7e68e01640 1 --2- 172.21.15.38:0/2156135019 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f7e540a3340 0x7f7e540a5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:53.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.466+0000 7f7e6d43e740 1 -- 172.21.15.38:0/3631125127 wait complete. 2024-09-16T00:33:53.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.466+0000 7f7e61ffb640 1 -- 172.21.15.38:0/2156135019 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f7e500cfca0 con 0x7f7e64102570 2024-09-16T00:33:53.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.466+0000 7f7e6d43e740 1 -- start start 2024-09-16T00:33:53.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.466+0000 7f7e6d43e740 1 Processor -- start 2024-09-16T00:33:53.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.466+0000 7f7e68e01640 1 --2- 172.21.15.38:0/2156135019 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f7e540a3340 0x7f7e540a5800 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7e58009230 tx=0x7f7e580058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:53.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.467+0000 7f7e6d43e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b9b547a80 0x556b9b555080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:53.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.467+0000 7f7e6d43e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b9b5483a0 0x556b9b5555c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:53.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.467+0000 7f7e6d43e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9b55b550 0x556b9b55ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:53.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.467+0000 7f7e63fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b9b5483a0 0x556b9b5555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:53.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.467+0000 7f7e63fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b9b5483a0 0x556b9b5555c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54182/0 (socket says 172.21.15.38:54182) 2024-09-16T00:33:53.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.467+0000 7f7e68e01640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b9b547a80 0x556b9b555080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:53.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.467+0000 7f7e63fff640 1 -- 172.21.15.38:0/3756512621 learned_addr learned my addr 172.21.15.38:0/3756512621 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:53.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.467+0000 7f7e6d43e740 1 -- 172.21.15.38:0/3756512621 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x556b9b54a9e0 con 0x556b9b5483a0 2024-09-16T00:33:53.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.467+0000 7f7e6d43e740 1 -- 172.21.15.38:0/3756512621 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x556b9b467f10 con 0x556b9b55b550 2024-09-16T00:33:53.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.467+0000 7f7e6d43e740 1 -- 172.21.15.38:0/3756512621 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x556b9b54ab60 con 0x556b9b547a80 2024-09-16T00:33:53.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.467+0000 7f7e6d43e740 10 client.0 _ll_register_callbacks cb 0x556b9b430a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:53.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.467+0000 7f7e69602640 1 --2- 172.21.15.38:0/3756512621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9b55b550 0x556b9b55ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:53.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.467+0000 7f7e68e01640 1 -- 172.21.15.38:0/3756512621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9b55b550 msgr2=0x556b9b55ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:53.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.467+0000 7f7e68e01640 1 --2- 172.21.15.38:0/3756512621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9b55b550 0x556b9b55ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:53.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.468+0000 7f7e68e01640 1 -- 172.21.15.38:0/3756512621 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b9b5483a0 msgr2=0x556b9b5555c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:53.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.468+0000 7f7e68e01640 1 --2- 172.21.15.38:0/3756512621 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b9b5483a0 0x556b9b5555c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:53.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.468+0000 7f7e68e01640 1 -- 172.21.15.38:0/3756512621 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b9b21e6a0 con 0x556b9b547a80 2024-09-16T00:33:53.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.468+0000 7f7e63fff640 1 --2- 172.21.15.38:0/3756512621 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b9b5483a0 0x556b9b5555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:33:53.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.468+0000 7f7e68e01640 1 --2- 172.21.15.38:0/3756512621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b9b547a80 0x556b9b555080 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7e58005940 tx=0x7f7e58009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7642fc7b7691efc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:53.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.468+0000 7f7e43fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-16T00:33:53.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.468+0000 7f7e43fff640 1 -- 172.21.15.38:0/3756512621 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e58035070 con 0x556b9b547a80 2024-09-16T00:33:53.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.468+0000 7f7e6d43e740 1 -- 172.21.15.38:0/3756512621 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e5c004050 con 0x556b9b547a80 2024-09-16T00:33:53.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.468+0000 7f7e43fff640 1 -- 172.21.15.38:0/3756512621 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f7e58031740 con 0x556b9b547a80 2024-09-16T00:33:53.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.468+0000 7f7e43fff640 1 -- 172.21.15.38:0/3756512621 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7e58031940 con 0x556b9b547a80 2024-09-16T00:33:53.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.468+0000 7f7e6d43e740 10 client.25198 fetch_fsmap learned FSMap version 26 2024-09-16T00:33:53.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.468+0000 7f7e6d43e740 1 -- 172.21.15.38:0/3756512621 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556b9b5726f0 con 0x556b9b547a80 2024-09-16T00:33:53.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.469+0000 7f7e43fff640 1 -- 172.21.15.38:0/3756512621 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e5c004050 con 0x556b9b547a80 2024-09-16T00:33:53.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.469+0000 7f7e43fff640 1 -- 172.21.15.38:0/3756512621 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7e58043030 con 0x556b9b547a80 2024-09-16T00:33:53.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.469+0000 7f7e6d43e740 10 client.25198 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:33:53.472 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.469+0000 7f7e6d43e740 10 client.25198 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:33:53.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.469+0000 7f7e6d43e740 20 client.25198 populate_metadata read hostname 'smithi038' 2024-09-16T00:33:53.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.469+0000 7f7e6d43e740 1 --2- 172.21.15.38:0/3756512621 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x556b9b573e80 0x556b9b5762b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:53.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.469+0000 7f7e6d43e740 4 client.25198 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:33:53.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.469+0000 7f7e6d43e740 1 -- 172.21.15.38:0/3756512621 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b9b4ba7f0 con 0x556b9b573e80 2024-09-16T00:33:53.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.470+0000 7f7e63fff640 1 --2- 172.21.15.38:0/3756512621 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x556b9b573e80 0x556b9b5762b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:53.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.470+0000 7f7e63fff640 1 --2- 172.21.15.38:0/3756512621 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x556b9b573e80 0x556b9b5762b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9918e0d1fc6c3f1 server_cookie=7dda17a2fb250774 in_seq=0 out_seq=0 2024-09-16T00:33:53.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.470+0000 7f7e43fff640 10 client.25198 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:33:53.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.471+0000 7f7e43fff640 1 -- 172.21.15.38:0/3756512621 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x556b9b4ba7f0 con 0x556b9b573e80 2024-09-16T00:33:53.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.471+0000 7f7e43fff640 10 client.25198 handle_command_reply: tid=0 2024-09-16T00:33:53.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.471+0000 7f7e6d43e740 1 client.25198 shutdown 2024-09-16T00:33:53.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.471+0000 7f7e6d43e740 1 -- 172.21.15.38:0/3756512621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b9b547a80 msgr2=0x556b9b555080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:53.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.471+0000 7f7e6d43e740 1 --2- 172.21.15.38:0/3756512621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b9b547a80 0x556b9b555080 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7e58005940 tx=0x7f7e58009260 comp rx=0 tx=0).stop 2024-09-16T00:33:53.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.471+0000 7f7e6d43e740 1 -- 172.21.15.38:0/3756512621 shutdown_connections 2024-09-16T00:33:53.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.471+0000 7f7e6d43e740 1 --2- 172.21.15.38:0/3756512621 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9b55b550 0x556b9b55ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:53.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.471+0000 7f7e6d43e740 1 --2- 172.21.15.38:0/3756512621 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x556b9b573e80 0x556b9b5762b0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:53.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.471+0000 7f7e6d43e740 1 --2- 172.21.15.38:0/3756512621 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b9b5483a0 0x556b9b5555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:53.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.471+0000 7f7e6d43e740 1 --2- 172.21.15.38:0/3756512621 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b9b547a80 0x556b9b555080 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:53.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.471+0000 7f7e6d43e740 1 -- 172.21.15.38:0/3756512621 >> 172.21.15.38:0/3756512621 conn(0x556b9b5440f0 msgr2=0x556b9b543270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:53.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.471+0000 7f7e43fff640 0 client.25198 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:33:53.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.471+0000 7f7e6d43e740 1 -- 172.21.15.38:0/3756512621 shutdown_connections 2024-09-16T00:33:53.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.472+0000 7f7e6d43e740 1 -- 172.21.15.38:0/3756512621 wait complete. 2024-09-16T00:33:53.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.472+0000 7f7e6d43e740 20 client.25198 trim_cache size 0 max 16384 2024-09-16T00:33:53.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.505+0000 7f7e6d43e740 1 Processor -- start 2024-09-16T00:33:53.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.506+0000 7f7e6d43e740 1 -- start start 2024-09-16T00:33:53.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.506+0000 7f7e6d43e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b9b547a80 0x556b9b55ace0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:53.506 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.506+0000 7f7e6d43e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9b5483a0 0x556b9b54ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:53.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.506+0000 7f7e6d43e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b9b55b550 0x556b9b54f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:53.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.506+0000 7f7e6d43e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7e641af230 con 0x556b9b55b550 2024-09-16T00:33:53.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.506+0000 7f7e6d43e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7e641acb00 con 0x556b9b5483a0 2024-09-16T00:33:53.507 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.506+0000 7f7e6d43e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7e641aa950 con 0x556b9b547a80 2024-09-16T00:33:53.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.506+0000 7f7e69602640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b9b55b550 0x556b9b54f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:53.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.506+0000 7f7e63fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9b5483a0 0x556b9b54ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:53.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.506+0000 7f7e68e01640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b9b547a80 0x556b9b55ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:53.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.506+0000 7f7e69602640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b9b55b550 0x556b9b54f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54198/0 (socket says 172.21.15.38:54198) 2024-09-16T00:33:53.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.506+0000 7f7e69602640 1 -- 172.21.15.38:0/3837084730 learned_addr learned my addr 172.21.15.38:0/3837084730 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:53.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.506+0000 7f7e63fff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9b5483a0 0x556b9b54ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.38:60552/0 (socket says 172.21.15.38:60552) 2024-09-16T00:33:53.508 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.507+0000 7f7e63fff640 1 -- 172.21.15.38:0/3837084730 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b9b547a80 msgr2=0x556b9b55ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:53.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.507+0000 7f7e63fff640 1 --2- 172.21.15.38:0/3837084730 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b9b547a80 0x556b9b55ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:53.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.507+0000 7f7e63fff640 1 -- 172.21.15.38:0/3837084730 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b9b55b550 msgr2=0x556b9b54f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:53.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.507+0000 7f7e63fff640 1 --2- 172.21.15.38:0/3837084730 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b9b55b550 0x556b9b54f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:53.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.507+0000 7f7e63fff640 1 -- 172.21.15.38:0/3837084730 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b9b5502a0 con 0x556b9b5483a0 2024-09-16T00:33:53.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.507+0000 7f7e69602640 1 --2- 172.21.15.38:0/3837084730 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b9b55b550 0x556b9b54f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:33:53.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.507+0000 7f7e68e01640 1 --2- 172.21.15.38:0/3837084730 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b9b547a80 0x556b9b55ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:33:53.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.507+0000 7f7e63fff640 1 --2- 172.21.15.38:0/3837084730 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9b5483a0 0x556b9b54ed30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7e5009be70 tx=0x7f7e5001a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13bcabbfc06d3b66 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:53.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.507+0000 7f7e4e7fc640 1 -- 172.21.15.38:0/3837084730 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5007f070 con 0x556b9b5483a0 2024-09-16T00:33:53.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.507+0000 7f7e4e7fc640 1 -- 172.21.15.38:0/3837084730 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e5009c5e0 con 0x556b9b5483a0 2024-09-16T00:33:53.509 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.507+0000 7f7e4e7fc640 1 -- 172.21.15.38:0/3837084730 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e500cd4d0 con 0x556b9b5483a0 2024-09-16T00:33:53.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.508+0000 7f7e6d43e740 1 -- 172.21.15.38:0/3837084730 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9b5483a0 msgr2=0x556b9b54ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:53.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.508+0000 7f7e6d43e740 1 --2- 172.21.15.38:0/3837084730 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9b5483a0 0x556b9b54ed30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7e5009be70 tx=0x7f7e5001a040 comp rx=0 tx=0).stop 2024-09-16T00:33:53.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.508+0000 7f7e6d43e740 1 -- 172.21.15.38:0/3837084730 shutdown_connections 2024-09-16T00:33:53.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.508+0000 7f7e6d43e740 1 --2- 172.21.15.38:0/3837084730 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b9b55b550 0x556b9b54f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:53.512 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.508+0000 7f7e6d43e740 1 --2- 172.21.15.38:0/3837084730 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9b5483a0 0x556b9b54ed30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:53.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.508+0000 7f7e6d43e740 1 --2- 172.21.15.38:0/3837084730 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b9b547a80 0x556b9b55ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:53.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.508+0000 7f7e6d43e740 1 -- 172.21.15.38:0/3837084730 >> 172.21.15.38:0/3837084730 conn(0x556b9b5440f0 msgr2=0x556b9b555080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:53.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.508+0000 7f7e6d43e740 1 -- 172.21.15.38:0/3837084730 shutdown_connections 2024-09-16T00:33:53.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.509+0000 7f7e6d43e740 1 -- 172.21.15.38:0/3837084730 wait complete. 2024-09-16T00:33:53.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.509+0000 7f7e6d43e740 1 -- start start 2024-09-16T00:33:53.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.509+0000 7f7e6d43e740 1 Processor -- start 2024-09-16T00:33:53.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.509+0000 7f7e6d43e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b9b547a80 0x556b9b587e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:53.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.510+0000 7f7e6d43e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9b5483a0 0x556b9b54cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:53.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.510+0000 7f7e6d43e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b9b55b550 0x556b9b54d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:53.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.510+0000 7f7e6d43e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f7e641aafc0 con 0x556b9b55b550 2024-09-16T00:33:53.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.510+0000 7f7e6d43e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f7e641ac430 con 0x556b9b5483a0 2024-09-16T00:33:53.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.510+0000 7f7e69602640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b9b55b550 0x556b9b54d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:53.513 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.510+0000 7f7e68e01640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b9b547a80 0x556b9b587e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:53.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.510+0000 7f7e6d43e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f7e641ab690 con 0x556b9b547a80 2024-09-16T00:33:53.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.510+0000 7f7e69602640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b9b55b550 0x556b9b54d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54200/0 (socket says 172.21.15.38:54200) 2024-09-16T00:33:53.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.510+0000 7f7e68e01640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b9b547a80 0x556b9b587e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.38:60270/0 (socket says 172.21.15.38:60270) 2024-09-16T00:33:53.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.510+0000 7f7e63fff640 1 --2- 172.21.15.38:0/1313415880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9b5483a0 0x556b9b54cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:53.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.510+0000 7f7e6d43e740 10 client.0 _ll_register_callbacks cb 0x7f7e50005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:53.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.510+0000 7f7e69602640 1 -- 172.21.15.38:0/1313415880 learned_addr learned my addr 172.21.15.38:0/1313415880 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:53.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.510+0000 7f7e63fff640 1 -- 172.21.15.38:0/1313415880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b9b547a80 msgr2=0x556b9b587e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:53.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.510+0000 7f7e63fff640 1 --2- 172.21.15.38:0/1313415880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b9b547a80 0x556b9b587e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:53.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.510+0000 7f7e63fff640 1 -- 172.21.15.38:0/1313415880 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b9b55b550 msgr2=0x556b9b54d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:53.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.510+0000 7f7e63fff640 1 --2- 172.21.15.38:0/1313415880 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b9b55b550 0x556b9b54d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:53.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.510+0000 7f7e63fff640 1 -- 172.21.15.38:0/1313415880 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b9b54da30 con 0x556b9b5483a0 2024-09-16T00:33:53.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.510+0000 7f7e69602640 1 --2- 172.21.15.38:0/1313415880 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b9b55b550 0x556b9b54d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:33:53.514 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.511+0000 7f7e63fff640 1 --2- 172.21.15.38:0/1313415880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9b5483a0 0x556b9b54cf00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000ed30 tx=0x7f7e50005780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee7d6f8320560ba2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:53.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.511+0000 7f7e43fff640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-09-16T00:33:53.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.511+0000 7f7e43fff640 1 -- 172.21.15.38:0/1313415880 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e5007f070 con 0x556b9b5483a0 2024-09-16T00:33:53.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.511+0000 7f7e43fff640 1 -- 172.21.15.38:0/1313415880 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f7e500cd4d0 con 0x556b9b5483a0 2024-09-16T00:33:53.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.511+0000 7f7e6d43e740 1 -- 172.21.15.38:0/1313415880 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7e5009c5e0 con 0x556b9b5483a0 2024-09-16T00:33:53.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.511+0000 7f7e43fff640 1 -- 172.21.15.38:0/1313415880 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f7e500ad050 con 0x556b9b5483a0 2024-09-16T00:33:53.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.511+0000 7f7e6d43e740 10 client.25265 fetch_fsmap learned FSMap version 26 2024-09-16T00:33:53.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.511+0000 7f7e6d43e740 1 -- 172.21.15.38:0/1313415880 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x556b9b5de7a0 con 0x556b9b5483a0 2024-09-16T00:33:53.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.511+0000 7f7e43fff640 1 -- 172.21.15.38:0/1313415880 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f7e5009c5e0 con 0x556b9b5483a0 2024-09-16T00:33:53.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.512+0000 7f7e43fff640 1 -- 172.21.15.38:0/1313415880 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f7e5001f030 con 0x556b9b5483a0 2024-09-16T00:33:53.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.512+0000 7f7e6d43e740 10 client.25265 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:33:53.515 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.512+0000 7f7e6d43e740 10 client.25265 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:33:53.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.512+0000 7f7e6d43e740 20 client.25265 populate_metadata read hostname 'smithi038' 2024-09-16T00:33:53.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.512+0000 7f7e6d43e740 1 --2- 172.21.15.38:0/1313415880 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x556b9b5deea0 0x556b9b5e12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:53.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.512+0000 7f7e6d43e740 4 client.25265 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:33:53.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.512+0000 7f7e6d43e740 1 -- 172.21.15.38:0/1313415880 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556b9b4b4d00 con 0x556b9b5deea0 2024-09-16T00:33:53.516 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.512+0000 7f7e68e01640 1 --2- 172.21.15.38:0/1313415880 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x556b9b5deea0 0x556b9b5e12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:53.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.512+0000 7f7e68e01640 1 --2- 172.21.15.38:0/1313415880 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x556b9b5deea0 0x556b9b5e12c0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=725591b8d64f8800 server_cookie=4c27e6c3eea0555f in_seq=0 out_seq=0 2024-09-16T00:33:53.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.512+0000 7f7e43fff640 10 client.25265 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:33:53.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.513+0000 7f7e43fff640 1 -- 172.21.15.38:0/1313415880 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556b9b4b4d00 con 0x556b9b5deea0 2024-09-16T00:33:53.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.513+0000 7f7e43fff640 10 client.25265 handle_command_reply: tid=0 2024-09-16T00:33:53.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.513+0000 7f7e6d43e740 1 client.25265 shutdown 2024-09-16T00:33:53.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.514+0000 7f7e6d43e740 1 -- 172.21.15.38:0/1313415880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9b5483a0 msgr2=0x556b9b54cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:53.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.514+0000 7f7e6d43e740 1 --2- 172.21.15.38:0/1313415880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9b5483a0 0x556b9b54cf00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000ed30 tx=0x7f7e50005780 comp rx=0 tx=0).stop 2024-09-16T00:33:53.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.514+0000 7f7e6d43e740 1 -- 172.21.15.38:0/1313415880 shutdown_connections 2024-09-16T00:33:53.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.514+0000 7f7e6d43e740 1 --2- 172.21.15.38:0/1313415880 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x556b9b5deea0 0x556b9b5e12c0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:53.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.514+0000 7f7e43fff640 0 client.25265 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:33:53.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.514+0000 7f7e6d43e740 1 --2- 172.21.15.38:0/1313415880 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x556b9b55b550 0x556b9b54d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:53.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.514+0000 7f7e6d43e740 1 --2- 172.21.15.38:0/1313415880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x556b9b5483a0 0x556b9b54cf00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:53.517 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.514+0000 7f7e6d43e740 1 --2- 172.21.15.38:0/1313415880 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x556b9b547a80 0x556b9b587e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:53.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.514+0000 7f7e6d43e740 1 -- 172.21.15.38:0/1313415880 >> 172.21.15.38:0/1313415880 conn(0x556b9b5440f0 msgr2=0x556b9b542a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:53.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.514+0000 7f7e6d43e740 1 -- 172.21.15.38:0/1313415880 shutdown_connections 2024-09-16T00:33:53.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.514+0000 7f7e6d43e740 1 -- 172.21.15.38:0/1313415880 wait complete. 2024-09-16T00:33:53.518 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.514+0000 7f7e6d43e740 20 client.25265 trim_cache size 0 max 16384 2024-09-16T00:33:53.518 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-16T00:33:53.518 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0, 2024-09-16T00:33:53.518 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_tag": "cc39a1cc-ffd9-456d-9280-78fe91cec833", 2024-09-16T00:33:53.518 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": "asynchronous" 2024-09-16T00:33:53.518 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-16T00:33:53.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.515+0000 7f7e6b065640 1 -- 172.21.15.38:0/2156135019 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f7e540a3340 msgr2=0x7f7e540a5800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:53.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.515+0000 7f7e6b065640 1 --2- 172.21.15.38:0/2156135019 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f7e540a3340 0x7f7e540a5800 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7e58009230 tx=0x7f7e580058d0 comp rx=0 tx=0).stop 2024-09-16T00:33:53.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.515+0000 7f7e6b065640 1 -- 172.21.15.38:0/2156135019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e64102570 msgr2=0x7f7e64198a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:53.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.515+0000 7f7e6b065640 1 --2- 172.21.15.38:0/2156135019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e64102570 0x7f7e64198a10 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f7e5000ed60 tx=0x7f7e5000c6a0 comp rx=0 tx=0).stop 2024-09-16T00:33:53.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.515+0000 7f7e6b065640 1 -- 172.21.15.38:0/2156135019 shutdown_connections 2024-09-16T00:33:53.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.515+0000 7f7e6b065640 1 --2- 172.21.15.38:0/2156135019 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f7e540a3340 0x7f7e540a5800 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:53.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.515+0000 7f7e6b065640 1 --2- 172.21.15.38:0/2156135019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e64102ef0 0x7f7e6419cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:53.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.515+0000 7f7e6b065640 1 --2- 172.21.15.38:0/2156135019 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f7e64102570 0x7f7e64198a10 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:53.519 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.515+0000 7f7e6b065640 1 --2- 172.21.15.38:0/2156135019 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f7e64100e20 0x7f7e641984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:53.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.515+0000 7f7e6b065640 1 -- 172.21.15.38:0/2156135019 >> 172.21.15.38:0/2156135019 conn(0x7f7e640fc7d0 msgr2=0x7f7e64069f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:53.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.516+0000 7f7e6b065640 1 -- 172.21.15.38:0/2156135019 shutdown_connections 2024-09-16T00:33:53.520 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:53.516+0000 7f7e6b065640 1 -- 172.21.15.38:0/2156135019 wait complete. 2024-09-16T00:33:54.251 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:33:54.534 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:54.621 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:33:55.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.124+0000 7fda427fc640 1 -- 172.21.15.38:0/2192494090 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3c02e320 con 0x7fda44102ef0 2024-09-16T00:33:55.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.125+0000 7fda4a0a7640 1 -- 172.21.15.38:0/2192494090 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda44102ef0 msgr2=0x7fda4410da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:55.126 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.126+0000 7fda4a0a7640 1 --2- 172.21.15.38:0/2192494090 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda44102ef0 0x7fda4410da40 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fda3c009ca0 tx=0x7fda3c02a970 comp rx=0 tx=0).stop 2024-09-16T00:33:55.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.126+0000 7fda4a0a7640 1 -- 172.21.15.38:0/2192494090 shutdown_connections 2024-09-16T00:33:55.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.126+0000 7fda4a0a7640 1 --2- 172.21.15.38:0/2192494090 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda44102ef0 0x7fda4410da40 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.126+0000 7fda4a0a7640 1 --2- 172.21.15.38:0/2192494090 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda44102570 0x7fda441029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.128 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.126+0000 7fda4a0a7640 1 --2- 172.21.15.38:0/2192494090 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda44100e20 0x7fda44101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.126+0000 7fda4a0a7640 1 -- 172.21.15.38:0/2192494090 >> 172.21.15.38:0/2192494090 conn(0x7fda440fc7d0 msgr2=0x7fda440febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:55.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.126+0000 7fda4a0a7640 1 -- 172.21.15.38:0/2192494090 shutdown_connections 2024-09-16T00:33:55.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.126+0000 7fda4a0a7640 1 -- 172.21.15.38:0/2192494090 wait complete. 2024-09-16T00:33:55.129 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.127+0000 7fda4a0a7640 1 Processor -- start 2024-09-16T00:33:55.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.127+0000 7fda4a0a7640 1 -- start start 2024-09-16T00:33:55.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.128+0000 7fda4a0a7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda44100e20 0x7fda4419a680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:55.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.128+0000 7fda4a0a7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda44102570 0x7fda4419abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:55.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.128+0000 7fda4a0a7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda44102ef0 0x7fda4419ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:55.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.128+0000 7fda4a0a7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fda44110fd0 con 0x7fda44102ef0 2024-09-16T00:33:55.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.128+0000 7fda4a0a7640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fda44110e50 con 0x7fda44100e20 2024-09-16T00:33:55.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.128+0000 7fda4a0a7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fda44111150 con 0x7fda44102570 2024-09-16T00:33:55.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.128+0000 7fda42ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda44102570 0x7fda4419abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:55.130 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.128+0000 7fda42ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda44102570 0x7fda4419abc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.38:60280/0 (socket says 172.21.15.38:60280) 2024-09-16T00:33:55.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.128+0000 7fda42ffd640 1 -- 172.21.15.38:0/1565736681 learned_addr learned my addr 172.21.15.38:0/1565736681 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:55.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.128+0000 7fda43fff640 1 --2- 172.21.15.38:0/1565736681 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda44102ef0 0x7fda4419ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:55.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.128+0000 7fda42ffd640 1 -- 172.21.15.38:0/1565736681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda44100e20 msgr2=0x7fda4419a680 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-16T00:33:55.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.128+0000 7fda42ffd640 1 --2- 172.21.15.38:0/1565736681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda44100e20 0x7fda4419a680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.131 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.128+0000 7fda42ffd640 1 -- 172.21.15.38:0/1565736681 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda44102ef0 msgr2=0x7fda4419ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:55.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.128+0000 7fda42ffd640 1 --2- 172.21.15.38:0/1565736681 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda44102ef0 0x7fda4419ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.128+0000 7fda42ffd640 1 -- 172.21.15.38:0/1565736681 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda4419f630 con 0x7fda44102570 2024-09-16T00:33:55.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.128+0000 7fda43fff640 1 --2- 172.21.15.38:0/1565736681 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda44102ef0 0x7fda4419ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:33:55.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.129+0000 7fda42ffd640 1 --2- 172.21.15.38:0/1565736681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda44102570 0x7fda4419abc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00ea60 tx=0x7fda2c00ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:55.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.130+0000 7fda40ff9640 1 -- 172.21.15.38:0/1565736681 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c00cde0 con 0x7fda44102570 2024-09-16T00:33:55.132 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.130+0000 7fda4a0a7640 1 -- 172.21.15.38:0/1565736681 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda4419f920 con 0x7fda44102570 2024-09-16T00:33:55.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.130+0000 7fda4a0a7640 1 -- 172.21.15.38:0/1565736681 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda441a7200 con 0x7fda44102570 2024-09-16T00:33:55.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.130+0000 7fda40ff9640 1 -- 172.21.15.38:0/1565736681 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda2c0035f0 con 0x7fda44102570 2024-09-16T00:33:55.133 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.130+0000 7fda40ff9640 1 -- 172.21.15.38:0/1565736681 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c010670 con 0x7fda44102570 2024-09-16T00:33:55.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.133+0000 7fda40ff9640 1 -- 172.21.15.38:0/1565736681 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fda2c0108d0 con 0x7fda44102570 2024-09-16T00:33:55.134 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.133+0000 7fda4c480740 1 Processor -- start 2024-09-16T00:33:55.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.133+0000 7fda4c480740 1 -- start start 2024-09-16T00:33:55.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.134+0000 7fda4c480740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583a05674a0 0x5583a0567880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:55.135 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.134+0000 7fda4c480740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583a0567dc0 0x5583a0572140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:55.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.134+0000 7fda4c480740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5583a0572680 0x5583a0574a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:55.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.134+0000 7fda4c480740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5583a04cd2e0 con 0x5583a0572680 2024-09-16T00:33:55.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.134+0000 7fda4c480740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5583a0474ce0 con 0x5583a05674a0 2024-09-16T00:33:55.139 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.134+0000 7fda4c480740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5583a04c92a0 con 0x5583a0567dc0 2024-09-16T00:33:55.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.134+0000 7fda437fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583a0567dc0 0x5583a0572140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:55.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.134+0000 7fda437fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583a0567dc0 0x5583a0572140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.38:60292/0 (socket says 172.21.15.38:60292) 2024-09-16T00:33:55.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.134+0000 7fda437fe640 1 -- 172.21.15.38:0/2220364263 learned_addr learned my addr 172.21.15.38:0/2220364263 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:55.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.134+0000 7fda42ffd640 1 --2- 172.21.15.38:0/2220364263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5583a0572680 0x5583a0574a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:55.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.134+0000 7fda43fff640 1 --2- 172.21.15.38:0/2220364263 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583a05674a0 0x5583a0567880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:55.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.135+0000 7fda40ff9640 1 --2- 172.21.15.38:0/1565736681 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fda300a3390 0x7fda300a5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:55.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.135+0000 7fda43fff640 1 --2- 172.21.15.38:0/1565736681 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fda300a3390 0x7fda300a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:55.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.135+0000 7fda43fff640 1 -- 172.21.15.38:0/2220364263 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583a0567dc0 msgr2=0x5583a0572140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:55.140 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.135+0000 7fda43fff640 1 --2- 172.21.15.38:0/2220364263 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583a0567dc0 0x5583a0572140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.135+0000 7fda43fff640 1 -- 172.21.15.38:0/2220364263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5583a0572680 msgr2=0x5583a0574a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:55.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.136+0000 7fda43fff640 1 --2- 172.21.15.38:0/2220364263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5583a0572680 0x5583a0574a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.136+0000 7fda43fff640 1 -- 172.21.15.38:0/2220364263 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583a0459440 con 0x5583a05674a0 2024-09-16T00:33:55.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.136+0000 7fda437fe640 1 --2- 172.21.15.38:0/2220364263 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583a0567dc0 0x5583a0572140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:33:55.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.136+0000 7fda42ffd640 1 --2- 172.21.15.38:0/2220364263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5583a0572680 0x5583a0574a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:55.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.136+0000 7fda40ff9640 1 -- 172.21.15.38:0/1565736681 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7fda2c0cfdf0 con 0x7fda44102570 2024-09-16T00:33:55.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.137+0000 7fda43fff640 1 --2- 172.21.15.38:0/2220364263 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583a05674a0 0x5583a0567880 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fda3c005cc0 tx=0x7fda3c009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d29c37d4d1bd8b18 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:55.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.137+0000 7fda39ffb640 1 -- 172.21.15.38:0/2220364263 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3c034040 con 0x5583a05674a0 2024-09-16T00:33:55.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.137+0000 7fda43fff640 1 --2- 172.21.15.38:0/1565736681 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fda300a3390 0x7fda300a5850 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fda3c005e30 tx=0x7fda3c047000 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:55.141 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.137+0000 7fda39ffb640 1 -- 172.21.15.38:0/2220364263 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda3c030b50 con 0x5583a05674a0 2024-09-16T00:33:55.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.137+0000 7fda39ffb640 1 -- 172.21.15.38:0/2220364263 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3c0043a0 con 0x5583a05674a0 2024-09-16T00:33:55.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.138+0000 7fda4c480740 1 -- 172.21.15.38:0/2220364263 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583a05674a0 msgr2=0x5583a0567880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:55.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.138+0000 7fda4c480740 1 --2- 172.21.15.38:0/2220364263 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583a05674a0 0x5583a0567880 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fda3c005cc0 tx=0x7fda3c009800 comp rx=0 tx=0).stop 2024-09-16T00:33:55.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.138+0000 7fda4c480740 1 -- 172.21.15.38:0/2220364263 shutdown_connections 2024-09-16T00:33:55.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.138+0000 7fda4c480740 1 --2- 172.21.15.38:0/2220364263 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5583a0572680 0x5583a0574a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.138+0000 7fda4c480740 1 --2- 172.21.15.38:0/2220364263 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583a0567dc0 0x5583a0572140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.139+0000 7fda4c480740 1 --2- 172.21.15.38:0/2220364263 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583a05674a0 0x5583a0567880 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.139+0000 7fda4c480740 1 -- 172.21.15.38:0/2220364263 >> 172.21.15.38:0/2220364263 conn(0x5583a0563a30 msgr2=0x5583a0563e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:55.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.139+0000 7fda4c480740 1 -- 172.21.15.38:0/2220364263 shutdown_connections 2024-09-16T00:33:55.142 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.139+0000 7fda4c480740 1 -- 172.21.15.38:0/2220364263 wait complete. 2024-09-16T00:33:55.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.139+0000 7fda4c480740 1 -- start start 2024-09-16T00:33:55.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.140+0000 7fda4c480740 1 Processor -- start 2024-09-16T00:33:55.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.141+0000 7fda4c480740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5583a05674a0 0x5583a0571f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:55.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.141+0000 7fda4c480740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583a0567dc0 0x5583a056f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:55.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.141+0000 7fda4c480740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583a0572680 0x5583a056f740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:55.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.141+0000 7fda4c480740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5583a05752a0 con 0x5583a05674a0 2024-09-16T00:33:55.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.141+0000 7fda4c480740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5583a0488050 con 0x5583a0567dc0 2024-09-16T00:33:55.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.141+0000 7fda42ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583a0567dc0 0x5583a056f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:55.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.141+0000 7fda4c480740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5583a0575420 con 0x5583a0572680 2024-09-16T00:33:55.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.141+0000 7fda42ffd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583a0567dc0 0x5583a056f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.38:60588/0 (socket says 172.21.15.38:60588) 2024-09-16T00:33:55.147 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.141+0000 7fda43fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583a0572680 0x5583a056f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:55.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.141+0000 7fda42ffd640 1 -- 172.21.15.38:0/2748527867 learned_addr learned my addr 172.21.15.38:0/2748527867 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:55.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.141+0000 7fda4c480740 10 client.0 _ll_register_callbacks cb 0x5583a04c8160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:55.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.142+0000 7fda437fe640 1 --2- 172.21.15.38:0/2748527867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5583a05674a0 0x5583a0571f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:55.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.142+0000 7fda437fe640 1 -- 172.21.15.38:0/2748527867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583a0572680 msgr2=0x5583a056f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:55.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.142+0000 7fda437fe640 1 --2- 172.21.15.38:0/2748527867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583a0572680 0x5583a056f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.142+0000 7fda437fe640 1 -- 172.21.15.38:0/2748527867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583a0567dc0 msgr2=0x5583a056f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:55.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.142+0000 7fda437fe640 1 --2- 172.21.15.38:0/2748527867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583a0567dc0 0x5583a056f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.142+0000 7fda437fe640 1 -- 172.21.15.38:0/2748527867 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583a023e3c0 con 0x5583a05674a0 2024-09-16T00:33:55.148 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.142+0000 7fda43fff640 1 --2- 172.21.15.38:0/2748527867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583a0572680 0x5583a056f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:55.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.143+0000 7fda437fe640 1 --2- 172.21.15.38:0/2748527867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5583a05674a0 0x5583a0571f50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fda340039b0 tx=0x7fda34003e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667f5aa5cc3495b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:55.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.143+0000 7fda22ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:33:55.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.143+0000 7fda22ffd640 1 -- 172.21.15.38:0/2748527867 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3400bdd0 con 0x5583a05674a0 2024-09-16T00:33:55.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.143+0000 7fda22ffd640 1 -- 172.21.15.38:0/2748527867 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7fda3401a470 con 0x5583a05674a0 2024-09-16T00:33:55.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.143+0000 7fda4c480740 1 -- 172.21.15.38:0/2748527867 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda3c030b50 con 0x5583a05674a0 2024-09-16T00:33:55.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.144+0000 7fda22ffd640 1 -- 172.21.15.38:0/2748527867 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fda34023050 con 0x5583a05674a0 2024-09-16T00:33:55.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.144+0000 7fda4c480740 10 client.15663 fetch_fsmap learned FSMap version 26 2024-09-16T00:33:55.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.144+0000 7fda4c480740 1 -- 172.21.15.38:0/2748527867 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5583a0580cb0 con 0x5583a05674a0 2024-09-16T00:33:55.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.145+0000 7fda22ffd640 1 -- 172.21.15.38:0/2748527867 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda3c030b50 con 0x5583a05674a0 2024-09-16T00:33:55.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.145+0000 7fda22ffd640 1 -- 172.21.15.38:0/2748527867 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fda34011930 con 0x5583a05674a0 2024-09-16T00:33:55.149 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.145+0000 7fda4c480740 10 client.15663 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:33:55.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.145+0000 7fda4c480740 10 client.15663 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:33:55.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.145+0000 7fda4c480740 20 client.15663 populate_metadata read hostname 'smithi038' 2024-09-16T00:33:55.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.146+0000 7fda4c480740 1 --2- 172.21.15.38:0/2748527867 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5583a0582170 0x5583a0584560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:55.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.146+0000 7fda4c480740 4 client.15663 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:33:55.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.146+0000 7fda4c480740 1 -- 172.21.15.38:0/2748527867 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5583a04da850 con 0x5583a0582170 2024-09-16T00:33:55.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.146+0000 7fda42ffd640 1 --2- 172.21.15.38:0/2748527867 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5583a0582170 0x5583a0584560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:55.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.147+0000 7fda42ffd640 1 --2- 172.21.15.38:0/2748527867 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5583a0582170 0x5583a0584560 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19d98fd8cd7ffbe5 server_cookie=4df4dacad4be763e in_seq=0 out_seq=0 2024-09-16T00:33:55.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.147+0000 7fda22ffd640 10 client.15663 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:33:55.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.147+0000 7fda22ffd640 1 -- 172.21.15.38:0/2748527867 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5583a04da850 con 0x5583a0582170 2024-09-16T00:33:55.151 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.147+0000 7fda22ffd640 10 client.15663 handle_command_reply: tid=0 2024-09-16T00:33:55.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.147+0000 7fda4c480740 1 client.15663 shutdown 2024-09-16T00:33:55.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.148+0000 7fda4c480740 1 -- 172.21.15.38:0/2748527867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5583a05674a0 msgr2=0x5583a0571f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:55.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.148+0000 7fda4c480740 1 --2- 172.21.15.38:0/2748527867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5583a05674a0 0x5583a0571f50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fda340039b0 tx=0x7fda34003e70 comp rx=0 tx=0).stop 2024-09-16T00:33:55.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.148+0000 7fda4c480740 1 -- 172.21.15.38:0/2748527867 shutdown_connections 2024-09-16T00:33:55.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.148+0000 7fda4c480740 1 --2- 172.21.15.38:0/2748527867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583a0572680 0x5583a056f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.148+0000 7fda4c480740 1 --2- 172.21.15.38:0/2748527867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583a0567dc0 0x5583a056f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.148+0000 7fda4c480740 1 --2- 172.21.15.38:0/2748527867 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5583a0582170 0x5583a0584560 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.148+0000 7fda4c480740 1 --2- 172.21.15.38:0/2748527867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5583a05674a0 0x5583a0571f50 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.152 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.148+0000 7fda4c480740 1 -- 172.21.15.38:0/2748527867 >> 172.21.15.38:0/2748527867 conn(0x5583a0563a30 msgr2=0x5583a0569710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:55.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.148+0000 7fda22ffd640 0 client.15663 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:33:55.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.148+0000 7fda4c480740 1 -- 172.21.15.38:0/2748527867 shutdown_connections 2024-09-16T00:33:55.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.148+0000 7fda4c480740 1 -- 172.21.15.38:0/2748527867 wait complete. 2024-09-16T00:33:55.153 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.148+0000 7fda4c480740 20 client.15663 trim_cache size 0 max 16384 2024-09-16T00:33:55.181 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:55.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.183+0000 7fda4c480740 1 Processor -- start 2024-09-16T00:33:55.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.183+0000 7fda4c480740 1 -- start start 2024-09-16T00:33:55.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.183+0000 7fda4c480740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583a0567dc0 0x5583a0589e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:55.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.183+0000 7fda4c480740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5583a0572680 0x5583a058a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:55.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.184+0000 7fda42ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5583a0572680 0x5583a058a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:55.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.184+0000 7fda42ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5583a0572680 0x5583a058a370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54238/0 (socket says 172.21.15.38:54238) 2024-09-16T00:33:55.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.184+0000 7fda437fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583a0567dc0 0x5583a0589e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:55.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.184+0000 7fda42ffd640 1 -- 172.21.15.38:0/3187168843 learned_addr learned my addr 172.21.15.38:0/3187168843 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:55.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.184+0000 7fda4c480740 1 --2- 172.21.15.38:0/3187168843 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583a056c460 0x5583a056c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:55.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.184+0000 7fda4c480740 1 -- 172.21.15.38:0/3187168843 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fda240123e0 con 0x5583a0572680 2024-09-16T00:33:55.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.184+0000 7fda4c480740 1 -- 172.21.15.38:0/3187168843 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fda2400fcb0 con 0x5583a056c460 2024-09-16T00:33:55.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.184+0000 7fda4c480740 1 -- 172.21.15.38:0/3187168843 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fda2400db00 con 0x5583a0567dc0 2024-09-16T00:33:55.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.184+0000 7fda43fff640 1 --2- 172.21.15.38:0/3187168843 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583a056c460 0x5583a056c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:55.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.184+0000 7fda42ffd640 1 -- 172.21.15.38:0/3187168843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583a0567dc0 msgr2=0x5583a0589e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:55.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.184+0000 7fda42ffd640 1 --2- 172.21.15.38:0/3187168843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583a0567dc0 0x5583a0589e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.184+0000 7fda42ffd640 1 -- 172.21.15.38:0/3187168843 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583a056c460 msgr2=0x5583a056c8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:55.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.184+0000 7fda42ffd640 1 --2- 172.21.15.38:0/3187168843 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583a056c460 0x5583a056c8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.184+0000 7fda42ffd640 1 -- 172.21.15.38:0/3187168843 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583a0563860 con 0x5583a0572680 2024-09-16T00:33:55.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.184+0000 7fda43fff640 1 --2- 172.21.15.38:0/3187168843 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583a056c460 0x5583a056c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:33:55.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.184+0000 7fda437fe640 1 --2- 172.21.15.38:0/3187168843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583a0567dc0 0x5583a0589e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:33:55.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.185+0000 7fda42ffd640 1 --2- 172.21.15.38:0/3187168843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5583a0572680 0x5583a058a370 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fda2c09d1d0 tx=0x7fda2c00ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86af0dc202a695c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:55.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.185+0000 7fda397fa640 1 -- 172.21.15.38:0/3187168843 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c09d940 con 0x5583a0572680 2024-09-16T00:33:55.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.185+0000 7fda397fa640 1 -- 172.21.15.38:0/3187168843 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fda2c09d240 con 0x5583a0572680 2024-09-16T00:33:55.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.185+0000 7fda397fa640 1 -- 172.21.15.38:0/3187168843 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda2c0a1160 con 0x5583a0572680 2024-09-16T00:33:55.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.186+0000 7fda4c480740 1 -- 172.21.15.38:0/3187168843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5583a0572680 msgr2=0x5583a058a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:55.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.186+0000 7fda4c480740 1 --2- 172.21.15.38:0/3187168843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5583a0572680 0x5583a058a370 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fda2c09d1d0 tx=0x7fda2c00ef90 comp rx=0 tx=0).stop 2024-09-16T00:33:55.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.186+0000 7fda4c480740 1 -- 172.21.15.38:0/3187168843 shutdown_connections 2024-09-16T00:33:55.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.186+0000 7fda4c480740 1 --2- 172.21.15.38:0/3187168843 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583a056c460 0x5583a056c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.186+0000 7fda4c480740 1 --2- 172.21.15.38:0/3187168843 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5583a0572680 0x5583a058a370 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.186+0000 7fda4c480740 1 --2- 172.21.15.38:0/3187168843 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583a0567dc0 0x5583a0589e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.186+0000 7fda4c480740 1 -- 172.21.15.38:0/3187168843 >> 172.21.15.38:0/3187168843 conn(0x5583a0563a30 msgr2=0x5583a0562b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:55.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.186+0000 7fda4c480740 1 -- 172.21.15.38:0/3187168843 shutdown_connections 2024-09-16T00:33:55.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.186+0000 7fda4c480740 1 -- 172.21.15.38:0/3187168843 wait complete. 2024-09-16T00:33:55.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.187+0000 7fda4c480740 1 -- start start 2024-09-16T00:33:55.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.187+0000 7fda4c480740 1 Processor -- start 2024-09-16T00:33:55.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.188+0000 7fda4c480740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5583a0567dc0 0x5583a057ad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:55.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.188+0000 7fda4c480740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583a056c460 0x5583a0579130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:55.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.188+0000 7fda437fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5583a0567dc0 0x5583a057ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:55.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.188+0000 7fda437fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5583a0567dc0 0x5583a057ad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54240/0 (socket says 172.21.15.38:54240) 2024-09-16T00:33:55.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.188+0000 7fda437fe640 1 -- 172.21.15.38:0/3183784059 learned_addr learned my addr 172.21.15.38:0/3183784059 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:55.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.188+0000 7fda4c480740 1 --2- 172.21.15.38:0/3183784059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583a0572680 0x5583a0579670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:55.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.188+0000 7fda4c480740 1 -- 172.21.15.38:0/3183784059 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fda2400e170 con 0x5583a0567dc0 2024-09-16T00:33:55.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.188+0000 7fda4c480740 1 -- 172.21.15.38:0/3183784059 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fda2400f5e0 con 0x5583a0572680 2024-09-16T00:33:55.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.188+0000 7fda42ffd640 1 --2- 172.21.15.38:0/3183784059 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583a056c460 0x5583a0579130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:55.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.188+0000 7fda4c480740 1 -- 172.21.15.38:0/3183784059 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fda2400e840 con 0x5583a056c460 2024-09-16T00:33:55.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.189+0000 7fda43fff640 1 --2- 172.21.15.38:0/3183784059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583a0572680 0x5583a0579670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:55.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.189+0000 7fda43fff640 1 -- 172.21.15.38:0/3183784059 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583a056c460 msgr2=0x5583a0579130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:55.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.189+0000 7fda43fff640 1 --2- 172.21.15.38:0/3183784059 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583a056c460 0x5583a0579130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.189+0000 7fda43fff640 1 -- 172.21.15.38:0/3183784059 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5583a0567dc0 msgr2=0x5583a057ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:55.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.189+0000 7fda43fff640 1 --2- 172.21.15.38:0/3183784059 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5583a0567dc0 0x5583a057ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.189+0000 7fda43fff640 1 -- 172.21.15.38:0/3183784059 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583a05646c0 con 0x5583a0572680 2024-09-16T00:33:55.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.189+0000 7fda437fe640 1 --2- 172.21.15.38:0/3183784059 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5583a0567dc0 0x5583a057ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:33:55.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.189+0000 7fda4c480740 10 client.0 _ll_register_callbacks cb 0x7fda34004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:55.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.189+0000 7fda4c480740 1 -- 172.21.15.38:0/3183784059 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fda2c09d240 con 0x5583a0572680 2024-09-16T00:33:55.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.190+0000 7fda43fff640 1 --2- 172.21.15.38:0/3183784059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583a0572680 0x5583a0579670 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x5583a055f630 tx=0x7fda3c0048e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6d4b9ce9496201a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:55.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.190+0000 7fda22ffd640 10 client.25280 ms_handle_connect on v2:172.21.15.73:3300/0 2024-09-16T00:33:55.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.190+0000 7fda22ffd640 1 -- 172.21.15.38:0/3183784059 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda3c034040 con 0x5583a0572680 2024-09-16T00:33:55.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.190+0000 7fda22ffd640 1 -- 172.21.15.38:0/3183784059 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7fda3c04d040 con 0x5583a0572680 2024-09-16T00:33:55.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.190+0000 7fda22ffd640 1 -- 172.21.15.38:0/3183784059 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fda3c004090 con 0x5583a0572680 2024-09-16T00:33:55.195 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.190+0000 7fda4c480740 10 client.25280 fetch_fsmap learned FSMap version 26 2024-09-16T00:33:55.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.190+0000 7fda4c480740 1 -- 172.21.15.38:0/3183784059 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({fsmap=26}) -- 0x5583a05ed920 con 0x5583a0572680 2024-09-16T00:33:55.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.191+0000 7fda22ffd640 1 -- 172.21.15.38:0/3183784059 <== mon.2 v2:172.21.15.73:3300/0 4 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fda3c004230 con 0x5583a0572680 2024-09-16T00:33:55.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.191+0000 7fda4c480740 10 client.25280 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:33:55.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.191+0000 7fda4c480740 10 client.25280 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:33:55.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.191+0000 7fda4c480740 20 client.25280 populate_metadata read hostname 'smithi038' 2024-09-16T00:33:55.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.191+0000 7fda4c480740 1 --2- 172.21.15.38:0/3183784059 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5583a05edb50 0x5583a05eff60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:55.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.191+0000 7fda4c480740 4 client.25280 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:33:55.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.191+0000 7fda4c480740 1 -- 172.21.15.38:0/3183784059 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5583a04d4d60 con 0x5583a05edb50 2024-09-16T00:33:55.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.191+0000 7fda437fe640 1 --2- 172.21.15.38:0/3183784059 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5583a05edb50 0x5583a05eff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:55.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.192+0000 7fda437fe640 1 --2- 172.21.15.38:0/3183784059 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5583a05edb50 0x5583a05eff60 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93306c8453898e53 server_cookie=9b01b5302e389c10 in_seq=0 out_seq=0 2024-09-16T00:33:55.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.192+0000 7fda22ffd640 10 client.25280 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:33:55.196 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.192+0000 7fda22ffd640 1 -- 172.21.15.38:0/3183784059 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5583a04d4d60 con 0x5583a05edb50 2024-09-16T00:33:55.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.192+0000 7fda22ffd640 10 client.25280 handle_command_reply: tid=0 2024-09-16T00:33:55.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.192+0000 7fda4c480740 1 client.25280 shutdown 2024-09-16T00:33:55.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.193+0000 7fda4c480740 1 -- 172.21.15.38:0/3183784059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583a0572680 msgr2=0x5583a0579670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:55.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.193+0000 7fda4c480740 1 --2- 172.21.15.38:0/3183784059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583a0572680 0x5583a0579670 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x5583a055f630 tx=0x7fda3c0048e0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.193+0000 7fda4c480740 1 -- 172.21.15.38:0/3183784059 shutdown_connections 2024-09-16T00:33:55.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.193+0000 7fda4c480740 1 --2- 172.21.15.38:0/3183784059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5583a0572680 0x5583a0579670 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.193+0000 7fda4c480740 1 --2- 172.21.15.38:0/3183784059 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5583a056c460 0x5583a0579130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.193+0000 7fda4c480740 1 --2- 172.21.15.38:0/3183784059 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5583a05edb50 0x5583a05eff60 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.193+0000 7fda22ffd640 0 client.25280 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:33:55.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.193+0000 7fda4c480740 1 --2- 172.21.15.38:0/3183784059 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5583a0567dc0 0x5583a057ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.193+0000 7fda4c480740 1 -- 172.21.15.38:0/3183784059 >> 172.21.15.38:0/3183784059 conn(0x5583a0563a30 msgr2=0x5583a056bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:55.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.193+0000 7fda4c480740 1 -- 172.21.15.38:0/3183784059 shutdown_connections 2024-09-16T00:33:55.197 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.193+0000 7fda4c480740 1 -- 172.21.15.38:0/3183784059 wait complete. 2024-09-16T00:33:55.198 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.193+0000 7fda4c480740 20 client.25280 trim_cache size 0 max 16384 2024-09-16T00:33:55.198 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-16T00:33:55.198 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:33:55.198 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": { 2024-09-16T00:33:55.198 INFO:teuthology.orchestra.run.smithi038.stdout: "cc39a1cc-ffd9-456d-9280-78fe91cec833": { 2024-09-16T00:33:55.198 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "/", 2024-09-16T00:33:55.198 INFO:teuthology.orchestra.run.smithi038.stdout: "tag": "cc39a1cc-ffd9-456d-9280-78fe91cec833", 2024-09-16T00:33:55.198 INFO:teuthology.orchestra.run.smithi038.stdout: "options": "recursive,force" 2024-09-16T00:33:55.199 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:33:55.199 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:33:55.199 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-16T00:33:55.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.194+0000 7fda4a0a7640 1 -- 172.21.15.38:0/1565736681 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fda300a3390 msgr2=0x7fda300a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:55.199 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.194+0000 7fda4a0a7640 1 --2- 172.21.15.38:0/1565736681 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fda300a3390 0x7fda300a5850 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fda3c005e30 tx=0x7fda3c047000 comp rx=0 tx=0).stop 2024-09-16T00:33:55.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.194+0000 7fda4a0a7640 1 -- 172.21.15.38:0/1565736681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda44102570 msgr2=0x7fda4419abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:55.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.194+0000 7fda4a0a7640 1 --2- 172.21.15.38:0/1565736681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda44102570 0x7fda4419abc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00ea60 tx=0x7fda2c00ef20 comp rx=0 tx=0).stop 2024-09-16T00:33:55.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.196+0000 7fda4a0a7640 1 -- 172.21.15.38:0/1565736681 shutdown_connections 2024-09-16T00:33:55.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.196+0000 7fda4a0a7640 1 --2- 172.21.15.38:0/1565736681 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fda300a3390 0x7fda300a5850 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.196+0000 7fda4a0a7640 1 --2- 172.21.15.38:0/1565736681 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fda44102ef0 0x7fda4419ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.196+0000 7fda4a0a7640 1 --2- 172.21.15.38:0/1565736681 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fda44102570 0x7fda4419abc0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.196+0000 7fda4a0a7640 1 --2- 172.21.15.38:0/1565736681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda44100e20 0x7fda4419a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:55.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.196+0000 7fda4a0a7640 1 -- 172.21.15.38:0/1565736681 >> 172.21.15.38:0/1565736681 conn(0x7fda440fc7d0 msgr2=0x7fda4406a4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:55.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.197+0000 7fda4a0a7640 1 -- 172.21.15.38:0/1565736681 shutdown_connections 2024-09-16T00:33:55.200 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:55.197+0000 7fda4a0a7640 1 -- 172.21.15.38:0/1565736681 wait complete. 2024-09-16T00:33:55.211 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:54 smithi038 ceph-mon[30945]: scrub queued for path: / 2024-09-16T00:33:55.211 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:54 smithi038 ceph-mon[30945]: scrub summary: idle+waiting paths [/] 2024-09-16T00:33:55.211 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:54 smithi038 ceph-mon[30945]: scrub summary: active paths [/] 2024-09-16T00:33:55.211 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:54 smithi038 ceph-mon[30945]: scrub summary: idle+waiting paths [/] 2024-09-16T00:33:55.211 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:54 smithi038 ceph-mon[30945]: pgmap v398: 129 pgs: 129 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-16T00:33:55.219 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:54 smithi122 ceph-mon[35412]: scrub queued for path: / 2024-09-16T00:33:55.219 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:54 smithi122 ceph-mon[35412]: scrub summary: idle+waiting paths [/] 2024-09-16T00:33:55.219 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:54 smithi122 ceph-mon[35412]: scrub summary: active paths [/] 2024-09-16T00:33:55.219 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:54 smithi122 ceph-mon[35412]: scrub summary: idle+waiting paths [/] 2024-09-16T00:33:55.219 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:54 smithi122 ceph-mon[35412]: pgmap v398: 129 pgs: 129 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-16T00:33:55.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:54 smithi073 ceph-mon[35254]: scrub queued for path: / 2024-09-16T00:33:55.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:54 smithi073 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-09-16T00:33:55.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:54 smithi073 ceph-mon[35254]: scrub summary: active paths [/] 2024-09-16T00:33:55.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:54 smithi073 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-09-16T00:33:55.295 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:54 smithi073 ceph-mon[35254]: pgmap v398: 129 pgs: 129 active+clean; 661 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 170 B/s wr, 6 op/s 2024-09-16T00:33:56.318 INFO:tasks.cephfs.filesystem:scrub status for tag:cc39a1cc-ffd9-456d-9280-78fe91cec833 - {'path': '/', 'tag': 'cc39a1cc-ffd9-456d-9280-78fe91cec833', 'options': 'recursive,force'} 2024-09-16T00:33:56.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:56.595+0000 7fb7250b7640 1 -- 172.21.15.38:0/1993389224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb72010c160 msgr2=0x7fb72010e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:56.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:56.595+0000 7fb7250b7640 1 --2- 172.21.15.38:0/1993389224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb72010c160 0x7fb72010e600 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb718009ca0 tx=0x7fb71802a950 comp rx=0 tx=0).stop 2024-09-16T00:33:56.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:56.595+0000 7fb7250b7640 1 -- 172.21.15.38:0/1993389224 shutdown_connections 2024-09-16T00:33:56.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:56.595+0000 7fb7250b7640 1 --2- 172.21.15.38:0/1993389224 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb72010c160 0x7fb72010e600 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:56.596 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:56.595+0000 7fb7250b7640 1 --2- 172.21.15.38:0/1993389224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb720103650 0x7fb72010bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:56.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:56.595+0000 7fb7250b7640 1 --2- 172.21.15.38:0/1993389224 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb720102d30 0x7fb720103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:56.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:56.595+0000 7fb7250b7640 1 -- 172.21.15.38:0/1993389224 >> 172.21.15.38:0/1993389224 conn(0x7fb7200fc790 msgr2=0x7fb7200febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:56.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:56.595+0000 7fb7250b7640 1 -- 172.21.15.38:0/1993389224 shutdown_connections 2024-09-16T00:33:56.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:56.595+0000 7fb7250b7640 1 -- 172.21.15.38:0/1993389224 wait complete. 2024-09-16T00:33:56.597 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:56.596+0000 7fb7250b7640 1 Processor -- start 2024-09-16T00:33:56.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:56.596+0000 7fb7250b7640 1 -- start start 2024-09-16T00:33:56.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:56.597+0000 7fb7250b7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb720102d30 0x7fb720100990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:56.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:56.597+0000 7fb7250b7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb720103650 0x7fb720100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:56.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:56.597+0000 7fb7250b7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb72010c160 0x7fb720101410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:56.598 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:56.597+0000 7fb7250b7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb720111f60 con 0x7fb72010c160 2024-09-16T00:33:56.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:56.597+0000 7fb7250b7640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb720111de0 con 0x7fb720102d30 2024-09-16T00:33:56.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:56.597+0000 7fb7250b7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb7201120e0 con 0x7fb720103650 2024-09-16T00:33:56.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:56.597+0000 7fb71e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb720103650 0x7fb720100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:56.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:56.597+0000 7fb71e59c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb720103650 0x7fb720100ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.38:50834/0 (socket says 172.21.15.38:50834) 2024-09-16T00:33:56.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:56.597+0000 7fb71e59c640 1 -- 172.21.15.38:0/1130751997 learned_addr learned my addr 172.21.15.38:0/1130751997 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:56.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:56.597+0000 7fb71f59e640 1 --2- 172.21.15.38:0/1130751997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb72010c160 0x7fb720101410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:56.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:56.597+0000 7fb71ed9d640 1 --2- 172.21.15.38:0/1130751997 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb720102d30 0x7fb720100990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:56.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:56.597+0000 7fb71e59c640 1 -- 172.21.15.38:0/1130751997 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb720102d30 msgr2=0x7fb720100990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:56.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:56.597+0000 7fb71e59c640 1 --2- 172.21.15.38:0/1130751997 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb720102d30 0x7fb720100990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:56.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:56.597+0000 7fb71e59c640 1 -- 172.21.15.38:0/1130751997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb72010c160 msgr2=0x7fb720101410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:56.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:56.597+0000 7fb71e59c640 1 --2- 172.21.15.38:0/1130751997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb72010c160 0x7fb720101410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:56.599 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:56.597+0000 7fb71e59c640 1 -- 172.21.15.38:0/1130751997 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7201aae20 con 0x7fb720103650 2024-09-16T00:33:56.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:56.597+0000 7fb71f59e640 1 --2- 172.21.15.38:0/1130751997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb72010c160 0x7fb720101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:33:56.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:56.598+0000 7fb71e59c640 1 --2- 172.21.15.38:0/1130751997 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb720103650 0x7fb720100ed0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb71400d740 tx=0x7fb71400dc00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:56.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:56.598+0000 7fb707fff640 1 -- 172.21.15.38:0/1130751997 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb714017070 con 0x7fb720103650 2024-09-16T00:33:56.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:56.598+0000 7fb707fff640 1 -- 172.21.15.38:0/1130751997 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb71400de50 con 0x7fb720103650 2024-09-16T00:33:56.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:56.598+0000 7fb7250b7640 1 -- 172.21.15.38:0/1130751997 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7201aaff0 con 0x7fb720103650 2024-09-16T00:33:56.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:56.598+0000 7fb707fff640 1 -- 172.21.15.38:0/1130751997 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb71400f620 con 0x7fb720103650 2024-09-16T00:33:56.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:56.598+0000 7fb7250b7640 1 -- 172.21.15.38:0/1130751997 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7201ab4f0 con 0x7fb720103650 2024-09-16T00:33:56.600 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:56.599+0000 7fb7250b7640 1 -- 172.21.15.38:0/1130751997 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb72006b070 con 0x7fb720103650 2024-09-16T00:33:56.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:56.600+0000 7fb707fff640 1 -- 172.21.15.38:0/1130751997 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fb714009c30 con 0x7fb720103650 2024-09-16T00:33:56.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:56.601+0000 7fb707fff640 1 --2- 172.21.15.38:0/1130751997 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb7080a32c0 0x7fb7080a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:56.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:56.601+0000 7fb707fff640 1 -- 172.21.15.38:0/1130751997 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7fb7140cf6b0 con 0x7fb720103650 2024-09-16T00:33:56.601 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:56.601+0000 7fb71ed9d640 1 --2- 172.21.15.38:0/1130751997 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb7080a32c0 0x7fb7080a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:56.602 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:56.602+0000 7fb71ed9d640 1 --2- 172.21.15.38:0/1130751997 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb7080a32c0 0x7fb7080a5780 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb710009bd0 tx=0x7fb710005f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:56.606 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:56.605+0000 7fb707fff640 1 -- 172.21.15.38:0/1130751997 <== mon.1 v2:172.21.15.122:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fb71409ba00 con 0x7fb720103650 2024-09-16T00:33:56.962 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:56 smithi038 ceph-mon[30945]: pgmap v399: 129 pgs: 129 active+clean; 662 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-16T00:33:57.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:57.019+0000 7fb7250b7640 1 -- 172.21.15.38:0/1130751997 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb72002d7c0 con 0x7fb720103650 2024-09-16T00:33:57.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:57.020+0000 7fb707fff640 1 -- 172.21.15.38:0/1130751997 <== mon.1 v2:172.21.15.122:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7fb71409fd00 con 0x7fb720103650 2024-09-16T00:33:57.020 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:33:57.020 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"btime":"2024-09-16T00:32:41:853514+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/367974527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":367974527},{"type":"v1","addr":"172.21.15.38:6835","nonce":367974527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/335077136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":335077136},{"type":"v1","addr":"172.21.15.38:6837","nonce":335077136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/3501211160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3501211160},{"type":"v1","addr":"172.21.15.38:6839","nonce":3501211160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6841/2623130915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":2623130915},{"type":"v1","addr":"172.21.15.38:6841","nonce":2623130915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/412277519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":412277519},{"type":"v1","addr":"172.21.15.122:6833","nonce":412277519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3987234757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3987234757},{"type":"v1","addr":"172.21.15.122:6835","nonce":3987234757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:30:26.764565+0000","modified":"2024-09-16T00:32:41.355828+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24440},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1210328177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1210328177},{"type":"v1","addr":"172.21.15.73:6833","nonce":1210328177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6837/1858299680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1858299680},{"type":"v1","addr":"172.21.15.122:6837","nonce":1858299680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1983975311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1983975311},{"type":"v1","addr":"172.21.15.122:6839","nonce":1983975311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.73:6835/2866363113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2866363113},{"type":"v1","addr":"172.21.15.73:6835","nonce":2866363113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.73:6837/2681081265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2681081265},{"type":"v1","addr":"172.21.15.73:6837","nonce":2681081265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.73:6839/2396224525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2396224525},{"type":"v1","addr":"172.21.15.73:6839","nonce":2396224525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24464,24440]},"id":1}]} 2024-09-16T00:33:57.021 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 26 2024-09-16T00:33:57.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:57.022+0000 7fb7250b7640 1 -- 172.21.15.38:0/1130751997 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb7080a32c0 msgr2=0x7fb7080a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:57.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:57.022+0000 7fb7250b7640 1 --2- 172.21.15.38:0/1130751997 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb7080a32c0 0x7fb7080a5780 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fb710009bd0 tx=0x7fb710005f90 comp rx=0 tx=0).stop 2024-09-16T00:33:57.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:57.022+0000 7fb7250b7640 1 -- 172.21.15.38:0/1130751997 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb720103650 msgr2=0x7fb720100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:57.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:57.022+0000 7fb7250b7640 1 --2- 172.21.15.38:0/1130751997 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb720103650 0x7fb720100ed0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fb71400d740 tx=0x7fb71400dc00 comp rx=0 tx=0).stop 2024-09-16T00:33:57.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:57.022+0000 7fb7250b7640 1 -- 172.21.15.38:0/1130751997 shutdown_connections 2024-09-16T00:33:57.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:57.022+0000 7fb7250b7640 1 --2- 172.21.15.38:0/1130751997 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb7080a32c0 0x7fb7080a5780 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:57.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:57.022+0000 7fb7250b7640 1 --2- 172.21.15.38:0/1130751997 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb72010c160 0x7fb720101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:57.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:57.022+0000 7fb7250b7640 1 --2- 172.21.15.38:0/1130751997 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb720103650 0x7fb720100ed0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:57.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:57.022+0000 7fb7250b7640 1 --2- 172.21.15.38:0/1130751997 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb720102d30 0x7fb720100990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:57.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:57.022+0000 7fb7250b7640 1 -- 172.21.15.38:0/1130751997 >> 172.21.15.38:0/1130751997 conn(0x7fb7200fc790 msgr2=0x7fb7200fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:57.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:57.022+0000 7fb7250b7640 1 -- 172.21.15.38:0/1130751997 shutdown_connections 2024-09-16T00:33:57.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:57.022+0000 7fb7250b7640 1 -- 172.21.15.38:0/1130751997 wait complete. 2024-09-16T00:33:57.216 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:56 smithi122 ceph-mon[35412]: pgmap v399: 129 pgs: 129 active+clean; 662 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-16T00:33:57.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:56 smithi073 ceph-mon[35254]: pgmap v399: 129 pgs: 129 active+clean; 662 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-16T00:33:57.756 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --set-id 6706a54b --release --await' 2024-09-16T00:33:57.756 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --set-id 6706a54b --release --await 2024-09-16T00:33:58.068 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:33:58.271 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:57 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/1130751997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:33:58.271 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:57 smithi038 ceph-mon[30945]: scrub summary: idle 2024-09-16T00:33:58.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:57 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/1130751997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:33:58.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:57 smithi073 ceph-mon[35254]: scrub summary: idle 2024-09-16T00:33:58.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:57 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/1130751997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:33:58.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:57 smithi122 ceph-mon[35412]: scrub summary: idle 2024-09-16T00:33:58.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.690+0000 7f20e22a2640 1 -- 172.21.15.38:0/3904031420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20dc100e20 msgr2=0x7f20dc101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:58.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.690+0000 7f20e22a2640 1 --2- 172.21.15.38:0/3904031420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20dc100e20 0x7f20dc101200 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f20cc009900 tx=0x7f20cc02a480 comp rx=0 tx=0).stop 2024-09-16T00:33:58.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.691+0000 7f20e22a2640 1 -- 172.21.15.38:0/3904031420 shutdown_connections 2024-09-16T00:33:58.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.691+0000 7f20e22a2640 1 --2- 172.21.15.38:0/3904031420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20dc102ef0 0x7f20dc10da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.691+0000 7f20e22a2640 1 --2- 172.21.15.38:0/3904031420 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20dc102570 0x7f20dc1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.691+0000 7f20e22a2640 1 --2- 172.21.15.38:0/3904031420 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20dc100e20 0x7f20dc101200 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.691+0000 7f20e22a2640 1 -- 172.21.15.38:0/3904031420 >> 172.21.15.38:0/3904031420 conn(0x7f20dc0fc7d0 msgr2=0x7f20dc0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:58.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.691+0000 7f20e22a2640 1 -- 172.21.15.38:0/3904031420 shutdown_connections 2024-09-16T00:33:58.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.691+0000 7f20e22a2640 1 -- 172.21.15.38:0/3904031420 wait complete. 2024-09-16T00:33:58.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.692+0000 7f20e22a2640 1 Processor -- start 2024-09-16T00:33:58.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.692+0000 7f20e22a2640 1 -- start start 2024-09-16T00:33:58.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.693+0000 7f20e22a2640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20dc100e20 0x7f20dc10e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:58.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.693+0000 7f20e22a2640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20dc102570 0x7f20dc10ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:58.693 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.693+0000 7f20e22a2640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20dc102ef0 0x7f20dc10ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:58.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.693+0000 7f20e083f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20dc102ef0 0x7f20dc10ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:58.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.693+0000 7f20dbfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20dc100e20 0x7f20dc10e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:58.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.693+0000 7f20dbfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20dc100e20 0x7f20dc10e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.38:48062/0 (socket says 172.21.15.38:48062) 2024-09-16T00:33:58.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.693+0000 7f20e083f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20dc102ef0 0x7f20dc10ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57050/0 (socket says 172.21.15.38:57050) 2024-09-16T00:33:58.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.693+0000 7f20db7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20dc102570 0x7f20dc10ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:58.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.693+0000 7f20dbfff640 1 -- 172.21.15.38:0/1507140804 learned_addr learned my addr 172.21.15.38:0/1507140804 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:58.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.693+0000 7f20e22a2640 1 -- 172.21.15.38:0/1507140804 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc1120d0 con 0x7f20dc102ef0 2024-09-16T00:33:58.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.693+0000 7f20e22a2640 1 -- 172.21.15.38:0/1507140804 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc111f50 con 0x7f20dc100e20 2024-09-16T00:33:58.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.693+0000 7f20e22a2640 1 -- 172.21.15.38:0/1507140804 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc112250 con 0x7f20dc102570 2024-09-16T00:33:58.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.694+0000 7f20dbfff640 1 -- 172.21.15.38:0/1507140804 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20dc102570 msgr2=0x7f20dc10ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:58.695 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.694+0000 7f20dbfff640 1 --2- 172.21.15.38:0/1507140804 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20dc102570 0x7f20dc10ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.694+0000 7f20dbfff640 1 -- 172.21.15.38:0/1507140804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20dc102ef0 msgr2=0x7f20dc10ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:58.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.694+0000 7f20dbfff640 1 --2- 172.21.15.38:0/1507140804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20dc102ef0 0x7f20dc10ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.694+0000 7f20dbfff640 1 -- 172.21.15.38:0/1507140804 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20dc1a69b0 con 0x7f20dc100e20 2024-09-16T00:33:58.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.694+0000 7f20e083f640 1 --2- 172.21.15.38:0/1507140804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20dc102ef0 0x7f20dc10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:33:58.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.694+0000 7f20db7fe640 1 --2- 172.21.15.38:0/1507140804 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20dc102570 0x7f20dc10ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:33:58.696 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.694+0000 7f20dbfff640 1 --2- 172.21.15.38:0/1507140804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20dc100e20 0x7f20dc10e4d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f20cc02aa70 tx=0x7f20cc003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:58.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.695+0000 7f20d97fa640 1 -- 172.21.15.38:0/1507140804 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20cc03f070 con 0x7f20dc100e20 2024-09-16T00:33:58.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.695+0000 7f20d97fa640 1 -- 172.21.15.38:0/1507140804 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20cc031c30 con 0x7f20dc100e20 2024-09-16T00:33:58.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.695+0000 7f20d97fa640 1 -- 172.21.15.38:0/1507140804 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20cc035030 con 0x7f20dc100e20 2024-09-16T00:33:58.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.695+0000 7f20e22a2640 1 -- 172.21.15.38:0/1507140804 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20dc198250 con 0x7f20dc100e20 2024-09-16T00:33:58.697 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.695+0000 7f20e22a2640 1 -- 172.21.15.38:0/1507140804 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20dc1986e0 con 0x7f20dc100e20 2024-09-16T00:33:58.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.698+0000 7f20d97fa640 1 -- 172.21.15.38:0/1507140804 <== mon.2 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f20cc042070 con 0x7f20dc100e20 2024-09-16T00:33:58.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.698+0000 7f20e467b740 1 Processor -- start 2024-09-16T00:33:58.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.698+0000 7f20e467b740 1 -- start start 2024-09-16T00:33:58.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.699+0000 7f20e467b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf659a4060 0x55bf659a4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:58.699 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.699+0000 7f20e467b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf659a4980 0x55bf659aed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:58.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.699+0000 7f20e467b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf659af240 0x55bf659b1630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:58.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.699+0000 7f20e467b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55bf65909620 con 0x55bf659af240 2024-09-16T00:33:58.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.699+0000 7f20e467b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55bf658b0e50 con 0x55bf659a4980 2024-09-16T00:33:58.700 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.699+0000 7f20e467b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55bf659055e0 con 0x55bf659a4060 2024-09-16T00:33:58.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.699+0000 7f20db7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf659af240 0x55bf659b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:58.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.699+0000 7f20dbfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf659a4980 0x55bf659aed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:58.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.699+0000 7f20d97fa640 1 --2- 172.21.15.38:0/1507140804 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f20c40a3240 0x7f20c40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:58.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.699+0000 7f20e083f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf659a4060 0x55bf659a4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:58.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.699+0000 7f20dbfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf659a4980 0x55bf659aed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.38:48070/0 (socket says 172.21.15.38:48070) 2024-09-16T00:33:58.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.699+0000 7f20dbfff640 1 -- 172.21.15.38:0/293449634 learned_addr learned my addr 172.21.15.38:0/293449634 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:58.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.700+0000 7f20d97fa640 1 -- 172.21.15.38:0/1507140804 <== mon.2 v2:172.21.15.73:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f20cc0f0460 con 0x7f20dc100e20 2024-09-16T00:33:58.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.700+0000 7f20e083f640 1 --2- 172.21.15.38:0/1507140804 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f20c40a3240 0x7f20c40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:58.701 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.700+0000 7f20db7fe640 1 -- 172.21.15.38:0/293449634 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf659a4060 msgr2=0x55bf659a4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:58.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.700+0000 7f20db7fe640 1 --2- 172.21.15.38:0/293449634 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf659a4060 0x55bf659a4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.700+0000 7f20db7fe640 1 -- 172.21.15.38:0/293449634 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf659a4980 msgr2=0x55bf659aed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:58.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.700+0000 7f20db7fe640 1 --2- 172.21.15.38:0/293449634 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf659a4980 0x55bf659aed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.700+0000 7f20db7fe640 1 -- 172.21.15.38:0/293449634 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf658a1d40 con 0x55bf659af240 2024-09-16T00:33:58.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.701+0000 7f20e083f640 1 --2- 172.21.15.38:0/1507140804 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f20c40a3240 0x7f20c40a5700 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f20d4005fd0 tx=0x7f20d40099c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:58.702 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.701+0000 7f20db7fe640 1 --2- 172.21.15.38:0/293449634 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf659af240 0x55bf659b1630 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f20d0009bb0 tx=0x7f20d000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c7f9571265ab69b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:58.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.701+0000 7f20c27fc640 1 -- 172.21.15.38:0/293449634 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d0016070 con 0x55bf659af240 2024-09-16T00:33:58.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.701+0000 7f20c27fc640 1 -- 172.21.15.38:0/293449634 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20d0011950 con 0x55bf659af240 2024-09-16T00:33:58.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.701+0000 7f20c27fc640 1 -- 172.21.15.38:0/293449634 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d0010850 con 0x55bf659af240 2024-09-16T00:33:58.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.702+0000 7f20e467b740 1 -- 172.21.15.38:0/293449634 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf659af240 msgr2=0x55bf659b1630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:58.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.702+0000 7f20e467b740 1 --2- 172.21.15.38:0/293449634 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf659af240 0x55bf659b1630 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f20d0009bb0 tx=0x7f20d000e6c0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.702+0000 7f20e467b740 1 -- 172.21.15.38:0/293449634 shutdown_connections 2024-09-16T00:33:58.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.702+0000 7f20e467b740 1 --2- 172.21.15.38:0/293449634 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf659af240 0x55bf659b1630 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.702+0000 7f20e467b740 1 --2- 172.21.15.38:0/293449634 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf659a4980 0x55bf659aed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.703 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.702+0000 7f20e467b740 1 --2- 172.21.15.38:0/293449634 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf659a4060 0x55bf659a4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.702+0000 7f20e467b740 1 -- 172.21.15.38:0/293449634 >> 172.21.15.38:0/293449634 conn(0x55bf659a05f0 msgr2=0x55bf659a09d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:58.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.703+0000 7f20e467b740 1 -- 172.21.15.38:0/293449634 shutdown_connections 2024-09-16T00:33:58.704 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.703+0000 7f20e467b740 1 -- 172.21.15.38:0/293449634 wait complete. 2024-09-16T00:33:58.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.703+0000 7f20e467b740 1 -- start start 2024-09-16T00:33:58.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.703+0000 7f20e467b740 1 Processor -- start 2024-09-16T00:33:58.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.704+0000 7f20e467b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf659a4060 0x55bf659ac3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:58.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.704+0000 7f20e467b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf659a4980 0x55bf659ac930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:58.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.704+0000 7f20e467b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf659acec0 0x55bf659bb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:58.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.704+0000 7f20e467b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55bf659b1e10 con 0x55bf659acec0 2024-09-16T00:33:58.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.704+0000 7f20e467b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55bf65891f10 con 0x55bf659a4060 2024-09-16T00:33:58.705 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.704+0000 7f20e467b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55bf659b1f90 con 0x55bf659a4980 2024-09-16T00:33:58.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.704+0000 7f20e467b740 10 client.0 _ll_register_callbacks cb 0x55bf658b5db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:58.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.704+0000 7f20e083f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf659acec0 0x55bf659bb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:58.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.704+0000 7f20e083f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf659acec0 0x55bf659bb2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57074/0 (socket says 172.21.15.38:57074) 2024-09-16T00:33:58.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.704+0000 7f20e083f640 1 -- 172.21.15.38:0/1006220576 learned_addr learned my addr 172.21.15.38:0/1006220576 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:58.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.705+0000 7f20e083f640 1 -- 172.21.15.38:0/1006220576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf659a4980 msgr2=0x55bf659ac930 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-16T00:33:58.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.705+0000 7f20e083f640 1 --2- 172.21.15.38:0/1006220576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf659a4980 0x55bf659ac930 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.705+0000 7f20e083f640 1 -- 172.21.15.38:0/1006220576 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf659a4060 msgr2=0x55bf659ac3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:58.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.705+0000 7f20dbfff640 1 --2- 172.21.15.38:0/1006220576 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf659a4060 0x55bf659ac3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:58.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.705+0000 7f20e083f640 1 --2- 172.21.15.38:0/1006220576 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf659a4060 0x55bf659ac3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.706 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.705+0000 7f20e083f640 1 -- 172.21.15.38:0/1006220576 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf65618580 con 0x55bf659acec0 2024-09-16T00:33:58.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.705+0000 7f20e083f640 1 --2- 172.21.15.38:0/1006220576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf659acec0 0x55bf659bb2d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f20d4022dc0 tx=0x7f20d4025000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f14f2b57442d390 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:58.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.705+0000 7f20bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:33:58.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.705+0000 7f20bb7fe640 1 -- 172.21.15.38:0/1006220576 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d4004b20 con 0x55bf659acec0 2024-09-16T00:33:58.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.706+0000 7f20bb7fe640 1 -- 172.21.15.38:0/1006220576 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f20d4004cc0 con 0x55bf659acec0 2024-09-16T00:33:58.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.706+0000 7f20e467b740 1 -- 172.21.15.38:0/1006220576 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20d0011950 con 0x55bf659acec0 2024-09-16T00:33:58.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.706+0000 7f20bb7fe640 1 -- 172.21.15.38:0/1006220576 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f20d4003970 con 0x55bf659acec0 2024-09-16T00:33:58.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.706+0000 7f20e467b740 10 client.15687 fetch_fsmap learned FSMap version 26 2024-09-16T00:33:58.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.706+0000 7f20e467b740 1 -- 172.21.15.38:0/1006220576 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bf656e2960 con 0x55bf659acec0 2024-09-16T00:33:58.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.706+0000 7f20bb7fe640 1 -- 172.21.15.38:0/1006220576 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20d4004020 con 0x55bf659acec0 2024-09-16T00:33:58.708 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.706+0000 7f20bb7fe640 1 -- 172.21.15.38:0/1006220576 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f20d402f070 con 0x55bf659acec0 2024-09-16T00:33:58.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.706+0000 7f20e467b740 10 client.15687 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:33:58.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.706+0000 7f20e467b740 10 client.15687 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-16T00:33:58.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.706+0000 7f20e467b740 20 client.15687 populate_metadata read hostname 'smithi038' 2024-09-16T00:33:58.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.706+0000 7f20e467b740 1 --2- 172.21.15.38:0/1006220576 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55bf659bf550 0x55bf659c1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:58.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.706+0000 7f20e467b740 4 client.15687 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:33:58.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.706+0000 7f20e467b740 1 -- 172.21.15.38:0/1006220576 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f20d0011950 con 0x55bf659bf550 2024-09-16T00:33:58.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.707+0000 7f20dbfff640 1 --2- 172.21.15.38:0/1006220576 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55bf659bf550 0x55bf659c1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:58.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.707+0000 7f20dbfff640 1 --2- 172.21.15.38:0/1006220576 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55bf659bf550 0x55bf659c1a10 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=647bb8badb4c84b9 server_cookie=877d3eb9c14f64a4 in_seq=0 out_seq=0 2024-09-16T00:33:58.709 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.707+0000 7f20bb7fe640 10 client.15687 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:33:58.710 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.708+0000 7f20bb7fe640 1 -- 172.21.15.38:0/1006220576 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f20d0011950 con 0x55bf659bf550 2024-09-16T00:33:58.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.708+0000 7f20bb7fe640 10 client.15687 handle_command_reply: tid=0 2024-09-16T00:33:58.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.708+0000 7f20e467b740 1 client.15687 shutdown 2024-09-16T00:33:58.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.708+0000 7f20e467b740 1 -- 172.21.15.38:0/1006220576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf659acec0 msgr2=0x55bf659bb2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:58.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.708+0000 7f20e467b740 1 --2- 172.21.15.38:0/1006220576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf659acec0 0x55bf659bb2d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f20d4022dc0 tx=0x7f20d4025000 comp rx=0 tx=0).stop 2024-09-16T00:33:58.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.708+0000 7f20e467b740 1 -- 172.21.15.38:0/1006220576 shutdown_connections 2024-09-16T00:33:58.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.708+0000 7f20e467b740 1 --2- 172.21.15.38:0/1006220576 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55bf659bf550 0x55bf659c1a10 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.708+0000 7f20e467b740 1 --2- 172.21.15.38:0/1006220576 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf659acec0 0x55bf659bb2d0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.708+0000 7f20e467b740 1 --2- 172.21.15.38:0/1006220576 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf659a4980 0x55bf659ac930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.708+0000 7f20e467b740 1 --2- 172.21.15.38:0/1006220576 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf659a4060 0x55bf659ac3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.708+0000 7f20bb7fe640 0 client.15687 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:33:58.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.708+0000 7f20e467b740 1 -- 172.21.15.38:0/1006220576 >> 172.21.15.38:0/1006220576 conn(0x55bf659a05f0 msgr2=0x55bf659a6350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:58.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.709+0000 7f20e467b740 1 -- 172.21.15.38:0/1006220576 shutdown_connections 2024-09-16T00:33:58.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.709+0000 7f20e467b740 1 -- 172.21.15.38:0/1006220576 wait complete. 2024-09-16T00:33:58.711 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.709+0000 7f20e467b740 20 client.15687 trim_cache size 0 max 16384 2024-09-16T00:33:58.726 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.726+0000 7f20e467b740 1 Processor -- start 2024-09-16T00:33:58.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.726+0000 7f20e467b740 1 -- start start 2024-09-16T00:33:58.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.726+0000 7f20e467b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf659a4060 0x55bf659ab350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:58.727 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.726+0000 7f20e467b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf659a4980 0x55bf659bb0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:58.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.726+0000 7f20e467b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf659acec0 0x55bf659c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:58.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.726+0000 7f20e467b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc1af2c0 con 0x55bf659a4980 2024-09-16T00:33:58.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.726+0000 7f20e467b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc1acb90 con 0x55bf659acec0 2024-09-16T00:33:58.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.726+0000 7f20e467b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc1aaa40 con 0x55bf659a4060 2024-09-16T00:33:58.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.726+0000 7f20db7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf659a4980 0x55bf659bb0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:58.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.726+0000 7f20db7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf659a4980 0x55bf659bb0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57088/0 (socket says 172.21.15.38:57088) 2024-09-16T00:33:58.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.726+0000 7f20db7fe640 1 -- 172.21.15.38:0/4282340069 learned_addr learned my addr 172.21.15.38:0/4282340069 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:58.728 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.726+0000 7f20e083f640 1 --2- 172.21.15.38:0/4282340069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf659acec0 0x55bf659c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:58.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.727+0000 7f20e083f640 1 -- 172.21.15.38:0/4282340069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf659a4060 msgr2=0x55bf659ab350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:58.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.727+0000 7f20dbfff640 1 --2- 172.21.15.38:0/4282340069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf659a4060 0x55bf659ab350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:58.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.727+0000 7f20e083f640 1 --2- 172.21.15.38:0/4282340069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf659a4060 0x55bf659ab350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.727+0000 7f20e083f640 1 -- 172.21.15.38:0/4282340069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf659a4980 msgr2=0x55bf659bb0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:58.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.727+0000 7f20e083f640 1 --2- 172.21.15.38:0/4282340069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf659a4980 0x55bf659bb0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.727+0000 7f20e083f640 1 -- 172.21.15.38:0/4282340069 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bf659c2090 con 0x55bf659acec0 2024-09-16T00:33:58.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.727+0000 7f20db7fe640 1 --2- 172.21.15.38:0/4282340069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf659a4980 0x55bf659bb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:33:58.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.727+0000 7f20e083f640 1 --2- 172.21.15.38:0/4282340069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf659acec0 0x55bf659c72e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f20d4014be0 tx=0x7f20d400b530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b31b305aafbea69 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:58.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.727+0000 7f20c1ffb640 1 -- 172.21.15.38:0/4282340069 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d4004be0 con 0x55bf659acec0 2024-09-16T00:33:58.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.727+0000 7f20c1ffb640 1 -- 172.21.15.38:0/4282340069 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20d4043070 con 0x55bf659acec0 2024-09-16T00:33:58.729 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.727+0000 7f20c1ffb640 1 -- 172.21.15.38:0/4282340069 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d403e770 con 0x55bf659acec0 2024-09-16T00:33:58.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.728+0000 7f20e467b740 1 -- 172.21.15.38:0/4282340069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf659acec0 msgr2=0x55bf659c72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:58.730 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.728+0000 7f20e467b740 1 --2- 172.21.15.38:0/4282340069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf659acec0 0x55bf659c72e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f20d4014be0 tx=0x7f20d400b530 comp rx=0 tx=0).stop 2024-09-16T00:33:58.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.728+0000 7f20e467b740 1 -- 172.21.15.38:0/4282340069 shutdown_connections 2024-09-16T00:33:58.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.728+0000 7f20e467b740 1 --2- 172.21.15.38:0/4282340069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf659acec0 0x55bf659c72e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.728+0000 7f20e467b740 1 --2- 172.21.15.38:0/4282340069 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf659a4980 0x55bf659bb0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.728+0000 7f20e467b740 1 --2- 172.21.15.38:0/4282340069 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf659a4060 0x55bf659ab350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.728+0000 7f20e467b740 1 -- 172.21.15.38:0/4282340069 >> 172.21.15.38:0/4282340069 conn(0x55bf659a05f0 msgr2=0x55bf659a6350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:58.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.728+0000 7f20e467b740 1 -- 172.21.15.38:0/4282340069 shutdown_connections 2024-09-16T00:33:58.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.728+0000 7f20e467b740 1 -- 172.21.15.38:0/4282340069 wait complete. 2024-09-16T00:33:58.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.729+0000 7f20e467b740 1 -- start start 2024-09-16T00:33:58.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.729+0000 7f20e467b740 1 Processor -- start 2024-09-16T00:33:58.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.729+0000 7f20e467b740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf659a4060 0x55bf659b78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:58.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.729+0000 7f20e467b740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf659a4980 0x55bf659b4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:58.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.729+0000 7f20e467b740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf659acec0 0x55bf659b4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:58.731 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.730+0000 7f20e467b740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc1ab090 con 0x55bf659a4980 2024-09-16T00:33:58.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.730+0000 7f20e467b740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc1ac4c0 con 0x55bf659acec0 2024-09-16T00:33:58.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.730+0000 7f20e467b740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f20dc1ab720 con 0x55bf659a4060 2024-09-16T00:33:58.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.730+0000 7f20db7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf659a4980 0x55bf659b4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:58.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.730+0000 7f20db7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf659a4980 0x55bf659b4200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:57104/0 (socket says 172.21.15.38:57104) 2024-09-16T00:33:58.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.730+0000 7f20db7fe640 1 -- 172.21.15.38:0/2920853747 learned_addr learned my addr 172.21.15.38:0/2920853747 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:33:58.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.730+0000 7f20e467b740 10 client.0 _ll_register_callbacks cb 0x7f20d4022cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:33:58.732 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.730+0000 7f20e083f640 1 --2- 172.21.15.38:0/2920853747 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf659acec0 0x55bf659b4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:58.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.730+0000 7f20db7fe640 1 -- 172.21.15.38:0/2920853747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf659a4060 msgr2=0x55bf659b78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:58.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.730+0000 7f20dbfff640 1 --2- 172.21.15.38:0/2920853747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf659a4060 0x55bf659b78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:58.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.730+0000 7f20db7fe640 1 --2- 172.21.15.38:0/2920853747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf659a4060 0x55bf659b78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.730+0000 7f20db7fe640 1 -- 172.21.15.38:0/2920853747 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf659acec0 msgr2=0x55bf659b4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:33:58.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.730+0000 7f20db7fe640 1 --2- 172.21.15.38:0/2920853747 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf659acec0 0x55bf659b4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.733 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.730+0000 7f20db7fe640 1 -- 172.21.15.38:0/2920853747 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bf659b4f20 con 0x55bf659a4980 2024-09-16T00:33:58.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.730+0000 7f20e083f640 1 --2- 172.21.15.38:0/2920853747 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf659acec0 0x55bf659b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:33:58.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.730+0000 7f20dbfff640 1 --2- 172.21.15.38:0/2920853747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf659a4060 0x55bf659b78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:33:58.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.730+0000 7f20db7fe640 1 --2- 172.21.15.38:0/2920853747 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf659a4980 0x55bf659b4200 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f20d0009bb0 tx=0x7f20d0005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27591dde7a0c297a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:33:58.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.730+0000 7f20bb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:33:58.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.731+0000 7f20bb7fe640 1 -- 172.21.15.38:0/2920853747 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d0016040 con 0x55bf659a4980 2024-09-16T00:33:58.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.731+0000 7f20bb7fe640 1 -- 172.21.15.38:0/2920853747 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f20d0004820 con 0x55bf659a4980 2024-09-16T00:33:58.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.731+0000 7f20e467b740 1 -- 172.21.15.38:0/2920853747 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20d4043070 con 0x55bf659a4980 2024-09-16T00:33:58.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.731+0000 7f20bb7fe640 1 -- 172.21.15.38:0/2920853747 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f20d00055f0 con 0x55bf659a4980 2024-09-16T00:33:58.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.731+0000 7f20e467b740 10 client.15693 fetch_fsmap learned FSMap version 26 2024-09-16T00:33:58.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.731+0000 7f20e467b740 1 -- 172.21.15.38:0/2920853747 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55bf65a2b230 con 0x55bf659a4980 2024-09-16T00:33:58.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.731+0000 7f20bb7fe640 1 -- 172.21.15.38:0/2920853747 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f20d0029070 con 0x55bf659a4980 2024-09-16T00:33:58.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.732+0000 7f20bb7fe640 1 -- 172.21.15.38:0/2920853747 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f20d000f040 con 0x55bf659a4980 2024-09-16T00:33:58.734 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.732+0000 7f20e467b740 10 client.15693 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:33:58.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.732+0000 7f20e467b740 10 client.15693 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-16T00:33:58.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.732+0000 7f20e467b740 20 client.15693 populate_metadata read hostname 'smithi038' 2024-09-16T00:33:58.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.732+0000 7f20e467b740 1 --2- 172.21.15.38:0/2920853747 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55bf65a2b850 0x55bf65a2dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:33:58.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.732+0000 7f20e467b740 4 client.15693 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "6706a54b", "release": true, "await": true}] 2024-09-16T00:33:58.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.732+0000 7f20e467b740 1 -- 172.21.15.38:0/2920853747 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "quiesce db", "set_id": "6706a54b", "release": true, "await": true}) -- 0x7f20d4043070 con 0x55bf65a2b850 2024-09-16T00:33:58.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.732+0000 7f20dbfff640 1 --2- 172.21.15.38:0/2920853747 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55bf65a2b850 0x55bf65a2dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:33:58.735 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.733+0000 7f20bb7fe640 10 client.15693 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:33:58.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.733+0000 7f20dbfff640 1 --2- 172.21.15.38:0/2920853747 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55bf65a2b850 0x55bf65a2dc70 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8068f4fad3e7925 server_cookie=1ecc85480f5301b4 in_seq=0 out_seq=0 2024-09-16T00:33:58.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.735+0000 7f20bb7fe640 1 -- 172.21.15.38:0/2920853747 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+589 (crc 0 0 0) 0x7f20d4043070 con 0x55bf65a2b850 2024-09-16T00:33:58.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.735+0000 7f20bb7fe640 10 client.15693 handle_command_reply: tid=0 2024-09-16T00:33:58.736 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.735+0000 7f20e467b740 1 client.15693 shutdown 2024-09-16T00:33:58.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.735+0000 7f20e467b740 1 -- 172.21.15.38:0/2920853747 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf659a4980 msgr2=0x55bf659b4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:58.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.735+0000 7f20e467b740 1 --2- 172.21.15.38:0/2920853747 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf659a4980 0x55bf659b4200 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f20d0009bb0 tx=0x7f20d0005270 comp rx=0 tx=0).stop 2024-09-16T00:33:58.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.736+0000 7f20e467b740 1 -- 172.21.15.38:0/2920853747 shutdown_connections 2024-09-16T00:33:58.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.736+0000 7f20e467b740 1 --2- 172.21.15.38:0/2920853747 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55bf659acec0 0x55bf659b4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.736+0000 7f20e467b740 1 --2- 172.21.15.38:0/2920853747 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55bf65a2b850 0x55bf65a2dc70 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.736+0000 7f20e467b740 1 --2- 172.21.15.38:0/2920853747 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55bf659a4980 0x55bf659b4200 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.736+0000 7f20e467b740 1 --2- 172.21.15.38:0/2920853747 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55bf659a4060 0x55bf659b78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.736+0000 7f20bb7fe640 0 client.15693 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:33:58.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.736+0000 7f20e467b740 1 -- 172.21.15.38:0/2920853747 >> 172.21.15.38:0/2920853747 conn(0x55bf659a05f0 msgr2=0x55bf659a5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:58.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.736+0000 7f20e467b740 1 -- 172.21.15.38:0/2920853747 shutdown_connections 2024-09-16T00:33:58.737 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.736+0000 7f20e467b740 1 -- 172.21.15.38:0/2920853747 wait complete. 2024-09-16T00:33:58.738 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.736+0000 7f20e467b740 20 client.15693 trim_cache size 0 max 16384 2024-09-16T00:33:58.738 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-16T00:33:58.738 INFO:teuthology.orchestra.run.smithi038.stdout: "epoch": 26, 2024-09-16T00:33:58.738 INFO:teuthology.orchestra.run.smithi038.stdout: "leader": 24494, 2024-09-16T00:33:58.738 INFO:teuthology.orchestra.run.smithi038.stdout: "set_version": 4, 2024-09-16T00:33:58.738 INFO:teuthology.orchestra.run.smithi038.stdout: "sets": { 2024-09-16T00:33:58.738 INFO:teuthology.orchestra.run.smithi038.stdout: "6706a54b": { 2024-09-16T00:33:58.738 INFO:teuthology.orchestra.run.smithi038.stdout: "version": 4, 2024-09-16T00:33:58.738 INFO:teuthology.orchestra.run.smithi038.stdout: "age_ref": 0.0, 2024-09-16T00:33:58.738 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-16T00:33:58.739 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RELEASED", 2024-09-16T00:33:58.739 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-16T00:33:58.739 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:33:58.739 INFO:teuthology.orchestra.run.smithi038.stdout: "timeout": 90.0, 2024-09-16T00:33:58.739 INFO:teuthology.orchestra.run.smithi038.stdout: "expiration": 150.2, 2024-09-16T00:33:58.739 INFO:teuthology.orchestra.run.smithi038.stdout: "members": { 2024-09-16T00:33:58.739 INFO:teuthology.orchestra.run.smithi038.stdout: "file:/": { 2024-09-16T00:33:58.739 INFO:teuthology.orchestra.run.smithi038.stdout: "excluded": false, 2024-09-16T00:33:58.739 INFO:teuthology.orchestra.run.smithi038.stdout: "state": { 2024-09-16T00:33:58.739 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RELEASED", 2024-09-16T00:33:58.739 INFO:teuthology.orchestra.run.smithi038.stdout: "age": 0.0 2024-09-16T00:33:58.739 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:33:58.739 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:33:58.739 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:33:58.740 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:33:58.740 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:33:58.740 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-16T00:33:58.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.738+0000 7f20e22a2640 1 -- 172.21.15.38:0/1507140804 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f20c40a3240 msgr2=0x7f20c40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:58.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.738+0000 7f20e22a2640 1 --2- 172.21.15.38:0/1507140804 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f20c40a3240 0x7f20c40a5700 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f20d4005fd0 tx=0x7f20d40099c0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.738+0000 7f20e22a2640 1 -- 172.21.15.38:0/1507140804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20dc100e20 msgr2=0x7f20dc10e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:33:58.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.738+0000 7f20e22a2640 1 --2- 172.21.15.38:0/1507140804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20dc100e20 0x7f20dc10e4d0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f20cc02aa70 tx=0x7f20cc003c10 comp rx=0 tx=0).stop 2024-09-16T00:33:58.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.738+0000 7f20e22a2640 1 -- 172.21.15.38:0/1507140804 shutdown_connections 2024-09-16T00:33:58.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.738+0000 7f20e22a2640 1 --2- 172.21.15.38:0/1507140804 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f20c40a3240 0x7f20c40a5700 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.738+0000 7f20e22a2640 1 --2- 172.21.15.38:0/1507140804 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f20dc102ef0 0x7f20dc10ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.741 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.738+0000 7f20e22a2640 1 --2- 172.21.15.38:0/1507140804 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f20dc102570 0x7f20dc10ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.738+0000 7f20e22a2640 1 --2- 172.21.15.38:0/1507140804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20dc100e20 0x7f20dc10e4d0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:33:58.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.738+0000 7f20e22a2640 1 -- 172.21.15.38:0/1507140804 >> 172.21.15.38:0/1507140804 conn(0x7f20dc0fc7d0 msgr2=0x7f20dc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:33:58.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.738+0000 7f20e22a2640 1 -- 172.21.15.38:0/1507140804 shutdown_connections 2024-09-16T00:33:58.742 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:33:58.738+0000 7f20e22a2640 1 -- 172.21.15.38:0/1507140804 wait complete. 2024-09-16T00:33:59.275 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:33:58 smithi038 ceph-mon[30945]: pgmap v400: 129 pgs: 129 active+clean; 662 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-16T00:33:59.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:33:58 smithi073 ceph-mon[35254]: pgmap v400: 129 pgs: 129 active+clean; 662 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-16T00:33:59.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:33:58 smithi122 ceph-mon[35412]: pgmap v400: 129 pgs: 129 active+clean; 662 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-16T00:33:59.477 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '6706a54b', total seconds elapsed: 35.0 2024-09-16T00:34:01.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:00 smithi073 ceph-mon[35254]: pgmap v401: 129 pgs: 129 active+clean; 662 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-16T00:34:01.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:00 smithi038 ceph-mon[30945]: pgmap v401: 129 pgs: 129 active+clean; 662 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-16T00:34:01.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:00 smithi122 ceph-mon[35412]: pgmap v401: 129 pgs: 129 active+clean; 662 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-16T00:34:03.269 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:02 smithi122 ceph-mon[35412]: pgmap v402: 129 pgs: 129 active+clean; 662 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-16T00:34:03.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:02 smithi073 ceph-mon[35254]: pgmap v402: 129 pgs: 129 active+clean; 662 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-16T00:34:03.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:02 smithi038 ceph-mon[30945]: pgmap v402: 129 pgs: 129 active+clean; 662 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-16T00:34:03.515 INFO:tasks.workunit.client.0.smithi038.stderr:Updating files: 94% (12214/12889) Updating files: 95% (12245/12889) Updating files: 96% (12374/12889) Updating files: 97% (12503/12889) Updating files: 98% (12632/12889) Updating files: 99% (12761/12889) Updating files: 100% (12889/12889) Updating files: 100% (12889/12889), done. 2024-09-16T00:34:04.279 INFO:tasks.workunit.client.0.smithi038.stderr:Note: switching to 'ad35db06e8924912f01e32b35576297c0f1352b4'. 2024-09-16T00:34:04.279 INFO:tasks.workunit.client.0.smithi038.stderr: 2024-09-16T00:34:04.279 INFO:tasks.workunit.client.0.smithi038.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-16T00:34:04.279 INFO:tasks.workunit.client.0.smithi038.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-16T00:34:04.279 INFO:tasks.workunit.client.0.smithi038.stderr:state without impacting any branches by switching back to a branch. 2024-09-16T00:34:04.279 INFO:tasks.workunit.client.0.smithi038.stderr: 2024-09-16T00:34:04.280 INFO:tasks.workunit.client.0.smithi038.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-16T00:34:04.280 INFO:tasks.workunit.client.0.smithi038.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-16T00:34:04.280 INFO:tasks.workunit.client.0.smithi038.stderr: 2024-09-16T00:34:04.280 INFO:tasks.workunit.client.0.smithi038.stderr: git switch -c 2024-09-16T00:34:04.280 INFO:tasks.workunit.client.0.smithi038.stderr: 2024-09-16T00:34:04.280 INFO:tasks.workunit.client.0.smithi038.stderr:Or undo this operation with: 2024-09-16T00:34:04.280 INFO:tasks.workunit.client.0.smithi038.stderr: 2024-09-16T00:34:04.280 INFO:tasks.workunit.client.0.smithi038.stderr: git switch - 2024-09-16T00:34:04.281 INFO:tasks.workunit.client.0.smithi038.stderr: 2024-09-16T00:34:04.281 INFO:tasks.workunit.client.0.smithi038.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-16T00:34:04.281 INFO:tasks.workunit.client.0.smithi038.stderr: 2024-09-16T00:34:04.281 INFO:tasks.workunit.client.0.smithi038.stderr:HEAD is now at ad35db06e89 Merge pull request #59794 from zdover23/wip-doc-2024-09-15-backport-59613-to-squid 2024-09-16T00:34:04.285 DEBUG:teuthology.orchestra.run.smithi038:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-16T00:34:04.319 INFO:tasks.workunit.client.0.smithi038.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-16T00:34:04.321 INFO:tasks.workunit.client.0.smithi038.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-16T00:34:04.321 INFO:tasks.workunit.client.0.smithi038.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-16T00:34:05.033 INFO:tasks.workunit.client.0.smithi038.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-16T00:34:05.113 INFO:tasks.workunit.client.0.smithi038.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-16T00:34:05.156 INFO:tasks.workunit.client.0.smithi038.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-16T00:34:05.157 INFO:tasks.workunit.client.0.smithi038.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-16T00:34:05.157 INFO:tasks.workunit.client.0.smithi038.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-16T00:34:05.198 INFO:tasks.workunit.client.0.smithi038.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-16T00:34:05.201 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-16T00:34:05.201 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-16T00:34:05.258 INFO:tasks.workunit:Running workunits matching direct_io on client.0... 2024-09-16T00:34:05.283 INFO:tasks.workunit:Running workunit direct_io/big.sh... 2024-09-16T00:34:05.284 DEBUG:teuthology.orchestra.run.smithi038:workunit test direct_io/big.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ad35db06e8924912f01e32b35576297c0f1352b4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/big.sh 2024-09-16T00:34:05.285 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:04 smithi038 ceph-mon[30945]: pgmap v403: 129 pgs: 129 active+clean; 662 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-16T00:34:05.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:04 smithi073 ceph-mon[35254]: pgmap v403: 129 pgs: 129 active+clean; 662 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-16T00:34:05.307 INFO:tasks.workunit.client.0.smithi038.stdout:test large (16MB) dio write 2024-09-16T00:34:05.308 INFO:tasks.workunit.client.0.smithi038.stderr:+ echo 'test large (16MB) dio write' 2024-09-16T00:34:05.308 INFO:tasks.workunit.client.0.smithi038.stderr:+ dd if=/dev/zero of=foo.big bs=16M count=1 oflag=direct 2024-09-16T00:34:05.390 INFO:tasks.workunit.client.0.smithi038.stderr:1+0 records in 2024-09-16T00:34:05.390 INFO:tasks.workunit.client.0.smithi038.stderr:1+0 records out 2024-09-16T00:34:05.390 INFO:tasks.workunit.client.0.smithi038.stderr:16777216 bytes (17 MB, 16 MiB) copied, 0.0804766 s, 208 MB/s 2024-09-16T00:34:05.391 INFO:tasks.workunit.client.0.smithi038.stdout:OK 2024-09-16T00:34:05.391 INFO:tasks.workunit.client.0.smithi038.stderr:+ echo OK 2024-09-16T00:34:05.392 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-16T00:34:05.392 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-16T00:34:05.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:04 smithi122 ceph-mon[35412]: pgmap v403: 129 pgs: 129 active+clean; 662 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-16T00:34:05.469 INFO:tasks.workunit:Running workunit direct_io/direct_io_test... 2024-09-16T00:34:05.469 DEBUG:teuthology.orchestra.run.smithi038:workunit test direct_io/direct_io_test> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ad35db06e8924912f01e32b35576297c0f1352b4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/direct_io_test 2024-09-16T00:34:07.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:06 smithi073 ceph-mon[35254]: pgmap v404: 129 pgs: 129 active+clean; 4.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 KiB/s wr, 7 op/s 2024-09-16T00:34:07.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:06 smithi038 ceph-mon[30945]: pgmap v404: 129 pgs: 129 active+clean; 4.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 KiB/s wr, 7 op/s 2024-09-16T00:34:07.470 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:06 smithi122 ceph-mon[35412]: pgmap v404: 129 pgs: 129 active+clean; 4.6 MiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 341 KiB/s wr, 7 op/s 2024-09-16T00:34:09.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:08 smithi073 ceph-mon[35254]: pgmap v405: 129 pgs: 129 active+clean; 13 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1024 KiB/s wr, 5 op/s 2024-09-16T00:34:09.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:08 smithi038 ceph-mon[30945]: pgmap v405: 129 pgs: 129 active+clean; 13 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1024 KiB/s wr, 5 op/s 2024-09-16T00:34:09.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:08 smithi122 ceph-mon[35412]: pgmap v405: 129 pgs: 129 active+clean; 13 MiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1024 KiB/s wr, 5 op/s 2024-09-16T00:34:11.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:11 smithi073 ceph-mon[35254]: pgmap v406: 129 pgs: 129 active+clean; 13 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 1.0 MiB/s wr, 144 op/s 2024-09-16T00:34:11.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:11 smithi038 ceph-mon[30945]: pgmap v406: 129 pgs: 129 active+clean; 13 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 1.0 MiB/s wr, 144 op/s 2024-09-16T00:34:11.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:11 smithi122 ceph-mon[35412]: pgmap v406: 129 pgs: 129 active+clean; 13 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 1.0 MiB/s wr, 144 op/s 2024-09-16T00:34:13.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:13 smithi073 ceph-mon[35254]: pgmap v407: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 1.4 MiB/s wr, 143 op/s 2024-09-16T00:34:13.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:13 smithi038 ceph-mon[30945]: pgmap v407: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 1.4 MiB/s wr, 143 op/s 2024-09-16T00:34:13.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:13 smithi122 ceph-mon[35412]: pgmap v407: 129 pgs: 129 active+clean; 1.1 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 550 KiB/s rd, 1.4 MiB/s wr, 143 op/s 2024-09-16T00:34:15.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:15 smithi073 ceph-mon[35254]: pgmap v408: 129 pgs: 129 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.4 MiB/s wr, 794 op/s 2024-09-16T00:34:15.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:15 smithi038 ceph-mon[30945]: pgmap v408: 129 pgs: 129 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.4 MiB/s wr, 794 op/s 2024-09-16T00:34:15.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:15 smithi122 ceph-mon[35412]: pgmap v408: 129 pgs: 129 active+clean; 1.1 MiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.4 MiB/s wr, 794 op/s 2024-09-16T00:34:17.000 INFO:tasks.workunit.client.0.smithi038.stdout:.......... 2024-09-16T00:34:17.001 INFO:tasks.workunit.client.0.smithi038.stdout:do_o_direct_reads: SUCCESS 2024-09-16T00:34:17.004 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-16T00:34:17.004 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-16T00:34:17.040 INFO:tasks.workunit:Running workunit direct_io/misc.sh... 2024-09-16T00:34:17.040 DEBUG:teuthology.orchestra.run.smithi038:workunit test direct_io/misc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ad35db06e8924912f01e32b35576297c0f1352b4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/misc.sh 2024-09-16T00:34:17.160 INFO:tasks.workunit.client.0.smithi038.stderr:+ echo 'test read from hole' 2024-09-16T00:34:17.161 INFO:tasks.workunit.client.0.smithi038.stdout:test read from hole 2024-09-16T00:34:17.161 INFO:tasks.workunit.client.0.smithi038.stderr:+ dd if=/dev/zero of=dd3 bs=1 seek=1048576 count=0 2024-09-16T00:34:17.162 INFO:tasks.workunit.client.0.smithi038.stderr:0+0 records in 2024-09-16T00:34:17.162 INFO:tasks.workunit.client.0.smithi038.stderr:0+0 records out 2024-09-16T00:34:17.162 INFO:tasks.workunit.client.0.smithi038.stderr:0 bytes copied, 3.5935e-05 s, 0.0 kB/s 2024-09-16T00:34:17.162 INFO:tasks.workunit.client.0.smithi038.stderr:+ dd if=dd3 of=/tmp/ddout1 skip=8 bs=512 count=2 iflag=direct 2024-09-16T00:34:17.164 INFO:tasks.workunit.client.0.smithi038.stderr:2+0 records in 2024-09-16T00:34:17.164 INFO:tasks.workunit.client.0.smithi038.stderr:2+0 records out 2024-09-16T00:34:17.165 INFO:tasks.workunit.client.0.smithi038.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 0.00137828 s, 743 kB/s 2024-09-16T00:34:17.165 INFO:tasks.workunit.client.0.smithi038.stderr:+ dd if=/dev/zero of=/tmp/dd3 bs=512 count=2 2024-09-16T00:34:17.165 INFO:tasks.workunit.client.0.smithi038.stderr:2+0 records in 2024-09-16T00:34:17.165 INFO:tasks.workunit.client.0.smithi038.stderr:2+0 records out 2024-09-16T00:34:17.165 INFO:tasks.workunit.client.0.smithi038.stderr:1024 bytes (1.0 kB, 1.0 KiB) copied, 4.169e-05 s, 24.6 MB/s 2024-09-16T00:34:17.165 INFO:tasks.workunit.client.0.smithi038.stderr:+ cmp /tmp/dd3 /tmp/ddout1 2024-09-16T00:34:17.166 INFO:tasks.workunit.client.0.smithi038.stdout:other thing 2024-09-16T00:34:17.166 INFO:tasks.workunit.client.0.smithi038.stderr:+ echo 'other thing' 2024-09-16T00:34:17.166 INFO:tasks.workunit.client.0.smithi038.stderr:+ dd if=/dev/urandom of=/tmp/dd10 bs=500 count=1 2024-09-16T00:34:17.167 INFO:tasks.workunit.client.0.smithi038.stderr:1+0 records in 2024-09-16T00:34:17.167 INFO:tasks.workunit.client.0.smithi038.stderr:1+0 records out 2024-09-16T00:34:17.167 INFO:tasks.workunit.client.0.smithi038.stderr:500 bytes copied, 4.0735e-05 s, 12.3 MB/s 2024-09-16T00:34:17.167 INFO:tasks.workunit.client.0.smithi038.stderr:+ dd if=/tmp/dd10 of=dd10 bs=512 seek=8388 count=1 2024-09-16T00:34:17.172 INFO:tasks.workunit.client.0.smithi038.stderr:0+1 records in 2024-09-16T00:34:17.172 INFO:tasks.workunit.client.0.smithi038.stderr:0+1 records out 2024-09-16T00:34:17.172 INFO:tasks.workunit.client.0.smithi038.stderr:500 bytes copied, 0.00390159 s, 128 kB/s 2024-09-16T00:34:17.173 INFO:tasks.workunit.client.0.smithi038.stderr:+ dd if=dd10 of=/tmp/dd10out bs=512 skip=8388 count=1 iflag=direct 2024-09-16T00:34:17.178 INFO:tasks.workunit.client.0.smithi038.stderr:0+1 records in 2024-09-16T00:34:17.178 INFO:tasks.workunit.client.0.smithi038.stderr:0+1 records out 2024-09-16T00:34:17.179 INFO:tasks.workunit.client.0.smithi038.stderr:500 bytes copied, 0.00534285 s, 93.6 kB/s 2024-09-16T00:34:17.179 INFO:tasks.workunit.client.0.smithi038.stderr:+ cmp /tmp/dd10 /tmp/dd10out 2024-09-16T00:34:17.179 INFO:tasks.workunit.client.0.smithi038.stdout:OK 2024-09-16T00:34:17.180 INFO:tasks.workunit.client.0.smithi038.stderr:+ echo OK 2024-09-16T00:34:17.180 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-16T00:34:17.180 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-16T00:34:17.219 INFO:tasks.workunit:Running workunit direct_io/test_short_dio_read... 2024-09-16T00:34:17.220 DEBUG:teuthology.orchestra.run.smithi038:workunit test direct_io/test_short_dio_read> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ad35db06e8924912f01e32b35576297c0f1352b4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/test_short_dio_read 2024-09-16T00:34:17.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:17 smithi073 ceph-mon[35254]: pgmap v409: 129 pgs: 129 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.4 MiB/s wr, 795 op/s 2024-09-16T00:34:17.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:17 smithi038 ceph-mon[30945]: pgmap v409: 129 pgs: 129 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.4 MiB/s wr, 795 op/s 2024-09-16T00:34:17.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:17 smithi122 ceph-mon[35412]: pgmap v409: 129 pgs: 129 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.4 MiB/s wr, 795 op/s 2024-09-16T00:34:19.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:19 smithi073 ceph-mon[35254]: pgmap v410: 129 pgs: 129 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.0 MiB/s wr, 793 op/s 2024-09-16T00:34:19.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:19 smithi038 ceph-mon[30945]: pgmap v410: 129 pgs: 129 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.0 MiB/s wr, 793 op/s 2024-09-16T00:34:19.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:19 smithi122 ceph-mon[35412]: pgmap v410: 129 pgs: 129 active+clean; 1.1 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.0 MiB/s wr, 793 op/s 2024-09-16T00:34:21.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:21 smithi073 ceph-mon[35254]: pgmap v411: 129 pgs: 129 active+clean; 1.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 382 KiB/s wr, 1.29k op/s 2024-09-16T00:34:21.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:21 smithi038 ceph-mon[30945]: pgmap v411: 129 pgs: 129 active+clean; 1.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 382 KiB/s wr, 1.29k op/s 2024-09-16T00:34:21.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:21 smithi122 ceph-mon[35412]: pgmap v411: 129 pgs: 129 active+clean; 1.2 MiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 MiB/s rd, 382 KiB/s wr, 1.29k op/s 2024-09-16T00:34:22.178 INFO:tasks.workunit.client.0.smithi038.stdout:writing first 3 bytes of 10k file 2024-09-16T00:34:22.178 INFO:tasks.workunit.client.0.smithi038.stdout:reading O_DIRECT 2024-09-16T00:34:22.178 INFO:tasks.workunit.client.0.smithi038.stdout:got 10000 2024-09-16T00:34:22.178 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-16T00:34:22.178 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-16T00:34:22.217 INFO:tasks.workunit:Running workunit direct_io/test_sync_io... 2024-09-16T00:34:22.218 DEBUG:teuthology.orchestra.run.smithi038:workunit test direct_io/test_sync_io> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=ad35db06e8924912f01e32b35576297c0f1352b4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io/test_sync_io 2024-09-16T00:34:23.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:23 smithi073 ceph-mon[35254]: pgmap v412: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 347 KiB/s wr, 1.15k op/s 2024-09-16T00:34:23.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:23 smithi038 ceph-mon[30945]: pgmap v412: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 347 KiB/s wr, 1.15k op/s 2024-09-16T00:34:23.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:23 smithi122 ceph-mon[35412]: pgmap v412: 129 pgs: 129 active+clean; 1.2 MiB data, 373 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 347 KiB/s wr, 1.15k op/s 2024-09-16T00:34:25.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:25 smithi073 ceph-mon[35254]: pgmap v413: 129 pgs: 129 active+clean; 732 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.7 KiB/s wr, 1.15k op/s 2024-09-16T00:34:25.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:25 smithi038 ceph-mon[30945]: pgmap v413: 129 pgs: 129 active+clean; 732 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.7 KiB/s wr, 1.15k op/s 2024-09-16T00:34:25.468 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:25 smithi122 ceph-mon[35412]: pgmap v413: 129 pgs: 129 active+clean; 732 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.7 KiB/s wr, 1.15k op/s 2024-09-16T00:34:26.319 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:34:26.595 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:34:27.099 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.070+0000 7f3941f9e640 1 -- 172.21.15.38:0/3678757239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f393c102df0 msgr2=0x7f393c1031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:27.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.070+0000 7f3941f9e640 1 --2- 172.21.15.38:0/3678757239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f393c102df0 0x7f393c1031d0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f392c009900 tx=0x7f392c02a410 comp rx=0 tx=0).stop 2024-09-16T00:34:27.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.070+0000 7f3941f9e640 1 -- 172.21.15.38:0/3678757239 shutdown_connections 2024-09-16T00:34:27.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.070+0000 7f3941f9e640 1 --2- 172.21.15.38:0/3678757239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f393c10c2f0 0x7f393c10e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.070+0000 7f3941f9e640 1 --2- 172.21.15.38:0/3678757239 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f393c1037a0 0x7f393c10bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.070+0000 7f3941f9e640 1 --2- 172.21.15.38:0/3678757239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f393c102df0 0x7f393c1031d0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.070+0000 7f3941f9e640 1 -- 172.21.15.38:0/3678757239 >> 172.21.15.38:0/3678757239 conn(0x7f393c0fc7d0 msgr2=0x7f393c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:27.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.070+0000 7f3941f9e640 1 -- 172.21.15.38:0/3678757239 shutdown_connections 2024-09-16T00:34:27.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.070+0000 7f3941f9e640 1 -- 172.21.15.38:0/3678757239 wait complete. 2024-09-16T00:34:27.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.071+0000 7f3941f9e640 1 Processor -- start 2024-09-16T00:34:27.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.071+0000 7f3941f9e640 1 -- start start 2024-09-16T00:34:27.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.071+0000 7f3941f9e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f393c102df0 0x7f393c198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.071+0000 7f3941f9e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f393c1037a0 0x7f393c198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.071+0000 7f3941f9e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f393c10c2f0 0x7f393c19cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.071+0000 7f3941f9e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f393c1120f0 con 0x7f393c102df0 2024-09-16T00:34:27.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.071+0000 7f3941f9e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f393c111f70 con 0x7f393c1037a0 2024-09-16T00:34:27.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.071+0000 7f3941f9e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f393c112270 con 0x7f393c10c2f0 2024-09-16T00:34:27.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.071+0000 7f393b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f393c102df0 0x7f393c198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.071+0000 7f393b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f393c102df0 0x7f393c198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51162/0 (socket says 172.21.15.38:51162) 2024-09-16T00:34:27.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.071+0000 7f393b7fe640 1 -- 172.21.15.38:0/1012773774 learned_addr learned my addr 172.21.15.38:0/1012773774 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:34:27.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.072+0000 7f3933fff640 1 --2- 172.21.15.38:0/1012773774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f393c1037a0 0x7f393c198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.101 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.072+0000 7f393bfff640 1 --2- 172.21.15.38:0/1012773774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f393c10c2f0 0x7f393c19cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.072+0000 7f393b7fe640 1 -- 172.21.15.38:0/1012773774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f393c10c2f0 msgr2=0x7f393c19cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:27.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.072+0000 7f393b7fe640 1 --2- 172.21.15.38:0/1012773774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f393c10c2f0 0x7f393c19cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.072+0000 7f393b7fe640 1 -- 172.21.15.38:0/1012773774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f393c1037a0 msgr2=0x7f393c198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:27.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.072+0000 7f393b7fe640 1 --2- 172.21.15.38:0/1012773774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f393c1037a0 0x7f393c198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.072+0000 7f393b7fe640 1 -- 172.21.15.38:0/1012773774 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f393c19d5d0 con 0x7f393c102df0 2024-09-16T00:34:27.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.072+0000 7f3933fff640 1 --2- 172.21.15.38:0/1012773774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f393c1037a0 0x7f393c198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:34:27.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.072+0000 7f393b7fe640 1 --2- 172.21.15.38:0/1012773774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f393c102df0 0x7f393c198620 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f392c006010 tx=0x7f392c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:27.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.072+0000 7f39397fa640 1 -- 172.21.15.38:0/1012773774 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c03f030 con 0x7f393c102df0 2024-09-16T00:34:27.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.072+0000 7f39397fa640 1 -- 172.21.15.38:0/1012773774 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f392c003ce0 con 0x7f393c102df0 2024-09-16T00:34:27.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.072+0000 7f3941f9e640 1 -- 172.21.15.38:0/1012773774 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f393c19d860 con 0x7f393c102df0 2024-09-16T00:34:27.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.072+0000 7f39397fa640 1 -- 172.21.15.38:0/1012773774 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c035070 con 0x7f393c102df0 2024-09-16T00:34:27.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.072+0000 7f3941f9e640 1 -- 172.21.15.38:0/1012773774 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f393c1a51a0 con 0x7f393c102df0 2024-09-16T00:34:27.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.075+0000 7f3944377740 1 Processor -- start 2024-09-16T00:34:27.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.075+0000 7f39397fa640 1 -- 172.21.15.38:0/1012773774 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f392c042030 con 0x7f393c102df0 2024-09-16T00:34:27.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.075+0000 7f3944377740 1 -- start start 2024-09-16T00:34:27.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.076+0000 7f3944377740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642592a0380 0x5642592a0760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.076+0000 7f3944377740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5642592a0ca0 0x5642592af550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.076+0000 7f3944377740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5642592afa90 0x5642592b1e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.076+0000 7f3944377740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5642592062e0 con 0x5642592a0ca0 2024-09-16T00:34:27.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.076+0000 7f3944377740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5642591adce0 con 0x5642592afa90 2024-09-16T00:34:27.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.076+0000 7f3944377740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5642592022a0 con 0x5642592a0380 2024-09-16T00:34:27.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.076+0000 7f393b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5642592a0ca0 0x5642592af550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.076+0000 7f393b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5642592a0ca0 0x5642592af550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51174/0 (socket says 172.21.15.38:51174) 2024-09-16T00:34:27.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.076+0000 7f393b7fe640 1 -- 172.21.15.38:0/2696167239 learned_addr learned my addr 172.21.15.38:0/2696167239 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:34:27.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.076+0000 7f393bfff640 1 --2- 172.21.15.38:0/2696167239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642592a0380 0x5642592a0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.076+0000 7f3933fff640 1 --2- 172.21.15.38:0/2696167239 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5642592afa90 0x5642592b1e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.076+0000 7f393b7fe640 1 -- 172.21.15.38:0/2696167239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642592a0380 msgr2=0x5642592a0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:27.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.076+0000 7f393b7fe640 1 --2- 172.21.15.38:0/2696167239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642592a0380 0x5642592a0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.076+0000 7f393b7fe640 1 -- 172.21.15.38:0/2696167239 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5642592afa90 msgr2=0x5642592b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:27.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.076+0000 7f39397fa640 1 --2- 172.21.15.38:0/1012773774 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f39280a3240 0x7f39280a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.076+0000 7f393b7fe640 1 --2- 172.21.15.38:0/2696167239 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5642592afa90 0x5642592b1e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.076+0000 7f393b7fe640 1 -- 172.21.15.38:0/2696167239 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564259192440 con 0x5642592a0ca0 2024-09-16T00:34:27.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.076+0000 7f393bfff640 1 --2- 172.21.15.38:0/1012773774 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f39280a3240 0x7f39280a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.077+0000 7f39397fa640 1 -- 172.21.15.38:0/1012773774 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f392c0f0360 con 0x7f393c102df0 2024-09-16T00:34:27.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.077+0000 7f393b7fe640 1 --2- 172.21.15.38:0/2696167239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5642592a0ca0 0x5642592af550 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f392c0bdc50 tx=0x7f392c0b8a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33e3789a6678a4af server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:27.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.077+0000 7f3931ffb640 1 -- 172.21.15.38:0/2696167239 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c049020 con 0x5642592a0ca0 2024-09-16T00:34:27.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.077+0000 7f3931ffb640 1 -- 172.21.15.38:0/2696167239 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f392c0bda90 con 0x5642592a0ca0 2024-09-16T00:34:27.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.077+0000 7f3931ffb640 1 -- 172.21.15.38:0/2696167239 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c0bc530 con 0x5642592a0ca0 2024-09-16T00:34:27.104 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.078+0000 7f3944377740 1 -- 172.21.15.38:0/2696167239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5642592a0ca0 msgr2=0x5642592af550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:27.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.078+0000 7f3944377740 1 --2- 172.21.15.38:0/2696167239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5642592a0ca0 0x5642592af550 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f392c0bdc50 tx=0x7f392c0b8a00 comp rx=0 tx=0).stop 2024-09-16T00:34:27.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.078+0000 7f393bfff640 1 --2- 172.21.15.38:0/1012773774 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f39280a3240 0x7f39280a5700 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3934007e50 tx=0x7f3934007600 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:27.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.078+0000 7f3944377740 1 -- 172.21.15.38:0/2696167239 shutdown_connections 2024-09-16T00:34:27.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.078+0000 7f3944377740 1 --2- 172.21.15.38:0/2696167239 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5642592afa90 0x5642592b1e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.078+0000 7f3944377740 1 --2- 172.21.15.38:0/2696167239 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5642592a0ca0 0x5642592af550 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.078+0000 7f3944377740 1 --2- 172.21.15.38:0/2696167239 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642592a0380 0x5642592a0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.078+0000 7f3944377740 1 -- 172.21.15.38:0/2696167239 >> 172.21.15.38:0/2696167239 conn(0x56425929ca30 msgr2=0x56425929ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:27.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.078+0000 7f3944377740 1 -- 172.21.15.38:0/2696167239 shutdown_connections 2024-09-16T00:34:27.105 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.078+0000 7f3944377740 1 -- 172.21.15.38:0/2696167239 wait complete. 2024-09-16T00:34:27.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.078+0000 7f3944377740 1 -- start start 2024-09-16T00:34:27.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.078+0000 7f3944377740 1 Processor -- start 2024-09-16T00:34:27.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.079+0000 7f3944377740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642592a0380 0x5642592af400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.079+0000 7f3944377740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5642592a0ca0 0x5642592ac700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.079+0000 7f3944377740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5642592afa90 0x5642592acd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.079+0000 7f3944377740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5642592b2420 con 0x5642592a0ca0 2024-09-16T00:34:27.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.079+0000 7f3944377740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5642591c1050 con 0x5642592afa90 2024-09-16T00:34:27.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.079+0000 7f3944377740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5642592b25a0 con 0x5642592a0380 2024-09-16T00:34:27.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.079+0000 7f3944377740 10 client.0 _ll_register_callbacks cb 0x564259201160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:27.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.079+0000 7f3933fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5642592a0ca0 0x5642592ac700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.079+0000 7f3933fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5642592a0ca0 0x5642592ac700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51188/0 (socket says 172.21.15.38:51188) 2024-09-16T00:34:27.106 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.079+0000 7f3933fff640 1 -- 172.21.15.38:0/3780075749 learned_addr learned my addr 172.21.15.38:0/3780075749 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:34:27.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.079+0000 7f393bfff640 1 --2- 172.21.15.38:0/3780075749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5642592afa90 0x5642592acd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.079+0000 7f393b7fe640 1 --2- 172.21.15.38:0/3780075749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642592a0380 0x5642592af400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.079+0000 7f3933fff640 1 -- 172.21.15.38:0/3780075749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642592a0380 msgr2=0x5642592af400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:27.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.079+0000 7f3933fff640 1 --2- 172.21.15.38:0/3780075749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642592a0380 0x5642592af400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.079+0000 7f3933fff640 1 -- 172.21.15.38:0/3780075749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5642592afa90 msgr2=0x5642592acd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:27.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.079+0000 7f3933fff640 1 --2- 172.21.15.38:0/3780075749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5642592afa90 0x5642592acd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.079+0000 7f3933fff640 1 -- 172.21.15.38:0/3780075749 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564258f773c0 con 0x5642592a0ca0 2024-09-16T00:34:27.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.080+0000 7f3933fff640 1 --2- 172.21.15.38:0/3780075749 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5642592a0ca0 0x5642592ac700 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f393c0697d0 tx=0x7f3924012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78484acfc32fd3c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:27.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.080+0000 7f391affd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:34:27.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.080+0000 7f391affd640 1 -- 172.21.15.38:0/3780075749 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3924004230 con 0x5642592a0ca0 2024-09-16T00:34:27.107 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.080+0000 7f391affd640 1 -- 172.21.15.38:0/3780075749 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f39240043d0 con 0x5642592a0ca0 2024-09-16T00:34:27.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.080+0000 7f3944377740 1 -- 172.21.15.38:0/3780075749 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564259213850 con 0x5642592a0ca0 2024-09-16T00:34:27.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.080+0000 7f391affd640 1 -- 172.21.15.38:0/3780075749 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3924010760 con 0x5642592a0ca0 2024-09-16T00:34:27.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.080+0000 7f3944377740 10 client.15705 fetch_fsmap learned FSMap version 26 2024-09-16T00:34:27.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.080+0000 7f3944377740 1 -- 172.21.15.38:0/3780075749 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5642592bde10 con 0x5642592a0ca0 2024-09-16T00:34:27.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.080+0000 7f391affd640 1 -- 172.21.15.38:0/3780075749 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x564259213850 con 0x5642592a0ca0 2024-09-16T00:34:27.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.080+0000 7f391affd640 1 -- 172.21.15.38:0/3780075749 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f3924003c60 con 0x5642592a0ca0 2024-09-16T00:34:27.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.081+0000 7f3944377740 10 client.15705 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:34:27.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.081+0000 7f3944377740 10 client.15705 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:34:27.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.081+0000 7f3944377740 20 client.15705 populate_metadata read hostname 'smithi038' 2024-09-16T00:34:27.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.081+0000 7f3944377740 1 --2- 172.21.15.38:0/3780075749 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5642592bf320 0x5642592c1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.081+0000 7f3944377740 4 client.15705 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:34:27.108 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.081+0000 7f3944377740 1 -- 172.21.15.38:0/3780075749 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56425920dd60 con 0x5642592bf320 2024-09-16T00:34:27.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.081+0000 7f393b7fe640 1 --2- 172.21.15.38:0/3780075749 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5642592bf320 0x5642592c1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.081+0000 7f393b7fe640 1 --2- 172.21.15.38:0/3780075749 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5642592bf320 0x5642592c1710 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=681124a0dab46ae2 server_cookie=435e9cd3639a99b1 in_seq=0 out_seq=0 2024-09-16T00:34:27.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.081+0000 7f391affd640 10 client.15705 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:34:27.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.082+0000 7f391affd640 1 -- 172.21.15.38:0/3780075749 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56425920dd60 con 0x5642592bf320 2024-09-16T00:34:27.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.082+0000 7f391affd640 10 client.15705 handle_command_reply: tid=0 2024-09-16T00:34:27.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.082+0000 7f3944377740 1 client.15705 shutdown 2024-09-16T00:34:27.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.082+0000 7f3944377740 1 -- 172.21.15.38:0/3780075749 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5642592a0ca0 msgr2=0x5642592ac700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:27.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.082+0000 7f3944377740 1 --2- 172.21.15.38:0/3780075749 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5642592a0ca0 0x5642592ac700 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f393c0697d0 tx=0x7f3924012820 comp rx=0 tx=0).stop 2024-09-16T00:34:27.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.082+0000 7f3944377740 1 -- 172.21.15.38:0/3780075749 shutdown_connections 2024-09-16T00:34:27.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.082+0000 7f3944377740 1 --2- 172.21.15.38:0/3780075749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5642592afa90 0x5642592acd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.082+0000 7f3944377740 1 --2- 172.21.15.38:0/3780075749 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5642592bf320 0x5642592c1710 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.109 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.082+0000 7f3944377740 1 --2- 172.21.15.38:0/3780075749 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5642592a0ca0 0x5642592ac700 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.082+0000 7f3944377740 1 --2- 172.21.15.38:0/3780075749 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642592a0380 0x5642592af400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.082+0000 7f3944377740 1 -- 172.21.15.38:0/3780075749 >> 172.21.15.38:0/3780075749 conn(0x56425929ca30 msgr2=0x5642592a2790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:27.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.082+0000 7f391affd640 0 client.15705 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:34:27.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.082+0000 7f3944377740 1 -- 172.21.15.38:0/3780075749 shutdown_connections 2024-09-16T00:34:27.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.082+0000 7f3944377740 1 -- 172.21.15.38:0/3780075749 wait complete. 2024-09-16T00:34:27.110 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.082+0000 7f3944377740 20 client.15705 trim_cache size 0 max 16384 2024-09-16T00:34:27.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.100+0000 7f3944377740 1 Processor -- start 2024-09-16T00:34:27.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.100+0000 7f3944377740 1 -- start start 2024-09-16T00:34:27.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.100+0000 7f3944377740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5642592a0380 0x5642592bb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.115 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.100+0000 7f3944377740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642592a0ca0 0x5642592c6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.100+0000 7f3944377740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5642592afa90 0x5642592c7520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.100+0000 7f3944377740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f393c1af310 con 0x5642592a0380 2024-09-16T00:34:27.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.100+0000 7f3944377740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f393c1acbe0 con 0x5642592afa90 2024-09-16T00:34:27.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.100+0000 7f3944377740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f393c1aaa30 con 0x5642592a0ca0 2024-09-16T00:34:27.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.100+0000 7f393b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5642592a0380 0x5642592bb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.100+0000 7f393b7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5642592a0380 0x5642592bb680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51200/0 (socket says 172.21.15.38:51200) 2024-09-16T00:34:27.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.100+0000 7f393b7fe640 1 -- 172.21.15.38:0/1430872740 learned_addr learned my addr 172.21.15.38:0/1430872740 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:34:27.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.100+0000 7f393bfff640 1 --2- 172.21.15.38:0/1430872740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5642592afa90 0x5642592c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.100+0000 7f3933fff640 1 --2- 172.21.15.38:0/1430872740 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642592a0ca0 0x5642592c6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.101+0000 7f393b7fe640 1 -- 172.21.15.38:0/1430872740 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642592a0ca0 msgr2=0x5642592c6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:27.116 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.101+0000 7f393b7fe640 1 --2- 172.21.15.38:0/1430872740 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642592a0ca0 0x5642592c6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.101+0000 7f393b7fe640 1 -- 172.21.15.38:0/1430872740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5642592afa90 msgr2=0x5642592c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:27.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.101+0000 7f393b7fe640 1 --2- 172.21.15.38:0/1430872740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5642592afa90 0x5642592c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.101+0000 7f393b7fe640 1 -- 172.21.15.38:0/1430872740 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642592c1a80 con 0x5642592a0380 2024-09-16T00:34:27.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.101+0000 7f393b7fe640 1 --2- 172.21.15.38:0/1430872740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5642592a0380 0x5642592bb680 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f392c0b8b70 tx=0x7f392c02a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9c0d01e3c3e3dfc server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:27.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.101+0000 7f39317fa640 1 -- 172.21.15.38:0/1430872740 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c049040 con 0x5642592a0380 2024-09-16T00:34:27.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.101+0000 7f39317fa640 1 -- 172.21.15.38:0/1430872740 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f392c0f3d20 con 0x5642592a0380 2024-09-16T00:34:27.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.101+0000 7f39317fa640 1 -- 172.21.15.38:0/1430872740 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f392c0044e0 con 0x5642592a0380 2024-09-16T00:34:27.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.102+0000 7f3944377740 1 -- 172.21.15.38:0/1430872740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5642592a0380 msgr2=0x5642592bb680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:27.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.102+0000 7f3944377740 1 --2- 172.21.15.38:0/1430872740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5642592a0380 0x5642592bb680 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f392c0b8b70 tx=0x7f392c02a410 comp rx=0 tx=0).stop 2024-09-16T00:34:27.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.102+0000 7f3944377740 1 -- 172.21.15.38:0/1430872740 shutdown_connections 2024-09-16T00:34:27.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.102+0000 7f3944377740 1 --2- 172.21.15.38:0/1430872740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5642592afa90 0x5642592c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.117 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.102+0000 7f3944377740 1 --2- 172.21.15.38:0/1430872740 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642592a0ca0 0x5642592c6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.102+0000 7f3944377740 1 --2- 172.21.15.38:0/1430872740 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5642592a0380 0x5642592bb680 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.102+0000 7f3944377740 1 -- 172.21.15.38:0/1430872740 >> 172.21.15.38:0/1430872740 conn(0x56425929ca30 msgr2=0x5642592a7d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:27.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.102+0000 7f3944377740 1 -- 172.21.15.38:0/1430872740 shutdown_connections 2024-09-16T00:34:27.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.102+0000 7f3944377740 1 -- 172.21.15.38:0/1430872740 wait complete. 2024-09-16T00:34:27.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.102+0000 7f3944377740 1 -- start start 2024-09-16T00:34:27.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.102+0000 7f3944377740 1 Processor -- start 2024-09-16T00:34:27.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.103+0000 7f3944377740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5642592a0380 0x5642592b7ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.103+0000 7f3944377740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642592a0ca0 0x5642592b31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.103+0000 7f3944377740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5642592afa90 0x5642592b3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.103+0000 7f3944377740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f393c1ab0a0 con 0x5642592afa90 2024-09-16T00:34:27.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.103+0000 7f3944377740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f393c1ac510 con 0x5642592a0380 2024-09-16T00:34:27.118 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.103+0000 7f3944377740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f393c1ab770 con 0x5642592a0ca0 2024-09-16T00:34:27.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.103+0000 7f3944377740 10 client.0 _ll_register_callbacks cb 0x7f392400b8e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:27.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.103+0000 7f393bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5642592afa90 0x5642592b3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.103+0000 7f393b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5642592a0380 0x5642592b7ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.103+0000 7f393bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5642592afa90 0x5642592b3710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51212/0 (socket says 172.21.15.38:51212) 2024-09-16T00:34:27.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.103+0000 7f393bfff640 1 -- 172.21.15.38:0/2564434949 learned_addr learned my addr 172.21.15.38:0/2564434949 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:34:27.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.103+0000 7f393b7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5642592a0380 0x5642592b7ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.38:57846/0 (socket says 172.21.15.38:57846) 2024-09-16T00:34:27.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.103+0000 7f3933fff640 1 --2- 172.21.15.38:0/2564434949 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642592a0ca0 0x5642592b31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.103+0000 7f393bfff640 1 -- 172.21.15.38:0/2564434949 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642592a0ca0 msgr2=0x5642592b31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:27.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.103+0000 7f393bfff640 1 --2- 172.21.15.38:0/2564434949 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642592a0ca0 0x5642592b31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.103+0000 7f393bfff640 1 -- 172.21.15.38:0/2564434949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5642592a0380 msgr2=0x5642592b7ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:27.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.103+0000 7f393bfff640 1 --2- 172.21.15.38:0/2564434949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5642592a0380 0x5642592b7ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.119 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.103+0000 7f393bfff640 1 -- 172.21.15.38:0/2564434949 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642592b3f10 con 0x5642592afa90 2024-09-16T00:34:27.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.103+0000 7f3933fff640 1 --2- 172.21.15.38:0/2564434949 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642592a0ca0 0x5642592b31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:34:27.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.103+0000 7f393b7fe640 1 --2- 172.21.15.38:0/2564434949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5642592a0380 0x5642592b7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:34:27.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.104+0000 7f393bfff640 1 --2- 172.21.15.38:0/2564434949 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5642592afa90 0x5642592b3710 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f393401dfa0 tx=0x7f3934004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dee5e299916bc3a5 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:27.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.104+0000 7f391affd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:34:27.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.104+0000 7f391affd640 1 -- 172.21.15.38:0/2564434949 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f393401cca0 con 0x5642592afa90 2024-09-16T00:34:27.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.104+0000 7f391affd640 1 -- 172.21.15.38:0/2564434949 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f393401ce40 con 0x5642592afa90 2024-09-16T00:34:27.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.104+0000 7f3944377740 1 -- 172.21.15.38:0/2564434949 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f392c0f3d20 con 0x5642592afa90 2024-09-16T00:34:27.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.104+0000 7f391affd640 1 -- 172.21.15.38:0/2564434949 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3934034050 con 0x5642592afa90 2024-09-16T00:34:27.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.104+0000 7f3944377740 10 client.15711 fetch_fsmap learned FSMap version 26 2024-09-16T00:34:27.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.104+0000 7f3944377740 1 -- 172.21.15.38:0/2564434949 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564259329f90 con 0x5642592afa90 2024-09-16T00:34:27.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.104+0000 7f391affd640 1 -- 172.21.15.38:0/2564434949 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f392c0f3d20 con 0x5642592afa90 2024-09-16T00:34:27.120 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.104+0000 7f391affd640 1 -- 172.21.15.38:0/2564434949 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f393400e5e0 con 0x5642592afa90 2024-09-16T00:34:27.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.105+0000 7f3944377740 10 client.15711 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:34:27.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.105+0000 7f3944377740 10 client.15711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:34:27.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.105+0000 7f3944377740 20 client.15711 populate_metadata read hostname 'smithi038' 2024-09-16T00:34:27.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.105+0000 7f3944377740 1 --2- 172.21.15.38:0/2564434949 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x56425932a710 0x56425932cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:27.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.105+0000 7f3944377740 4 client.15711 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:34:27.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.105+0000 7f3944377740 1 -- 172.21.15.38:0/2564434949 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56425918a210 con 0x56425932a710 2024-09-16T00:34:27.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.105+0000 7f393b7fe640 1 --2- 172.21.15.38:0/2564434949 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x56425932a710 0x56425932cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:27.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.105+0000 7f393b7fe640 1 --2- 172.21.15.38:0/2564434949 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x56425932a710 0x56425932cb30 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd6d3466e3675ba server_cookie=5bbcae6791cb06c3 in_seq=0 out_seq=0 2024-09-16T00:34:27.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.105+0000 7f391affd640 10 client.15711 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:34:27.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.106+0000 7f391affd640 1 -- 172.21.15.38:0/2564434949 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56425918a210 con 0x56425932a710 2024-09-16T00:34:27.121 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.106+0000 7f391affd640 10 client.15711 handle_command_reply: tid=0 2024-09-16T00:34:27.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.106+0000 7f3944377740 1 client.15711 shutdown 2024-09-16T00:34:27.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.106+0000 7f3944377740 1 -- 172.21.15.38:0/2564434949 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5642592afa90 msgr2=0x5642592b3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:27.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.106+0000 7f3944377740 1 --2- 172.21.15.38:0/2564434949 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5642592afa90 0x5642592b3710 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f393401dfa0 tx=0x7f3934004a40 comp rx=0 tx=0).stop 2024-09-16T00:34:27.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.106+0000 7f3944377740 1 -- 172.21.15.38:0/2564434949 shutdown_connections 2024-09-16T00:34:27.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.106+0000 7f3944377740 1 --2- 172.21.15.38:0/2564434949 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x56425932a710 0x56425932cb30 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.106+0000 7f3944377740 1 --2- 172.21.15.38:0/2564434949 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5642592afa90 0x5642592b3710 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.106+0000 7f3944377740 1 --2- 172.21.15.38:0/2564434949 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5642592a0ca0 0x5642592b31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.106+0000 7f3944377740 1 --2- 172.21.15.38:0/2564434949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5642592a0380 0x5642592b7ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.106+0000 7f3944377740 1 -- 172.21.15.38:0/2564434949 >> 172.21.15.38:0/2564434949 conn(0x56425929ca30 msgr2=0x56425929b350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:27.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.106+0000 7f391affd640 0 client.15711 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:34:27.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.106+0000 7f3944377740 1 -- 172.21.15.38:0/2564434949 shutdown_connections 2024-09-16T00:34:27.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.106+0000 7f3944377740 1 -- 172.21.15.38:0/2564434949 wait complete. 2024-09-16T00:34:27.122 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.106+0000 7f3944377740 20 client.15711 trim_cache size 0 max 16384 2024-09-16T00:34:27.123 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-16T00:34:27.123 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "no active scrubs running", 2024-09-16T00:34:27.123 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": {} 2024-09-16T00:34:27.123 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-16T00:34:27.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.107+0000 7f3941f9e640 1 -- 172.21.15.38:0/1012773774 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f39280a3240 msgr2=0x7f39280a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:27.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.107+0000 7f3941f9e640 1 --2- 172.21.15.38:0/1012773774 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f39280a3240 0x7f39280a5700 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3934007e50 tx=0x7f3934007600 comp rx=0 tx=0).stop 2024-09-16T00:34:27.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.107+0000 7f3941f9e640 1 -- 172.21.15.38:0/1012773774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f393c102df0 msgr2=0x7f393c198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:27.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.107+0000 7f3941f9e640 1 --2- 172.21.15.38:0/1012773774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f393c102df0 0x7f393c198620 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f392c006010 tx=0x7f392c0042f0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.107+0000 7f3941f9e640 1 -- 172.21.15.38:0/1012773774 shutdown_connections 2024-09-16T00:34:27.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.107+0000 7f3941f9e640 1 --2- 172.21.15.38:0/1012773774 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f39280a3240 0x7f39280a5700 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.107+0000 7f3941f9e640 1 --2- 172.21.15.38:0/1012773774 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f393c10c2f0 0x7f393c19cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.124 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.107+0000 7f3941f9e640 1 --2- 172.21.15.38:0/1012773774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f393c1037a0 0x7f393c198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.107+0000 7f3941f9e640 1 --2- 172.21.15.38:0/1012773774 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f393c102df0 0x7f393c198620 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:27.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.107+0000 7f3941f9e640 1 -- 172.21.15.38:0/1012773774 >> 172.21.15.38:0/1012773774 conn(0x7f393c0fc7d0 msgr2=0x7f393c0fd4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:27.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.107+0000 7f3941f9e640 1 -- 172.21.15.38:0/1012773774 shutdown_connections 2024-09-16T00:34:27.125 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:27.107+0000 7f3941f9e640 1 -- 172.21.15.38:0/1012773774 wait complete. 2024-09-16T00:34:27.265 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:27 smithi038 ceph-mon[30945]: pgmap v414: 129 pgs: 129 active+clean; 761 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.2 KiB/s wr, 500 op/s 2024-09-16T00:34:27.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:27 smithi073 ceph-mon[35254]: pgmap v414: 129 pgs: 129 active+clean; 761 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.2 KiB/s wr, 500 op/s 2024-09-16T00:34:27.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:27 smithi122 ceph-mon[35412]: pgmap v414: 129 pgs: 129 active+clean; 761 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.2 KiB/s wr, 500 op/s 2024-09-16T00:34:27.840 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:34:27.840 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:34:28.107 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:34:28.650 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.624+0000 7f042cc6f640 1 -- 172.21.15.38:0/3038487555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0428102dd0 msgr2=0x7f04281031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:28.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.624+0000 7f042cc6f640 1 --2- 172.21.15.38:0/3038487555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0428102dd0 0x7f04281031b0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0418009930 tx=0x7f041802a420 comp rx=0 tx=0).stop 2024-09-16T00:34:28.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.624+0000 7f042cc6f640 1 -- 172.21.15.38:0/3038487555 shutdown_connections 2024-09-16T00:34:28.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.624+0000 7f042cc6f640 1 --2- 172.21.15.38:0/3038487555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0428108300 0x7f042810ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.624+0000 7f042cc6f640 1 --2- 172.21.15.38:0/3038487555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0428103780 0x7f0428107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.624+0000 7f042cc6f640 1 --2- 172.21.15.38:0/3038487555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0428102dd0 0x7f04281031b0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.624+0000 7f042cc6f640 1 -- 172.21.15.38:0/3038487555 >> 172.21.15.38:0/3038487555 conn(0x7f04280fc7d0 msgr2=0x7f04280febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:28.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.624+0000 7f042cc6f640 1 -- 172.21.15.38:0/3038487555 shutdown_connections 2024-09-16T00:34:28.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.624+0000 7f042cc6f640 1 -- 172.21.15.38:0/3038487555 wait complete. 2024-09-16T00:34:28.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.625+0000 7f042cc6f640 1 Processor -- start 2024-09-16T00:34:28.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.625+0000 7f042cc6f640 1 -- start start 2024-09-16T00:34:28.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.625+0000 7f042cc6f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0428102dd0 0x7f0428100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:28.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.625+0000 7f042cc6f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0428103780 0x7f0428100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:28.651 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.625+0000 7f042cc6f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0428108300 0x7f04281014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:28.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.625+0000 7f042cc6f640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f04281120b0 con 0x7f0428103780 2024-09-16T00:34:28.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.625+0000 7f042cc6f640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0428111f30 con 0x7f0428108300 2024-09-16T00:34:28.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.625+0000 7f042cc6f640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0428112230 con 0x7f0428102dd0 2024-09-16T00:34:28.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.625+0000 7f0425d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0428103780 0x7f0428100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:28.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.625+0000 7f0425d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0428103780 0x7f0428100f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51226/0 (socket says 172.21.15.38:51226) 2024-09-16T00:34:28.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.625+0000 7f0425d9b640 1 -- 172.21.15.38:0/1009667578 learned_addr learned my addr 172.21.15.38:0/1009667578 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:34:28.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.626+0000 7f0426d9d640 1 --2- 172.21.15.38:0/1009667578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0428108300 0x7f04281014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:28.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.626+0000 7f042659c640 1 --2- 172.21.15.38:0/1009667578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0428102dd0 0x7f0428100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:28.652 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.626+0000 7f0425d9b640 1 -- 172.21.15.38:0/1009667578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0428102dd0 msgr2=0x7f0428100a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:28.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.626+0000 7f0425d9b640 1 --2- 172.21.15.38:0/1009667578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0428102dd0 0x7f0428100a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.626+0000 7f0425d9b640 1 -- 172.21.15.38:0/1009667578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0428108300 msgr2=0x7f04281014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:28.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.626+0000 7f0425d9b640 1 --2- 172.21.15.38:0/1009667578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0428108300 0x7f04281014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.626+0000 7f0425d9b640 1 -- 172.21.15.38:0/1009667578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04281a6b30 con 0x7f0428103780 2024-09-16T00:34:28.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.626+0000 7f0425d9b640 1 --2- 172.21.15.38:0/1009667578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0428103780 0x7f0428100f70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f041c00d740 tx=0x7f041c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:28.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.626+0000 7f04177fe640 1 -- 172.21.15.38:0/1009667578 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c017070 con 0x7f0428103780 2024-09-16T00:34:28.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.626+0000 7f04177fe640 1 -- 172.21.15.38:0/1009667578 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f041c00de50 con 0x7f0428103780 2024-09-16T00:34:28.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.626+0000 7f042cc6f640 1 -- 172.21.15.38:0/1009667578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04281a6e20 con 0x7f0428103780 2024-09-16T00:34:28.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.626+0000 7f04177fe640 1 -- 172.21.15.38:0/1009667578 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041c00f6b0 con 0x7f0428103780 2024-09-16T00:34:28.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.626+0000 7f042cc6f640 1 -- 172.21.15.38:0/1009667578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04281a72d0 con 0x7f0428103780 2024-09-16T00:34:28.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.628+0000 7f042cc6f640 1 -- 172.21.15.38:0/1009667578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f042806a7b0 con 0x7f0428103780 2024-09-16T00:34:28.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.629+0000 7f04177fe640 1 -- 172.21.15.38:0/1009667578 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f041c01a070 con 0x7f0428103780 2024-09-16T00:34:28.653 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.630+0000 7f04177fe640 1 --2- 172.21.15.38:0/1009667578 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f04100a3240 0x7f04100a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:28.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.630+0000 7f042659c640 1 --2- 172.21.15.38:0/1009667578 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f04100a3240 0x7f04100a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:28.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.630+0000 7f04177fe640 1 -- 172.21.15.38:0/1009667578 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f041c0cf5f0 con 0x7f0428103780 2024-09-16T00:34:28.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.631+0000 7f042659c640 1 --2- 172.21.15.38:0/1009667578 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f04100a3240 0x7f04100a5700 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0418009900 tx=0x7f04180058d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:28.654 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.635+0000 7f04177fe640 1 -- 172.21.15.38:0/1009667578 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f041c09b940 con 0x7f0428103780 2024-09-16T00:34:28.710 INFO:tasks.workunit.client.0.smithi038.stdout:writing pattern 2024-09-16T00:34:28.710 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4190208 len 1024 2024-09-16T00:34:28.710 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4190208 len 1024 2024-09-16T00:34:28.710 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4190208 len 1024 2024-09-16T00:34:28.710 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4190208 len 1024 2024-09-16T00:34:28.710 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4190208 len 2048 2024-09-16T00:34:28.710 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4190208 len 2048 2024-09-16T00:34:28.711 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4190208 len 2048 2024-09-16T00:34:28.711 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4190208 len 2048 2024-09-16T00:34:28.711 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4190208 len 4096 2024-09-16T00:34:28.711 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4190208 len 4096 2024-09-16T00:34:28.711 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4190208 len 4096 2024-09-16T00:34:28.711 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4190208 len 4096 2024-09-16T00:34:28.711 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4190208 len 8192 2024-09-16T00:34:28.711 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4190208 len 8192 2024-09-16T00:34:28.711 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4190208 len 8192 2024-09-16T00:34:28.711 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4190208 len 8192 2024-09-16T00:34:28.711 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4190208 len 16384 2024-09-16T00:34:28.711 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4190208 len 16384 2024-09-16T00:34:28.711 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4190208 len 16384 2024-09-16T00:34:28.712 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4190208 len 16384 2024-09-16T00:34:28.712 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4190720 len 1024 2024-09-16T00:34:28.712 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4190720 len 1024 2024-09-16T00:34:28.712 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4190720 len 1024 2024-09-16T00:34:28.712 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4190720 len 1024 2024-09-16T00:34:28.712 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4190720 len 2048 2024-09-16T00:34:28.712 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4190720 len 2048 2024-09-16T00:34:28.712 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4190720 len 2048 2024-09-16T00:34:28.712 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4190720 len 2048 2024-09-16T00:34:28.712 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4190720 len 4096 2024-09-16T00:34:28.712 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4190720 len 4096 2024-09-16T00:34:28.713 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4190720 len 4096 2024-09-16T00:34:28.713 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4190720 len 4096 2024-09-16T00:34:28.713 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4190720 len 8192 2024-09-16T00:34:28.713 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4190720 len 8192 2024-09-16T00:34:28.713 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4190720 len 8192 2024-09-16T00:34:28.713 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4190720 len 8192 2024-09-16T00:34:28.713 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4190720 len 16384 2024-09-16T00:34:28.713 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4190720 len 16384 2024-09-16T00:34:28.713 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4190720 len 16384 2024-09-16T00:34:28.713 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4190720 len 16384 2024-09-16T00:34:28.713 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4191232 len 1024 2024-09-16T00:34:28.713 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4191232 len 1024 2024-09-16T00:34:28.713 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4191232 len 1024 2024-09-16T00:34:28.714 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4191232 len 1024 2024-09-16T00:34:28.714 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4191232 len 2048 2024-09-16T00:34:28.714 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4191232 len 2048 2024-09-16T00:34:28.714 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4191232 len 2048 2024-09-16T00:34:28.714 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4191232 len 2048 2024-09-16T00:34:28.714 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4191232 len 4096 2024-09-16T00:34:28.714 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4191232 len 4096 2024-09-16T00:34:28.714 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4191232 len 4096 2024-09-16T00:34:28.714 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4191232 len 4096 2024-09-16T00:34:28.715 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4191232 len 8192 2024-09-16T00:34:28.715 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4191232 len 8192 2024-09-16T00:34:28.715 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4191232 len 8192 2024-09-16T00:34:28.715 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4191232 len 8192 2024-09-16T00:34:28.715 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4191232 len 16384 2024-09-16T00:34:28.715 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4191232 len 16384 2024-09-16T00:34:28.715 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4191232 len 16384 2024-09-16T00:34:28.715 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4191232 len 16384 2024-09-16T00:34:28.715 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4191744 len 1024 2024-09-16T00:34:28.715 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4191744 len 1024 2024-09-16T00:34:28.715 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4191744 len 1024 2024-09-16T00:34:28.715 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4191744 len 1024 2024-09-16T00:34:28.716 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4191744 len 2048 2024-09-16T00:34:28.716 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4191744 len 2048 2024-09-16T00:34:28.716 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4191744 len 2048 2024-09-16T00:34:28.716 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4191744 len 2048 2024-09-16T00:34:28.716 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4191744 len 4096 2024-09-16T00:34:28.716 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4191744 len 4096 2024-09-16T00:34:28.716 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4191744 len 4096 2024-09-16T00:34:28.716 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4191744 len 4096 2024-09-16T00:34:28.716 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4191744 len 8192 2024-09-16T00:34:28.717 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4191744 len 8192 2024-09-16T00:34:28.717 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4191744 len 8192 2024-09-16T00:34:28.717 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4191744 len 8192 2024-09-16T00:34:28.717 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4191744 len 16384 2024-09-16T00:34:28.717 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4191744 len 16384 2024-09-16T00:34:28.717 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4191744 len 16384 2024-09-16T00:34:28.717 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4191744 len 16384 2024-09-16T00:34:28.717 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4192256 len 1024 2024-09-16T00:34:28.717 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4192256 len 1024 2024-09-16T00:34:28.717 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4192256 len 1024 2024-09-16T00:34:28.717 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4192256 len 1024 2024-09-16T00:34:28.717 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4192256 len 2048 2024-09-16T00:34:28.718 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4192256 len 2048 2024-09-16T00:34:28.718 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4192256 len 2048 2024-09-16T00:34:28.758 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4192256 len 2048 2024-09-16T00:34:28.759 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4192256 len 4096 2024-09-16T00:34:28.759 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4192256 len 4096 2024-09-16T00:34:28.759 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4192256 len 4096 2024-09-16T00:34:28.759 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4192256 len 4096 2024-09-16T00:34:28.759 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4192256 len 8192 2024-09-16T00:34:28.759 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4192256 len 8192 2024-09-16T00:34:28.759 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4192256 len 8192 2024-09-16T00:34:28.760 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4192256 len 8192 2024-09-16T00:34:28.760 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4192256 len 16384 2024-09-16T00:34:28.760 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4192256 len 16384 2024-09-16T00:34:28.760 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4192256 len 16384 2024-09-16T00:34:28.760 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4192256 len 16384 2024-09-16T00:34:28.760 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4192768 len 1024 2024-09-16T00:34:28.760 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4192768 len 1024 2024-09-16T00:34:28.760 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4192768 len 1024 2024-09-16T00:34:28.760 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4192768 len 1024 2024-09-16T00:34:28.760 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4192768 len 2048 2024-09-16T00:34:28.760 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4192768 len 2048 2024-09-16T00:34:28.760 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4192768 len 2048 2024-09-16T00:34:28.760 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4192768 len 2048 2024-09-16T00:34:28.760 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4192768 len 4096 2024-09-16T00:34:28.761 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4192768 len 4096 2024-09-16T00:34:28.761 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4192768 len 4096 2024-09-16T00:34:28.761 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4192768 len 4096 2024-09-16T00:34:28.761 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4192768 len 8192 2024-09-16T00:34:28.761 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4192768 len 8192 2024-09-16T00:34:28.761 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4192768 len 8192 2024-09-16T00:34:28.761 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4192768 len 8192 2024-09-16T00:34:28.761 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4192768 len 16384 2024-09-16T00:34:28.761 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4192768 len 16384 2024-09-16T00:34:28.761 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4192768 len 16384 2024-09-16T00:34:28.761 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4192768 len 16384 2024-09-16T00:34:28.761 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4193280 len 1024 2024-09-16T00:34:28.761 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4193280 len 1024 2024-09-16T00:34:28.761 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4193280 len 1024 2024-09-16T00:34:28.762 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4193280 len 1024 2024-09-16T00:34:28.762 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4193280 len 2048 2024-09-16T00:34:28.762 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4193280 len 2048 2024-09-16T00:34:28.762 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4193280 len 2048 2024-09-16T00:34:28.762 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4193280 len 2048 2024-09-16T00:34:28.762 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4193280 len 4096 2024-09-16T00:34:28.762 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4193280 len 4096 2024-09-16T00:34:28.762 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4193280 len 4096 2024-09-16T00:34:28.762 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4193280 len 4096 2024-09-16T00:34:28.762 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4193280 len 8192 2024-09-16T00:34:28.762 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4193280 len 8192 2024-09-16T00:34:28.762 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4193280 len 8192 2024-09-16T00:34:28.762 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4193280 len 8192 2024-09-16T00:34:28.762 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4193280 len 16384 2024-09-16T00:34:28.763 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4193280 len 16384 2024-09-16T00:34:28.763 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4193280 len 16384 2024-09-16T00:34:28.763 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4193280 len 16384 2024-09-16T00:34:28.763 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4193792 len 1024 2024-09-16T00:34:28.763 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4193792 len 1024 2024-09-16T00:34:28.763 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4193792 len 1024 2024-09-16T00:34:28.763 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4193792 len 1024 2024-09-16T00:34:28.763 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4193792 len 2048 2024-09-16T00:34:28.763 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4193792 len 2048 2024-09-16T00:34:28.763 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4193792 len 2048 2024-09-16T00:34:28.763 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4193792 len 2048 2024-09-16T00:34:28.764 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4193792 len 4096 2024-09-16T00:34:28.764 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4193792 len 4096 2024-09-16T00:34:28.764 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4193792 len 4096 2024-09-16T00:34:28.764 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4193792 len 4096 2024-09-16T00:34:28.764 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4193792 len 8192 2024-09-16T00:34:28.764 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4193792 len 8192 2024-09-16T00:34:28.764 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4193792 len 8192 2024-09-16T00:34:28.764 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4193792 len 8192 2024-09-16T00:34:28.764 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4193792 len 16384 2024-09-16T00:34:28.764 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4193792 len 16384 2024-09-16T00:34:28.764 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4193792 len 16384 2024-09-16T00:34:28.764 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4193792 len 16384 2024-09-16T00:34:28.764 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4194304 len 1024 2024-09-16T00:34:28.765 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4194304 len 1024 2024-09-16T00:34:28.765 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4194304 len 1024 2024-09-16T00:34:28.765 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4194304 len 1024 2024-09-16T00:34:28.765 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4194304 len 2048 2024-09-16T00:34:28.765 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4194304 len 2048 2024-09-16T00:34:28.765 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4194304 len 2048 2024-09-16T00:34:28.765 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4194304 len 2048 2024-09-16T00:34:28.765 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4194304 len 4096 2024-09-16T00:34:28.765 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4194304 len 4096 2024-09-16T00:34:28.765 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4194304 len 4096 2024-09-16T00:34:28.765 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4194304 len 4096 2024-09-16T00:34:28.765 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4194304 len 8192 2024-09-16T00:34:28.765 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4194304 len 8192 2024-09-16T00:34:28.765 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4194304 len 8192 2024-09-16T00:34:28.784 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4194304 len 8192 2024-09-16T00:34:28.784 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4194304 len 16384 2024-09-16T00:34:28.784 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4194304 len 16384 2024-09-16T00:34:28.785 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4194304 len 16384 2024-09-16T00:34:28.785 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4194304 len 16384 2024-09-16T00:34:28.785 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4194816 len 1024 2024-09-16T00:34:28.785 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4194816 len 1024 2024-09-16T00:34:28.785 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4194816 len 1024 2024-09-16T00:34:28.785 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4194816 len 1024 2024-09-16T00:34:28.785 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4194816 len 2048 2024-09-16T00:34:28.785 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4194816 len 2048 2024-09-16T00:34:28.785 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4194816 len 2048 2024-09-16T00:34:28.785 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4194816 len 2048 2024-09-16T00:34:28.785 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4194816 len 4096 2024-09-16T00:34:28.785 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4194816 len 4096 2024-09-16T00:34:28.785 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4194816 len 4096 2024-09-16T00:34:28.786 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4194816 len 4096 2024-09-16T00:34:28.786 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4194816 len 8192 2024-09-16T00:34:28.786 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4194816 len 8192 2024-09-16T00:34:28.786 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4194816 len 8192 2024-09-16T00:34:28.786 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4194816 len 8192 2024-09-16T00:34:28.786 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4194816 len 16384 2024-09-16T00:34:28.786 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4194816 len 16384 2024-09-16T00:34:28.786 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4194816 len 16384 2024-09-16T00:34:28.786 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4194816 len 16384 2024-09-16T00:34:28.786 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4195328 len 1024 2024-09-16T00:34:28.786 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4195328 len 1024 2024-09-16T00:34:28.786 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4195328 len 1024 2024-09-16T00:34:28.786 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4195328 len 1024 2024-09-16T00:34:28.786 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4195328 len 2048 2024-09-16T00:34:28.787 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4195328 len 2048 2024-09-16T00:34:28.787 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4195328 len 2048 2024-09-16T00:34:28.787 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4195328 len 2048 2024-09-16T00:34:28.787 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4195328 len 4096 2024-09-16T00:34:28.787 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4195328 len 4096 2024-09-16T00:34:28.787 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4195328 len 4096 2024-09-16T00:34:28.787 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4195328 len 4096 2024-09-16T00:34:28.787 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4195328 len 8192 2024-09-16T00:34:28.787 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4195328 len 8192 2024-09-16T00:34:28.787 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4195328 len 8192 2024-09-16T00:34:28.787 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4195328 len 8192 2024-09-16T00:34:28.787 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4195328 len 16384 2024-09-16T00:34:28.787 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4195328 len 16384 2024-09-16T00:34:28.787 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4195328 len 16384 2024-09-16T00:34:28.788 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4195328 len 16384 2024-09-16T00:34:28.788 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4195840 len 1024 2024-09-16T00:34:28.788 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4195840 len 1024 2024-09-16T00:34:28.788 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4195840 len 1024 2024-09-16T00:34:28.788 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4195840 len 1024 2024-09-16T00:34:28.788 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4195840 len 2048 2024-09-16T00:34:28.788 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4195840 len 2048 2024-09-16T00:34:28.788 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4195840 len 2048 2024-09-16T00:34:28.788 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4195840 len 2048 2024-09-16T00:34:28.788 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4195840 len 4096 2024-09-16T00:34:28.788 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4195840 len 4096 2024-09-16T00:34:28.789 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4195840 len 4096 2024-09-16T00:34:28.789 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4195840 len 4096 2024-09-16T00:34:28.789 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4195840 len 8192 2024-09-16T00:34:28.789 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4195840 len 8192 2024-09-16T00:34:28.789 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4195840 len 8192 2024-09-16T00:34:28.789 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4195840 len 8192 2024-09-16T00:34:28.789 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4195840 len 16384 2024-09-16T00:34:28.789 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4195840 len 16384 2024-09-16T00:34:28.789 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4195840 len 16384 2024-09-16T00:34:28.789 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4195840 len 16384 2024-09-16T00:34:28.790 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4196352 len 1024 2024-09-16T00:34:28.790 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4196352 len 1024 2024-09-16T00:34:28.790 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4196352 len 1024 2024-09-16T00:34:28.790 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4196352 len 1024 2024-09-16T00:34:28.790 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4196352 len 2048 2024-09-16T00:34:28.790 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4196352 len 2048 2024-09-16T00:34:28.790 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4196352 len 2048 2024-09-16T00:34:28.790 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4196352 len 2048 2024-09-16T00:34:28.790 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4196352 len 4096 2024-09-16T00:34:28.791 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4196352 len 4096 2024-09-16T00:34:28.791 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4196352 len 4096 2024-09-16T00:34:28.791 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4196352 len 4096 2024-09-16T00:34:28.791 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4196352 len 8192 2024-09-16T00:34:28.791 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4196352 len 8192 2024-09-16T00:34:28.791 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4196352 len 8192 2024-09-16T00:34:28.791 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4196352 len 8192 2024-09-16T00:34:28.791 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4196352 len 16384 2024-09-16T00:34:28.791 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4196352 len 16384 2024-09-16T00:34:28.791 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4196352 len 16384 2024-09-16T00:34:28.791 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4196352 len 16384 2024-09-16T00:34:28.791 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4196864 len 1024 2024-09-16T00:34:28.791 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4196864 len 1024 2024-09-16T00:34:28.812 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4196864 len 1024 2024-09-16T00:34:28.812 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4196864 len 1024 2024-09-16T00:34:28.812 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4196864 len 2048 2024-09-16T00:34:28.812 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4196864 len 2048 2024-09-16T00:34:28.812 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4196864 len 2048 2024-09-16T00:34:28.813 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4196864 len 2048 2024-09-16T00:34:28.813 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4196864 len 4096 2024-09-16T00:34:28.813 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4196864 len 4096 2024-09-16T00:34:28.813 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4196864 len 4096 2024-09-16T00:34:28.813 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4196864 len 4096 2024-09-16T00:34:28.813 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4196864 len 8192 2024-09-16T00:34:28.813 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4196864 len 8192 2024-09-16T00:34:28.813 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4196864 len 8192 2024-09-16T00:34:28.813 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4196864 len 8192 2024-09-16T00:34:28.813 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4196864 len 16384 2024-09-16T00:34:28.813 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4196864 len 16384 2024-09-16T00:34:28.813 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4196864 len 16384 2024-09-16T00:34:28.813 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4196864 len 16384 2024-09-16T00:34:28.813 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4197376 len 1024 2024-09-16T00:34:28.814 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4197376 len 1024 2024-09-16T00:34:28.814 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4197376 len 1024 2024-09-16T00:34:28.814 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4197376 len 1024 2024-09-16T00:34:28.814 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4197376 len 2048 2024-09-16T00:34:28.814 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4197376 len 2048 2024-09-16T00:34:28.814 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4197376 len 2048 2024-09-16T00:34:28.814 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4197376 len 2048 2024-09-16T00:34:28.814 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4197376 len 4096 2024-09-16T00:34:28.814 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4197376 len 4096 2024-09-16T00:34:28.814 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4197376 len 4096 2024-09-16T00:34:28.814 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4197376 len 4096 2024-09-16T00:34:28.814 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4197376 len 8192 2024-09-16T00:34:28.814 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4197376 len 8192 2024-09-16T00:34:28.814 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4197376 len 8192 2024-09-16T00:34:28.815 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4197376 len 8192 2024-09-16T00:34:28.815 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4197376 len 16384 2024-09-16T00:34:28.815 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4197376 len 16384 2024-09-16T00:34:28.815 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4197376 len 16384 2024-09-16T00:34:28.815 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4197376 len 16384 2024-09-16T00:34:28.815 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4197888 len 1024 2024-09-16T00:34:28.815 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4197888 len 1024 2024-09-16T00:34:28.815 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4197888 len 1024 2024-09-16T00:34:28.815 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4197888 len 1024 2024-09-16T00:34:28.815 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4197888 len 2048 2024-09-16T00:34:28.815 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4197888 len 2048 2024-09-16T00:34:28.815 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4197888 len 2048 2024-09-16T00:34:28.815 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4197888 len 2048 2024-09-16T00:34:28.816 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4197888 len 4096 2024-09-16T00:34:28.816 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4197888 len 4096 2024-09-16T00:34:28.816 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4197888 len 4096 2024-09-16T00:34:28.816 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4197888 len 4096 2024-09-16T00:34:28.816 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4197888 len 8192 2024-09-16T00:34:28.816 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4197888 len 8192 2024-09-16T00:34:28.816 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4197888 len 8192 2024-09-16T00:34:28.816 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4197888 len 8192 2024-09-16T00:34:28.816 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 0 offset 4197888 len 16384 2024-09-16T00:34:28.816 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4197888 len 16384 2024-09-16T00:34:28.816 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 0 offset 4197888 len 16384 2024-09-16T00:34:28.816 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 0 offset 4197888 len 16384 2024-09-16T00:34:28.816 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4190208 len 1024 2024-09-16T00:34:28.816 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4190208 len 1024 2024-09-16T00:34:28.817 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4190208 len 1024 2024-09-16T00:34:28.817 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4190208 len 1024 2024-09-16T00:34:28.817 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4190208 len 2048 2024-09-16T00:34:28.817 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4190208 len 2048 2024-09-16T00:34:28.817 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4190208 len 2048 2024-09-16T00:34:28.817 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4190208 len 2048 2024-09-16T00:34:28.817 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4190208 len 4096 2024-09-16T00:34:28.817 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4190208 len 4096 2024-09-16T00:34:28.817 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4190208 len 4096 2024-09-16T00:34:28.817 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4190208 len 4096 2024-09-16T00:34:28.817 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4190208 len 8192 2024-09-16T00:34:28.817 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4190208 len 8192 2024-09-16T00:34:28.817 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4190208 len 8192 2024-09-16T00:34:28.817 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4190208 len 8192 2024-09-16T00:34:28.818 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4190208 len 16384 2024-09-16T00:34:28.818 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4190208 len 16384 2024-09-16T00:34:28.818 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4190208 len 16384 2024-09-16T00:34:28.818 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4190208 len 16384 2024-09-16T00:34:28.818 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4190720 len 1024 2024-09-16T00:34:28.818 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4190720 len 1024 2024-09-16T00:34:28.818 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4190720 len 1024 2024-09-16T00:34:28.818 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4190720 len 1024 2024-09-16T00:34:28.818 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4190720 len 2048 2024-09-16T00:34:28.818 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4190720 len 2048 2024-09-16T00:34:28.818 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4190720 len 2048 2024-09-16T00:34:28.818 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4190720 len 2048 2024-09-16T00:34:28.818 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4190720 len 4096 2024-09-16T00:34:28.854 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4190720 len 4096 2024-09-16T00:34:28.854 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4190720 len 4096 2024-09-16T00:34:28.854 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4190720 len 4096 2024-09-16T00:34:28.855 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4190720 len 8192 2024-09-16T00:34:28.855 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4190720 len 8192 2024-09-16T00:34:28.855 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4190720 len 8192 2024-09-16T00:34:28.855 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4190720 len 8192 2024-09-16T00:34:28.855 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4190720 len 16384 2024-09-16T00:34:28.855 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4190720 len 16384 2024-09-16T00:34:28.855 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4190720 len 16384 2024-09-16T00:34:28.855 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4190720 len 16384 2024-09-16T00:34:28.855 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4191232 len 1024 2024-09-16T00:34:28.855 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4191232 len 1024 2024-09-16T00:34:28.855 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4191232 len 1024 2024-09-16T00:34:28.855 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4191232 len 1024 2024-09-16T00:34:28.855 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4191232 len 2048 2024-09-16T00:34:28.855 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4191232 len 2048 2024-09-16T00:34:28.856 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4191232 len 2048 2024-09-16T00:34:28.856 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4191232 len 2048 2024-09-16T00:34:28.856 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4191232 len 4096 2024-09-16T00:34:28.856 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4191232 len 4096 2024-09-16T00:34:28.856 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4191232 len 4096 2024-09-16T00:34:28.856 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4191232 len 4096 2024-09-16T00:34:28.856 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4191232 len 8192 2024-09-16T00:34:28.856 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4191232 len 8192 2024-09-16T00:34:28.856 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4191232 len 8192 2024-09-16T00:34:28.856 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4191232 len 8192 2024-09-16T00:34:28.856 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4191232 len 16384 2024-09-16T00:34:28.856 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4191232 len 16384 2024-09-16T00:34:28.856 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4191232 len 16384 2024-09-16T00:34:28.857 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4191232 len 16384 2024-09-16T00:34:28.857 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4191744 len 1024 2024-09-16T00:34:28.857 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4191744 len 1024 2024-09-16T00:34:28.857 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4191744 len 1024 2024-09-16T00:34:28.857 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4191744 len 1024 2024-09-16T00:34:28.857 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4191744 len 2048 2024-09-16T00:34:28.857 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4191744 len 2048 2024-09-16T00:34:28.857 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4191744 len 2048 2024-09-16T00:34:28.857 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4191744 len 2048 2024-09-16T00:34:28.857 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4191744 len 4096 2024-09-16T00:34:28.857 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4191744 len 4096 2024-09-16T00:34:28.857 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4191744 len 4096 2024-09-16T00:34:28.857 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4191744 len 4096 2024-09-16T00:34:28.857 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4191744 len 8192 2024-09-16T00:34:28.858 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4191744 len 8192 2024-09-16T00:34:28.858 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4191744 len 8192 2024-09-16T00:34:28.858 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4191744 len 8192 2024-09-16T00:34:28.858 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4191744 len 16384 2024-09-16T00:34:28.858 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4191744 len 16384 2024-09-16T00:34:28.858 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4191744 len 16384 2024-09-16T00:34:28.858 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4191744 len 16384 2024-09-16T00:34:28.858 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4192256 len 1024 2024-09-16T00:34:28.858 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4192256 len 1024 2024-09-16T00:34:28.858 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4192256 len 1024 2024-09-16T00:34:28.858 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4192256 len 1024 2024-09-16T00:34:28.858 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4192256 len 2048 2024-09-16T00:34:28.858 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4192256 len 2048 2024-09-16T00:34:28.859 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4192256 len 2048 2024-09-16T00:34:28.859 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4192256 len 2048 2024-09-16T00:34:28.859 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4192256 len 4096 2024-09-16T00:34:28.859 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4192256 len 4096 2024-09-16T00:34:28.859 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4192256 len 4096 2024-09-16T00:34:28.859 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4192256 len 4096 2024-09-16T00:34:28.859 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4192256 len 8192 2024-09-16T00:34:28.859 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4192256 len 8192 2024-09-16T00:34:28.859 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4192256 len 8192 2024-09-16T00:34:28.859 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4192256 len 8192 2024-09-16T00:34:28.859 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4192256 len 16384 2024-09-16T00:34:28.859 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4192256 len 16384 2024-09-16T00:34:28.859 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4192256 len 16384 2024-09-16T00:34:28.859 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4192256 len 16384 2024-09-16T00:34:28.860 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4192768 len 1024 2024-09-16T00:34:28.860 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4192768 len 1024 2024-09-16T00:34:28.860 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4192768 len 1024 2024-09-16T00:34:28.860 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4192768 len 1024 2024-09-16T00:34:28.860 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4192768 len 2048 2024-09-16T00:34:28.860 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4192768 len 2048 2024-09-16T00:34:28.860 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4192768 len 2048 2024-09-16T00:34:28.860 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4192768 len 2048 2024-09-16T00:34:28.860 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4192768 len 4096 2024-09-16T00:34:28.860 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4192768 len 4096 2024-09-16T00:34:28.860 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4192768 len 4096 2024-09-16T00:34:28.860 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4192768 len 4096 2024-09-16T00:34:28.860 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4192768 len 8192 2024-09-16T00:34:28.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.855+0000 7f042cc6f640 1 -- 172.21.15.38:0/1009667578 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f04281031b0 con 0x7f0428103780 2024-09-16T00:34:28.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.856+0000 7f04177fe640 1 -- 172.21.15.38:0/1009667578 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f041c09fc40 con 0x7f0428103780 2024-09-16T00:34:28.861 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:34:28.862 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"btime":"2024-09-16T00:32:41:853514+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/367974527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":367974527},{"type":"v1","addr":"172.21.15.38:6835","nonce":367974527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/335077136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":335077136},{"type":"v1","addr":"172.21.15.38:6837","nonce":335077136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/3501211160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3501211160},{"type":"v1","addr":"172.21.15.38:6839","nonce":3501211160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6841/2623130915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":2623130915},{"type":"v1","addr":"172.21.15.38:6841","nonce":2623130915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/412277519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":412277519},{"type":"v1","addr":"172.21.15.122:6833","nonce":412277519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3987234757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3987234757},{"type":"v1","addr":"172.21.15.122:6835","nonce":3987234757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:30:26.764565+0000","modified":"2024-09-16T00:32:41.355828+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24440},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1210328177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1210328177},{"type":"v1","addr":"172.21.15.73:6833","nonce":1210328177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6837/1858299680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1858299680},{"type":"v1","addr":"172.21.15.122:6837","nonce":1858299680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1983975311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1983975311},{"type":"v1","addr":"172.21.15.122:6839","nonce":1983975311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.73:6835/2866363113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2866363113},{"type":"v1","addr":"172.21.15.73:6835","nonce":2866363113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.73:6837/2681081265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2681081265},{"type":"v1","addr":"172.21.15.73:6837","nonce":2681081265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.73:6839/2396224525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2396224525},{"type":"v1","addr":"172.21.15.73:6839","nonce":2396224525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24464,24440]},"id":1}]} 2024-09-16T00:34:28.862 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 26 2024-09-16T00:34:28.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.860+0000 7f042cc6f640 1 -- 172.21.15.38:0/1009667578 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f04100a3240 msgr2=0x7f04100a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:28.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.860+0000 7f042cc6f640 1 --2- 172.21.15.38:0/1009667578 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f04100a3240 0x7f04100a5700 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0418009900 tx=0x7f04180058d0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.860+0000 7f042cc6f640 1 -- 172.21.15.38:0/1009667578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0428103780 msgr2=0x7f0428100f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:28.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.860+0000 7f042cc6f640 1 --2- 172.21.15.38:0/1009667578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0428103780 0x7f0428100f70 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f041c00d740 tx=0x7f041c00dc00 comp rx=0 tx=0).stop 2024-09-16T00:34:28.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.860+0000 7f042cc6f640 1 -- 172.21.15.38:0/1009667578 shutdown_connections 2024-09-16T00:34:28.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.860+0000 7f042cc6f640 1 --2- 172.21.15.38:0/1009667578 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f04100a3240 0x7f04100a5700 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.860+0000 7f042cc6f640 1 --2- 172.21.15.38:0/1009667578 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0428108300 0x7f04281014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.860+0000 7f042cc6f640 1 --2- 172.21.15.38:0/1009667578 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0428103780 0x7f0428100f70 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.860+0000 7f042cc6f640 1 --2- 172.21.15.38:0/1009667578 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0428102dd0 0x7f0428100a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:28.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.860+0000 7f042cc6f640 1 -- 172.21.15.38:0/1009667578 >> 172.21.15.38:0/1009667578 conn(0x7f04280fc7d0 msgr2=0x7f0428105e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:28.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.860+0000 7f042cc6f640 1 -- 172.21.15.38:0/1009667578 shutdown_connections 2024-09-16T00:34:28.863 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:28.860+0000 7f042cc6f640 1 -- 172.21.15.38:0/1009667578 wait complete. 2024-09-16T00:34:28.897 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4192768 len 8192 2024-09-16T00:34:28.897 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4192768 len 8192 2024-09-16T00:34:28.897 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4192768 len 8192 2024-09-16T00:34:28.897 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4192768 len 16384 2024-09-16T00:34:28.898 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4192768 len 16384 2024-09-16T00:34:28.898 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4192768 len 16384 2024-09-16T00:34:28.898 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4192768 len 16384 2024-09-16T00:34:28.898 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4193280 len 1024 2024-09-16T00:34:28.898 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4193280 len 1024 2024-09-16T00:34:28.898 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4193280 len 1024 2024-09-16T00:34:28.898 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4193280 len 1024 2024-09-16T00:34:28.898 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4193280 len 2048 2024-09-16T00:34:28.898 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4193280 len 2048 2024-09-16T00:34:28.898 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4193280 len 2048 2024-09-16T00:34:28.898 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4193280 len 2048 2024-09-16T00:34:28.898 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4193280 len 4096 2024-09-16T00:34:28.898 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4193280 len 4096 2024-09-16T00:34:28.899 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4193280 len 4096 2024-09-16T00:34:28.899 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4193280 len 4096 2024-09-16T00:34:28.899 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4193280 len 8192 2024-09-16T00:34:28.899 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4193280 len 8192 2024-09-16T00:34:28.899 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4193280 len 8192 2024-09-16T00:34:28.899 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4193280 len 8192 2024-09-16T00:34:28.899 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4193280 len 16384 2024-09-16T00:34:28.899 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4193280 len 16384 2024-09-16T00:34:28.899 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4193280 len 16384 2024-09-16T00:34:28.899 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4193280 len 16384 2024-09-16T00:34:28.899 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4193792 len 1024 2024-09-16T00:34:28.899 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4193792 len 1024 2024-09-16T00:34:28.899 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4193792 len 1024 2024-09-16T00:34:28.899 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4193792 len 1024 2024-09-16T00:34:28.900 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4193792 len 2048 2024-09-16T00:34:28.900 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4193792 len 2048 2024-09-16T00:34:28.900 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4193792 len 2048 2024-09-16T00:34:28.900 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4193792 len 2048 2024-09-16T00:34:28.900 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4193792 len 4096 2024-09-16T00:34:28.900 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4193792 len 4096 2024-09-16T00:34:28.900 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4193792 len 4096 2024-09-16T00:34:28.900 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4193792 len 4096 2024-09-16T00:34:28.900 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4193792 len 8192 2024-09-16T00:34:28.900 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4193792 len 8192 2024-09-16T00:34:28.900 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4193792 len 8192 2024-09-16T00:34:28.901 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4193792 len 8192 2024-09-16T00:34:28.901 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4193792 len 16384 2024-09-16T00:34:28.901 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4193792 len 16384 2024-09-16T00:34:28.901 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4193792 len 16384 2024-09-16T00:34:28.901 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4193792 len 16384 2024-09-16T00:34:28.901 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4194304 len 1024 2024-09-16T00:34:28.901 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4194304 len 1024 2024-09-16T00:34:28.901 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4194304 len 1024 2024-09-16T00:34:28.901 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4194304 len 1024 2024-09-16T00:34:28.901 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4194304 len 2048 2024-09-16T00:34:28.901 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4194304 len 2048 2024-09-16T00:34:28.901 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4194304 len 2048 2024-09-16T00:34:28.901 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4194304 len 2048 2024-09-16T00:34:28.901 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4194304 len 4096 2024-09-16T00:34:28.902 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4194304 len 4096 2024-09-16T00:34:28.902 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4194304 len 4096 2024-09-16T00:34:28.902 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4194304 len 4096 2024-09-16T00:34:28.902 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4194304 len 8192 2024-09-16T00:34:28.902 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4194304 len 8192 2024-09-16T00:34:28.902 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4194304 len 8192 2024-09-16T00:34:28.902 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4194304 len 8192 2024-09-16T00:34:28.902 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4194304 len 16384 2024-09-16T00:34:28.902 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4194304 len 16384 2024-09-16T00:34:28.902 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4194304 len 16384 2024-09-16T00:34:28.902 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4194304 len 16384 2024-09-16T00:34:28.902 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4194816 len 1024 2024-09-16T00:34:28.902 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4194816 len 1024 2024-09-16T00:34:28.902 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4194816 len 1024 2024-09-16T00:34:28.903 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4194816 len 1024 2024-09-16T00:34:28.903 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4194816 len 2048 2024-09-16T00:34:28.903 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4194816 len 2048 2024-09-16T00:34:28.903 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4194816 len 2048 2024-09-16T00:34:28.903 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4194816 len 2048 2024-09-16T00:34:28.903 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4194816 len 4096 2024-09-16T00:34:28.903 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4194816 len 4096 2024-09-16T00:34:28.903 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4194816 len 4096 2024-09-16T00:34:28.903 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4194816 len 4096 2024-09-16T00:34:28.903 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4194816 len 8192 2024-09-16T00:34:28.904 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4194816 len 8192 2024-09-16T00:34:28.904 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4194816 len 8192 2024-09-16T00:34:28.904 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4194816 len 8192 2024-09-16T00:34:28.904 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4194816 len 16384 2024-09-16T00:34:28.925 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4194816 len 16384 2024-09-16T00:34:28.928 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4194816 len 16384 2024-09-16T00:34:28.928 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4194816 len 16384 2024-09-16T00:34:28.928 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4195328 len 1024 2024-09-16T00:34:28.928 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4195328 len 1024 2024-09-16T00:34:28.928 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4195328 len 1024 2024-09-16T00:34:28.928 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4195328 len 1024 2024-09-16T00:34:28.928 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4195328 len 2048 2024-09-16T00:34:28.928 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4195328 len 2048 2024-09-16T00:34:28.928 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4195328 len 2048 2024-09-16T00:34:28.928 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4195328 len 2048 2024-09-16T00:34:28.929 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4195328 len 4096 2024-09-16T00:34:28.929 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4195328 len 4096 2024-09-16T00:34:28.929 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4195328 len 4096 2024-09-16T00:34:28.929 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4195328 len 4096 2024-09-16T00:34:28.929 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4195328 len 8192 2024-09-16T00:34:28.929 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4195328 len 8192 2024-09-16T00:34:28.929 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4195328 len 8192 2024-09-16T00:34:28.929 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4195328 len 8192 2024-09-16T00:34:28.929 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4195328 len 16384 2024-09-16T00:34:28.929 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4195328 len 16384 2024-09-16T00:34:28.929 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4195328 len 16384 2024-09-16T00:34:28.929 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4195328 len 16384 2024-09-16T00:34:28.929 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4195840 len 1024 2024-09-16T00:34:28.929 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4195840 len 1024 2024-09-16T00:34:28.930 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4195840 len 1024 2024-09-16T00:34:28.930 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4195840 len 1024 2024-09-16T00:34:28.930 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4195840 len 2048 2024-09-16T00:34:28.930 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4195840 len 2048 2024-09-16T00:34:28.930 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4195840 len 2048 2024-09-16T00:34:28.930 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4195840 len 2048 2024-09-16T00:34:28.930 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4195840 len 4096 2024-09-16T00:34:28.930 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4195840 len 4096 2024-09-16T00:34:28.930 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4195840 len 4096 2024-09-16T00:34:28.930 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4195840 len 4096 2024-09-16T00:34:28.930 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4195840 len 8192 2024-09-16T00:34:28.930 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4195840 len 8192 2024-09-16T00:34:28.930 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4195840 len 8192 2024-09-16T00:34:28.930 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4195840 len 8192 2024-09-16T00:34:28.931 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4195840 len 16384 2024-09-16T00:34:28.931 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4195840 len 16384 2024-09-16T00:34:28.931 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4195840 len 16384 2024-09-16T00:34:28.931 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4195840 len 16384 2024-09-16T00:34:28.931 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4196352 len 1024 2024-09-16T00:34:28.931 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4196352 len 1024 2024-09-16T00:34:28.931 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4196352 len 1024 2024-09-16T00:34:28.931 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4196352 len 1024 2024-09-16T00:34:28.931 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4196352 len 2048 2024-09-16T00:34:28.931 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4196352 len 2048 2024-09-16T00:34:28.931 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4196352 len 2048 2024-09-16T00:34:28.931 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4196352 len 2048 2024-09-16T00:34:28.931 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4196352 len 4096 2024-09-16T00:34:28.931 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4196352 len 4096 2024-09-16T00:34:28.932 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4196352 len 4096 2024-09-16T00:34:28.932 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4196352 len 4096 2024-09-16T00:34:28.932 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4196352 len 8192 2024-09-16T00:34:28.932 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4196352 len 8192 2024-09-16T00:34:28.932 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4196352 len 8192 2024-09-16T00:34:28.932 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4196352 len 8192 2024-09-16T00:34:28.932 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4196352 len 16384 2024-09-16T00:34:28.932 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4196352 len 16384 2024-09-16T00:34:28.932 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4196352 len 16384 2024-09-16T00:34:28.932 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4196352 len 16384 2024-09-16T00:34:28.932 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4196864 len 1024 2024-09-16T00:34:28.932 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4196864 len 1024 2024-09-16T00:34:28.932 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4196864 len 1024 2024-09-16T00:34:28.933 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4196864 len 1024 2024-09-16T00:34:28.933 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4196864 len 2048 2024-09-16T00:34:28.933 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4196864 len 2048 2024-09-16T00:34:28.933 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4196864 len 2048 2024-09-16T00:34:28.933 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4196864 len 2048 2024-09-16T00:34:28.933 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4196864 len 4096 2024-09-16T00:34:28.933 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4196864 len 4096 2024-09-16T00:34:28.933 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4196864 len 4096 2024-09-16T00:34:28.933 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4196864 len 4096 2024-09-16T00:34:28.933 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4196864 len 8192 2024-09-16T00:34:28.933 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4196864 len 8192 2024-09-16T00:34:28.933 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4196864 len 8192 2024-09-16T00:34:28.934 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4196864 len 8192 2024-09-16T00:34:28.934 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4196864 len 16384 2024-09-16T00:34:28.934 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4196864 len 16384 2024-09-16T00:34:28.934 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4196864 len 16384 2024-09-16T00:34:28.934 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4196864 len 16384 2024-09-16T00:34:28.934 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4197376 len 1024 2024-09-16T00:34:28.962 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4197376 len 1024 2024-09-16T00:34:28.962 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4197376 len 1024 2024-09-16T00:34:28.962 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4197376 len 1024 2024-09-16T00:34:28.962 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4197376 len 2048 2024-09-16T00:34:28.962 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4197376 len 2048 2024-09-16T00:34:28.963 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4197376 len 2048 2024-09-16T00:34:28.963 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4197376 len 2048 2024-09-16T00:34:28.963 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4197376 len 4096 2024-09-16T00:34:28.963 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4197376 len 4096 2024-09-16T00:34:28.963 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4197376 len 4096 2024-09-16T00:34:28.963 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4197376 len 4096 2024-09-16T00:34:28.963 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4197376 len 8192 2024-09-16T00:34:28.963 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4197376 len 8192 2024-09-16T00:34:28.963 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4197376 len 8192 2024-09-16T00:34:28.963 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4197376 len 8192 2024-09-16T00:34:28.963 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4197376 len 16384 2024-09-16T00:34:28.963 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4197376 len 16384 2024-09-16T00:34:28.963 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4197376 len 16384 2024-09-16T00:34:28.963 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4197376 len 16384 2024-09-16T00:34:28.964 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4197888 len 1024 2024-09-16T00:34:28.964 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4197888 len 1024 2024-09-16T00:34:28.964 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4197888 len 1024 2024-09-16T00:34:28.964 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4197888 len 1024 2024-09-16T00:34:28.964 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4197888 len 2048 2024-09-16T00:34:28.964 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4197888 len 2048 2024-09-16T00:34:28.964 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4197888 len 2048 2024-09-16T00:34:28.964 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4197888 len 2048 2024-09-16T00:34:28.964 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4197888 len 4096 2024-09-16T00:34:28.964 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4197888 len 4096 2024-09-16T00:34:28.964 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4197888 len 4096 2024-09-16T00:34:28.964 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4197888 len 4096 2024-09-16T00:34:28.964 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4197888 len 8192 2024-09-16T00:34:28.965 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4197888 len 8192 2024-09-16T00:34:28.965 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4197888 len 8192 2024-09-16T00:34:28.965 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4197888 len 8192 2024-09-16T00:34:28.965 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 512 offset 4197888 len 16384 2024-09-16T00:34:28.965 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4197888 len 16384 2024-09-16T00:34:28.965 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 512 offset 4197888 len 16384 2024-09-16T00:34:28.965 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 512 offset 4197888 len 16384 2024-09-16T00:34:28.965 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4190208 len 1024 2024-09-16T00:34:28.965 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4190208 len 1024 2024-09-16T00:34:28.965 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4190208 len 1024 2024-09-16T00:34:28.965 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4190208 len 1024 2024-09-16T00:34:28.965 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4190208 len 2048 2024-09-16T00:34:28.965 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4190208 len 2048 2024-09-16T00:34:28.965 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4190208 len 2048 2024-09-16T00:34:28.966 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4190208 len 2048 2024-09-16T00:34:28.966 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4190208 len 4096 2024-09-16T00:34:28.966 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4190208 len 4096 2024-09-16T00:34:28.966 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4190208 len 4096 2024-09-16T00:34:28.966 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4190208 len 4096 2024-09-16T00:34:28.966 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4190208 len 8192 2024-09-16T00:34:28.966 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4190208 len 8192 2024-09-16T00:34:28.966 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4190208 len 8192 2024-09-16T00:34:28.966 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4190208 len 8192 2024-09-16T00:34:28.966 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4190208 len 16384 2024-09-16T00:34:28.966 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4190208 len 16384 2024-09-16T00:34:28.966 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4190208 len 16384 2024-09-16T00:34:28.966 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4190208 len 16384 2024-09-16T00:34:28.966 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4190720 len 1024 2024-09-16T00:34:28.967 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4190720 len 1024 2024-09-16T00:34:28.967 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4190720 len 1024 2024-09-16T00:34:28.967 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4190720 len 1024 2024-09-16T00:34:28.967 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4190720 len 2048 2024-09-16T00:34:28.967 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4190720 len 2048 2024-09-16T00:34:28.967 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4190720 len 2048 2024-09-16T00:34:28.967 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4190720 len 2048 2024-09-16T00:34:28.967 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4190720 len 4096 2024-09-16T00:34:28.967 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4190720 len 4096 2024-09-16T00:34:28.967 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4190720 len 4096 2024-09-16T00:34:28.967 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4190720 len 4096 2024-09-16T00:34:28.967 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4190720 len 8192 2024-09-16T00:34:28.967 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4190720 len 8192 2024-09-16T00:34:28.967 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4190720 len 8192 2024-09-16T00:34:28.968 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4190720 len 8192 2024-09-16T00:34:28.968 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4190720 len 16384 2024-09-16T00:34:28.968 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4190720 len 16384 2024-09-16T00:34:28.968 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4190720 len 16384 2024-09-16T00:34:28.968 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4190720 len 16384 2024-09-16T00:34:28.968 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4191232 len 1024 2024-09-16T00:34:28.968 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4191232 len 1024 2024-09-16T00:34:28.968 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4191232 len 1024 2024-09-16T00:34:28.968 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4191232 len 1024 2024-09-16T00:34:29.012 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4191232 len 2048 2024-09-16T00:34:29.013 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4191232 len 2048 2024-09-16T00:34:29.013 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4191232 len 2048 2024-09-16T00:34:29.013 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4191232 len 2048 2024-09-16T00:34:29.013 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4191232 len 4096 2024-09-16T00:34:29.013 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4191232 len 4096 2024-09-16T00:34:29.013 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4191232 len 4096 2024-09-16T00:34:29.013 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4191232 len 4096 2024-09-16T00:34:29.013 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4191232 len 8192 2024-09-16T00:34:29.013 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4191232 len 8192 2024-09-16T00:34:29.013 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4191232 len 8192 2024-09-16T00:34:29.013 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4191232 len 8192 2024-09-16T00:34:29.013 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4191232 len 16384 2024-09-16T00:34:29.013 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4191232 len 16384 2024-09-16T00:34:29.014 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4191232 len 16384 2024-09-16T00:34:29.014 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4191232 len 16384 2024-09-16T00:34:29.014 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4191744 len 1024 2024-09-16T00:34:29.014 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4191744 len 1024 2024-09-16T00:34:29.014 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4191744 len 1024 2024-09-16T00:34:29.014 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4191744 len 1024 2024-09-16T00:34:29.014 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4191744 len 2048 2024-09-16T00:34:29.014 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4191744 len 2048 2024-09-16T00:34:29.014 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4191744 len 2048 2024-09-16T00:34:29.014 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4191744 len 2048 2024-09-16T00:34:29.014 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4191744 len 4096 2024-09-16T00:34:29.014 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4191744 len 4096 2024-09-16T00:34:29.014 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4191744 len 4096 2024-09-16T00:34:29.014 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4191744 len 4096 2024-09-16T00:34:29.015 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4191744 len 8192 2024-09-16T00:34:29.015 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4191744 len 8192 2024-09-16T00:34:29.015 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4191744 len 8192 2024-09-16T00:34:29.015 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4191744 len 8192 2024-09-16T00:34:29.015 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4191744 len 16384 2024-09-16T00:34:29.015 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4191744 len 16384 2024-09-16T00:34:29.015 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4191744 len 16384 2024-09-16T00:34:29.015 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4191744 len 16384 2024-09-16T00:34:29.015 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4192256 len 1024 2024-09-16T00:34:29.015 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4192256 len 1024 2024-09-16T00:34:29.015 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4192256 len 1024 2024-09-16T00:34:29.015 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4192256 len 1024 2024-09-16T00:34:29.015 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4192256 len 2048 2024-09-16T00:34:29.016 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4192256 len 2048 2024-09-16T00:34:29.016 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4192256 len 2048 2024-09-16T00:34:29.016 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4192256 len 2048 2024-09-16T00:34:29.016 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4192256 len 4096 2024-09-16T00:34:29.016 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4192256 len 4096 2024-09-16T00:34:29.016 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4192256 len 4096 2024-09-16T00:34:29.016 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4192256 len 4096 2024-09-16T00:34:29.016 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4192256 len 8192 2024-09-16T00:34:29.016 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4192256 len 8192 2024-09-16T00:34:29.016 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4192256 len 8192 2024-09-16T00:34:29.016 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4192256 len 8192 2024-09-16T00:34:29.016 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4192256 len 16384 2024-09-16T00:34:29.016 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4192256 len 16384 2024-09-16T00:34:29.016 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4192256 len 16384 2024-09-16T00:34:29.017 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4192256 len 16384 2024-09-16T00:34:29.017 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4192768 len 1024 2024-09-16T00:34:29.017 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4192768 len 1024 2024-09-16T00:34:29.017 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4192768 len 1024 2024-09-16T00:34:29.017 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4192768 len 1024 2024-09-16T00:34:29.017 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4192768 len 2048 2024-09-16T00:34:29.017 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4192768 len 2048 2024-09-16T00:34:29.017 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4192768 len 2048 2024-09-16T00:34:29.017 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4192768 len 2048 2024-09-16T00:34:29.017 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4192768 len 4096 2024-09-16T00:34:29.017 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4192768 len 4096 2024-09-16T00:34:29.017 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4192768 len 4096 2024-09-16T00:34:29.017 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4192768 len 4096 2024-09-16T00:34:29.018 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4192768 len 8192 2024-09-16T00:34:29.018 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4192768 len 8192 2024-09-16T00:34:29.018 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4192768 len 8192 2024-09-16T00:34:29.018 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4192768 len 8192 2024-09-16T00:34:29.018 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4192768 len 16384 2024-09-16T00:34:29.018 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4192768 len 16384 2024-09-16T00:34:29.018 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4192768 len 16384 2024-09-16T00:34:29.018 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4192768 len 16384 2024-09-16T00:34:29.018 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4193280 len 1024 2024-09-16T00:34:29.018 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4193280 len 1024 2024-09-16T00:34:29.018 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4193280 len 1024 2024-09-16T00:34:29.018 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4193280 len 1024 2024-09-16T00:34:29.018 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4193280 len 2048 2024-09-16T00:34:29.018 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4193280 len 2048 2024-09-16T00:34:29.018 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4193280 len 2048 2024-09-16T00:34:29.054 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4193280 len 2048 2024-09-16T00:34:29.054 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4193280 len 4096 2024-09-16T00:34:29.054 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4193280 len 4096 2024-09-16T00:34:29.054 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4193280 len 4096 2024-09-16T00:34:29.054 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4193280 len 4096 2024-09-16T00:34:29.054 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4193280 len 8192 2024-09-16T00:34:29.054 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4193280 len 8192 2024-09-16T00:34:29.054 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4193280 len 8192 2024-09-16T00:34:29.054 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4193280 len 8192 2024-09-16T00:34:29.054 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4193280 len 16384 2024-09-16T00:34:29.054 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4193280 len 16384 2024-09-16T00:34:29.055 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4193280 len 16384 2024-09-16T00:34:29.055 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4193280 len 16384 2024-09-16T00:34:29.055 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4193792 len 1024 2024-09-16T00:34:29.055 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4193792 len 1024 2024-09-16T00:34:29.055 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4193792 len 1024 2024-09-16T00:34:29.055 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4193792 len 1024 2024-09-16T00:34:29.055 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4193792 len 2048 2024-09-16T00:34:29.055 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4193792 len 2048 2024-09-16T00:34:29.055 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4193792 len 2048 2024-09-16T00:34:29.055 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4193792 len 2048 2024-09-16T00:34:29.055 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4193792 len 4096 2024-09-16T00:34:29.055 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4193792 len 4096 2024-09-16T00:34:29.055 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4193792 len 4096 2024-09-16T00:34:29.056 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4193792 len 4096 2024-09-16T00:34:29.056 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4193792 len 8192 2024-09-16T00:34:29.056 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4193792 len 8192 2024-09-16T00:34:29.056 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4193792 len 8192 2024-09-16T00:34:29.056 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4193792 len 8192 2024-09-16T00:34:29.056 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4193792 len 16384 2024-09-16T00:34:29.056 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4193792 len 16384 2024-09-16T00:34:29.056 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4193792 len 16384 2024-09-16T00:34:29.056 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4193792 len 16384 2024-09-16T00:34:29.056 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4194304 len 1024 2024-09-16T00:34:29.056 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4194304 len 1024 2024-09-16T00:34:29.056 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4194304 len 1024 2024-09-16T00:34:29.056 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4194304 len 1024 2024-09-16T00:34:29.056 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4194304 len 2048 2024-09-16T00:34:29.056 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4194304 len 2048 2024-09-16T00:34:29.057 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4194304 len 2048 2024-09-16T00:34:29.057 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4194304 len 2048 2024-09-16T00:34:29.057 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4194304 len 4096 2024-09-16T00:34:29.057 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4194304 len 4096 2024-09-16T00:34:29.057 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4194304 len 4096 2024-09-16T00:34:29.057 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4194304 len 4096 2024-09-16T00:34:29.057 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4194304 len 8192 2024-09-16T00:34:29.057 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4194304 len 8192 2024-09-16T00:34:29.057 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4194304 len 8192 2024-09-16T00:34:29.057 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4194304 len 8192 2024-09-16T00:34:29.057 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4194304 len 16384 2024-09-16T00:34:29.057 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4194304 len 16384 2024-09-16T00:34:29.057 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4194304 len 16384 2024-09-16T00:34:29.058 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4194304 len 16384 2024-09-16T00:34:29.058 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4194816 len 1024 2024-09-16T00:34:29.058 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4194816 len 1024 2024-09-16T00:34:29.058 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4194816 len 1024 2024-09-16T00:34:29.058 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4194816 len 1024 2024-09-16T00:34:29.058 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4194816 len 2048 2024-09-16T00:34:29.058 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4194816 len 2048 2024-09-16T00:34:29.058 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4194816 len 2048 2024-09-16T00:34:29.058 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4194816 len 2048 2024-09-16T00:34:29.058 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4194816 len 4096 2024-09-16T00:34:29.058 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4194816 len 4096 2024-09-16T00:34:29.058 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4194816 len 4096 2024-09-16T00:34:29.058 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4194816 len 4096 2024-09-16T00:34:29.058 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4194816 len 8192 2024-09-16T00:34:29.059 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4194816 len 8192 2024-09-16T00:34:29.059 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4194816 len 8192 2024-09-16T00:34:29.059 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4194816 len 8192 2024-09-16T00:34:29.059 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4194816 len 16384 2024-09-16T00:34:29.059 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4194816 len 16384 2024-09-16T00:34:29.059 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4194816 len 16384 2024-09-16T00:34:29.059 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4194816 len 16384 2024-09-16T00:34:29.059 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4195328 len 1024 2024-09-16T00:34:29.059 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4195328 len 1024 2024-09-16T00:34:29.059 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4195328 len 1024 2024-09-16T00:34:29.059 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4195328 len 1024 2024-09-16T00:34:29.059 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4195328 len 2048 2024-09-16T00:34:29.059 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4195328 len 2048 2024-09-16T00:34:29.059 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4195328 len 2048 2024-09-16T00:34:29.059 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4195328 len 2048 2024-09-16T00:34:29.060 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4195328 len 4096 2024-09-16T00:34:29.080 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4195328 len 4096 2024-09-16T00:34:29.080 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4195328 len 4096 2024-09-16T00:34:29.080 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4195328 len 4096 2024-09-16T00:34:29.080 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4195328 len 8192 2024-09-16T00:34:29.080 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4195328 len 8192 2024-09-16T00:34:29.081 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4195328 len 8192 2024-09-16T00:34:29.081 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4195328 len 8192 2024-09-16T00:34:29.081 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4195328 len 16384 2024-09-16T00:34:29.081 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4195328 len 16384 2024-09-16T00:34:29.081 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4195328 len 16384 2024-09-16T00:34:29.081 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4195328 len 16384 2024-09-16T00:34:29.081 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4195840 len 1024 2024-09-16T00:34:29.081 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4195840 len 1024 2024-09-16T00:34:29.081 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4195840 len 1024 2024-09-16T00:34:29.081 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4195840 len 1024 2024-09-16T00:34:29.081 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4195840 len 2048 2024-09-16T00:34:29.081 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4195840 len 2048 2024-09-16T00:34:29.081 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4195840 len 2048 2024-09-16T00:34:29.081 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4195840 len 2048 2024-09-16T00:34:29.082 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4195840 len 4096 2024-09-16T00:34:29.082 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4195840 len 4096 2024-09-16T00:34:29.082 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4195840 len 4096 2024-09-16T00:34:29.082 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4195840 len 4096 2024-09-16T00:34:29.082 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4195840 len 8192 2024-09-16T00:34:29.082 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4195840 len 8192 2024-09-16T00:34:29.082 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4195840 len 8192 2024-09-16T00:34:29.082 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4195840 len 8192 2024-09-16T00:34:29.082 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4195840 len 16384 2024-09-16T00:34:29.082 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4195840 len 16384 2024-09-16T00:34:29.082 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4195840 len 16384 2024-09-16T00:34:29.082 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4195840 len 16384 2024-09-16T00:34:29.082 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4196352 len 1024 2024-09-16T00:34:29.082 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4196352 len 1024 2024-09-16T00:34:29.083 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4196352 len 1024 2024-09-16T00:34:29.083 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4196352 len 1024 2024-09-16T00:34:29.083 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4196352 len 2048 2024-09-16T00:34:29.083 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4196352 len 2048 2024-09-16T00:34:29.083 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4196352 len 2048 2024-09-16T00:34:29.083 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4196352 len 2048 2024-09-16T00:34:29.083 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4196352 len 4096 2024-09-16T00:34:29.083 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4196352 len 4096 2024-09-16T00:34:29.083 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4196352 len 4096 2024-09-16T00:34:29.083 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4196352 len 4096 2024-09-16T00:34:29.083 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4196352 len 8192 2024-09-16T00:34:29.083 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4196352 len 8192 2024-09-16T00:34:29.083 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4196352 len 8192 2024-09-16T00:34:29.084 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4196352 len 8192 2024-09-16T00:34:29.084 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4196352 len 16384 2024-09-16T00:34:29.084 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4196352 len 16384 2024-09-16T00:34:29.084 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4196352 len 16384 2024-09-16T00:34:29.084 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4196352 len 16384 2024-09-16T00:34:29.084 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4196864 len 1024 2024-09-16T00:34:29.084 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4196864 len 1024 2024-09-16T00:34:29.084 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4196864 len 1024 2024-09-16T00:34:29.084 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4196864 len 1024 2024-09-16T00:34:29.084 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4196864 len 2048 2024-09-16T00:34:29.084 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4196864 len 2048 2024-09-16T00:34:29.084 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4196864 len 2048 2024-09-16T00:34:29.084 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4196864 len 2048 2024-09-16T00:34:29.084 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4196864 len 4096 2024-09-16T00:34:29.085 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4196864 len 4096 2024-09-16T00:34:29.085 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4196864 len 4096 2024-09-16T00:34:29.085 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4196864 len 4096 2024-09-16T00:34:29.085 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4196864 len 8192 2024-09-16T00:34:29.085 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4196864 len 8192 2024-09-16T00:34:29.085 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4196864 len 8192 2024-09-16T00:34:29.085 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4196864 len 8192 2024-09-16T00:34:29.085 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4196864 len 16384 2024-09-16T00:34:29.085 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4196864 len 16384 2024-09-16T00:34:29.085 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4196864 len 16384 2024-09-16T00:34:29.085 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4196864 len 16384 2024-09-16T00:34:29.085 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4197376 len 1024 2024-09-16T00:34:29.085 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4197376 len 1024 2024-09-16T00:34:29.085 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4197376 len 1024 2024-09-16T00:34:29.086 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4197376 len 1024 2024-09-16T00:34:29.086 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4197376 len 2048 2024-09-16T00:34:29.086 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4197376 len 2048 2024-09-16T00:34:29.086 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4197376 len 2048 2024-09-16T00:34:29.086 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4197376 len 2048 2024-09-16T00:34:29.086 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4197376 len 4096 2024-09-16T00:34:29.086 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4197376 len 4096 2024-09-16T00:34:29.086 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4197376 len 4096 2024-09-16T00:34:29.086 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4197376 len 4096 2024-09-16T00:34:29.119 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4197376 len 8192 2024-09-16T00:34:29.119 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4197376 len 8192 2024-09-16T00:34:29.120 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4197376 len 8192 2024-09-16T00:34:29.120 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4197376 len 8192 2024-09-16T00:34:29.120 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4197376 len 16384 2024-09-16T00:34:29.120 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4197376 len 16384 2024-09-16T00:34:29.120 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4197376 len 16384 2024-09-16T00:34:29.120 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4197376 len 16384 2024-09-16T00:34:29.120 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4197888 len 1024 2024-09-16T00:34:29.120 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4197888 len 1024 2024-09-16T00:34:29.120 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4197888 len 1024 2024-09-16T00:34:29.120 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4197888 len 1024 2024-09-16T00:34:29.120 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4197888 len 2048 2024-09-16T00:34:29.120 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4197888 len 2048 2024-09-16T00:34:29.120 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4197888 len 2048 2024-09-16T00:34:29.121 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4197888 len 2048 2024-09-16T00:34:29.121 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4197888 len 4096 2024-09-16T00:34:29.121 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4197888 len 4096 2024-09-16T00:34:29.121 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4197888 len 4096 2024-09-16T00:34:29.121 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4197888 len 4096 2024-09-16T00:34:29.121 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4197888 len 8192 2024-09-16T00:34:29.121 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4197888 len 8192 2024-09-16T00:34:29.121 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4197888 len 8192 2024-09-16T00:34:29.121 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4197888 len 8192 2024-09-16T00:34:29.121 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1024 offset 4197888 len 16384 2024-09-16T00:34:29.121 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4197888 len 16384 2024-09-16T00:34:29.121 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1024 offset 4197888 len 16384 2024-09-16T00:34:29.121 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1024 offset 4197888 len 16384 2024-09-16T00:34:29.121 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4190208 len 1024 2024-09-16T00:34:29.122 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4190208 len 1024 2024-09-16T00:34:29.122 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4190208 len 1024 2024-09-16T00:34:29.122 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4190208 len 1024 2024-09-16T00:34:29.122 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4190208 len 2048 2024-09-16T00:34:29.122 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4190208 len 2048 2024-09-16T00:34:29.122 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4190208 len 2048 2024-09-16T00:34:29.122 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4190208 len 2048 2024-09-16T00:34:29.122 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4190208 len 4096 2024-09-16T00:34:29.122 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4190208 len 4096 2024-09-16T00:34:29.122 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4190208 len 4096 2024-09-16T00:34:29.122 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4190208 len 4096 2024-09-16T00:34:29.122 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4190208 len 8192 2024-09-16T00:34:29.122 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4190208 len 8192 2024-09-16T00:34:29.122 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4190208 len 8192 2024-09-16T00:34:29.123 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4190208 len 8192 2024-09-16T00:34:29.123 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4190208 len 16384 2024-09-16T00:34:29.123 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4190208 len 16384 2024-09-16T00:34:29.123 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4190208 len 16384 2024-09-16T00:34:29.123 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4190208 len 16384 2024-09-16T00:34:29.123 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4190720 len 1024 2024-09-16T00:34:29.123 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4190720 len 1024 2024-09-16T00:34:29.123 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4190720 len 1024 2024-09-16T00:34:29.123 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4190720 len 1024 2024-09-16T00:34:29.123 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4190720 len 2048 2024-09-16T00:34:29.123 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4190720 len 2048 2024-09-16T00:34:29.123 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4190720 len 2048 2024-09-16T00:34:29.123 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4190720 len 2048 2024-09-16T00:34:29.123 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4190720 len 4096 2024-09-16T00:34:29.124 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4190720 len 4096 2024-09-16T00:34:29.124 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4190720 len 4096 2024-09-16T00:34:29.124 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4190720 len 4096 2024-09-16T00:34:29.124 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4190720 len 8192 2024-09-16T00:34:29.124 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4190720 len 8192 2024-09-16T00:34:29.124 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4190720 len 8192 2024-09-16T00:34:29.124 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4190720 len 8192 2024-09-16T00:34:29.124 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4190720 len 16384 2024-09-16T00:34:29.124 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4190720 len 16384 2024-09-16T00:34:29.124 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4190720 len 16384 2024-09-16T00:34:29.124 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4190720 len 16384 2024-09-16T00:34:29.124 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4191232 len 1024 2024-09-16T00:34:29.124 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4191232 len 1024 2024-09-16T00:34:29.125 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4191232 len 1024 2024-09-16T00:34:29.125 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4191232 len 1024 2024-09-16T00:34:29.125 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4191232 len 2048 2024-09-16T00:34:29.125 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4191232 len 2048 2024-09-16T00:34:29.125 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4191232 len 2048 2024-09-16T00:34:29.125 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4191232 len 2048 2024-09-16T00:34:29.125 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4191232 len 4096 2024-09-16T00:34:29.125 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4191232 len 4096 2024-09-16T00:34:29.125 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4191232 len 4096 2024-09-16T00:34:29.125 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4191232 len 4096 2024-09-16T00:34:29.125 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4191232 len 8192 2024-09-16T00:34:29.125 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4191232 len 8192 2024-09-16T00:34:29.169 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4191232 len 8192 2024-09-16T00:34:29.169 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4191232 len 8192 2024-09-16T00:34:29.169 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4191232 len 16384 2024-09-16T00:34:29.169 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4191232 len 16384 2024-09-16T00:34:29.169 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4191232 len 16384 2024-09-16T00:34:29.169 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4191232 len 16384 2024-09-16T00:34:29.169 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4191744 len 1024 2024-09-16T00:34:29.169 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4191744 len 1024 2024-09-16T00:34:29.170 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4191744 len 1024 2024-09-16T00:34:29.170 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4191744 len 1024 2024-09-16T00:34:29.170 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4191744 len 2048 2024-09-16T00:34:29.170 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4191744 len 2048 2024-09-16T00:34:29.170 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4191744 len 2048 2024-09-16T00:34:29.170 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4191744 len 2048 2024-09-16T00:34:29.170 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4191744 len 4096 2024-09-16T00:34:29.170 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4191744 len 4096 2024-09-16T00:34:29.170 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4191744 len 4096 2024-09-16T00:34:29.170 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4191744 len 4096 2024-09-16T00:34:29.170 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4191744 len 8192 2024-09-16T00:34:29.170 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4191744 len 8192 2024-09-16T00:34:29.170 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4191744 len 8192 2024-09-16T00:34:29.170 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4191744 len 8192 2024-09-16T00:34:29.171 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4191744 len 16384 2024-09-16T00:34:29.171 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4191744 len 16384 2024-09-16T00:34:29.171 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4191744 len 16384 2024-09-16T00:34:29.171 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4191744 len 16384 2024-09-16T00:34:29.171 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4192256 len 1024 2024-09-16T00:34:29.171 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4192256 len 1024 2024-09-16T00:34:29.171 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4192256 len 1024 2024-09-16T00:34:29.171 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4192256 len 1024 2024-09-16T00:34:29.171 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4192256 len 2048 2024-09-16T00:34:29.171 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4192256 len 2048 2024-09-16T00:34:29.171 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4192256 len 2048 2024-09-16T00:34:29.171 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4192256 len 2048 2024-09-16T00:34:29.171 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4192256 len 4096 2024-09-16T00:34:29.171 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4192256 len 4096 2024-09-16T00:34:29.172 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4192256 len 4096 2024-09-16T00:34:29.172 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4192256 len 4096 2024-09-16T00:34:29.172 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4192256 len 8192 2024-09-16T00:34:29.172 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4192256 len 8192 2024-09-16T00:34:29.172 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4192256 len 8192 2024-09-16T00:34:29.172 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4192256 len 8192 2024-09-16T00:34:29.172 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4192256 len 16384 2024-09-16T00:34:29.172 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4192256 len 16384 2024-09-16T00:34:29.172 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4192256 len 16384 2024-09-16T00:34:29.172 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4192256 len 16384 2024-09-16T00:34:29.172 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4192768 len 1024 2024-09-16T00:34:29.172 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4192768 len 1024 2024-09-16T00:34:29.172 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4192768 len 1024 2024-09-16T00:34:29.172 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4192768 len 1024 2024-09-16T00:34:29.173 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4192768 len 2048 2024-09-16T00:34:29.173 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4192768 len 2048 2024-09-16T00:34:29.173 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4192768 len 2048 2024-09-16T00:34:29.173 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4192768 len 2048 2024-09-16T00:34:29.173 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4192768 len 4096 2024-09-16T00:34:29.173 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4192768 len 4096 2024-09-16T00:34:29.173 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4192768 len 4096 2024-09-16T00:34:29.173 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4192768 len 4096 2024-09-16T00:34:29.173 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4192768 len 8192 2024-09-16T00:34:29.173 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4192768 len 8192 2024-09-16T00:34:29.173 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4192768 len 8192 2024-09-16T00:34:29.173 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4192768 len 8192 2024-09-16T00:34:29.173 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4192768 len 16384 2024-09-16T00:34:29.173 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4192768 len 16384 2024-09-16T00:34:29.174 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4192768 len 16384 2024-09-16T00:34:29.174 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4192768 len 16384 2024-09-16T00:34:29.174 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4193280 len 1024 2024-09-16T00:34:29.174 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4193280 len 1024 2024-09-16T00:34:29.174 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4193280 len 1024 2024-09-16T00:34:29.174 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4193280 len 1024 2024-09-16T00:34:29.174 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4193280 len 2048 2024-09-16T00:34:29.174 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4193280 len 2048 2024-09-16T00:34:29.174 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4193280 len 2048 2024-09-16T00:34:29.174 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4193280 len 2048 2024-09-16T00:34:29.174 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4193280 len 4096 2024-09-16T00:34:29.174 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4193280 len 4096 2024-09-16T00:34:29.174 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4193280 len 4096 2024-09-16T00:34:29.174 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4193280 len 4096 2024-09-16T00:34:29.175 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4193280 len 8192 2024-09-16T00:34:29.175 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4193280 len 8192 2024-09-16T00:34:29.175 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4193280 len 8192 2024-09-16T00:34:29.175 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4193280 len 8192 2024-09-16T00:34:29.202 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4193280 len 16384 2024-09-16T00:34:29.202 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4193280 len 16384 2024-09-16T00:34:29.202 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4193280 len 16384 2024-09-16T00:34:29.202 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4193280 len 16384 2024-09-16T00:34:29.202 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4193792 len 1024 2024-09-16T00:34:29.202 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4193792 len 1024 2024-09-16T00:34:29.202 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4193792 len 1024 2024-09-16T00:34:29.202 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4193792 len 1024 2024-09-16T00:34:29.203 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4193792 len 2048 2024-09-16T00:34:29.203 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4193792 len 2048 2024-09-16T00:34:29.203 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4193792 len 2048 2024-09-16T00:34:29.203 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4193792 len 2048 2024-09-16T00:34:29.203 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4193792 len 4096 2024-09-16T00:34:29.203 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4193792 len 4096 2024-09-16T00:34:29.203 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4193792 len 4096 2024-09-16T00:34:29.203 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4193792 len 4096 2024-09-16T00:34:29.203 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4193792 len 8192 2024-09-16T00:34:29.203 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4193792 len 8192 2024-09-16T00:34:29.204 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4193792 len 8192 2024-09-16T00:34:29.204 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4193792 len 8192 2024-09-16T00:34:29.204 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4193792 len 16384 2024-09-16T00:34:29.204 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4193792 len 16384 2024-09-16T00:34:29.204 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4193792 len 16384 2024-09-16T00:34:29.204 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4193792 len 16384 2024-09-16T00:34:29.204 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4194304 len 1024 2024-09-16T00:34:29.204 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4194304 len 1024 2024-09-16T00:34:29.204 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4194304 len 1024 2024-09-16T00:34:29.204 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4194304 len 1024 2024-09-16T00:34:29.205 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4194304 len 2048 2024-09-16T00:34:29.205 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4194304 len 2048 2024-09-16T00:34:29.205 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4194304 len 2048 2024-09-16T00:34:29.205 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4194304 len 2048 2024-09-16T00:34:29.205 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4194304 len 4096 2024-09-16T00:34:29.205 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4194304 len 4096 2024-09-16T00:34:29.205 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4194304 len 4096 2024-09-16T00:34:29.205 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4194304 len 4096 2024-09-16T00:34:29.205 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4194304 len 8192 2024-09-16T00:34:29.205 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4194304 len 8192 2024-09-16T00:34:29.206 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4194304 len 8192 2024-09-16T00:34:29.206 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4194304 len 8192 2024-09-16T00:34:29.206 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4194304 len 16384 2024-09-16T00:34:29.206 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4194304 len 16384 2024-09-16T00:34:29.206 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4194304 len 16384 2024-09-16T00:34:29.206 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4194304 len 16384 2024-09-16T00:34:29.206 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4194816 len 1024 2024-09-16T00:34:29.206 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4194816 len 1024 2024-09-16T00:34:29.206 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4194816 len 1024 2024-09-16T00:34:29.206 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4194816 len 1024 2024-09-16T00:34:29.206 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4194816 len 2048 2024-09-16T00:34:29.207 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4194816 len 2048 2024-09-16T00:34:29.207 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4194816 len 2048 2024-09-16T00:34:29.207 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4194816 len 2048 2024-09-16T00:34:29.207 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4194816 len 4096 2024-09-16T00:34:29.207 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4194816 len 4096 2024-09-16T00:34:29.207 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4194816 len 4096 2024-09-16T00:34:29.207 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4194816 len 4096 2024-09-16T00:34:29.207 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4194816 len 8192 2024-09-16T00:34:29.207 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4194816 len 8192 2024-09-16T00:34:29.207 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4194816 len 8192 2024-09-16T00:34:29.208 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4194816 len 8192 2024-09-16T00:34:29.208 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4194816 len 16384 2024-09-16T00:34:29.208 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4194816 len 16384 2024-09-16T00:34:29.208 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4194816 len 16384 2024-09-16T00:34:29.208 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4194816 len 16384 2024-09-16T00:34:29.208 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4195328 len 1024 2024-09-16T00:34:29.208 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4195328 len 1024 2024-09-16T00:34:29.208 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4195328 len 1024 2024-09-16T00:34:29.208 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4195328 len 1024 2024-09-16T00:34:29.208 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4195328 len 2048 2024-09-16T00:34:29.209 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4195328 len 2048 2024-09-16T00:34:29.209 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4195328 len 2048 2024-09-16T00:34:29.209 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4195328 len 2048 2024-09-16T00:34:29.209 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4195328 len 4096 2024-09-16T00:34:29.209 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4195328 len 4096 2024-09-16T00:34:29.209 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4195328 len 4096 2024-09-16T00:34:29.209 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4195328 len 4096 2024-09-16T00:34:29.209 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4195328 len 8192 2024-09-16T00:34:29.209 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4195328 len 8192 2024-09-16T00:34:29.209 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4195328 len 8192 2024-09-16T00:34:29.210 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4195328 len 8192 2024-09-16T00:34:29.210 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4195328 len 16384 2024-09-16T00:34:29.210 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4195328 len 16384 2024-09-16T00:34:29.210 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4195328 len 16384 2024-09-16T00:34:29.214 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:29 smithi038 ceph-mon[30945]: pgmap v415: 129 pgs: 129 active+clean; 761 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.6 KiB/s wr, 498 op/s 2024-09-16T00:34:29.214 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:29 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/1009667578' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:34:29.226 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4195328 len 16384 2024-09-16T00:34:29.226 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4195840 len 1024 2024-09-16T00:34:29.226 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4195840 len 1024 2024-09-16T00:34:29.226 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4195840 len 1024 2024-09-16T00:34:29.226 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4195840 len 1024 2024-09-16T00:34:29.226 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4195840 len 2048 2024-09-16T00:34:29.227 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4195840 len 2048 2024-09-16T00:34:29.227 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4195840 len 2048 2024-09-16T00:34:29.227 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4195840 len 2048 2024-09-16T00:34:29.227 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4195840 len 4096 2024-09-16T00:34:29.227 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4195840 len 4096 2024-09-16T00:34:29.227 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4195840 len 4096 2024-09-16T00:34:29.227 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4195840 len 4096 2024-09-16T00:34:29.227 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4195840 len 8192 2024-09-16T00:34:29.227 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4195840 len 8192 2024-09-16T00:34:29.228 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4195840 len 8192 2024-09-16T00:34:29.228 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4195840 len 8192 2024-09-16T00:34:29.228 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4195840 len 16384 2024-09-16T00:34:29.228 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4195840 len 16384 2024-09-16T00:34:29.228 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4195840 len 16384 2024-09-16T00:34:29.228 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4195840 len 16384 2024-09-16T00:34:29.228 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4196352 len 1024 2024-09-16T00:34:29.228 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4196352 len 1024 2024-09-16T00:34:29.228 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4196352 len 1024 2024-09-16T00:34:29.228 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4196352 len 1024 2024-09-16T00:34:29.229 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4196352 len 2048 2024-09-16T00:34:29.229 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4196352 len 2048 2024-09-16T00:34:29.229 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4196352 len 2048 2024-09-16T00:34:29.229 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4196352 len 2048 2024-09-16T00:34:29.229 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4196352 len 4096 2024-09-16T00:34:29.229 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4196352 len 4096 2024-09-16T00:34:29.229 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4196352 len 4096 2024-09-16T00:34:29.229 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4196352 len 4096 2024-09-16T00:34:29.229 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4196352 len 8192 2024-09-16T00:34:29.229 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4196352 len 8192 2024-09-16T00:34:29.230 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4196352 len 8192 2024-09-16T00:34:29.230 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4196352 len 8192 2024-09-16T00:34:29.230 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4196352 len 16384 2024-09-16T00:34:29.230 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4196352 len 16384 2024-09-16T00:34:29.230 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4196352 len 16384 2024-09-16T00:34:29.230 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4196352 len 16384 2024-09-16T00:34:29.230 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4196864 len 1024 2024-09-16T00:34:29.230 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4196864 len 1024 2024-09-16T00:34:29.230 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4196864 len 1024 2024-09-16T00:34:29.230 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4196864 len 1024 2024-09-16T00:34:29.231 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4196864 len 2048 2024-09-16T00:34:29.231 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4196864 len 2048 2024-09-16T00:34:29.231 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4196864 len 2048 2024-09-16T00:34:29.231 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4196864 len 2048 2024-09-16T00:34:29.231 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4196864 len 4096 2024-09-16T00:34:29.231 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4196864 len 4096 2024-09-16T00:34:29.231 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4196864 len 4096 2024-09-16T00:34:29.231 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4196864 len 4096 2024-09-16T00:34:29.231 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4196864 len 8192 2024-09-16T00:34:29.231 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4196864 len 8192 2024-09-16T00:34:29.232 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4196864 len 8192 2024-09-16T00:34:29.232 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4196864 len 8192 2024-09-16T00:34:29.232 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4196864 len 16384 2024-09-16T00:34:29.232 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4196864 len 16384 2024-09-16T00:34:29.232 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4196864 len 16384 2024-09-16T00:34:29.232 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4196864 len 16384 2024-09-16T00:34:29.232 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4197376 len 1024 2024-09-16T00:34:29.232 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4197376 len 1024 2024-09-16T00:34:29.232 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4197376 len 1024 2024-09-16T00:34:29.232 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4197376 len 1024 2024-09-16T00:34:29.233 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4197376 len 2048 2024-09-16T00:34:29.233 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4197376 len 2048 2024-09-16T00:34:29.233 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4197376 len 2048 2024-09-16T00:34:29.233 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4197376 len 2048 2024-09-16T00:34:29.233 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4197376 len 4096 2024-09-16T00:34:29.233 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4197376 len 4096 2024-09-16T00:34:29.233 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4197376 len 4096 2024-09-16T00:34:29.233 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4197376 len 4096 2024-09-16T00:34:29.234 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4197376 len 8192 2024-09-16T00:34:29.234 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4197376 len 8192 2024-09-16T00:34:29.234 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4197376 len 8192 2024-09-16T00:34:29.234 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4197376 len 8192 2024-09-16T00:34:29.234 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4197376 len 16384 2024-09-16T00:34:29.234 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4197376 len 16384 2024-09-16T00:34:29.234 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4197376 len 16384 2024-09-16T00:34:29.234 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4197376 len 16384 2024-09-16T00:34:29.234 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4197888 len 1024 2024-09-16T00:34:29.270 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4197888 len 1024 2024-09-16T00:34:29.270 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4197888 len 1024 2024-09-16T00:34:29.270 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4197888 len 1024 2024-09-16T00:34:29.270 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4197888 len 2048 2024-09-16T00:34:29.271 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4197888 len 2048 2024-09-16T00:34:29.271 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4197888 len 2048 2024-09-16T00:34:29.271 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4197888 len 2048 2024-09-16T00:34:29.271 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4197888 len 4096 2024-09-16T00:34:29.271 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4197888 len 4096 2024-09-16T00:34:29.271 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4197888 len 4096 2024-09-16T00:34:29.271 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4197888 len 4096 2024-09-16T00:34:29.271 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4197888 len 8192 2024-09-16T00:34:29.271 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4197888 len 8192 2024-09-16T00:34:29.271 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4197888 len 8192 2024-09-16T00:34:29.271 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4197888 len 8192 2024-09-16T00:34:29.271 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 1536 offset 4197888 len 16384 2024-09-16T00:34:29.271 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4197888 len 16384 2024-09-16T00:34:29.271 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 1536 offset 4197888 len 16384 2024-09-16T00:34:29.272 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 1536 offset 4197888 len 16384 2024-09-16T00:34:29.272 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4190208 len 1024 2024-09-16T00:34:29.272 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4190208 len 1024 2024-09-16T00:34:29.272 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4190208 len 1024 2024-09-16T00:34:29.272 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4190208 len 1024 2024-09-16T00:34:29.272 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4190208 len 2048 2024-09-16T00:34:29.272 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4190208 len 2048 2024-09-16T00:34:29.272 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4190208 len 2048 2024-09-16T00:34:29.272 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4190208 len 2048 2024-09-16T00:34:29.272 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4190208 len 4096 2024-09-16T00:34:29.272 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4190208 len 4096 2024-09-16T00:34:29.272 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4190208 len 4096 2024-09-16T00:34:29.272 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4190208 len 4096 2024-09-16T00:34:29.273 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4190208 len 8192 2024-09-16T00:34:29.273 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4190208 len 8192 2024-09-16T00:34:29.273 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4190208 len 8192 2024-09-16T00:34:29.273 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4190208 len 8192 2024-09-16T00:34:29.273 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4190208 len 16384 2024-09-16T00:34:29.273 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4190208 len 16384 2024-09-16T00:34:29.273 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4190208 len 16384 2024-09-16T00:34:29.273 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4190208 len 16384 2024-09-16T00:34:29.273 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4190720 len 1024 2024-09-16T00:34:29.273 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4190720 len 1024 2024-09-16T00:34:29.273 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4190720 len 1024 2024-09-16T00:34:29.273 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4190720 len 1024 2024-09-16T00:34:29.273 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4190720 len 2048 2024-09-16T00:34:29.273 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4190720 len 2048 2024-09-16T00:34:29.274 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4190720 len 2048 2024-09-16T00:34:29.274 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4190720 len 2048 2024-09-16T00:34:29.274 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4190720 len 4096 2024-09-16T00:34:29.274 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4190720 len 4096 2024-09-16T00:34:29.274 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4190720 len 4096 2024-09-16T00:34:29.274 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4190720 len 4096 2024-09-16T00:34:29.274 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4190720 len 8192 2024-09-16T00:34:29.274 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4190720 len 8192 2024-09-16T00:34:29.274 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4190720 len 8192 2024-09-16T00:34:29.274 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4190720 len 8192 2024-09-16T00:34:29.274 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4190720 len 16384 2024-09-16T00:34:29.274 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4190720 len 16384 2024-09-16T00:34:29.274 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4190720 len 16384 2024-09-16T00:34:29.274 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4190720 len 16384 2024-09-16T00:34:29.275 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4191232 len 1024 2024-09-16T00:34:29.275 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4191232 len 1024 2024-09-16T00:34:29.275 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4191232 len 1024 2024-09-16T00:34:29.275 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4191232 len 1024 2024-09-16T00:34:29.275 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4191232 len 2048 2024-09-16T00:34:29.275 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4191232 len 2048 2024-09-16T00:34:29.275 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4191232 len 2048 2024-09-16T00:34:29.275 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4191232 len 2048 2024-09-16T00:34:29.275 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4191232 len 4096 2024-09-16T00:34:29.275 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4191232 len 4096 2024-09-16T00:34:29.275 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4191232 len 4096 2024-09-16T00:34:29.275 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4191232 len 4096 2024-09-16T00:34:29.275 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4191232 len 8192 2024-09-16T00:34:29.275 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4191232 len 8192 2024-09-16T00:34:29.275 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4191232 len 8192 2024-09-16T00:34:29.276 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4191232 len 8192 2024-09-16T00:34:29.276 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4191232 len 16384 2024-09-16T00:34:29.276 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4191232 len 16384 2024-09-16T00:34:29.276 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4191232 len 16384 2024-09-16T00:34:29.276 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4191232 len 16384 2024-09-16T00:34:29.276 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4191744 len 1024 2024-09-16T00:34:29.276 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4191744 len 1024 2024-09-16T00:34:29.276 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4191744 len 1024 2024-09-16T00:34:29.276 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4191744 len 1024 2024-09-16T00:34:29.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:29 smithi073 ceph-mon[35254]: pgmap v415: 129 pgs: 129 active+clean; 761 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.6 KiB/s wr, 498 op/s 2024-09-16T00:34:29.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:29 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/1009667578' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:34:29.320 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4191744 len 2048 2024-09-16T00:34:29.321 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4191744 len 2048 2024-09-16T00:34:29.321 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4191744 len 2048 2024-09-16T00:34:29.321 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4191744 len 2048 2024-09-16T00:34:29.321 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4191744 len 4096 2024-09-16T00:34:29.321 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4191744 len 4096 2024-09-16T00:34:29.321 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4191744 len 4096 2024-09-16T00:34:29.321 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4191744 len 4096 2024-09-16T00:34:29.321 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4191744 len 8192 2024-09-16T00:34:29.321 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4191744 len 8192 2024-09-16T00:34:29.321 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4191744 len 8192 2024-09-16T00:34:29.322 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4191744 len 8192 2024-09-16T00:34:29.322 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4191744 len 16384 2024-09-16T00:34:29.322 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4191744 len 16384 2024-09-16T00:34:29.322 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4191744 len 16384 2024-09-16T00:34:29.322 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4191744 len 16384 2024-09-16T00:34:29.322 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4192256 len 1024 2024-09-16T00:34:29.322 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4192256 len 1024 2024-09-16T00:34:29.322 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4192256 len 1024 2024-09-16T00:34:29.322 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4192256 len 1024 2024-09-16T00:34:29.322 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4192256 len 2048 2024-09-16T00:34:29.322 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4192256 len 2048 2024-09-16T00:34:29.322 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4192256 len 2048 2024-09-16T00:34:29.323 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4192256 len 2048 2024-09-16T00:34:29.323 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4192256 len 4096 2024-09-16T00:34:29.323 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4192256 len 4096 2024-09-16T00:34:29.323 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4192256 len 4096 2024-09-16T00:34:29.323 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4192256 len 4096 2024-09-16T00:34:29.323 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4192256 len 8192 2024-09-16T00:34:29.323 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4192256 len 8192 2024-09-16T00:34:29.323 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4192256 len 8192 2024-09-16T00:34:29.323 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4192256 len 8192 2024-09-16T00:34:29.323 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4192256 len 16384 2024-09-16T00:34:29.323 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4192256 len 16384 2024-09-16T00:34:29.323 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4192256 len 16384 2024-09-16T00:34:29.324 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4192256 len 16384 2024-09-16T00:34:29.324 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4192768 len 1024 2024-09-16T00:34:29.324 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4192768 len 1024 2024-09-16T00:34:29.324 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4192768 len 1024 2024-09-16T00:34:29.324 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4192768 len 1024 2024-09-16T00:34:29.324 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4192768 len 2048 2024-09-16T00:34:29.324 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4192768 len 2048 2024-09-16T00:34:29.324 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4192768 len 2048 2024-09-16T00:34:29.324 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4192768 len 2048 2024-09-16T00:34:29.324 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4192768 len 4096 2024-09-16T00:34:29.324 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4192768 len 4096 2024-09-16T00:34:29.324 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4192768 len 4096 2024-09-16T00:34:29.325 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4192768 len 4096 2024-09-16T00:34:29.325 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4192768 len 8192 2024-09-16T00:34:29.325 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4192768 len 8192 2024-09-16T00:34:29.325 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4192768 len 8192 2024-09-16T00:34:29.325 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4192768 len 8192 2024-09-16T00:34:29.325 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4192768 len 16384 2024-09-16T00:34:29.325 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4192768 len 16384 2024-09-16T00:34:29.325 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4192768 len 16384 2024-09-16T00:34:29.325 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4192768 len 16384 2024-09-16T00:34:29.325 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4193280 len 1024 2024-09-16T00:34:29.325 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4193280 len 1024 2024-09-16T00:34:29.325 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4193280 len 1024 2024-09-16T00:34:29.325 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4193280 len 1024 2024-09-16T00:34:29.326 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4193280 len 2048 2024-09-16T00:34:29.326 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4193280 len 2048 2024-09-16T00:34:29.326 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4193280 len 2048 2024-09-16T00:34:29.326 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4193280 len 2048 2024-09-16T00:34:29.326 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4193280 len 4096 2024-09-16T00:34:29.326 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4193280 len 4096 2024-09-16T00:34:29.326 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4193280 len 4096 2024-09-16T00:34:29.326 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4193280 len 4096 2024-09-16T00:34:29.326 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4193280 len 8192 2024-09-16T00:34:29.326 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4193280 len 8192 2024-09-16T00:34:29.326 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4193280 len 8192 2024-09-16T00:34:29.326 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4193280 len 8192 2024-09-16T00:34:29.327 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4193280 len 16384 2024-09-16T00:34:29.327 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4193280 len 16384 2024-09-16T00:34:29.327 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4193280 len 16384 2024-09-16T00:34:29.327 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4193280 len 16384 2024-09-16T00:34:29.327 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4193792 len 1024 2024-09-16T00:34:29.327 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4193792 len 1024 2024-09-16T00:34:29.327 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4193792 len 1024 2024-09-16T00:34:29.327 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4193792 len 1024 2024-09-16T00:34:29.327 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4193792 len 2048 2024-09-16T00:34:29.327 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4193792 len 2048 2024-09-16T00:34:29.352 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4193792 len 2048 2024-09-16T00:34:29.352 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4193792 len 2048 2024-09-16T00:34:29.353 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4193792 len 4096 2024-09-16T00:34:29.353 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4193792 len 4096 2024-09-16T00:34:29.353 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4193792 len 4096 2024-09-16T00:34:29.353 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4193792 len 4096 2024-09-16T00:34:29.353 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4193792 len 8192 2024-09-16T00:34:29.353 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4193792 len 8192 2024-09-16T00:34:29.353 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4193792 len 8192 2024-09-16T00:34:29.353 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4193792 len 8192 2024-09-16T00:34:29.353 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4193792 len 16384 2024-09-16T00:34:29.353 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4193792 len 16384 2024-09-16T00:34:29.353 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4193792 len 16384 2024-09-16T00:34:29.353 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4193792 len 16384 2024-09-16T00:34:29.354 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4194304 len 1024 2024-09-16T00:34:29.354 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4194304 len 1024 2024-09-16T00:34:29.354 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4194304 len 1024 2024-09-16T00:34:29.354 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4194304 len 1024 2024-09-16T00:34:29.354 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4194304 len 2048 2024-09-16T00:34:29.354 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4194304 len 2048 2024-09-16T00:34:29.354 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4194304 len 2048 2024-09-16T00:34:29.354 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4194304 len 2048 2024-09-16T00:34:29.354 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4194304 len 4096 2024-09-16T00:34:29.354 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4194304 len 4096 2024-09-16T00:34:29.354 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4194304 len 4096 2024-09-16T00:34:29.354 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4194304 len 4096 2024-09-16T00:34:29.355 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4194304 len 8192 2024-09-16T00:34:29.355 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4194304 len 8192 2024-09-16T00:34:29.355 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4194304 len 8192 2024-09-16T00:34:29.355 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4194304 len 8192 2024-09-16T00:34:29.355 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4194304 len 16384 2024-09-16T00:34:29.355 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4194304 len 16384 2024-09-16T00:34:29.355 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4194304 len 16384 2024-09-16T00:34:29.355 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4194304 len 16384 2024-09-16T00:34:29.355 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4194816 len 1024 2024-09-16T00:34:29.355 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4194816 len 1024 2024-09-16T00:34:29.355 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4194816 len 1024 2024-09-16T00:34:29.355 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4194816 len 1024 2024-09-16T00:34:29.356 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4194816 len 2048 2024-09-16T00:34:29.356 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4194816 len 2048 2024-09-16T00:34:29.356 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4194816 len 2048 2024-09-16T00:34:29.356 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4194816 len 2048 2024-09-16T00:34:29.356 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4194816 len 4096 2024-09-16T00:34:29.356 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4194816 len 4096 2024-09-16T00:34:29.356 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4194816 len 4096 2024-09-16T00:34:29.356 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4194816 len 4096 2024-09-16T00:34:29.356 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4194816 len 8192 2024-09-16T00:34:29.356 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4194816 len 8192 2024-09-16T00:34:29.356 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4194816 len 8192 2024-09-16T00:34:29.356 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4194816 len 8192 2024-09-16T00:34:29.357 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4194816 len 16384 2024-09-16T00:34:29.357 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4194816 len 16384 2024-09-16T00:34:29.357 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4194816 len 16384 2024-09-16T00:34:29.357 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4194816 len 16384 2024-09-16T00:34:29.357 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4195328 len 1024 2024-09-16T00:34:29.357 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4195328 len 1024 2024-09-16T00:34:29.357 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4195328 len 1024 2024-09-16T00:34:29.357 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4195328 len 1024 2024-09-16T00:34:29.357 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4195328 len 2048 2024-09-16T00:34:29.357 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4195328 len 2048 2024-09-16T00:34:29.357 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4195328 len 2048 2024-09-16T00:34:29.357 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4195328 len 2048 2024-09-16T00:34:29.357 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4195328 len 4096 2024-09-16T00:34:29.358 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4195328 len 4096 2024-09-16T00:34:29.358 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4195328 len 4096 2024-09-16T00:34:29.358 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4195328 len 4096 2024-09-16T00:34:29.358 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4195328 len 8192 2024-09-16T00:34:29.358 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4195328 len 8192 2024-09-16T00:34:29.358 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4195328 len 8192 2024-09-16T00:34:29.358 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4195328 len 8192 2024-09-16T00:34:29.358 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4195328 len 16384 2024-09-16T00:34:29.358 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4195328 len 16384 2024-09-16T00:34:29.358 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4195328 len 16384 2024-09-16T00:34:29.358 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4195328 len 16384 2024-09-16T00:34:29.358 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4195840 len 1024 2024-09-16T00:34:29.359 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4195840 len 1024 2024-09-16T00:34:29.359 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4195840 len 1024 2024-09-16T00:34:29.359 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4195840 len 1024 2024-09-16T00:34:29.359 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4195840 len 2048 2024-09-16T00:34:29.359 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4195840 len 2048 2024-09-16T00:34:29.359 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4195840 len 2048 2024-09-16T00:34:29.359 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4195840 len 2048 2024-09-16T00:34:29.376 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4195840 len 4096 2024-09-16T00:34:29.376 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4195840 len 4096 2024-09-16T00:34:29.376 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4195840 len 4096 2024-09-16T00:34:29.376 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4195840 len 4096 2024-09-16T00:34:29.376 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4195840 len 8192 2024-09-16T00:34:29.376 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4195840 len 8192 2024-09-16T00:34:29.377 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4195840 len 8192 2024-09-16T00:34:29.377 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4195840 len 8192 2024-09-16T00:34:29.377 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4195840 len 16384 2024-09-16T00:34:29.377 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4195840 len 16384 2024-09-16T00:34:29.377 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4195840 len 16384 2024-09-16T00:34:29.377 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4195840 len 16384 2024-09-16T00:34:29.377 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4196352 len 1024 2024-09-16T00:34:29.377 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4196352 len 1024 2024-09-16T00:34:29.377 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4196352 len 1024 2024-09-16T00:34:29.377 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4196352 len 1024 2024-09-16T00:34:29.377 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4196352 len 2048 2024-09-16T00:34:29.377 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4196352 len 2048 2024-09-16T00:34:29.377 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4196352 len 2048 2024-09-16T00:34:29.377 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4196352 len 2048 2024-09-16T00:34:29.378 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4196352 len 4096 2024-09-16T00:34:29.378 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4196352 len 4096 2024-09-16T00:34:29.378 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4196352 len 4096 2024-09-16T00:34:29.378 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4196352 len 4096 2024-09-16T00:34:29.378 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4196352 len 8192 2024-09-16T00:34:29.378 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4196352 len 8192 2024-09-16T00:34:29.378 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4196352 len 8192 2024-09-16T00:34:29.378 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4196352 len 8192 2024-09-16T00:34:29.378 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4196352 len 16384 2024-09-16T00:34:29.378 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4196352 len 16384 2024-09-16T00:34:29.378 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4196352 len 16384 2024-09-16T00:34:29.378 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4196352 len 16384 2024-09-16T00:34:29.378 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4196864 len 1024 2024-09-16T00:34:29.378 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4196864 len 1024 2024-09-16T00:34:29.379 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4196864 len 1024 2024-09-16T00:34:29.379 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4196864 len 1024 2024-09-16T00:34:29.379 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4196864 len 2048 2024-09-16T00:34:29.379 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4196864 len 2048 2024-09-16T00:34:29.379 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4196864 len 2048 2024-09-16T00:34:29.379 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4196864 len 2048 2024-09-16T00:34:29.379 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4196864 len 4096 2024-09-16T00:34:29.379 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4196864 len 4096 2024-09-16T00:34:29.379 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4196864 len 4096 2024-09-16T00:34:29.379 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4196864 len 4096 2024-09-16T00:34:29.379 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4196864 len 8192 2024-09-16T00:34:29.379 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4196864 len 8192 2024-09-16T00:34:29.379 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4196864 len 8192 2024-09-16T00:34:29.379 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4196864 len 8192 2024-09-16T00:34:29.379 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4196864 len 16384 2024-09-16T00:34:29.380 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4196864 len 16384 2024-09-16T00:34:29.380 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4196864 len 16384 2024-09-16T00:34:29.380 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4196864 len 16384 2024-09-16T00:34:29.380 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4197376 len 1024 2024-09-16T00:34:29.380 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4197376 len 1024 2024-09-16T00:34:29.380 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4197376 len 1024 2024-09-16T00:34:29.380 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4197376 len 1024 2024-09-16T00:34:29.380 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4197376 len 2048 2024-09-16T00:34:29.380 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4197376 len 2048 2024-09-16T00:34:29.380 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4197376 len 2048 2024-09-16T00:34:29.380 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4197376 len 2048 2024-09-16T00:34:29.380 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4197376 len 4096 2024-09-16T00:34:29.380 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4197376 len 4096 2024-09-16T00:34:29.380 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4197376 len 4096 2024-09-16T00:34:29.381 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4197376 len 4096 2024-09-16T00:34:29.381 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4197376 len 8192 2024-09-16T00:34:29.381 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4197376 len 8192 2024-09-16T00:34:29.381 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4197376 len 8192 2024-09-16T00:34:29.381 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4197376 len 8192 2024-09-16T00:34:29.381 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4197376 len 16384 2024-09-16T00:34:29.381 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4197376 len 16384 2024-09-16T00:34:29.381 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4197376 len 16384 2024-09-16T00:34:29.381 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4197376 len 16384 2024-09-16T00:34:29.381 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4197888 len 1024 2024-09-16T00:34:29.381 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4197888 len 1024 2024-09-16T00:34:29.381 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4197888 len 1024 2024-09-16T00:34:29.381 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4197888 len 1024 2024-09-16T00:34:29.381 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4197888 len 2048 2024-09-16T00:34:29.381 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4197888 len 2048 2024-09-16T00:34:29.382 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4197888 len 2048 2024-09-16T00:34:29.382 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4197888 len 2048 2024-09-16T00:34:29.382 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4197888 len 4096 2024-09-16T00:34:29.382 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4197888 len 4096 2024-09-16T00:34:29.382 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4197888 len 4096 2024-09-16T00:34:29.417 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4197888 len 4096 2024-09-16T00:34:29.417 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4197888 len 8192 2024-09-16T00:34:29.417 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4197888 len 8192 2024-09-16T00:34:29.417 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4197888 len 8192 2024-09-16T00:34:29.417 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4197888 len 8192 2024-09-16T00:34:29.417 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2048 offset 4197888 len 16384 2024-09-16T00:34:29.417 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4197888 len 16384 2024-09-16T00:34:29.417 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2048 offset 4197888 len 16384 2024-09-16T00:34:29.417 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2048 offset 4197888 len 16384 2024-09-16T00:34:29.417 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4190208 len 1024 2024-09-16T00:34:29.418 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4190208 len 1024 2024-09-16T00:34:29.418 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4190208 len 1024 2024-09-16T00:34:29.418 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4190208 len 1024 2024-09-16T00:34:29.418 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4190208 len 2048 2024-09-16T00:34:29.418 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4190208 len 2048 2024-09-16T00:34:29.418 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4190208 len 2048 2024-09-16T00:34:29.418 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4190208 len 2048 2024-09-16T00:34:29.418 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4190208 len 4096 2024-09-16T00:34:29.418 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4190208 len 4096 2024-09-16T00:34:29.418 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4190208 len 4096 2024-09-16T00:34:29.418 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4190208 len 4096 2024-09-16T00:34:29.418 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4190208 len 8192 2024-09-16T00:34:29.418 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4190208 len 8192 2024-09-16T00:34:29.419 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4190208 len 8192 2024-09-16T00:34:29.419 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4190208 len 8192 2024-09-16T00:34:29.419 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4190208 len 16384 2024-09-16T00:34:29.419 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4190208 len 16384 2024-09-16T00:34:29.419 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4190208 len 16384 2024-09-16T00:34:29.419 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4190208 len 16384 2024-09-16T00:34:29.419 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4190720 len 1024 2024-09-16T00:34:29.419 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4190720 len 1024 2024-09-16T00:34:29.419 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4190720 len 1024 2024-09-16T00:34:29.419 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4190720 len 1024 2024-09-16T00:34:29.419 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4190720 len 2048 2024-09-16T00:34:29.419 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4190720 len 2048 2024-09-16T00:34:29.419 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4190720 len 2048 2024-09-16T00:34:29.419 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4190720 len 2048 2024-09-16T00:34:29.420 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4190720 len 4096 2024-09-16T00:34:29.420 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4190720 len 4096 2024-09-16T00:34:29.420 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4190720 len 4096 2024-09-16T00:34:29.420 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4190720 len 4096 2024-09-16T00:34:29.420 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4190720 len 8192 2024-09-16T00:34:29.420 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4190720 len 8192 2024-09-16T00:34:29.420 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4190720 len 8192 2024-09-16T00:34:29.420 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4190720 len 8192 2024-09-16T00:34:29.420 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4190720 len 16384 2024-09-16T00:34:29.420 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4190720 len 16384 2024-09-16T00:34:29.420 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4190720 len 16384 2024-09-16T00:34:29.420 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4190720 len 16384 2024-09-16T00:34:29.420 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4191232 len 1024 2024-09-16T00:34:29.421 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4191232 len 1024 2024-09-16T00:34:29.421 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4191232 len 1024 2024-09-16T00:34:29.421 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4191232 len 1024 2024-09-16T00:34:29.421 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4191232 len 2048 2024-09-16T00:34:29.421 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4191232 len 2048 2024-09-16T00:34:29.421 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4191232 len 2048 2024-09-16T00:34:29.421 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4191232 len 2048 2024-09-16T00:34:29.421 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4191232 len 4096 2024-09-16T00:34:29.421 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4191232 len 4096 2024-09-16T00:34:29.421 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4191232 len 4096 2024-09-16T00:34:29.421 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4191232 len 4096 2024-09-16T00:34:29.421 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4191232 len 8192 2024-09-16T00:34:29.421 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4191232 len 8192 2024-09-16T00:34:29.421 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4191232 len 8192 2024-09-16T00:34:29.422 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4191232 len 8192 2024-09-16T00:34:29.422 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4191232 len 16384 2024-09-16T00:34:29.422 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4191232 len 16384 2024-09-16T00:34:29.422 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4191232 len 16384 2024-09-16T00:34:29.422 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4191232 len 16384 2024-09-16T00:34:29.422 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4191744 len 1024 2024-09-16T00:34:29.422 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4191744 len 1024 2024-09-16T00:34:29.422 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4191744 len 1024 2024-09-16T00:34:29.422 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4191744 len 1024 2024-09-16T00:34:29.422 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4191744 len 2048 2024-09-16T00:34:29.423 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4191744 len 2048 2024-09-16T00:34:29.423 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4191744 len 2048 2024-09-16T00:34:29.423 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4191744 len 2048 2024-09-16T00:34:29.423 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4191744 len 4096 2024-09-16T00:34:29.423 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4191744 len 4096 2024-09-16T00:34:29.423 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4191744 len 4096 2024-09-16T00:34:29.423 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4191744 len 4096 2024-09-16T00:34:29.423 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4191744 len 8192 2024-09-16T00:34:29.464 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4191744 len 8192 2024-09-16T00:34:29.464 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4191744 len 8192 2024-09-16T00:34:29.464 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4191744 len 8192 2024-09-16T00:34:29.464 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4191744 len 16384 2024-09-16T00:34:29.464 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4191744 len 16384 2024-09-16T00:34:29.464 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4191744 len 16384 2024-09-16T00:34:29.464 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4191744 len 16384 2024-09-16T00:34:29.464 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4192256 len 1024 2024-09-16T00:34:29.464 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4192256 len 1024 2024-09-16T00:34:29.464 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4192256 len 1024 2024-09-16T00:34:29.465 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4192256 len 1024 2024-09-16T00:34:29.465 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4192256 len 2048 2024-09-16T00:34:29.465 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4192256 len 2048 2024-09-16T00:34:29.465 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4192256 len 2048 2024-09-16T00:34:29.465 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4192256 len 2048 2024-09-16T00:34:29.465 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4192256 len 4096 2024-09-16T00:34:29.465 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4192256 len 4096 2024-09-16T00:34:29.465 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4192256 len 4096 2024-09-16T00:34:29.465 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4192256 len 4096 2024-09-16T00:34:29.465 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4192256 len 8192 2024-09-16T00:34:29.465 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4192256 len 8192 2024-09-16T00:34:29.465 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4192256 len 8192 2024-09-16T00:34:29.465 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4192256 len 8192 2024-09-16T00:34:29.466 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4192256 len 16384 2024-09-16T00:34:29.466 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4192256 len 16384 2024-09-16T00:34:29.466 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4192256 len 16384 2024-09-16T00:34:29.466 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4192256 len 16384 2024-09-16T00:34:29.466 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4192768 len 1024 2024-09-16T00:34:29.466 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4192768 len 1024 2024-09-16T00:34:29.466 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4192768 len 1024 2024-09-16T00:34:29.466 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4192768 len 1024 2024-09-16T00:34:29.466 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4192768 len 2048 2024-09-16T00:34:29.466 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4192768 len 2048 2024-09-16T00:34:29.466 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4192768 len 2048 2024-09-16T00:34:29.466 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4192768 len 2048 2024-09-16T00:34:29.467 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4192768 len 4096 2024-09-16T00:34:29.467 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4192768 len 4096 2024-09-16T00:34:29.467 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4192768 len 4096 2024-09-16T00:34:29.467 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4192768 len 4096 2024-09-16T00:34:29.467 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4192768 len 8192 2024-09-16T00:34:29.467 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4192768 len 8192 2024-09-16T00:34:29.467 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4192768 len 8192 2024-09-16T00:34:29.467 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4192768 len 8192 2024-09-16T00:34:29.467 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4192768 len 16384 2024-09-16T00:34:29.467 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4192768 len 16384 2024-09-16T00:34:29.467 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4192768 len 16384 2024-09-16T00:34:29.467 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4192768 len 16384 2024-09-16T00:34:29.467 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4193280 len 1024 2024-09-16T00:34:29.467 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4193280 len 1024 2024-09-16T00:34:29.468 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4193280 len 1024 2024-09-16T00:34:29.468 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4193280 len 1024 2024-09-16T00:34:29.468 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4193280 len 2048 2024-09-16T00:34:29.468 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4193280 len 2048 2024-09-16T00:34:29.468 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4193280 len 2048 2024-09-16T00:34:29.468 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4193280 len 2048 2024-09-16T00:34:29.468 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4193280 len 4096 2024-09-16T00:34:29.468 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4193280 len 4096 2024-09-16T00:34:29.468 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4193280 len 4096 2024-09-16T00:34:29.468 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4193280 len 4096 2024-09-16T00:34:29.468 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4193280 len 8192 2024-09-16T00:34:29.468 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4193280 len 8192 2024-09-16T00:34:29.468 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4193280 len 8192 2024-09-16T00:34:29.468 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4193280 len 8192 2024-09-16T00:34:29.469 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4193280 len 16384 2024-09-16T00:34:29.469 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4193280 len 16384 2024-09-16T00:34:29.469 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4193280 len 16384 2024-09-16T00:34:29.469 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4193280 len 16384 2024-09-16T00:34:29.469 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4193792 len 1024 2024-09-16T00:34:29.469 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4193792 len 1024 2024-09-16T00:34:29.469 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4193792 len 1024 2024-09-16T00:34:29.469 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4193792 len 1024 2024-09-16T00:34:29.469 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4193792 len 2048 2024-09-16T00:34:29.469 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4193792 len 2048 2024-09-16T00:34:29.469 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4193792 len 2048 2024-09-16T00:34:29.469 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4193792 len 2048 2024-09-16T00:34:29.469 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4193792 len 4096 2024-09-16T00:34:29.470 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4193792 len 4096 2024-09-16T00:34:29.470 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4193792 len 4096 2024-09-16T00:34:29.470 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4193792 len 4096 2024-09-16T00:34:29.470 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4193792 len 8192 2024-09-16T00:34:29.470 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4193792 len 8192 2024-09-16T00:34:29.470 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4193792 len 8192 2024-09-16T00:34:29.470 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4193792 len 8192 2024-09-16T00:34:29.470 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:29 smithi122 ceph-mon[35412]: pgmap v415: 129 pgs: 129 active+clean; 761 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.6 KiB/s wr, 498 op/s 2024-09-16T00:34:29.470 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:29 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/1009667578' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:34:29.493 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4193792 len 16384 2024-09-16T00:34:29.494 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4193792 len 16384 2024-09-16T00:34:29.494 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4193792 len 16384 2024-09-16T00:34:29.494 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4193792 len 16384 2024-09-16T00:34:29.494 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4194304 len 1024 2024-09-16T00:34:29.494 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4194304 len 1024 2024-09-16T00:34:29.494 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4194304 len 1024 2024-09-16T00:34:29.494 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4194304 len 1024 2024-09-16T00:34:29.494 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4194304 len 2048 2024-09-16T00:34:29.494 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4194304 len 2048 2024-09-16T00:34:29.494 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4194304 len 2048 2024-09-16T00:34:29.494 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4194304 len 2048 2024-09-16T00:34:29.494 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4194304 len 4096 2024-09-16T00:34:29.495 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4194304 len 4096 2024-09-16T00:34:29.495 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4194304 len 4096 2024-09-16T00:34:29.495 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4194304 len 4096 2024-09-16T00:34:29.495 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4194304 len 8192 2024-09-16T00:34:29.495 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4194304 len 8192 2024-09-16T00:34:29.495 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4194304 len 8192 2024-09-16T00:34:29.495 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4194304 len 8192 2024-09-16T00:34:29.495 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4194304 len 16384 2024-09-16T00:34:29.495 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4194304 len 16384 2024-09-16T00:34:29.495 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4194304 len 16384 2024-09-16T00:34:29.496 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4194304 len 16384 2024-09-16T00:34:29.496 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4194816 len 1024 2024-09-16T00:34:29.496 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4194816 len 1024 2024-09-16T00:34:29.496 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4194816 len 1024 2024-09-16T00:34:29.496 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4194816 len 1024 2024-09-16T00:34:29.496 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4194816 len 2048 2024-09-16T00:34:29.496 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4194816 len 2048 2024-09-16T00:34:29.496 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4194816 len 2048 2024-09-16T00:34:29.496 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4194816 len 2048 2024-09-16T00:34:29.496 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4194816 len 4096 2024-09-16T00:34:29.496 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4194816 len 4096 2024-09-16T00:34:29.497 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4194816 len 4096 2024-09-16T00:34:29.497 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4194816 len 4096 2024-09-16T00:34:29.497 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4194816 len 8192 2024-09-16T00:34:29.497 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4194816 len 8192 2024-09-16T00:34:29.497 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4194816 len 8192 2024-09-16T00:34:29.497 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4194816 len 8192 2024-09-16T00:34:29.497 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4194816 len 16384 2024-09-16T00:34:29.497 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4194816 len 16384 2024-09-16T00:34:29.497 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4194816 len 16384 2024-09-16T00:34:29.497 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4194816 len 16384 2024-09-16T00:34:29.497 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4195328 len 1024 2024-09-16T00:34:29.497 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4195328 len 1024 2024-09-16T00:34:29.497 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4195328 len 1024 2024-09-16T00:34:29.497 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4195328 len 1024 2024-09-16T00:34:29.498 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4195328 len 2048 2024-09-16T00:34:29.498 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4195328 len 2048 2024-09-16T00:34:29.498 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4195328 len 2048 2024-09-16T00:34:29.498 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4195328 len 2048 2024-09-16T00:34:29.498 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4195328 len 4096 2024-09-16T00:34:29.498 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4195328 len 4096 2024-09-16T00:34:29.499 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4195328 len 4096 2024-09-16T00:34:29.499 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4195328 len 4096 2024-09-16T00:34:29.499 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4195328 len 8192 2024-09-16T00:34:29.499 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4195328 len 8192 2024-09-16T00:34:29.499 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4195328 len 8192 2024-09-16T00:34:29.499 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4195328 len 8192 2024-09-16T00:34:29.499 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4195328 len 16384 2024-09-16T00:34:29.499 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4195328 len 16384 2024-09-16T00:34:29.499 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4195328 len 16384 2024-09-16T00:34:29.499 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4195328 len 16384 2024-09-16T00:34:29.499 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4195840 len 1024 2024-09-16T00:34:29.499 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4195840 len 1024 2024-09-16T00:34:29.500 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4195840 len 1024 2024-09-16T00:34:29.500 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4195840 len 1024 2024-09-16T00:34:29.500 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4195840 len 2048 2024-09-16T00:34:29.500 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4195840 len 2048 2024-09-16T00:34:29.500 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4195840 len 2048 2024-09-16T00:34:29.500 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4195840 len 2048 2024-09-16T00:34:29.500 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4195840 len 4096 2024-09-16T00:34:29.500 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4195840 len 4096 2024-09-16T00:34:29.500 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4195840 len 4096 2024-09-16T00:34:29.500 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4195840 len 4096 2024-09-16T00:34:29.500 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4195840 len 8192 2024-09-16T00:34:29.500 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4195840 len 8192 2024-09-16T00:34:29.500 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4195840 len 8192 2024-09-16T00:34:29.500 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4195840 len 8192 2024-09-16T00:34:29.501 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4195840 len 16384 2024-09-16T00:34:29.501 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4195840 len 16384 2024-09-16T00:34:29.522 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4195840 len 16384 2024-09-16T00:34:29.522 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4195840 len 16384 2024-09-16T00:34:29.522 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4196352 len 1024 2024-09-16T00:34:29.522 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4196352 len 1024 2024-09-16T00:34:29.522 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4196352 len 1024 2024-09-16T00:34:29.522 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4196352 len 1024 2024-09-16T00:34:29.522 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4196352 len 2048 2024-09-16T00:34:29.522 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4196352 len 2048 2024-09-16T00:34:29.522 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4196352 len 2048 2024-09-16T00:34:29.522 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4196352 len 2048 2024-09-16T00:34:29.522 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4196352 len 4096 2024-09-16T00:34:29.523 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4196352 len 4096 2024-09-16T00:34:29.523 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4196352 len 4096 2024-09-16T00:34:29.523 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4196352 len 4096 2024-09-16T00:34:29.523 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4196352 len 8192 2024-09-16T00:34:29.523 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4196352 len 8192 2024-09-16T00:34:29.523 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4196352 len 8192 2024-09-16T00:34:29.523 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4196352 len 8192 2024-09-16T00:34:29.523 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4196352 len 16384 2024-09-16T00:34:29.523 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4196352 len 16384 2024-09-16T00:34:29.523 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4196352 len 16384 2024-09-16T00:34:29.523 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4196352 len 16384 2024-09-16T00:34:29.523 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4196864 len 1024 2024-09-16T00:34:29.523 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4196864 len 1024 2024-09-16T00:34:29.523 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4196864 len 1024 2024-09-16T00:34:29.524 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4196864 len 1024 2024-09-16T00:34:29.524 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4196864 len 2048 2024-09-16T00:34:29.524 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4196864 len 2048 2024-09-16T00:34:29.524 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4196864 len 2048 2024-09-16T00:34:29.524 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4196864 len 2048 2024-09-16T00:34:29.524 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4196864 len 4096 2024-09-16T00:34:29.524 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4196864 len 4096 2024-09-16T00:34:29.524 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4196864 len 4096 2024-09-16T00:34:29.524 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4196864 len 4096 2024-09-16T00:34:29.524 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4196864 len 8192 2024-09-16T00:34:29.524 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4196864 len 8192 2024-09-16T00:34:29.524 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4196864 len 8192 2024-09-16T00:34:29.524 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4196864 len 8192 2024-09-16T00:34:29.524 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4196864 len 16384 2024-09-16T00:34:29.525 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4196864 len 16384 2024-09-16T00:34:29.525 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4196864 len 16384 2024-09-16T00:34:29.525 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4196864 len 16384 2024-09-16T00:34:29.525 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4197376 len 1024 2024-09-16T00:34:29.525 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4197376 len 1024 2024-09-16T00:34:29.525 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4197376 len 1024 2024-09-16T00:34:29.525 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4197376 len 1024 2024-09-16T00:34:29.525 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4197376 len 2048 2024-09-16T00:34:29.525 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4197376 len 2048 2024-09-16T00:34:29.525 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4197376 len 2048 2024-09-16T00:34:29.525 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4197376 len 2048 2024-09-16T00:34:29.525 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4197376 len 4096 2024-09-16T00:34:29.525 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4197376 len 4096 2024-09-16T00:34:29.525 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4197376 len 4096 2024-09-16T00:34:29.526 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4197376 len 4096 2024-09-16T00:34:29.526 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4197376 len 8192 2024-09-16T00:34:29.526 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4197376 len 8192 2024-09-16T00:34:29.526 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4197376 len 8192 2024-09-16T00:34:29.526 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4197376 len 8192 2024-09-16T00:34:29.526 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4197376 len 16384 2024-09-16T00:34:29.526 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4197376 len 16384 2024-09-16T00:34:29.526 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4197376 len 16384 2024-09-16T00:34:29.526 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4197376 len 16384 2024-09-16T00:34:29.526 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4197888 len 1024 2024-09-16T00:34:29.526 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4197888 len 1024 2024-09-16T00:34:29.526 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4197888 len 1024 2024-09-16T00:34:29.526 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4197888 len 1024 2024-09-16T00:34:29.526 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4197888 len 2048 2024-09-16T00:34:29.527 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4197888 len 2048 2024-09-16T00:34:29.527 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4197888 len 2048 2024-09-16T00:34:29.527 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4197888 len 2048 2024-09-16T00:34:29.527 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4197888 len 4096 2024-09-16T00:34:29.527 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4197888 len 4096 2024-09-16T00:34:29.527 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4197888 len 4096 2024-09-16T00:34:29.527 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4197888 len 4096 2024-09-16T00:34:29.527 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4197888 len 8192 2024-09-16T00:34:29.527 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4197888 len 8192 2024-09-16T00:34:29.527 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4197888 len 8192 2024-09-16T00:34:29.527 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4197888 len 8192 2024-09-16T00:34:29.527 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 2560 offset 4197888 len 16384 2024-09-16T00:34:29.527 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4197888 len 16384 2024-09-16T00:34:29.527 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 2560 offset 4197888 len 16384 2024-09-16T00:34:29.528 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 2560 offset 4197888 len 16384 2024-09-16T00:34:29.562 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4190208 len 1024 2024-09-16T00:34:29.563 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4190208 len 1024 2024-09-16T00:34:29.563 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4190208 len 1024 2024-09-16T00:34:29.563 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4190208 len 1024 2024-09-16T00:34:29.563 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4190208 len 2048 2024-09-16T00:34:29.563 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4190208 len 2048 2024-09-16T00:34:29.563 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4190208 len 2048 2024-09-16T00:34:29.563 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4190208 len 2048 2024-09-16T00:34:29.563 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4190208 len 4096 2024-09-16T00:34:29.563 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4190208 len 4096 2024-09-16T00:34:29.563 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4190208 len 4096 2024-09-16T00:34:29.563 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4190208 len 4096 2024-09-16T00:34:29.563 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4190208 len 8192 2024-09-16T00:34:29.563 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4190208 len 8192 2024-09-16T00:34:29.563 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4190208 len 8192 2024-09-16T00:34:29.564 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4190208 len 8192 2024-09-16T00:34:29.564 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4190208 len 16384 2024-09-16T00:34:29.564 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4190208 len 16384 2024-09-16T00:34:29.564 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4190208 len 16384 2024-09-16T00:34:29.564 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4190208 len 16384 2024-09-16T00:34:29.564 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4190720 len 1024 2024-09-16T00:34:29.564 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4190720 len 1024 2024-09-16T00:34:29.564 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4190720 len 1024 2024-09-16T00:34:29.564 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4190720 len 1024 2024-09-16T00:34:29.564 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4190720 len 2048 2024-09-16T00:34:29.564 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4190720 len 2048 2024-09-16T00:34:29.564 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4190720 len 2048 2024-09-16T00:34:29.564 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4190720 len 2048 2024-09-16T00:34:29.565 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4190720 len 4096 2024-09-16T00:34:29.565 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4190720 len 4096 2024-09-16T00:34:29.565 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4190720 len 4096 2024-09-16T00:34:29.565 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4190720 len 4096 2024-09-16T00:34:29.565 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4190720 len 8192 2024-09-16T00:34:29.565 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4190720 len 8192 2024-09-16T00:34:29.565 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4190720 len 8192 2024-09-16T00:34:29.565 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4190720 len 8192 2024-09-16T00:34:29.565 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4190720 len 16384 2024-09-16T00:34:29.565 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4190720 len 16384 2024-09-16T00:34:29.565 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4190720 len 16384 2024-09-16T00:34:29.565 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4190720 len 16384 2024-09-16T00:34:29.565 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4191232 len 1024 2024-09-16T00:34:29.565 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4191232 len 1024 2024-09-16T00:34:29.566 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4191232 len 1024 2024-09-16T00:34:29.566 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4191232 len 1024 2024-09-16T00:34:29.566 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4191232 len 2048 2024-09-16T00:34:29.566 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4191232 len 2048 2024-09-16T00:34:29.566 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4191232 len 2048 2024-09-16T00:34:29.566 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4191232 len 2048 2024-09-16T00:34:29.566 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4191232 len 4096 2024-09-16T00:34:29.566 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4191232 len 4096 2024-09-16T00:34:29.566 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4191232 len 4096 2024-09-16T00:34:29.566 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4191232 len 4096 2024-09-16T00:34:29.566 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4191232 len 8192 2024-09-16T00:34:29.566 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4191232 len 8192 2024-09-16T00:34:29.566 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4191232 len 8192 2024-09-16T00:34:29.567 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4191232 len 8192 2024-09-16T00:34:29.567 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4191232 len 16384 2024-09-16T00:34:29.567 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4191232 len 16384 2024-09-16T00:34:29.567 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4191232 len 16384 2024-09-16T00:34:29.567 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4191232 len 16384 2024-09-16T00:34:29.567 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4191744 len 1024 2024-09-16T00:34:29.567 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4191744 len 1024 2024-09-16T00:34:29.567 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4191744 len 1024 2024-09-16T00:34:29.567 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4191744 len 1024 2024-09-16T00:34:29.567 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4191744 len 2048 2024-09-16T00:34:29.567 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4191744 len 2048 2024-09-16T00:34:29.567 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4191744 len 2048 2024-09-16T00:34:29.567 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4191744 len 2048 2024-09-16T00:34:29.567 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4191744 len 4096 2024-09-16T00:34:29.568 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4191744 len 4096 2024-09-16T00:34:29.568 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4191744 len 4096 2024-09-16T00:34:29.568 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4191744 len 4096 2024-09-16T00:34:29.568 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4191744 len 8192 2024-09-16T00:34:29.568 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4191744 len 8192 2024-09-16T00:34:29.568 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4191744 len 8192 2024-09-16T00:34:29.568 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4191744 len 8192 2024-09-16T00:34:29.568 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4191744 len 16384 2024-09-16T00:34:29.568 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4191744 len 16384 2024-09-16T00:34:29.568 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4191744 len 16384 2024-09-16T00:34:29.568 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4191744 len 16384 2024-09-16T00:34:29.568 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4192256 len 1024 2024-09-16T00:34:29.568 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4192256 len 1024 2024-09-16T00:34:29.568 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4192256 len 1024 2024-09-16T00:34:29.610 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4192256 len 1024 2024-09-16T00:34:29.610 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4192256 len 2048 2024-09-16T00:34:29.610 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4192256 len 2048 2024-09-16T00:34:29.610 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4192256 len 2048 2024-09-16T00:34:29.610 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4192256 len 2048 2024-09-16T00:34:29.610 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4192256 len 4096 2024-09-16T00:34:29.610 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4192256 len 4096 2024-09-16T00:34:29.610 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4192256 len 4096 2024-09-16T00:34:29.610 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4192256 len 4096 2024-09-16T00:34:29.610 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4192256 len 8192 2024-09-16T00:34:29.610 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4192256 len 8192 2024-09-16T00:34:29.611 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4192256 len 8192 2024-09-16T00:34:29.611 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4192256 len 8192 2024-09-16T00:34:29.611 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4192256 len 16384 2024-09-16T00:34:29.611 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4192256 len 16384 2024-09-16T00:34:29.611 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4192256 len 16384 2024-09-16T00:34:29.611 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4192256 len 16384 2024-09-16T00:34:29.611 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4192768 len 1024 2024-09-16T00:34:29.611 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4192768 len 1024 2024-09-16T00:34:29.611 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4192768 len 1024 2024-09-16T00:34:29.611 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4192768 len 1024 2024-09-16T00:34:29.611 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4192768 len 2048 2024-09-16T00:34:29.611 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4192768 len 2048 2024-09-16T00:34:29.611 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4192768 len 2048 2024-09-16T00:34:29.611 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4192768 len 2048 2024-09-16T00:34:29.612 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4192768 len 4096 2024-09-16T00:34:29.612 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4192768 len 4096 2024-09-16T00:34:29.612 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4192768 len 4096 2024-09-16T00:34:29.612 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4192768 len 4096 2024-09-16T00:34:29.612 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4192768 len 8192 2024-09-16T00:34:29.612 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4192768 len 8192 2024-09-16T00:34:29.612 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4192768 len 8192 2024-09-16T00:34:29.612 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4192768 len 8192 2024-09-16T00:34:29.612 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4192768 len 16384 2024-09-16T00:34:29.612 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4192768 len 16384 2024-09-16T00:34:29.613 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4192768 len 16384 2024-09-16T00:34:29.613 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4192768 len 16384 2024-09-16T00:34:29.613 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4193280 len 1024 2024-09-16T00:34:29.613 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4193280 len 1024 2024-09-16T00:34:29.613 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4193280 len 1024 2024-09-16T00:34:29.613 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4193280 len 1024 2024-09-16T00:34:29.613 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4193280 len 2048 2024-09-16T00:34:29.613 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4193280 len 2048 2024-09-16T00:34:29.613 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4193280 len 2048 2024-09-16T00:34:29.613 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4193280 len 2048 2024-09-16T00:34:29.613 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4193280 len 4096 2024-09-16T00:34:29.613 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4193280 len 4096 2024-09-16T00:34:29.613 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4193280 len 4096 2024-09-16T00:34:29.614 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4193280 len 4096 2024-09-16T00:34:29.614 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4193280 len 8192 2024-09-16T00:34:29.614 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4193280 len 8192 2024-09-16T00:34:29.614 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4193280 len 8192 2024-09-16T00:34:29.614 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4193280 len 8192 2024-09-16T00:34:29.614 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4193280 len 16384 2024-09-16T00:34:29.614 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4193280 len 16384 2024-09-16T00:34:29.614 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4193280 len 16384 2024-09-16T00:34:29.614 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4193280 len 16384 2024-09-16T00:34:29.614 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4193792 len 1024 2024-09-16T00:34:29.614 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4193792 len 1024 2024-09-16T00:34:29.614 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4193792 len 1024 2024-09-16T00:34:29.614 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4193792 len 1024 2024-09-16T00:34:29.615 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4193792 len 2048 2024-09-16T00:34:29.615 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4193792 len 2048 2024-09-16T00:34:29.615 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4193792 len 2048 2024-09-16T00:34:29.615 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4193792 len 2048 2024-09-16T00:34:29.615 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4193792 len 4096 2024-09-16T00:34:29.615 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4193792 len 4096 2024-09-16T00:34:29.615 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4193792 len 4096 2024-09-16T00:34:29.615 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4193792 len 4096 2024-09-16T00:34:29.615 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4193792 len 8192 2024-09-16T00:34:29.615 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4193792 len 8192 2024-09-16T00:34:29.615 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4193792 len 8192 2024-09-16T00:34:29.615 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4193792 len 8192 2024-09-16T00:34:29.615 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4193792 len 16384 2024-09-16T00:34:29.615 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4193792 len 16384 2024-09-16T00:34:29.615 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4193792 len 16384 2024-09-16T00:34:29.616 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4193792 len 16384 2024-09-16T00:34:29.616 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4194304 len 1024 2024-09-16T00:34:29.616 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4194304 len 1024 2024-09-16T00:34:29.616 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4194304 len 1024 2024-09-16T00:34:29.616 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4194304 len 1024 2024-09-16T00:34:29.616 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4194304 len 2048 2024-09-16T00:34:29.637 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4194304 len 2048 2024-09-16T00:34:29.637 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4194304 len 2048 2024-09-16T00:34:29.637 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4194304 len 2048 2024-09-16T00:34:29.637 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4194304 len 4096 2024-09-16T00:34:29.637 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4194304 len 4096 2024-09-16T00:34:29.638 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4194304 len 4096 2024-09-16T00:34:29.638 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4194304 len 4096 2024-09-16T00:34:29.638 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4194304 len 8192 2024-09-16T00:34:29.638 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4194304 len 8192 2024-09-16T00:34:29.638 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4194304 len 8192 2024-09-16T00:34:29.638 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4194304 len 8192 2024-09-16T00:34:29.638 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4194304 len 16384 2024-09-16T00:34:29.638 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4194304 len 16384 2024-09-16T00:34:29.638 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4194304 len 16384 2024-09-16T00:34:29.638 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4194304 len 16384 2024-09-16T00:34:29.639 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4194816 len 1024 2024-09-16T00:34:29.639 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4194816 len 1024 2024-09-16T00:34:29.639 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4194816 len 1024 2024-09-16T00:34:29.639 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4194816 len 1024 2024-09-16T00:34:29.639 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4194816 len 2048 2024-09-16T00:34:29.639 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4194816 len 2048 2024-09-16T00:34:29.639 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4194816 len 2048 2024-09-16T00:34:29.639 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4194816 len 2048 2024-09-16T00:34:29.639 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4194816 len 4096 2024-09-16T00:34:29.639 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4194816 len 4096 2024-09-16T00:34:29.639 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4194816 len 4096 2024-09-16T00:34:29.639 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4194816 len 4096 2024-09-16T00:34:29.639 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4194816 len 8192 2024-09-16T00:34:29.640 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4194816 len 8192 2024-09-16T00:34:29.640 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4194816 len 8192 2024-09-16T00:34:29.640 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4194816 len 8192 2024-09-16T00:34:29.640 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4194816 len 16384 2024-09-16T00:34:29.640 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4194816 len 16384 2024-09-16T00:34:29.640 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4194816 len 16384 2024-09-16T00:34:29.640 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4194816 len 16384 2024-09-16T00:34:29.640 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4195328 len 1024 2024-09-16T00:34:29.640 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4195328 len 1024 2024-09-16T00:34:29.640 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4195328 len 1024 2024-09-16T00:34:29.640 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4195328 len 1024 2024-09-16T00:34:29.640 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4195328 len 2048 2024-09-16T00:34:29.640 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4195328 len 2048 2024-09-16T00:34:29.640 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4195328 len 2048 2024-09-16T00:34:29.641 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4195328 len 2048 2024-09-16T00:34:29.641 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4195328 len 4096 2024-09-16T00:34:29.641 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4195328 len 4096 2024-09-16T00:34:29.641 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4195328 len 4096 2024-09-16T00:34:29.641 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4195328 len 4096 2024-09-16T00:34:29.641 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4195328 len 8192 2024-09-16T00:34:29.641 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4195328 len 8192 2024-09-16T00:34:29.641 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4195328 len 8192 2024-09-16T00:34:29.641 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4195328 len 8192 2024-09-16T00:34:29.641 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4195328 len 16384 2024-09-16T00:34:29.641 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4195328 len 16384 2024-09-16T00:34:29.641 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4195328 len 16384 2024-09-16T00:34:29.641 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4195328 len 16384 2024-09-16T00:34:29.641 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4195840 len 1024 2024-09-16T00:34:29.642 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4195840 len 1024 2024-09-16T00:34:29.642 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4195840 len 1024 2024-09-16T00:34:29.642 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4195840 len 1024 2024-09-16T00:34:29.642 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4195840 len 2048 2024-09-16T00:34:29.642 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4195840 len 2048 2024-09-16T00:34:29.642 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4195840 len 2048 2024-09-16T00:34:29.642 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4195840 len 2048 2024-09-16T00:34:29.642 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4195840 len 4096 2024-09-16T00:34:29.642 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4195840 len 4096 2024-09-16T00:34:29.642 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4195840 len 4096 2024-09-16T00:34:29.642 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4195840 len 4096 2024-09-16T00:34:29.642 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4195840 len 8192 2024-09-16T00:34:29.642 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4195840 len 8192 2024-09-16T00:34:29.642 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4195840 len 8192 2024-09-16T00:34:29.642 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4195840 len 8192 2024-09-16T00:34:29.643 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4195840 len 16384 2024-09-16T00:34:29.643 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4195840 len 16384 2024-09-16T00:34:29.643 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4195840 len 16384 2024-09-16T00:34:29.643 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4195840 len 16384 2024-09-16T00:34:29.643 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4196352 len 1024 2024-09-16T00:34:29.643 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4196352 len 1024 2024-09-16T00:34:29.643 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4196352 len 1024 2024-09-16T00:34:29.643 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4196352 len 1024 2024-09-16T00:34:29.643 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4196352 len 2048 2024-09-16T00:34:29.643 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4196352 len 2048 2024-09-16T00:34:29.643 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4196352 len 2048 2024-09-16T00:34:29.644 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4196352 len 2048 2024-09-16T00:34:29.662 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4196352 len 4096 2024-09-16T00:34:29.662 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4196352 len 4096 2024-09-16T00:34:29.662 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4196352 len 4096 2024-09-16T00:34:29.662 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4196352 len 4096 2024-09-16T00:34:29.662 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4196352 len 8192 2024-09-16T00:34:29.662 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4196352 len 8192 2024-09-16T00:34:29.662 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4196352 len 8192 2024-09-16T00:34:29.663 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4196352 len 8192 2024-09-16T00:34:29.663 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4196352 len 16384 2024-09-16T00:34:29.663 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4196352 len 16384 2024-09-16T00:34:29.663 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4196352 len 16384 2024-09-16T00:34:29.663 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4196352 len 16384 2024-09-16T00:34:29.663 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4196864 len 1024 2024-09-16T00:34:29.663 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4196864 len 1024 2024-09-16T00:34:29.663 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4196864 len 1024 2024-09-16T00:34:29.663 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4196864 len 1024 2024-09-16T00:34:29.663 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4196864 len 2048 2024-09-16T00:34:29.663 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4196864 len 2048 2024-09-16T00:34:29.664 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4196864 len 2048 2024-09-16T00:34:29.664 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4196864 len 2048 2024-09-16T00:34:29.664 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4196864 len 4096 2024-09-16T00:34:29.664 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4196864 len 4096 2024-09-16T00:34:29.664 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4196864 len 4096 2024-09-16T00:34:29.664 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4196864 len 4096 2024-09-16T00:34:29.664 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4196864 len 8192 2024-09-16T00:34:29.664 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4196864 len 8192 2024-09-16T00:34:29.664 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4196864 len 8192 2024-09-16T00:34:29.664 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4196864 len 8192 2024-09-16T00:34:29.664 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4196864 len 16384 2024-09-16T00:34:29.664 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4196864 len 16384 2024-09-16T00:34:29.664 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4196864 len 16384 2024-09-16T00:34:29.665 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4196864 len 16384 2024-09-16T00:34:29.665 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4197376 len 1024 2024-09-16T00:34:29.665 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4197376 len 1024 2024-09-16T00:34:29.665 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4197376 len 1024 2024-09-16T00:34:29.665 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4197376 len 1024 2024-09-16T00:34:29.665 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4197376 len 2048 2024-09-16T00:34:29.665 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4197376 len 2048 2024-09-16T00:34:29.665 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4197376 len 2048 2024-09-16T00:34:29.665 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4197376 len 2048 2024-09-16T00:34:29.665 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4197376 len 4096 2024-09-16T00:34:29.665 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4197376 len 4096 2024-09-16T00:34:29.665 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4197376 len 4096 2024-09-16T00:34:29.665 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4197376 len 4096 2024-09-16T00:34:29.666 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4197376 len 8192 2024-09-16T00:34:29.666 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4197376 len 8192 2024-09-16T00:34:29.666 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4197376 len 8192 2024-09-16T00:34:29.666 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4197376 len 8192 2024-09-16T00:34:29.666 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4197376 len 16384 2024-09-16T00:34:29.666 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4197376 len 16384 2024-09-16T00:34:29.666 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4197376 len 16384 2024-09-16T00:34:29.666 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4197376 len 16384 2024-09-16T00:34:29.666 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4197888 len 1024 2024-09-16T00:34:29.666 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4197888 len 1024 2024-09-16T00:34:29.666 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4197888 len 1024 2024-09-16T00:34:29.666 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4197888 len 1024 2024-09-16T00:34:29.666 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4197888 len 2048 2024-09-16T00:34:29.666 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4197888 len 2048 2024-09-16T00:34:29.667 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4197888 len 2048 2024-09-16T00:34:29.667 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4197888 len 2048 2024-09-16T00:34:29.667 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4197888 len 4096 2024-09-16T00:34:29.667 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4197888 len 4096 2024-09-16T00:34:29.667 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4197888 len 4096 2024-09-16T00:34:29.667 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4197888 len 4096 2024-09-16T00:34:29.667 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4197888 len 8192 2024-09-16T00:34:29.667 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4197888 len 8192 2024-09-16T00:34:29.667 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4197888 len 8192 2024-09-16T00:34:29.667 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4197888 len 8192 2024-09-16T00:34:29.667 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3072 offset 4197888 len 16384 2024-09-16T00:34:29.667 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4197888 len 16384 2024-09-16T00:34:29.667 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3072 offset 4197888 len 16384 2024-09-16T00:34:29.667 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3072 offset 4197888 len 16384 2024-09-16T00:34:29.667 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4190208 len 1024 2024-09-16T00:34:29.668 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4190208 len 1024 2024-09-16T00:34:29.668 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4190208 len 1024 2024-09-16T00:34:29.668 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4190208 len 1024 2024-09-16T00:34:29.668 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4190208 len 2048 2024-09-16T00:34:29.668 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4190208 len 2048 2024-09-16T00:34:29.668 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4190208 len 2048 2024-09-16T00:34:29.668 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4190208 len 2048 2024-09-16T00:34:29.668 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4190208 len 4096 2024-09-16T00:34:29.668 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4190208 len 4096 2024-09-16T00:34:29.677 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:34:29.706 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4190208 len 4096 2024-09-16T00:34:29.706 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4190208 len 4096 2024-09-16T00:34:29.706 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4190208 len 8192 2024-09-16T00:34:29.706 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4190208 len 8192 2024-09-16T00:34:29.706 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4190208 len 8192 2024-09-16T00:34:29.707 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4190208 len 8192 2024-09-16T00:34:29.707 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4190208 len 16384 2024-09-16T00:34:29.707 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4190208 len 16384 2024-09-16T00:34:29.707 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4190208 len 16384 2024-09-16T00:34:29.707 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4190208 len 16384 2024-09-16T00:34:29.707 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4190720 len 1024 2024-09-16T00:34:29.707 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4190720 len 1024 2024-09-16T00:34:29.707 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4190720 len 1024 2024-09-16T00:34:29.707 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4190720 len 1024 2024-09-16T00:34:29.707 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4190720 len 2048 2024-09-16T00:34:29.707 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4190720 len 2048 2024-09-16T00:34:29.707 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4190720 len 2048 2024-09-16T00:34:29.707 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4190720 len 2048 2024-09-16T00:34:29.707 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4190720 len 4096 2024-09-16T00:34:29.708 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4190720 len 4096 2024-09-16T00:34:29.708 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4190720 len 4096 2024-09-16T00:34:29.708 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4190720 len 4096 2024-09-16T00:34:29.708 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4190720 len 8192 2024-09-16T00:34:29.708 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4190720 len 8192 2024-09-16T00:34:29.708 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4190720 len 8192 2024-09-16T00:34:29.708 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4190720 len 8192 2024-09-16T00:34:29.708 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4190720 len 16384 2024-09-16T00:34:29.708 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4190720 len 16384 2024-09-16T00:34:29.708 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4190720 len 16384 2024-09-16T00:34:29.708 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4190720 len 16384 2024-09-16T00:34:29.708 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4191232 len 1024 2024-09-16T00:34:29.708 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4191232 len 1024 2024-09-16T00:34:29.708 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4191232 len 1024 2024-09-16T00:34:29.709 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4191232 len 1024 2024-09-16T00:34:29.709 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4191232 len 2048 2024-09-16T00:34:29.709 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4191232 len 2048 2024-09-16T00:34:29.709 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4191232 len 2048 2024-09-16T00:34:29.709 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4191232 len 2048 2024-09-16T00:34:29.709 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4191232 len 4096 2024-09-16T00:34:29.709 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4191232 len 4096 2024-09-16T00:34:29.709 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4191232 len 4096 2024-09-16T00:34:29.709 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4191232 len 4096 2024-09-16T00:34:29.709 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4191232 len 8192 2024-09-16T00:34:29.710 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4191232 len 8192 2024-09-16T00:34:29.710 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4191232 len 8192 2024-09-16T00:34:29.710 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4191232 len 8192 2024-09-16T00:34:29.710 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4191232 len 16384 2024-09-16T00:34:29.710 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4191232 len 16384 2024-09-16T00:34:29.710 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4191232 len 16384 2024-09-16T00:34:29.710 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4191232 len 16384 2024-09-16T00:34:29.710 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4191744 len 1024 2024-09-16T00:34:29.710 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4191744 len 1024 2024-09-16T00:34:29.710 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4191744 len 1024 2024-09-16T00:34:29.710 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4191744 len 1024 2024-09-16T00:34:29.710 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4191744 len 2048 2024-09-16T00:34:29.710 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4191744 len 2048 2024-09-16T00:34:29.710 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4191744 len 2048 2024-09-16T00:34:29.711 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4191744 len 2048 2024-09-16T00:34:29.711 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4191744 len 4096 2024-09-16T00:34:29.711 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4191744 len 4096 2024-09-16T00:34:29.711 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4191744 len 4096 2024-09-16T00:34:29.711 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4191744 len 4096 2024-09-16T00:34:29.711 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4191744 len 8192 2024-09-16T00:34:29.711 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4191744 len 8192 2024-09-16T00:34:29.711 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4191744 len 8192 2024-09-16T00:34:29.711 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4191744 len 8192 2024-09-16T00:34:29.711 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4191744 len 16384 2024-09-16T00:34:29.711 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4191744 len 16384 2024-09-16T00:34:29.711 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4191744 len 16384 2024-09-16T00:34:29.711 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4191744 len 16384 2024-09-16T00:34:29.712 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4192256 len 1024 2024-09-16T00:34:29.712 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4192256 len 1024 2024-09-16T00:34:29.712 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4192256 len 1024 2024-09-16T00:34:29.712 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4192256 len 1024 2024-09-16T00:34:29.712 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4192256 len 2048 2024-09-16T00:34:29.712 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4192256 len 2048 2024-09-16T00:34:29.712 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4192256 len 2048 2024-09-16T00:34:29.712 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4192256 len 2048 2024-09-16T00:34:29.712 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4192256 len 4096 2024-09-16T00:34:29.712 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4192256 len 4096 2024-09-16T00:34:29.713 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4192256 len 4096 2024-09-16T00:34:29.713 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4192256 len 4096 2024-09-16T00:34:29.713 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4192256 len 8192 2024-09-16T00:34:29.750 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4192256 len 8192 2024-09-16T00:34:29.750 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4192256 len 8192 2024-09-16T00:34:29.750 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4192256 len 8192 2024-09-16T00:34:29.750 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4192256 len 16384 2024-09-16T00:34:29.750 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4192256 len 16384 2024-09-16T00:34:29.750 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4192256 len 16384 2024-09-16T00:34:29.750 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4192256 len 16384 2024-09-16T00:34:29.750 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4192768 len 1024 2024-09-16T00:34:29.750 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4192768 len 1024 2024-09-16T00:34:29.750 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4192768 len 1024 2024-09-16T00:34:29.750 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4192768 len 1024 2024-09-16T00:34:29.750 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4192768 len 2048 2024-09-16T00:34:29.751 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4192768 len 2048 2024-09-16T00:34:29.751 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4192768 len 2048 2024-09-16T00:34:29.751 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4192768 len 2048 2024-09-16T00:34:29.751 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4192768 len 4096 2024-09-16T00:34:29.751 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4192768 len 4096 2024-09-16T00:34:29.751 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4192768 len 4096 2024-09-16T00:34:29.751 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4192768 len 4096 2024-09-16T00:34:29.751 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4192768 len 8192 2024-09-16T00:34:29.751 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4192768 len 8192 2024-09-16T00:34:29.751 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4192768 len 8192 2024-09-16T00:34:29.751 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4192768 len 8192 2024-09-16T00:34:29.751 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4192768 len 16384 2024-09-16T00:34:29.751 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4192768 len 16384 2024-09-16T00:34:29.751 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4192768 len 16384 2024-09-16T00:34:29.751 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4192768 len 16384 2024-09-16T00:34:29.752 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4193280 len 1024 2024-09-16T00:34:29.752 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4193280 len 1024 2024-09-16T00:34:29.752 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4193280 len 1024 2024-09-16T00:34:29.752 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4193280 len 1024 2024-09-16T00:34:29.752 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4193280 len 2048 2024-09-16T00:34:29.752 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4193280 len 2048 2024-09-16T00:34:29.752 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4193280 len 2048 2024-09-16T00:34:29.752 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4193280 len 2048 2024-09-16T00:34:29.753 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4193280 len 4096 2024-09-16T00:34:29.753 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4193280 len 4096 2024-09-16T00:34:29.753 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4193280 len 4096 2024-09-16T00:34:29.753 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4193280 len 4096 2024-09-16T00:34:29.753 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4193280 len 8192 2024-09-16T00:34:29.753 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4193280 len 8192 2024-09-16T00:34:29.753 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4193280 len 8192 2024-09-16T00:34:29.753 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4193280 len 8192 2024-09-16T00:34:29.753 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4193280 len 16384 2024-09-16T00:34:29.753 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4193280 len 16384 2024-09-16T00:34:29.753 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4193280 len 16384 2024-09-16T00:34:29.753 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4193280 len 16384 2024-09-16T00:34:29.753 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4193792 len 1024 2024-09-16T00:34:29.753 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4193792 len 1024 2024-09-16T00:34:29.754 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4193792 len 1024 2024-09-16T00:34:29.754 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4193792 len 1024 2024-09-16T00:34:29.754 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4193792 len 2048 2024-09-16T00:34:29.754 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4193792 len 2048 2024-09-16T00:34:29.754 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4193792 len 2048 2024-09-16T00:34:29.754 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4193792 len 2048 2024-09-16T00:34:29.754 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4193792 len 4096 2024-09-16T00:34:29.754 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4193792 len 4096 2024-09-16T00:34:29.754 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4193792 len 4096 2024-09-16T00:34:29.754 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4193792 len 4096 2024-09-16T00:34:29.754 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4193792 len 8192 2024-09-16T00:34:29.756 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4193792 len 8192 2024-09-16T00:34:29.756 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4193792 len 8192 2024-09-16T00:34:29.756 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4193792 len 8192 2024-09-16T00:34:29.756 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4193792 len 16384 2024-09-16T00:34:29.756 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4193792 len 16384 2024-09-16T00:34:29.756 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4193792 len 16384 2024-09-16T00:34:29.757 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4193792 len 16384 2024-09-16T00:34:29.757 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4194304 len 1024 2024-09-16T00:34:29.757 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4194304 len 1024 2024-09-16T00:34:29.757 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4194304 len 1024 2024-09-16T00:34:29.757 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4194304 len 1024 2024-09-16T00:34:29.757 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4194304 len 2048 2024-09-16T00:34:29.757 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4194304 len 2048 2024-09-16T00:34:29.757 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4194304 len 2048 2024-09-16T00:34:29.757 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4194304 len 2048 2024-09-16T00:34:29.757 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4194304 len 4096 2024-09-16T00:34:29.758 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4194304 len 4096 2024-09-16T00:34:29.758 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4194304 len 4096 2024-09-16T00:34:29.758 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4194304 len 4096 2024-09-16T00:34:29.758 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4194304 len 8192 2024-09-16T00:34:29.758 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4194304 len 8192 2024-09-16T00:34:29.758 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4194304 len 8192 2024-09-16T00:34:29.773 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4194304 len 8192 2024-09-16T00:34:29.773 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4194304 len 16384 2024-09-16T00:34:29.773 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4194304 len 16384 2024-09-16T00:34:29.773 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4194304 len 16384 2024-09-16T00:34:29.773 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4194304 len 16384 2024-09-16T00:34:29.773 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4194816 len 1024 2024-09-16T00:34:29.773 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4194816 len 1024 2024-09-16T00:34:29.773 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4194816 len 1024 2024-09-16T00:34:29.773 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4194816 len 1024 2024-09-16T00:34:29.773 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4194816 len 2048 2024-09-16T00:34:29.773 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4194816 len 2048 2024-09-16T00:34:29.774 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4194816 len 2048 2024-09-16T00:34:29.774 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4194816 len 2048 2024-09-16T00:34:29.774 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4194816 len 4096 2024-09-16T00:34:29.774 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4194816 len 4096 2024-09-16T00:34:29.774 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4194816 len 4096 2024-09-16T00:34:29.774 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4194816 len 4096 2024-09-16T00:34:29.774 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4194816 len 8192 2024-09-16T00:34:29.774 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4194816 len 8192 2024-09-16T00:34:29.774 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4194816 len 8192 2024-09-16T00:34:29.774 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4194816 len 8192 2024-09-16T00:34:29.774 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4194816 len 16384 2024-09-16T00:34:29.774 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4194816 len 16384 2024-09-16T00:34:29.774 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4194816 len 16384 2024-09-16T00:34:29.774 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4194816 len 16384 2024-09-16T00:34:29.775 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4195328 len 1024 2024-09-16T00:34:29.775 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4195328 len 1024 2024-09-16T00:34:29.775 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4195328 len 1024 2024-09-16T00:34:29.775 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4195328 len 1024 2024-09-16T00:34:29.775 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4195328 len 2048 2024-09-16T00:34:29.775 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4195328 len 2048 2024-09-16T00:34:29.775 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4195328 len 2048 2024-09-16T00:34:29.775 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4195328 len 2048 2024-09-16T00:34:29.775 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4195328 len 4096 2024-09-16T00:34:29.775 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4195328 len 4096 2024-09-16T00:34:29.775 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4195328 len 4096 2024-09-16T00:34:29.775 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4195328 len 4096 2024-09-16T00:34:29.775 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4195328 len 8192 2024-09-16T00:34:29.775 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4195328 len 8192 2024-09-16T00:34:29.776 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4195328 len 8192 2024-09-16T00:34:29.776 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4195328 len 8192 2024-09-16T00:34:29.776 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4195328 len 16384 2024-09-16T00:34:29.776 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4195328 len 16384 2024-09-16T00:34:29.776 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4195328 len 16384 2024-09-16T00:34:29.776 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4195328 len 16384 2024-09-16T00:34:29.776 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4195840 len 1024 2024-09-16T00:34:29.776 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4195840 len 1024 2024-09-16T00:34:29.776 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4195840 len 1024 2024-09-16T00:34:29.776 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4195840 len 1024 2024-09-16T00:34:29.776 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4195840 len 2048 2024-09-16T00:34:29.776 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4195840 len 2048 2024-09-16T00:34:29.776 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4195840 len 2048 2024-09-16T00:34:29.776 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4195840 len 2048 2024-09-16T00:34:29.777 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4195840 len 4096 2024-09-16T00:34:29.777 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4195840 len 4096 2024-09-16T00:34:29.777 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4195840 len 4096 2024-09-16T00:34:29.777 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4195840 len 4096 2024-09-16T00:34:29.777 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4195840 len 8192 2024-09-16T00:34:29.777 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4195840 len 8192 2024-09-16T00:34:29.777 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4195840 len 8192 2024-09-16T00:34:29.777 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4195840 len 8192 2024-09-16T00:34:29.777 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4195840 len 16384 2024-09-16T00:34:29.777 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4195840 len 16384 2024-09-16T00:34:29.777 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4195840 len 16384 2024-09-16T00:34:29.777 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4195840 len 16384 2024-09-16T00:34:29.777 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4196352 len 1024 2024-09-16T00:34:29.777 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4196352 len 1024 2024-09-16T00:34:29.778 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4196352 len 1024 2024-09-16T00:34:29.778 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4196352 len 1024 2024-09-16T00:34:29.778 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4196352 len 2048 2024-09-16T00:34:29.778 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4196352 len 2048 2024-09-16T00:34:29.778 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4196352 len 2048 2024-09-16T00:34:29.778 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4196352 len 2048 2024-09-16T00:34:29.778 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4196352 len 4096 2024-09-16T00:34:29.778 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4196352 len 4096 2024-09-16T00:34:29.778 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4196352 len 4096 2024-09-16T00:34:29.778 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4196352 len 4096 2024-09-16T00:34:29.778 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4196352 len 8192 2024-09-16T00:34:29.778 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4196352 len 8192 2024-09-16T00:34:29.778 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4196352 len 8192 2024-09-16T00:34:29.778 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4196352 len 8192 2024-09-16T00:34:29.779 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4196352 len 16384 2024-09-16T00:34:29.954 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:34:30.083 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4196352 len 16384 2024-09-16T00:34:30.084 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4196352 len 16384 2024-09-16T00:34:30.084 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4196352 len 16384 2024-09-16T00:34:30.084 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4196864 len 1024 2024-09-16T00:34:30.084 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4196864 len 1024 2024-09-16T00:34:30.084 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4196864 len 1024 2024-09-16T00:34:30.084 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4196864 len 1024 2024-09-16T00:34:30.084 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4196864 len 2048 2024-09-16T00:34:30.084 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4196864 len 2048 2024-09-16T00:34:30.084 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4196864 len 2048 2024-09-16T00:34:30.084 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4196864 len 2048 2024-09-16T00:34:30.084 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4196864 len 4096 2024-09-16T00:34:30.084 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4196864 len 4096 2024-09-16T00:34:30.084 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4196864 len 4096 2024-09-16T00:34:30.085 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4196864 len 4096 2024-09-16T00:34:30.085 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4196864 len 8192 2024-09-16T00:34:30.085 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4196864 len 8192 2024-09-16T00:34:30.085 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4196864 len 8192 2024-09-16T00:34:30.085 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4196864 len 8192 2024-09-16T00:34:30.085 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4196864 len 16384 2024-09-16T00:34:30.085 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4196864 len 16384 2024-09-16T00:34:30.085 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4196864 len 16384 2024-09-16T00:34:30.085 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4196864 len 16384 2024-09-16T00:34:30.085 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4197376 len 1024 2024-09-16T00:34:30.085 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4197376 len 1024 2024-09-16T00:34:30.085 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4197376 len 1024 2024-09-16T00:34:30.085 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4197376 len 1024 2024-09-16T00:34:30.085 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4197376 len 2048 2024-09-16T00:34:30.086 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4197376 len 2048 2024-09-16T00:34:30.086 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4197376 len 2048 2024-09-16T00:34:30.086 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4197376 len 2048 2024-09-16T00:34:30.086 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4197376 len 4096 2024-09-16T00:34:30.086 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4197376 len 4096 2024-09-16T00:34:30.086 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4197376 len 4096 2024-09-16T00:34:30.086 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4197376 len 4096 2024-09-16T00:34:30.086 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4197376 len 8192 2024-09-16T00:34:30.086 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4197376 len 8192 2024-09-16T00:34:30.086 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4197376 len 8192 2024-09-16T00:34:30.087 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4197376 len 8192 2024-09-16T00:34:30.087 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4197376 len 16384 2024-09-16T00:34:30.087 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4197376 len 16384 2024-09-16T00:34:30.087 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4197376 len 16384 2024-09-16T00:34:30.087 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4197376 len 16384 2024-09-16T00:34:30.087 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4197888 len 1024 2024-09-16T00:34:30.087 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4197888 len 1024 2024-09-16T00:34:30.087 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4197888 len 1024 2024-09-16T00:34:30.087 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4197888 len 1024 2024-09-16T00:34:30.087 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4197888 len 2048 2024-09-16T00:34:30.087 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4197888 len 2048 2024-09-16T00:34:30.087 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4197888 len 2048 2024-09-16T00:34:30.087 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4197888 len 2048 2024-09-16T00:34:30.088 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4197888 len 4096 2024-09-16T00:34:30.088 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4197888 len 4096 2024-09-16T00:34:30.088 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4197888 len 4096 2024-09-16T00:34:30.088 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4197888 len 4096 2024-09-16T00:34:30.088 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4197888 len 8192 2024-09-16T00:34:30.088 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4197888 len 8192 2024-09-16T00:34:30.088 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4197888 len 8192 2024-09-16T00:34:30.088 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4197888 len 8192 2024-09-16T00:34:30.088 INFO:tasks.workunit.client.0.smithi038.stdout:read_direct buf_align 3584 offset 4197888 len 16384 2024-09-16T00:34:30.088 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4197888 len 16384 2024-09-16T00:34:30.088 INFO:tasks.workunit.client.0.smithi038.stdout:read_sync buf_align 3584 offset 4197888 len 16384 2024-09-16T00:34:30.088 INFO:tasks.workunit.client.0.smithi038.stdout:read_file buf_align 3584 offset 4197888 len 16384 2024-09-16T00:34:30.088 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4190720 len 1024 2024-09-16T00:34:30.089 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4190720 len 1024 2024-09-16T00:34:30.089 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4190720 len 1024 2024-09-16T00:34:30.089 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4190720 len 1024 2024-09-16T00:34:30.089 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4190720 len 2048 2024-09-16T00:34:30.089 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4190720 len 2048 2024-09-16T00:34:30.089 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4190720 len 2048 2024-09-16T00:34:30.089 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4190720 len 2048 2024-09-16T00:34:30.089 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4190720 len 4096 2024-09-16T00:34:30.089 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4190720 len 4096 2024-09-16T00:34:30.089 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4190720 len 4096 2024-09-16T00:34:30.089 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4190720 len 4096 2024-09-16T00:34:30.089 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4190720 len 8192 2024-09-16T00:34:30.089 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4190720 len 8192 2024-09-16T00:34:30.089 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4190720 len 8192 2024-09-16T00:34:30.090 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4190720 len 8192 2024-09-16T00:34:30.090 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4190720 len 16384 2024-09-16T00:34:30.090 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4190720 len 16384 2024-09-16T00:34:30.090 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4190720 len 16384 2024-09-16T00:34:30.090 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4190720 len 16384 2024-09-16T00:34:30.090 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4191232 len 1024 2024-09-16T00:34:30.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.455+0000 7f4b52ec3640 1 -- 172.21.15.38:0/410838794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b4c106a90 msgr2=0x7f4b4c106e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:30.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.455+0000 7f4b52ec3640 1 --2- 172.21.15.38:0/410838794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b4c106a90 0x7f4b4c106e70 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f4b40009930 tx=0x7f4b4002a420 comp rx=0 tx=0).stop 2024-09-16T00:34:30.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.455+0000 7f4b52ec3640 1 -- 172.21.15.38:0/410838794 shutdown_connections 2024-09-16T00:34:30.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.455+0000 7f4b52ec3640 1 --2- 172.21.15.38:0/410838794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b4c101430 0x7f4b4c10d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.455+0000 7f4b52ec3640 1 --2- 172.21.15.38:0/410838794 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b4c100a90 0x7f4b4c100ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.455+0000 7f4b52ec3640 1 --2- 172.21.15.38:0/410838794 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b4c106a90 0x7f4b4c106e70 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.455+0000 7f4b52ec3640 1 -- 172.21.15.38:0/410838794 >> 172.21.15.38:0/410838794 conn(0x7f4b4c0fc7d0 msgr2=0x7f4b4c0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:30.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.455+0000 7f4b52ec3640 1 -- 172.21.15.38:0/410838794 shutdown_connections 2024-09-16T00:34:30.473 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.455+0000 7f4b52ec3640 1 -- 172.21.15.38:0/410838794 wait complete. 2024-09-16T00:34:30.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.456+0000 7f4b52ec3640 1 Processor -- start 2024-09-16T00:34:30.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.456+0000 7f4b52ec3640 1 -- start start 2024-09-16T00:34:30.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.457+0000 7f4b52ec3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b4c100a90 0x7f4b4c1992a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:30.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.457+0000 7f4b52ec3640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b4c101430 0x7f4b4c1997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:30.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.457+0000 7f4b52ec3640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b4c106a90 0x7f4b4c1a4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:30.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.457+0000 7f4b52ec3640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c110e90 con 0x7f4b4c100a90 2024-09-16T00:34:30.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.457+0000 7f4b52ec3640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c110d10 con 0x7f4b4c101430 2024-09-16T00:34:30.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.457+0000 7f4b52ec3640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c111010 con 0x7f4b4c106a90 2024-09-16T00:34:30.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.457+0000 7f4b50c5f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b4c100a90 0x7f4b4c1992a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:30.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.457+0000 7f4b50c5f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b4c100a90 0x7f4b4c1992a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51232/0 (socket says 172.21.15.38:51232) 2024-09-16T00:34:30.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.457+0000 7f4b50c5f640 1 -- 172.21.15.38:0/1572203649 learned_addr learned my addr 172.21.15.38:0/1572203649 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:34:30.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.457+0000 7f4b51460640 1 --2- 172.21.15.38:0/1572203649 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b4c106a90 0x7f4b4c1a4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:30.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.457+0000 7f4b4bfff640 1 --2- 172.21.15.38:0/1572203649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b4c101430 0x7f4b4c1997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:30.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.457+0000 7f4b50c5f640 1 -- 172.21.15.38:0/1572203649 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b4c106a90 msgr2=0x7f4b4c1a4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:30.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.457+0000 7f4b50c5f640 1 --2- 172.21.15.38:0/1572203649 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b4c106a90 0x7f4b4c1a4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.457+0000 7f4b50c5f640 1 -- 172.21.15.38:0/1572203649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b4c101430 msgr2=0x7f4b4c1997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:30.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.457+0000 7f4b50c5f640 1 --2- 172.21.15.38:0/1572203649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b4c101430 0x7f4b4c1997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.457+0000 7f4b50c5f640 1 -- 172.21.15.38:0/1572203649 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b4c197090 con 0x7f4b4c100a90 2024-09-16T00:34:30.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.458+0000 7f4b50c5f640 1 --2- 172.21.15.38:0/1572203649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b4c100a90 0x7f4b4c1992a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f4b40009230 tx=0x7f4b40004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:30.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.458+0000 7f4b49ffb640 1 -- 172.21.15.38:0/1572203649 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4003f030 con 0x7f4b4c100a90 2024-09-16T00:34:30.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.458+0000 7f4b49ffb640 1 -- 172.21.15.38:0/1572203649 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b40003ad0 con 0x7f4b4c100a90 2024-09-16T00:34:30.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.458+0000 7f4b49ffb640 1 -- 172.21.15.38:0/1572203649 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b40035070 con 0x7f4b4c100a90 2024-09-16T00:34:30.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.458+0000 7f4b52ec3640 1 -- 172.21.15.38:0/1572203649 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b4c197320 con 0x7f4b4c100a90 2024-09-16T00:34:30.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.458+0000 7f4b52ec3640 1 -- 172.21.15.38:0/1572203649 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b4c1976f0 con 0x7f4b4c100a90 2024-09-16T00:34:30.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.461+0000 7f4b49ffb640 1 -- 172.21.15.38:0/1572203649 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f4b40042030 con 0x7f4b4c100a90 2024-09-16T00:34:30.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.461+0000 7f4b5529c740 1 Processor -- start 2024-09-16T00:34:30.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.461+0000 7f4b5529c740 1 -- start start 2024-09-16T00:34:30.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.462+0000 7f4b49ffb640 1 --2- 172.21.15.38:0/1572203649 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f4b3c0a32c0 0x7f4b3c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:30.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.462+0000 7f4b5529c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56283ff41b40 0x56283ff40090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:30.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.462+0000 7f4b5529c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283ff405d0 0x56283ff40a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:30.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.462+0000 7f4b5529c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56283ff4b370 0x56283ff4d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:30.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.462+0000 7f4b5529c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56283fea62f0 con 0x56283ff4b370 2024-09-16T00:34:30.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.462+0000 7f4b5529c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56283fe4dcf0 con 0x56283ff41b40 2024-09-16T00:34:30.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.462+0000 7f4b5529c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56283fea22b0 con 0x56283ff405d0 2024-09-16T00:34:30.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.462+0000 7f4b51460640 1 --2- 172.21.15.38:0/1572203649 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f4b3c0a32c0 0x7f4b3c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:30.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.462+0000 7f4b4bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283ff405d0 0x56283ff40a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:30.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.462+0000 7f4b50c5f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56283ff41b40 0x56283ff40090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:30.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.462+0000 7f4b4bfff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283ff405d0 0x56283ff40a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.38:59510/0 (socket says 172.21.15.38:59510) 2024-09-16T00:34:30.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.462+0000 7f4b4bfff640 1 -- 172.21.15.38:0/4246264845 learned_addr learned my addr 172.21.15.38:0/4246264845 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:34:30.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.462+0000 7f4b49ffb640 1 -- 172.21.15.38:0/1572203649 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f4b400f0220 con 0x7f4b4c100a90 2024-09-16T00:34:30.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.462+0000 7f4b51460640 1 --2- 172.21.15.38:0/4246264845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56283ff4b370 0x56283ff4d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:30.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.462+0000 7f4b4bfff640 1 -- 172.21.15.38:0/4246264845 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56283ff41b40 msgr2=0x56283ff40090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:30.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.462+0000 7f4b4bfff640 1 --2- 172.21.15.38:0/4246264845 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56283ff41b40 0x56283ff40090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.462+0000 7f4b4bfff640 1 -- 172.21.15.38:0/4246264845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56283ff4b370 msgr2=0x56283ff4d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:30.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.462+0000 7f4b4bfff640 1 --2- 172.21.15.38:0/4246264845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56283ff4b370 0x56283ff4d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.462+0000 7f4b4bfff640 1 -- 172.21.15.38:0/4246264845 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56283fe323a0 con 0x56283ff405d0 2024-09-16T00:34:30.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.462+0000 7f4b50c5f640 1 --2- 172.21.15.38:0/4246264845 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56283ff41b40 0x56283ff40090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:34:30.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.462+0000 7f4b4bfff640 1 --2- 172.21.15.38:0/4246264845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283ff405d0 0x56283ff40a10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4b3800bcb0 tx=0x7f4b380107a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=79144901613aa360 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:30.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.462+0000 7f4b51460640 1 --2- 172.21.15.38:0/1572203649 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f4b3c0a32c0 0x7f4b3c0a5780 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4b44005fd0 tx=0x7f4b44009e90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:30.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.463+0000 7f4b36ffd640 1 -- 172.21.15.38:0/4246264845 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b38004030 con 0x56283ff405d0 2024-09-16T00:34:30.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.463+0000 7f4b36ffd640 1 -- 172.21.15.38:0/4246264845 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b380041d0 con 0x56283ff405d0 2024-09-16T00:34:30.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.463+0000 7f4b36ffd640 1 -- 172.21.15.38:0/4246264845 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b38011790 con 0x56283ff405d0 2024-09-16T00:34:30.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.463+0000 7f4b5529c740 1 -- 172.21.15.38:0/4246264845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283ff405d0 msgr2=0x56283ff40a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:30.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.463+0000 7f4b5529c740 1 --2- 172.21.15.38:0/4246264845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283ff405d0 0x56283ff40a10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4b3800bcb0 tx=0x7f4b380107a0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.464+0000 7f4b5529c740 1 -- 172.21.15.38:0/4246264845 shutdown_connections 2024-09-16T00:34:30.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.464+0000 7f4b5529c740 1 --2- 172.21.15.38:0/4246264845 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56283ff4b370 0x56283ff4d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.464+0000 7f4b5529c740 1 --2- 172.21.15.38:0/4246264845 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283ff405d0 0x56283ff40a10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.464+0000 7f4b5529c740 1 --2- 172.21.15.38:0/4246264845 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56283ff41b40 0x56283ff40090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.464+0000 7f4b5529c740 1 -- 172.21.15.38:0/4246264845 >> 172.21.15.38:0/4246264845 conn(0x56283ff09310 msgr2=0x56283ff3cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:30.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.464+0000 7f4b5529c740 1 -- 172.21.15.38:0/4246264845 shutdown_connections 2024-09-16T00:34:30.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.464+0000 7f4b5529c740 1 -- 172.21.15.38:0/4246264845 wait complete. 2024-09-16T00:34:30.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.464+0000 7f4b5529c740 1 -- start start 2024-09-16T00:34:30.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.464+0000 7f4b5529c740 1 Processor -- start 2024-09-16T00:34:30.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.465+0000 7f4b5529c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283ff405d0 0x56283ff4ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:30.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.465+0000 7f4b5529c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56283ff41b40 0x56283ff48000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:30.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.465+0000 7f4b5529c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56283ff4b370 0x56283ff485d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:30.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.465+0000 7f4b5529c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56283ff4e070 con 0x56283ff41b40 2024-09-16T00:34:30.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.465+0000 7f4b5529c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56283fe61060 con 0x56283ff4b370 2024-09-16T00:34:30.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.465+0000 7f4b5529c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56283ff4e1f0 con 0x56283ff405d0 2024-09-16T00:34:30.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.465+0000 7f4b50c5f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283ff405d0 0x56283ff4ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:30.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.465+0000 7f4b50c5f640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283ff405d0 0x56283ff4ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.38:59514/0 (socket says 172.21.15.38:59514) 2024-09-16T00:34:30.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.465+0000 7f4b5529c740 10 client.0 _ll_register_callbacks cb 0x56283fbb48c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:30.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.465+0000 7f4b50c5f640 1 -- 172.21.15.38:0/1998498374 learned_addr learned my addr 172.21.15.38:0/1998498374 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:34:30.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.465+0000 7f4b4bfff640 1 --2- 172.21.15.38:0/1998498374 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56283ff41b40 0x56283ff48000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:30.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.465+0000 7f4b51460640 1 --2- 172.21.15.38:0/1998498374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56283ff4b370 0x56283ff485d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:30.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.465+0000 7f4b50c5f640 1 -- 172.21.15.38:0/1998498374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56283ff4b370 msgr2=0x56283ff485d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:30.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.465+0000 7f4b50c5f640 1 --2- 172.21.15.38:0/1998498374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56283ff4b370 0x56283ff485d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.465+0000 7f4b50c5f640 1 -- 172.21.15.38:0/1998498374 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56283ff41b40 msgr2=0x56283ff48000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:30.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.465+0000 7f4b50c5f640 1 --2- 172.21.15.38:0/1998498374 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56283ff41b40 0x56283ff48000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.465+0000 7f4b50c5f640 1 -- 172.21.15.38:0/1998498374 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56283fc173c0 con 0x56283ff405d0 2024-09-16T00:34:30.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.466+0000 7f4b50c5f640 1 --2- 172.21.15.38:0/1998498374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283ff405d0 0x56283ff4ad50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4b400ed940 tx=0x7f4b40041f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=370fe21c8b4cad84 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:30.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.466+0000 7f4b2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-16T00:34:30.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.466+0000 7f4b2bfff640 1 -- 172.21.15.38:0/1998498374 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b40049040 con 0x56283ff405d0 2024-09-16T00:34:30.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.466+0000 7f4b2bfff640 1 -- 172.21.15.38:0/1998498374 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f4b400eda50 con 0x56283ff405d0 2024-09-16T00:34:30.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.466+0000 7f4b5529c740 1 -- 172.21.15.38:0/1998498374 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b380041d0 con 0x56283ff405d0 2024-09-16T00:34:30.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.466+0000 7f4b2bfff640 1 -- 172.21.15.38:0/1998498374 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4b400bbe70 con 0x56283ff405d0 2024-09-16T00:34:30.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.466+0000 7f4b5529c740 10 client.25264 fetch_fsmap learned FSMap version 26 2024-09-16T00:34:30.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.466+0000 7f4b5529c740 1 -- 172.21.15.38:0/1998498374 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56283ff4e0d0 con 0x56283ff405d0 2024-09-16T00:34:30.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.466+0000 7f4b2bfff640 1 -- 172.21.15.38:0/1998498374 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b380041d0 con 0x56283ff405d0 2024-09-16T00:34:30.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.466+0000 7f4b2bfff640 1 -- 172.21.15.38:0/1998498374 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4b40003c80 con 0x56283ff405d0 2024-09-16T00:34:30.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.466+0000 7f4b5529c740 10 client.25264 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:34:30.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.466+0000 7f4b5529c740 10 client.25264 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-16T00:34:30.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.466+0000 7f4b5529c740 20 client.25264 populate_metadata read hostname 'smithi038' 2024-09-16T00:34:30.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.466+0000 7f4b5529c740 1 --2- 172.21.15.38:0/1998498374 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x56283ff5afc0 0x56283ff5d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:30.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.467+0000 7f4b5529c740 4 client.25264 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:34:30.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.467+0000 7f4b5529c740 1 -- 172.21.15.38:0/1998498374 --> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56283feb3860 con 0x56283ff5afc0 2024-09-16T00:34:30.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.467+0000 7f4b4bfff640 1 --2- 172.21.15.38:0/1998498374 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x56283ff5afc0 0x56283ff5d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:30.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.467+0000 7f4b2bfff640 10 client.25264 ms_handle_connect on v2:172.21.15.73:6834/2866363113 2024-09-16T00:34:30.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.467+0000 7f4b4bfff640 1 --2- 172.21.15.38:0/1998498374 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x56283ff5afc0 0x56283ff5d3b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1ea15eff6e42956d server_cookie=608e6f544c25d591 in_seq=0 out_seq=0 2024-09-16T00:34:30.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.468+0000 7f4b2bfff640 1 -- 172.21.15.38:0/1998498374 <== mds.2 v2:172.21.15.73:6834/2866363113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56283feb3860 con 0x56283ff5afc0 2024-09-16T00:34:30.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.468+0000 7f4b2bfff640 10 client.25264 handle_command_reply: tid=0 2024-09-16T00:34:30.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.468+0000 7f4b5529c740 1 client.25264 shutdown 2024-09-16T00:34:30.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.468+0000 7f4b5529c740 1 -- 172.21.15.38:0/1998498374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283ff405d0 msgr2=0x56283ff4ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:30.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.468+0000 7f4b5529c740 1 --2- 172.21.15.38:0/1998498374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283ff405d0 0x56283ff4ad50 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4b400ed940 tx=0x7f4b40041f70 comp rx=0 tx=0).stop 2024-09-16T00:34:30.481 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.468+0000 7f4b5529c740 1 -- 172.21.15.38:0/1998498374 shutdown_connections 2024-09-16T00:34:30.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.468+0000 7f4b5529c740 1 --2- 172.21.15.38:0/1998498374 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x56283ff5afc0 0x56283ff5d3b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.468+0000 7f4b5529c740 1 --2- 172.21.15.38:0/1998498374 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56283ff4b370 0x56283ff485d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.468+0000 7f4b5529c740 1 --2- 172.21.15.38:0/1998498374 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56283ff41b40 0x56283ff48000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.468+0000 7f4b2bfff640 0 client.25264 ms_handle_reset on v2:172.21.15.73:6834/2866363113 2024-09-16T00:34:30.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.468+0000 7f4b5529c740 1 --2- 172.21.15.38:0/1998498374 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283ff405d0 0x56283ff4ad50 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.468+0000 7f4b5529c740 1 -- 172.21.15.38:0/1998498374 >> 172.21.15.38:0/1998498374 conn(0x56283ff09310 msgr2=0x56283ff42960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:30.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.468+0000 7f4b5529c740 1 -- 172.21.15.38:0/1998498374 shutdown_connections 2024-09-16T00:34:30.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.468+0000 7f4b5529c740 1 -- 172.21.15.38:0/1998498374 wait complete. 2024-09-16T00:34:30.482 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.468+0000 7f4b5529c740 20 client.25264 trim_cache size 0 max 16384 2024-09-16T00:34:30.485 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.483+0000 7f4b5529c740 1 Processor -- start 2024-09-16T00:34:30.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.483+0000 7f4b5529c740 1 -- start start 2024-09-16T00:34:30.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.484+0000 7f4b5529c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283ff405d0 0x56283ff57310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:30.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.484+0000 7f4b5529c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56283ff41b40 0x56283ff4c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:30.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.484+0000 7f4b5529c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56283ff4b370 0x56283ff4cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:30.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.484+0000 7f4b5529c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c1b3740 con 0x56283ff4b370 2024-09-16T00:34:30.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.484+0000 7f4b5529c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c1b1010 con 0x56283ff41b40 2024-09-16T00:34:30.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.484+0000 7f4b5529c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c1aee60 con 0x56283ff405d0 2024-09-16T00:34:30.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.484+0000 7f4b51460640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56283ff4b370 0x56283ff4cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:30.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.484+0000 7f4b4bfff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56283ff41b40 0x56283ff4c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:30.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.484+0000 7f4b51460640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56283ff4b370 0x56283ff4cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51254/0 (socket says 172.21.15.38:51254) 2024-09-16T00:34:30.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.484+0000 7f4b51460640 1 -- 172.21.15.38:0/2751590927 learned_addr learned my addr 172.21.15.38:0/2751590927 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:34:30.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.484+0000 7f4b50c5f640 1 --2- 172.21.15.38:0/2751590927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283ff405d0 0x56283ff57310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:30.486 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.484+0000 7f4b51460640 1 -- 172.21.15.38:0/2751590927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283ff405d0 msgr2=0x56283ff57310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:30.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.484+0000 7f4b51460640 1 --2- 172.21.15.38:0/2751590927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283ff405d0 0x56283ff57310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.484+0000 7f4b51460640 1 -- 172.21.15.38:0/2751590927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56283ff41b40 msgr2=0x56283ff4c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:30.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.484+0000 7f4b51460640 1 --2- 172.21.15.38:0/2751590927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56283ff41b40 0x56283ff4c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.484+0000 7f4b51460640 1 -- 172.21.15.38:0/2751590927 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56283ff48bd0 con 0x56283ff4b370 2024-09-16T00:34:30.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.484+0000 7f4b4bfff640 1 --2- 172.21.15.38:0/2751590927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56283ff41b40 0x56283ff4c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:34:30.487 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.484+0000 7f4b51460640 1 --2- 172.21.15.38:0/2751590927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56283ff4b370 0x56283ff4cca0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4b440115e0 tx=0x7f4b44004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76f105f2b7b8d37e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:30.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.485+0000 7f4b367fc640 1 -- 172.21.15.38:0/2751590927 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b44004050 con 0x56283ff4b370 2024-09-16T00:34:30.491 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.485+0000 7f4b367fc640 1 -- 172.21.15.38:0/2751590927 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b440046d0 con 0x56283ff4b370 2024-09-16T00:34:30.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.485+0000 7f4b367fc640 1 -- 172.21.15.38:0/2751590927 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b44020ae0 con 0x56283ff4b370 2024-09-16T00:34:30.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.485+0000 7f4b5529c740 1 -- 172.21.15.38:0/2751590927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56283ff4b370 msgr2=0x56283ff4cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:30.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.485+0000 7f4b5529c740 1 --2- 172.21.15.38:0/2751590927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56283ff4b370 0x56283ff4cca0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f4b440115e0 tx=0x7f4b44004ab0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.486+0000 7f4b5529c740 1 -- 172.21.15.38:0/2751590927 shutdown_connections 2024-09-16T00:34:30.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.486+0000 7f4b5529c740 1 --2- 172.21.15.38:0/2751590927 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56283ff4b370 0x56283ff4cca0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.486+0000 7f4b5529c740 1 --2- 172.21.15.38:0/2751590927 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56283ff41b40 0x56283ff4c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.486+0000 7f4b5529c740 1 --2- 172.21.15.38:0/2751590927 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283ff405d0 0x56283ff57310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.486+0000 7f4b5529c740 1 -- 172.21.15.38:0/2751590927 >> 172.21.15.38:0/2751590927 conn(0x56283ff09310 msgr2=0x56283ff4d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:30.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.486+0000 7f4b5529c740 1 -- 172.21.15.38:0/2751590927 shutdown_connections 2024-09-16T00:34:30.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.486+0000 7f4b5529c740 1 -- 172.21.15.38:0/2751590927 wait complete. 2024-09-16T00:34:30.492 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.486+0000 7f4b5529c740 1 -- start start 2024-09-16T00:34:30.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.486+0000 7f4b5529c740 1 Processor -- start 2024-09-16T00:34:30.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.486+0000 7f4b5529c740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56283ff405d0 0x56283ff62840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:30.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.486+0000 7f4b5529c740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56283ff41b40 0x56283ff62d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:30.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.486+0000 7f4b5529c740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283ff4b370 0x56283ff632c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:30.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.487+0000 7f4b5529c740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c1af4d0 con 0x56283ff41b40 2024-09-16T00:34:30.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.487+0000 7f4b5529c740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c1b0940 con 0x56283ff405d0 2024-09-16T00:34:30.493 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.487+0000 7f4b5529c740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4b4c1afba0 con 0x56283ff4b370 2024-09-16T00:34:30.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.487+0000 7f4b5529c740 10 client.0 _ll_register_callbacks cb 0x7f4b400b8090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:30.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.487+0000 7f4b50c5f640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56283ff405d0 0x56283ff62840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:30.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.487+0000 7f4b4bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56283ff41b40 0x56283ff62d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:30.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.487+0000 7f4b51460640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283ff4b370 0x56283ff632c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:30.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.487+0000 7f4b4bfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56283ff41b40 0x56283ff62d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51262/0 (socket says 172.21.15.38:51262) 2024-09-16T00:34:30.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.487+0000 7f4b4bfff640 1 -- 172.21.15.38:0/882901867 learned_addr learned my addr 172.21.15.38:0/882901867 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:34:30.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.487+0000 7f4b50c5f640 1 -- 172.21.15.38:0/882901867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283ff4b370 msgr2=0x56283ff632c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:30.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.487+0000 7f4b50c5f640 1 --2- 172.21.15.38:0/882901867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283ff4b370 0x56283ff632c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.487+0000 7f4b50c5f640 1 -- 172.21.15.38:0/882901867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56283ff41b40 msgr2=0x56283ff62d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:30.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.487+0000 7f4b50c5f640 1 --2- 172.21.15.38:0/882901867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56283ff41b40 0x56283ff62d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.487+0000 7f4b50c5f640 1 -- 172.21.15.38:0/882901867 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56283ff519d0 con 0x56283ff405d0 2024-09-16T00:34:30.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.487+0000 7f4b4bfff640 1 --2- 172.21.15.38:0/882901867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56283ff41b40 0x56283ff62d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:34:30.494 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.487+0000 7f4b50c5f640 1 --2- 172.21.15.38:0/882901867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56283ff405d0 0x56283ff62840 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4b40009a50 tx=0x7f4b400f0150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41360a0180c1415 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:30.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.487+0000 7f4b2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-09-16T00:34:30.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.487+0000 7f4b2bfff640 1 -- 172.21.15.38:0/882901867 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b40049040 con 0x56283ff405d0 2024-09-16T00:34:30.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.487+0000 7f4b2bfff640 1 -- 172.21.15.38:0/882901867 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f4b400b8b80 con 0x56283ff405d0 2024-09-16T00:34:30.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.487+0000 7f4b5529c740 1 -- 172.21.15.38:0/882901867 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b440046d0 con 0x56283ff405d0 2024-09-16T00:34:30.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.488+0000 7f4b2bfff640 1 -- 172.21.15.38:0/882901867 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f4b40049e40 con 0x56283ff405d0 2024-09-16T00:34:30.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.488+0000 7f4b5529c740 10 client.25343 fetch_fsmap learned FSMap version 26 2024-09-16T00:34:30.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.488+0000 7f4b5529c740 1 -- 172.21.15.38:0/882901867 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56283ffc5f70 con 0x56283ff405d0 2024-09-16T00:34:30.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.488+0000 7f4b2bfff640 1 -- 172.21.15.38:0/882901867 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4b440046d0 con 0x56283ff405d0 2024-09-16T00:34:30.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.488+0000 7f4b2bfff640 1 -- 172.21.15.38:0/882901867 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f4b40036cf0 con 0x56283ff405d0 2024-09-16T00:34:30.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.488+0000 7f4b5529c740 10 client.25343 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:34:30.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.488+0000 7f4b5529c740 10 client.25343 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-16T00:34:30.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.488+0000 7f4b5529c740 20 client.25343 populate_metadata read hostname 'smithi038' 2024-09-16T00:34:30.495 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.488+0000 7f4b5529c740 1 --2- 172.21.15.38:0/882901867 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x56283ffc6500 0x56283ffc88f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:30.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.488+0000 7f4b5529c740 4 client.25343 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:34:30.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.488+0000 7f4b5529c740 1 -- 172.21.15.38:0/882901867 --> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56283feadd70 con 0x56283ffc6500 2024-09-16T00:34:30.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.488+0000 7f4b4bfff640 1 --2- 172.21.15.38:0/882901867 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x56283ffc6500 0x56283ffc88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:30.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.489+0000 7f4b4bfff640 1 --2- 172.21.15.38:0/882901867 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x56283ffc6500 0x56283ffc88f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a67b7a78cf5fc84a server_cookie=db555fd316ba5f1c in_seq=0 out_seq=0 2024-09-16T00:34:30.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.489+0000 7f4b2bfff640 10 client.25343 ms_handle_connect on v2:172.21.15.73:6834/2866363113 2024-09-16T00:34:30.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.489+0000 7f4b2bfff640 1 -- 172.21.15.38:0/882901867 <== mds.2 v2:172.21.15.73:6834/2866363113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56283feadd70 con 0x56283ffc6500 2024-09-16T00:34:30.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.489+0000 7f4b2bfff640 10 client.25343 handle_command_reply: tid=0 2024-09-16T00:34:30.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.489+0000 7f4b5529c740 1 client.25343 shutdown 2024-09-16T00:34:30.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.489+0000 7f4b5529c740 1 -- 172.21.15.38:0/882901867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56283ff405d0 msgr2=0x56283ff62840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:30.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.489+0000 7f4b5529c740 1 --2- 172.21.15.38:0/882901867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56283ff405d0 0x56283ff62840 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f4b40009a50 tx=0x7f4b400f0150 comp rx=0 tx=0).stop 2024-09-16T00:34:30.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.489+0000 7f4b5529c740 1 -- 172.21.15.38:0/882901867 shutdown_connections 2024-09-16T00:34:30.496 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.489+0000 7f4b5529c740 1 --2- 172.21.15.38:0/882901867 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x56283ffc6500 0x56283ffc88f0 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.489+0000 7f4b2bfff640 0 client.25343 ms_handle_reset on v2:172.21.15.73:6834/2866363113 2024-09-16T00:34:30.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.489+0000 7f4b5529c740 1 --2- 172.21.15.38:0/882901867 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56283ff4b370 0x56283ff632c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.489+0000 7f4b5529c740 1 --2- 172.21.15.38:0/882901867 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56283ff41b40 0x56283ff62d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.489+0000 7f4b5529c740 1 --2- 172.21.15.38:0/882901867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56283ff405d0 0x56283ff62840 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.489+0000 7f4b5529c740 1 -- 172.21.15.38:0/882901867 >> 172.21.15.38:0/882901867 conn(0x56283ff09310 msgr2=0x56283ff3b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:30.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.489+0000 7f4b5529c740 1 -- 172.21.15.38:0/882901867 shutdown_connections 2024-09-16T00:34:30.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.489+0000 7f4b5529c740 1 -- 172.21.15.38:0/882901867 wait complete. 2024-09-16T00:34:30.497 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.489+0000 7f4b5529c740 20 client.25343 trim_cache size 0 max 16384 2024-09-16T00:34:30.497 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-09-16T00:34:30.498 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.490+0000 7f4b52ec3640 1 -- 172.21.15.38:0/1572203649 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f4b3c0a32c0 msgr2=0x7f4b3c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:30.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.490+0000 7f4b52ec3640 1 --2- 172.21.15.38:0/1572203649 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f4b3c0a32c0 0x7f4b3c0a5780 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f4b44005fd0 tx=0x7f4b44009e90 comp rx=0 tx=0).stop 2024-09-16T00:34:30.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.490+0000 7f4b52ec3640 1 -- 172.21.15.38:0/1572203649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b4c100a90 msgr2=0x7f4b4c1992a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:30.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.490+0000 7f4b52ec3640 1 --2- 172.21.15.38:0/1572203649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b4c100a90 0x7f4b4c1992a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f4b40009230 tx=0x7f4b40004270 comp rx=0 tx=0).stop 2024-09-16T00:34:30.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.490+0000 7f4b52ec3640 1 -- 172.21.15.38:0/1572203649 shutdown_connections 2024-09-16T00:34:30.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.490+0000 7f4b52ec3640 1 --2- 172.21.15.38:0/1572203649 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f4b3c0a32c0 0x7f4b3c0a5780 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.490+0000 7f4b52ec3640 1 --2- 172.21.15.38:0/1572203649 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4b4c106a90 0x7f4b4c1a4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.490+0000 7f4b52ec3640 1 --2- 172.21.15.38:0/1572203649 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b4c101430 0x7f4b4c1997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.490+0000 7f4b52ec3640 1 --2- 172.21.15.38:0/1572203649 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4b4c100a90 0x7f4b4c1992a0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:30.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.490+0000 7f4b52ec3640 1 -- 172.21.15.38:0/1572203649 >> 172.21.15.38:0/1572203649 conn(0x7f4b4c0fc7d0 msgr2=0x7f4b4c0fe040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:30.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.491+0000 7f4b52ec3640 1 -- 172.21.15.38:0/1572203649 shutdown_connections 2024-09-16T00:34:30.499 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:30.491+0000 7f4b52ec3640 1 -- 172.21.15.38:0/1572203649 wait complete. 2024-09-16T00:34:31.214 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:31 smithi038 ceph-mon[30945]: pgmap v416: 129 pgs: 129 active+clean; 778 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.4 KiB/s wr, 501 op/s 2024-09-16T00:34:31.293 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:31 smithi073 ceph-mon[35254]: pgmap v416: 129 pgs: 129 active+clean; 778 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.4 KiB/s wr, 501 op/s 2024-09-16T00:34:31.311 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4191232 len 1024 2024-09-16T00:34:31.311 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4191232 len 1024 2024-09-16T00:34:31.311 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4191232 len 1024 2024-09-16T00:34:31.311 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4191232 len 2048 2024-09-16T00:34:31.311 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4191232 len 2048 2024-09-16T00:34:31.311 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4191232 len 2048 2024-09-16T00:34:31.311 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4191232 len 2048 2024-09-16T00:34:31.311 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4191232 len 4096 2024-09-16T00:34:31.312 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4191232 len 4096 2024-09-16T00:34:31.312 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4191232 len 4096 2024-09-16T00:34:31.312 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4191232 len 4096 2024-09-16T00:34:31.312 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4191232 len 8192 2024-09-16T00:34:31.312 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4191232 len 8192 2024-09-16T00:34:31.312 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4191232 len 8192 2024-09-16T00:34:31.312 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4191232 len 8192 2024-09-16T00:34:31.312 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4191232 len 16384 2024-09-16T00:34:31.312 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4191232 len 16384 2024-09-16T00:34:31.312 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4191232 len 16384 2024-09-16T00:34:31.312 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4191232 len 16384 2024-09-16T00:34:31.312 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4191744 len 1024 2024-09-16T00:34:31.312 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4191744 len 1024 2024-09-16T00:34:31.312 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4191744 len 1024 2024-09-16T00:34:31.313 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4191744 len 1024 2024-09-16T00:34:31.313 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4191744 len 2048 2024-09-16T00:34:31.313 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4191744 len 2048 2024-09-16T00:34:31.313 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4191744 len 2048 2024-09-16T00:34:31.313 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4191744 len 2048 2024-09-16T00:34:31.313 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4191744 len 4096 2024-09-16T00:34:31.313 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4191744 len 4096 2024-09-16T00:34:31.313 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4191744 len 4096 2024-09-16T00:34:31.313 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4191744 len 4096 2024-09-16T00:34:31.313 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4191744 len 8192 2024-09-16T00:34:31.313 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4191744 len 8192 2024-09-16T00:34:31.313 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4191744 len 8192 2024-09-16T00:34:31.313 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4191744 len 8192 2024-09-16T00:34:31.314 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4191744 len 16384 2024-09-16T00:34:31.314 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4191744 len 16384 2024-09-16T00:34:31.314 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4191744 len 16384 2024-09-16T00:34:31.314 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4191744 len 16384 2024-09-16T00:34:31.314 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4192256 len 1024 2024-09-16T00:34:31.314 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4192256 len 1024 2024-09-16T00:34:31.314 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4192256 len 1024 2024-09-16T00:34:31.314 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4192256 len 1024 2024-09-16T00:34:31.314 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4192256 len 2048 2024-09-16T00:34:31.314 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4192256 len 2048 2024-09-16T00:34:31.314 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4192256 len 2048 2024-09-16T00:34:31.314 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4192256 len 2048 2024-09-16T00:34:31.314 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4192256 len 4096 2024-09-16T00:34:31.314 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4192256 len 4096 2024-09-16T00:34:31.315 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4192256 len 4096 2024-09-16T00:34:31.315 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4192256 len 4096 2024-09-16T00:34:31.315 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4192256 len 8192 2024-09-16T00:34:31.315 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4192256 len 8192 2024-09-16T00:34:31.315 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4192256 len 8192 2024-09-16T00:34:31.315 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4192256 len 8192 2024-09-16T00:34:31.315 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4192256 len 16384 2024-09-16T00:34:31.315 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4192256 len 16384 2024-09-16T00:34:31.315 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4192256 len 16384 2024-09-16T00:34:31.315 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4192256 len 16384 2024-09-16T00:34:31.315 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4192768 len 1024 2024-09-16T00:34:31.315 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4192768 len 1024 2024-09-16T00:34:31.315 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4192768 len 1024 2024-09-16T00:34:31.315 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4192768 len 1024 2024-09-16T00:34:31.316 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4192768 len 2048 2024-09-16T00:34:31.316 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4192768 len 2048 2024-09-16T00:34:31.316 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4192768 len 2048 2024-09-16T00:34:31.316 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4192768 len 2048 2024-09-16T00:34:31.316 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4192768 len 4096 2024-09-16T00:34:31.316 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4192768 len 4096 2024-09-16T00:34:31.316 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4192768 len 4096 2024-09-16T00:34:31.316 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4192768 len 4096 2024-09-16T00:34:31.316 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4192768 len 8192 2024-09-16T00:34:31.316 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4192768 len 8192 2024-09-16T00:34:31.316 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4192768 len 8192 2024-09-16T00:34:31.316 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4192768 len 8192 2024-09-16T00:34:31.316 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4192768 len 16384 2024-09-16T00:34:31.317 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4192768 len 16384 2024-09-16T00:34:31.317 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4192768 len 16384 2024-09-16T00:34:31.317 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4192768 len 16384 2024-09-16T00:34:31.317 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4193280 len 1024 2024-09-16T00:34:31.317 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4193280 len 1024 2024-09-16T00:34:31.317 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4193280 len 1024 2024-09-16T00:34:31.317 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4193280 len 1024 2024-09-16T00:34:31.317 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4193280 len 2048 2024-09-16T00:34:31.317 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4193280 len 2048 2024-09-16T00:34:31.317 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4193280 len 2048 2024-09-16T00:34:31.318 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:34:31.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:31 smithi122 ceph-mon[35412]: pgmap v416: 129 pgs: 129 active+clean; 778 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.4 KiB/s wr, 501 op/s 2024-09-16T00:34:31.639 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:34:32.148 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4193280 len 2048 2024-09-16T00:34:32.148 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4193280 len 4096 2024-09-16T00:34:32.148 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4193280 len 4096 2024-09-16T00:34:32.149 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4193280 len 4096 2024-09-16T00:34:32.149 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4193280 len 4096 2024-09-16T00:34:32.149 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4193280 len 8192 2024-09-16T00:34:32.149 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4193280 len 8192 2024-09-16T00:34:32.149 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4193280 len 8192 2024-09-16T00:34:32.149 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4193280 len 8192 2024-09-16T00:34:32.149 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4193280 len 16384 2024-09-16T00:34:32.149 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4193280 len 16384 2024-09-16T00:34:32.149 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4193280 len 16384 2024-09-16T00:34:32.149 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4193280 len 16384 2024-09-16T00:34:32.149 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4193792 len 1024 2024-09-16T00:34:32.149 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4193792 len 1024 2024-09-16T00:34:32.149 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4193792 len 1024 2024-09-16T00:34:32.149 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4193792 len 1024 2024-09-16T00:34:32.150 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4193792 len 2048 2024-09-16T00:34:32.150 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4193792 len 2048 2024-09-16T00:34:32.150 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4193792 len 2048 2024-09-16T00:34:32.150 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4193792 len 2048 2024-09-16T00:34:32.150 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4193792 len 4096 2024-09-16T00:34:32.150 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4193792 len 4096 2024-09-16T00:34:32.150 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4193792 len 4096 2024-09-16T00:34:32.150 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4193792 len 4096 2024-09-16T00:34:32.150 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4193792 len 8192 2024-09-16T00:34:32.150 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4193792 len 8192 2024-09-16T00:34:32.150 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4193792 len 8192 2024-09-16T00:34:32.150 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4193792 len 8192 2024-09-16T00:34:32.150 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4193792 len 16384 2024-09-16T00:34:32.150 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4193792 len 16384 2024-09-16T00:34:32.151 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4193792 len 16384 2024-09-16T00:34:32.151 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4193792 len 16384 2024-09-16T00:34:32.151 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4194304 len 1024 2024-09-16T00:34:32.151 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4194304 len 1024 2024-09-16T00:34:32.151 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4194304 len 1024 2024-09-16T00:34:32.151 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4194304 len 1024 2024-09-16T00:34:32.151 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4194304 len 2048 2024-09-16T00:34:32.151 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4194304 len 2048 2024-09-16T00:34:32.151 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4194304 len 2048 2024-09-16T00:34:32.151 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4194304 len 2048 2024-09-16T00:34:32.151 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4194304 len 4096 2024-09-16T00:34:32.151 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4194304 len 4096 2024-09-16T00:34:32.151 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4194304 len 4096 2024-09-16T00:34:32.151 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4194304 len 4096 2024-09-16T00:34:32.152 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4194304 len 8192 2024-09-16T00:34:32.152 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4194304 len 8192 2024-09-16T00:34:32.152 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4194304 len 8192 2024-09-16T00:34:32.152 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4194304 len 8192 2024-09-16T00:34:32.152 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4194304 len 16384 2024-09-16T00:34:32.152 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4194304 len 16384 2024-09-16T00:34:32.152 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4194304 len 16384 2024-09-16T00:34:32.152 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4194304 len 16384 2024-09-16T00:34:32.153 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4194816 len 1024 2024-09-16T00:34:32.153 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4194816 len 1024 2024-09-16T00:34:32.153 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4194816 len 1024 2024-09-16T00:34:32.153 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4194816 len 1024 2024-09-16T00:34:32.153 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4194816 len 2048 2024-09-16T00:34:32.153 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4194816 len 2048 2024-09-16T00:34:32.153 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4194816 len 2048 2024-09-16T00:34:32.153 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4194816 len 2048 2024-09-16T00:34:32.153 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4194816 len 4096 2024-09-16T00:34:32.153 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4194816 len 4096 2024-09-16T00:34:32.154 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4194816 len 4096 2024-09-16T00:34:32.154 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4194816 len 4096 2024-09-16T00:34:32.154 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4194816 len 8192 2024-09-16T00:34:32.154 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4194816 len 8192 2024-09-16T00:34:32.154 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4194816 len 8192 2024-09-16T00:34:32.154 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4194816 len 8192 2024-09-16T00:34:32.154 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4194816 len 16384 2024-09-16T00:34:32.154 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4194816 len 16384 2024-09-16T00:34:32.154 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4194816 len 16384 2024-09-16T00:34:32.154 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4194816 len 16384 2024-09-16T00:34:32.154 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4195328 len 1024 2024-09-16T00:34:32.154 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4195328 len 1024 2024-09-16T00:34:32.154 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4195328 len 1024 2024-09-16T00:34:32.154 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4195328 len 1024 2024-09-16T00:34:32.154 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4195328 len 2048 2024-09-16T00:34:32.155 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4195328 len 2048 2024-09-16T00:34:32.155 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4195328 len 2048 2024-09-16T00:34:32.155 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4195328 len 2048 2024-09-16T00:34:32.155 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4195328 len 4096 2024-09-16T00:34:32.155 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4195328 len 4096 2024-09-16T00:34:32.155 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4195328 len 4096 2024-09-16T00:34:32.155 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4195328 len 4096 2024-09-16T00:34:32.157 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.144+0000 7f46cda45640 1 -- 172.21.15.38:0/164640573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46c8100e20 msgr2=0x7f46c8101200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:32.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.144+0000 7f46cda45640 1 --2- 172.21.15.38:0/164640573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46c8100e20 0x7f46c8101200 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f46b8009900 tx=0x7f46b802a480 comp rx=0 tx=0).stop 2024-09-16T00:34:32.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.144+0000 7f46cda45640 1 -- 172.21.15.38:0/164640573 shutdown_connections 2024-09-16T00:34:32.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.144+0000 7f46cda45640 1 --2- 172.21.15.38:0/164640573 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f46c8102ef0 0x7f46c810da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:32.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.144+0000 7f46cda45640 1 --2- 172.21.15.38:0/164640573 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46c8102570 0x7f46c81029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:32.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.144+0000 7f46cda45640 1 --2- 172.21.15.38:0/164640573 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46c8100e20 0x7f46c8101200 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:32.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.144+0000 7f46cda45640 1 -- 172.21.15.38:0/164640573 >> 172.21.15.38:0/164640573 conn(0x7f46c80fc7d0 msgr2=0x7f46c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:32.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.144+0000 7f46cda45640 1 -- 172.21.15.38:0/164640573 shutdown_connections 2024-09-16T00:34:32.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.144+0000 7f46cda45640 1 -- 172.21.15.38:0/164640573 wait complete. 2024-09-16T00:34:32.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.144+0000 7f46cda45640 1 Processor -- start 2024-09-16T00:34:32.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.145+0000 7f46cda45640 1 -- start start 2024-09-16T00:34:32.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.145+0000 7f46cda45640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f46c8100e20 0x7f46c8198490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:32.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.145+0000 7f46cda45640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46c8102570 0x7f46c81989d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:32.158 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.145+0000 7f46cda45640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46c8102ef0 0x7f46c819cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:32.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.145+0000 7f46cda45640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f46c8112030 con 0x7f46c8102570 2024-09-16T00:34:32.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.145+0000 7f46cda45640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f46c8111eb0 con 0x7f46c8102ef0 2024-09-16T00:34:32.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.145+0000 7f46cda45640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f46c81121b0 con 0x7f46c8100e20 2024-09-16T00:34:32.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.145+0000 7f46c67fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46c8102570 0x7f46c81989d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:32.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.145+0000 7f46c6ffd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f46c8100e20 0x7f46c8198490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:32.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.145+0000 7f46c67fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46c8102570 0x7f46c81989d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51274/0 (socket says 172.21.15.38:51274) 2024-09-16T00:34:32.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.145+0000 7f46c67fc640 1 -- 172.21.15.38:0/1400130706 learned_addr learned my addr 172.21.15.38:0/1400130706 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:34:32.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.145+0000 7f46c77fe640 1 --2- 172.21.15.38:0/1400130706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46c8102ef0 0x7f46c819cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:32.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.145+0000 7f46c67fc640 1 -- 172.21.15.38:0/1400130706 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f46c8100e20 msgr2=0x7f46c8198490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:32.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.145+0000 7f46c67fc640 1 --2- 172.21.15.38:0/1400130706 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f46c8100e20 0x7f46c8198490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:32.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.145+0000 7f46c67fc640 1 -- 172.21.15.38:0/1400130706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46c8102ef0 msgr2=0x7f46c819cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:32.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.145+0000 7f46c67fc640 1 --2- 172.21.15.38:0/1400130706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46c8102ef0 0x7f46c819cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:32.159 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.145+0000 7f46c67fc640 1 -- 172.21.15.38:0/1400130706 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46c819d440 con 0x7f46c8102570 2024-09-16T00:34:32.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.145+0000 7f46c6ffd640 1 --2- 172.21.15.38:0/1400130706 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f46c8100e20 0x7f46c8198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:34:32.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.146+0000 7f46c67fc640 1 --2- 172.21.15.38:0/1400130706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46c8102570 0x7f46c81989d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00e9f0 tx=0x7f46bc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:32.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.146+0000 7f46cca43640 1 -- 172.21.15.38:0/1400130706 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc00cd80 con 0x7f46c8102570 2024-09-16T00:34:32.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.146+0000 7f46cca43640 1 -- 172.21.15.38:0/1400130706 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46bc010910 con 0x7f46c8102570 2024-09-16T00:34:32.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.146+0000 7f46cda45640 1 -- 172.21.15.38:0/1400130706 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46c819d730 con 0x7f46c8102570 2024-09-16T00:34:32.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.146+0000 7f46cca43640 1 -- 172.21.15.38:0/1400130706 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc018990 con 0x7f46c8102570 2024-09-16T00:34:32.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.146+0000 7f46cda45640 1 -- 172.21.15.38:0/1400130706 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46c81a5010 con 0x7f46c8102570 2024-09-16T00:34:32.160 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.148+0000 7f46cca43640 1 -- 172.21.15.38:0/1400130706 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f46bc010ac0 con 0x7f46c8102570 2024-09-16T00:34:32.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.148+0000 7f46cfe1e740 1 Processor -- start 2024-09-16T00:34:32.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.149+0000 7f46cfe1e740 1 -- start start 2024-09-16T00:34:32.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.149+0000 7f46cfe1e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557c17138b40 0x557c17137090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:32.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.149+0000 7f46cfe1e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c171375d0 0x557c17137a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:32.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.149+0000 7f46cfe1e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557c17142370 0x557c17144760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:32.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.149+0000 7f46cfe1e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557c1709d2f0 con 0x557c171375d0 2024-09-16T00:34:32.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.149+0000 7f46cfe1e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557c17044cf0 con 0x557c17142370 2024-09-16T00:34:32.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.149+0000 7f46cfe1e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557c170992b0 con 0x557c17138b40 2024-09-16T00:34:32.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.149+0000 7f46c6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c171375d0 0x557c17137a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:32.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.149+0000 7f46c77fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557c17138b40 0x557c17137090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:32.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.149+0000 7f46c6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c171375d0 0x557c17137a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51284/0 (socket says 172.21.15.38:51284) 2024-09-16T00:34:32.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.149+0000 7f46c6ffd640 1 -- 172.21.15.38:0/3748717999 learned_addr learned my addr 172.21.15.38:0/3748717999 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:34:32.161 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.149+0000 7f46c67fc640 1 --2- 172.21.15.38:0/3748717999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557c17142370 0x557c17144760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:32.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.149+0000 7f46cca43640 1 --2- 172.21.15.38:0/1400130706 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f46b00a3240 0x7f46b00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:32.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.149+0000 7f46c6ffd640 1 -- 172.21.15.38:0/3748717999 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557c17138b40 msgr2=0x557c17137090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:32.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.149+0000 7f46c6ffd640 1 --2- 172.21.15.38:0/3748717999 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557c17138b40 0x557c17137090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:32.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.149+0000 7f46c6ffd640 1 -- 172.21.15.38:0/3748717999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557c17142370 msgr2=0x557c17144760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:32.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.149+0000 7f46c6ffd640 1 --2- 172.21.15.38:0/3748717999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557c17142370 0x557c17144760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:32.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.149+0000 7f46c6ffd640 1 -- 172.21.15.38:0/3748717999 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c170293a0 con 0x557c171375d0 2024-09-16T00:34:32.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.149+0000 7f46c77fe640 1 --2- 172.21.15.38:0/3748717999 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557c17138b40 0x557c17137090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:34:32.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.149+0000 7f46c77fe640 1 --2- 172.21.15.38:0/1400130706 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f46b00a3240 0x7f46b00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:32.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.149+0000 7f46cca43640 1 -- 172.21.15.38:0/1400130706 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f46bc0cffa0 con 0x7f46c8102570 2024-09-16T00:34:32.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.150+0000 7f46c6ffd640 1 --2- 172.21.15.38:0/3748717999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c171375d0 0x557c17137a10 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f46b802a9a0 tx=0x7f46b8005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bcd512ef9e5d8f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:32.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.150+0000 7f46adffb640 1 -- 172.21.15.38:0/3748717999 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b8035030 con 0x557c171375d0 2024-09-16T00:34:32.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.150+0000 7f46adffb640 1 -- 172.21.15.38:0/3748717999 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46b8040070 con 0x557c171375d0 2024-09-16T00:34:32.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.150+0000 7f46adffb640 1 -- 172.21.15.38:0/3748717999 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b8031bc0 con 0x557c171375d0 2024-09-16T00:34:32.162 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.151+0000 7f46cfe1e740 1 -- 172.21.15.38:0/3748717999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c171375d0 msgr2=0x557c17137a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:32.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.151+0000 7f46c77fe640 1 --2- 172.21.15.38:0/1400130706 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f46b00a3240 0x7f46b00a5700 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f46c0005fd0 tx=0x7f46c0004a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:32.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.151+0000 7f46cfe1e740 1 --2- 172.21.15.38:0/3748717999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c171375d0 0x557c17137a10 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f46b802a9a0 tx=0x7f46b8005dd0 comp rx=0 tx=0).stop 2024-09-16T00:34:32.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.151+0000 7f46cfe1e740 1 -- 172.21.15.38:0/3748717999 shutdown_connections 2024-09-16T00:34:32.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.151+0000 7f46cfe1e740 1 --2- 172.21.15.38:0/3748717999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557c17142370 0x557c17144760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:32.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.151+0000 7f46cfe1e740 1 --2- 172.21.15.38:0/3748717999 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c171375d0 0x557c17137a10 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:32.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.151+0000 7f46cfe1e740 1 --2- 172.21.15.38:0/3748717999 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557c17138b40 0x557c17137090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:32.163 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.151+0000 7f46cfe1e740 1 -- 172.21.15.38:0/3748717999 >> 172.21.15.38:0/3748717999 conn(0x557c17100310 msgr2=0x557c17133c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:32.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.151+0000 7f46cfe1e740 1 -- 172.21.15.38:0/3748717999 shutdown_connections 2024-09-16T00:34:32.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.151+0000 7f46cfe1e740 1 -- 172.21.15.38:0/3748717999 wait complete. 2024-09-16T00:34:32.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.151+0000 7f46cfe1e740 1 -- start start 2024-09-16T00:34:32.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.151+0000 7f46cfe1e740 1 Processor -- start 2024-09-16T00:34:32.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.152+0000 7f46cfe1e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557c171375d0 0x557c17141d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:32.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.152+0000 7f46cfe1e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c17138b40 0x557c1713eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:32.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.152+0000 7f46cfe1e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557c17142370 0x557c1713f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:32.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.152+0000 7f46cfe1e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557c17145010 con 0x557c17138b40 2024-09-16T00:34:32.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.152+0000 7f46cfe1e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557c17058060 con 0x557c171375d0 2024-09-16T00:34:32.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.152+0000 7f46cfe1e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x557c17145190 con 0x557c17142370 2024-09-16T00:34:32.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.152+0000 7f46cfe1e740 10 client.0 _ll_register_callbacks cb 0x557c16dab8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:32.164 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.152+0000 7f46c67fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c17138b40 0x557c1713eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:32.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.152+0000 7f46c67fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c17138b40 0x557c1713eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51292/0 (socket says 172.21.15.38:51292) 2024-09-16T00:34:32.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.152+0000 7f46c67fc640 1 -- 172.21.15.38:0/2540332412 learned_addr learned my addr 172.21.15.38:0/2540332412 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:34:32.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.152+0000 7f46c77fe640 1 --2- 172.21.15.38:0/2540332412 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557c17142370 0x557c1713f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:32.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.152+0000 7f46c6ffd640 1 --2- 172.21.15.38:0/2540332412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557c171375d0 0x557c17141d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:32.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.152+0000 7f46c67fc640 1 -- 172.21.15.38:0/2540332412 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557c17142370 msgr2=0x557c1713f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:32.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.152+0000 7f46c67fc640 1 --2- 172.21.15.38:0/2540332412 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557c17142370 0x557c1713f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:32.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.152+0000 7f46c67fc640 1 -- 172.21.15.38:0/2540332412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557c171375d0 msgr2=0x557c17141d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:32.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.152+0000 7f46c67fc640 1 --2- 172.21.15.38:0/2540332412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557c171375d0 0x557c17141d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:32.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.152+0000 7f46c67fc640 1 -- 172.21.15.38:0/2540332412 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c16e0e3c0 con 0x557c17138b40 2024-09-16T00:34:32.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.152+0000 7f46c6ffd640 1 --2- 172.21.15.38:0/2540332412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557c171375d0 0x557c17141d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:34:32.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.152+0000 7f46c67fc640 1 --2- 172.21.15.38:0/2540332412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c17138b40 0x557c1713eff0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f46bc0cda90 tx=0x7f46bc098690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ca7a993b43e303d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:32.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.152+0000 7f46a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:34:32.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.153+0000 7f46a6ffd640 1 -- 172.21.15.38:0/2540332412 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc09c2f0 con 0x557c17138b40 2024-09-16T00:34:32.165 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.153+0000 7f46a6ffd640 1 -- 172.21.15.38:0/2540332412 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f46bc0d20f0 con 0x557c17138b40 2024-09-16T00:34:32.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.153+0000 7f46cfe1e740 1 -- 172.21.15.38:0/2540332412 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46b8040070 con 0x557c17138b40 2024-09-16T00:34:32.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.153+0000 7f46a6ffd640 1 -- 172.21.15.38:0/2540332412 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f46bc0a05f0 con 0x557c17138b40 2024-09-16T00:34:32.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.153+0000 7f46cfe1e740 10 client.15747 fetch_fsmap learned FSMap version 26 2024-09-16T00:34:32.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.153+0000 7f46cfe1e740 1 -- 172.21.15.38:0/2540332412 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557c17150ad0 con 0x557c17138b40 2024-09-16T00:34:32.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.153+0000 7f46a6ffd640 1 -- 172.21.15.38:0/2540332412 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46b8040070 con 0x557c17138b40 2024-09-16T00:34:32.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.153+0000 7f46a6ffd640 1 -- 172.21.15.38:0/2540332412 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f46bc0cdc70 con 0x557c17138b40 2024-09-16T00:34:32.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.153+0000 7f46cfe1e740 10 client.15747 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:34:32.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.153+0000 7f46cfe1e740 10 client.15747 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:34:32.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.153+0000 7f46cfe1e740 20 client.15747 populate_metadata read hostname 'smithi038' 2024-09-16T00:34:32.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.153+0000 7f46cfe1e740 1 --2- 172.21.15.38:0/2540332412 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x557c17151ee0 0x557c17154330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:32.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.153+0000 7f46cfe1e740 4 client.15747 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:34:32.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.153+0000 7f46cfe1e740 1 -- 172.21.15.38:0/2540332412 --> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557c170aa860 con 0x557c17151ee0 2024-09-16T00:34:32.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.154+0000 7f46c6ffd640 1 --2- 172.21.15.38:0/2540332412 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x557c17151ee0 0x557c17154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:32.166 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.154+0000 7f46c6ffd640 1 --2- 172.21.15.38:0/2540332412 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x557c17151ee0 0x557c17154330 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=96e481500c3bb772 server_cookie=9119cdd0a5c99026 in_seq=0 out_seq=0 2024-09-16T00:34:32.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.154+0000 7f46a6ffd640 10 client.15747 ms_handle_connect on v2:172.21.15.73:6836/2681081265 2024-09-16T00:34:32.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.155+0000 7f46a6ffd640 1 -- 172.21.15.38:0/2540332412 <== mds.1 v2:172.21.15.73:6836/2681081265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x557c170aa860 con 0x557c17151ee0 2024-09-16T00:34:32.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.155+0000 7f46a6ffd640 10 client.15747 handle_command_reply: tid=0 2024-09-16T00:34:32.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.155+0000 7f46cfe1e740 1 client.15747 shutdown 2024-09-16T00:34:32.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.155+0000 7f46cfe1e740 1 -- 172.21.15.38:0/2540332412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c17138b40 msgr2=0x557c1713eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:32.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.155+0000 7f46cfe1e740 1 --2- 172.21.15.38:0/2540332412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c17138b40 0x557c1713eff0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f46bc0cda90 tx=0x7f46bc098690 comp rx=0 tx=0).stop 2024-09-16T00:34:32.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.155+0000 7f46cfe1e740 1 -- 172.21.15.38:0/2540332412 shutdown_connections 2024-09-16T00:34:32.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.155+0000 7f46cfe1e740 1 --2- 172.21.15.38:0/2540332412 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557c17142370 0x557c1713f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:32.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.155+0000 7f46cfe1e740 1 --2- 172.21.15.38:0/2540332412 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x557c17151ee0 0x557c17154330 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:32.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.155+0000 7f46cfe1e740 1 --2- 172.21.15.38:0/2540332412 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c17138b40 0x557c1713eff0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:32.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.155+0000 7f46cfe1e740 1 --2- 172.21.15.38:0/2540332412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557c171375d0 0x557c17141d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:32.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.155+0000 7f46cfe1e740 1 -- 172.21.15.38:0/2540332412 >> 172.21.15.38:0/2540332412 conn(0x557c17100310 msgr2=0x557c17139960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:32.167 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.155+0000 7f46a6ffd640 0 client.15747 ms_handle_reset on v2:172.21.15.73:6836/2681081265 2024-09-16T00:34:32.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.155+0000 7f46cfe1e740 1 -- 172.21.15.38:0/2540332412 shutdown_connections 2024-09-16T00:34:32.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.155+0000 7f46cfe1e740 1 -- 172.21.15.38:0/2540332412 wait complete. 2024-09-16T00:34:32.168 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.155+0000 7f46cfe1e740 20 client.15747 trim_cache size 0 max 16384 2024-09-16T00:34:32.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.169+0000 7f46cfe1e740 1 Processor -- start 2024-09-16T00:34:32.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.169+0000 7f46cfe1e740 1 -- start start 2024-09-16T00:34:32.171 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.169+0000 7f46cfe1e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557c171375d0 0x557c1714e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:32.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.169+0000 7f46cfe1e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557c17138b40 0x557c1713a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:32.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.169+0000 7f46cfe1e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c17142370 0x557c1713a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:32.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.169+0000 7f46cfe1e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f46c81b1420 con 0x557c17142370 2024-09-16T00:34:32.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.169+0000 7f46cfe1e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f46c81aecf0 con 0x557c171375d0 2024-09-16T00:34:32.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.169+0000 7f46cfe1e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f46c81acb40 con 0x557c17138b40 2024-09-16T00:34:32.172 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.169+0000 7f46c77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c17142370 0x557c1713a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:32.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.169+0000 7f46c77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c17142370 0x557c1713a660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51296/0 (socket says 172.21.15.38:51296) 2024-09-16T00:34:32.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.169+0000 7f46c67fc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557c17138b40 0x557c1713a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:32.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.169+0000 7f46c77fe640 1 -- 172.21.15.38:0/2304832745 learned_addr learned my addr 172.21.15.38:0/2304832745 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:34:32.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.169+0000 7f46c6ffd640 1 --2- 172.21.15.38:0/2304832745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557c171375d0 0x557c1714e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:32.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.170+0000 7f46c67fc640 1 -- 172.21.15.38:0/2304832745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557c171375d0 msgr2=0x557c1714e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:32.173 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.170+0000 7f46c67fc640 1 --2- 172.21.15.38:0/2304832745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557c171375d0 0x557c1714e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:32.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.170+0000 7f46c67fc640 1 -- 172.21.15.38:0/2304832745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c17142370 msgr2=0x557c1713a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:32.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.170+0000 7f46c67fc640 1 --2- 172.21.15.38:0/2304832745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c17142370 0x557c1713a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:32.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.170+0000 7f46c67fc640 1 -- 172.21.15.38:0/2304832745 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c17141960 con 0x557c17138b40 2024-09-16T00:34:32.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.170+0000 7f46c67fc640 1 --2- 172.21.15.38:0/2304832745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557c17138b40 0x557c1713a120 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f46bc09bff0 tx=0x7f46bc09a800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2aa7024aca737815 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:32.174 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.170+0000 7f46ad7fa640 1 -- 172.21.15.38:0/2304832745 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc09be30 con 0x557c17138b40 2024-09-16T00:34:32.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.170+0000 7f46ad7fa640 1 -- 172.21.15.38:0/2304832745 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46bc0a4eb0 con 0x557c17138b40 2024-09-16T00:34:32.175 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.170+0000 7f46ad7fa640 1 -- 172.21.15.38:0/2304832745 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46bc0d3160 con 0x557c17138b40 2024-09-16T00:34:32.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.171+0000 7f46cfe1e740 1 -- 172.21.15.38:0/2304832745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557c17138b40 msgr2=0x557c1713a120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:32.181 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.171+0000 7f46cfe1e740 1 --2- 172.21.15.38:0/2304832745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557c17138b40 0x557c1713a120 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f46bc09bff0 tx=0x7f46bc09a800 comp rx=0 tx=0).stop 2024-09-16T00:34:32.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.171+0000 7f46cfe1e740 1 -- 172.21.15.38:0/2304832745 shutdown_connections 2024-09-16T00:34:32.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.171+0000 7f46cfe1e740 1 --2- 172.21.15.38:0/2304832745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c17142370 0x557c1713a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:32.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.171+0000 7f46cfe1e740 1 --2- 172.21.15.38:0/2304832745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557c17138b40 0x557c1713a120 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:32.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.171+0000 7f46cfe1e740 1 --2- 172.21.15.38:0/2304832745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557c171375d0 0x557c1714e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:32.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.171+0000 7f46cfe1e740 1 -- 172.21.15.38:0/2304832745 >> 172.21.15.38:0/2304832745 conn(0x557c17100310 msgr2=0x557c1713e100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:32.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.171+0000 7f46cfe1e740 1 -- 172.21.15.38:0/2304832745 shutdown_connections 2024-09-16T00:34:32.182 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.171+0000 7f46cfe1e740 1 -- 172.21.15.38:0/2304832745 wait complete. 2024-09-16T00:34:32.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.171+0000 7f46cfe1e740 1 -- start start 2024-09-16T00:34:32.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.171+0000 7f46cfe1e740 1 Processor -- start 2024-09-16T00:34:32.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.172+0000 7f46cfe1e740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c171375d0 0x557c1714ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:32.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.172+0000 7f46cfe1e740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557c17138b40 0x557c1713bd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:32.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.172+0000 7f46cfe1e740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557c17142370 0x557c1713c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:32.183 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.172+0000 7f46cfe1e740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f46c81ad1b0 con 0x557c171375d0 2024-09-16T00:34:32.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.172+0000 7f46cfe1e740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f46c81ae620 con 0x557c17138b40 2024-09-16T00:34:32.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.172+0000 7f46cfe1e740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f46c81ad880 con 0x557c17142370 2024-09-16T00:34:32.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.172+0000 7f46cfe1e740 10 client.0 _ll_register_callbacks cb 0x7f46bc009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:32.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.172+0000 7f46c6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c171375d0 0x557c1714ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:32.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.172+0000 7f46c67fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557c17138b40 0x557c1713bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:32.184 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.172+0000 7f46c6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c171375d0 0x557c1714ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51304/0 (socket says 172.21.15.38:51304) 2024-09-16T00:34:32.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.172+0000 7f46c6ffd640 1 -- 172.21.15.38:0/4013575157 learned_addr learned my addr 172.21.15.38:0/4013575157 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:34:32.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.172+0000 7f46c77fe640 1 --2- 172.21.15.38:0/4013575157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557c17142370 0x557c1713c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:32.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.172+0000 7f46c6ffd640 1 -- 172.21.15.38:0/4013575157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557c17142370 msgr2=0x557c1713c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:32.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.172+0000 7f46c6ffd640 1 --2- 172.21.15.38:0/4013575157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557c17142370 0x557c1713c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:32.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.172+0000 7f46c6ffd640 1 -- 172.21.15.38:0/4013575157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557c17138b40 msgr2=0x557c1713bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:32.185 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.172+0000 7f46c6ffd640 1 --2- 172.21.15.38:0/4013575157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557c17138b40 0x557c1713bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:32.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.172+0000 7f46c6ffd640 1 -- 172.21.15.38:0/4013575157 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c1713caa0 con 0x557c171375d0 2024-09-16T00:34:32.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.172+0000 7f46c67fc640 1 --2- 172.21.15.38:0/4013575157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557c17138b40 0x557c1713bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:34:32.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.172+0000 7f46c6ffd640 1 --2- 172.21.15.38:0/4013575157 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c171375d0 0x557c1714ab30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f46b80314e0 tx=0x7f46b80317a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7017a3867fbaadd server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:32.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.172+0000 7f46a6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:34:32.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.173+0000 7f46a6ffd640 1 -- 172.21.15.38:0/4013575157 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46b8035070 con 0x557c171375d0 2024-09-16T00:34:32.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.173+0000 7f46a6ffd640 1 -- 172.21.15.38:0/4013575157 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f46b8031ce0 con 0x557c171375d0 2024-09-16T00:34:32.186 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.173+0000 7f46cfe1e740 1 -- 172.21.15.38:0/4013575157 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f46bc0a4eb0 con 0x557c171375d0 2024-09-16T00:34:32.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.173+0000 7f46a6ffd640 1 -- 172.21.15.38:0/4013575157 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f46b8060050 con 0x557c171375d0 2024-09-16T00:34:32.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.173+0000 7f46cfe1e740 10 client.15753 fetch_fsmap learned FSMap version 26 2024-09-16T00:34:32.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.173+0000 7f46cfe1e740 1 -- 172.21.15.38:0/4013575157 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x557c171bcc70 con 0x557c171375d0 2024-09-16T00:34:32.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.173+0000 7f46a6ffd640 1 -- 172.21.15.38:0/4013575157 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f46bc0a4eb0 con 0x557c171375d0 2024-09-16T00:34:32.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.173+0000 7f46a6ffd640 1 -- 172.21.15.38:0/4013575157 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f46b80514b0 con 0x557c171375d0 2024-09-16T00:34:32.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.173+0000 7f46cfe1e740 10 client.15753 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:34:32.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.173+0000 7f46cfe1e740 10 client.15753 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:34:32.187 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.173+0000 7f46cfe1e740 20 client.15753 populate_metadata read hostname 'smithi038' 2024-09-16T00:34:32.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.173+0000 7f46cfe1e740 1 --2- 172.21.15.38:0/4013575157 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x557c171bd070 0x557c171bf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:32.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.173+0000 7f46cfe1e740 4 client.15753 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:34:32.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.173+0000 7f46cfe1e740 1 -- 172.21.15.38:0/4013575157 --> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557c170a4d70 con 0x557c171bd070 2024-09-16T00:34:32.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.174+0000 7f46c67fc640 1 --2- 172.21.15.38:0/4013575157 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x557c171bd070 0x557c171bf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:32.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.174+0000 7f46c67fc640 1 --2- 172.21.15.38:0/4013575157 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x557c171bd070 0x557c171bf460 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a389e732d72e6109 server_cookie=655979e5953f5d0b in_seq=0 out_seq=0 2024-09-16T00:34:32.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.174+0000 7f46a6ffd640 10 client.15753 ms_handle_connect on v2:172.21.15.73:6836/2681081265 2024-09-16T00:34:32.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.174+0000 7f46a6ffd640 1 -- 172.21.15.38:0/4013575157 <== mds.1 v2:172.21.15.73:6836/2681081265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557c170a4d70 con 0x557c171bd070 2024-09-16T00:34:32.188 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.174+0000 7f46a6ffd640 10 client.15753 handle_command_reply: tid=0 2024-09-16T00:34:32.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.174+0000 7f46cfe1e740 1 client.15753 shutdown 2024-09-16T00:34:32.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.174+0000 7f46cfe1e740 1 -- 172.21.15.38:0/4013575157 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c171375d0 msgr2=0x557c1714ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:32.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.174+0000 7f46cfe1e740 1 --2- 172.21.15.38:0/4013575157 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c171375d0 0x557c1714ab30 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f46b80314e0 tx=0x7f46b80317a0 comp rx=0 tx=0).stop 2024-09-16T00:34:32.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.174+0000 7f46cfe1e740 1 -- 172.21.15.38:0/4013575157 shutdown_connections 2024-09-16T00:34:32.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.174+0000 7f46cfe1e740 1 --2- 172.21.15.38:0/4013575157 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557c17142370 0x557c1713c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:32.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.174+0000 7f46cfe1e740 1 --2- 172.21.15.38:0/4013575157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557c17138b40 0x557c1713bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:32.189 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.174+0000 7f46cfe1e740 1 --2- 172.21.15.38:0/4013575157 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x557c171bd070 0x557c171bf460 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:32.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.174+0000 7f46cfe1e740 1 --2- 172.21.15.38:0/4013575157 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557c171375d0 0x557c1714ab30 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:32.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.174+0000 7f46a6ffd640 0 client.15753 ms_handle_reset on v2:172.21.15.73:6836/2681081265 2024-09-16T00:34:32.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.174+0000 7f46cfe1e740 1 -- 172.21.15.38:0/4013575157 >> 172.21.15.38:0/4013575157 conn(0x557c17100310 msgr2=0x557c17140850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:32.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.175+0000 7f46cfe1e740 1 -- 172.21.15.38:0/4013575157 shutdown_connections 2024-09-16T00:34:32.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.175+0000 7f46cfe1e740 1 -- 172.21.15.38:0/4013575157 wait complete. 2024-09-16T00:34:32.190 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.175+0000 7f46cfe1e740 20 client.15753 trim_cache size 0 max 16384 2024-09-16T00:34:32.191 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-09-16T00:34:32.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.175+0000 7f46cda45640 1 -- 172.21.15.38:0/1400130706 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f46b00a3240 msgr2=0x7f46b00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:32.192 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.175+0000 7f46cda45640 1 --2- 172.21.15.38:0/1400130706 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f46b00a3240 0x7f46b00a5700 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f46c0005fd0 tx=0x7f46c0004a40 comp rx=0 tx=0).stop 2024-09-16T00:34:32.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.175+0000 7f46cda45640 1 -- 172.21.15.38:0/1400130706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46c8102570 msgr2=0x7f46c81989d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:32.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.175+0000 7f46cda45640 1 --2- 172.21.15.38:0/1400130706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46c8102570 0x7f46c81989d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f46bc00e9f0 tx=0x7f46bc00eeb0 comp rx=0 tx=0).stop 2024-09-16T00:34:32.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.175+0000 7f46cda45640 1 -- 172.21.15.38:0/1400130706 shutdown_connections 2024-09-16T00:34:32.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.175+0000 7f46cda45640 1 --2- 172.21.15.38:0/1400130706 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f46b00a3240 0x7f46b00a5700 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:32.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.175+0000 7f46cda45640 1 --2- 172.21.15.38:0/1400130706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46c8102ef0 0x7f46c819cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:32.193 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.175+0000 7f46cda45640 1 --2- 172.21.15.38:0/1400130706 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f46c8102570 0x7f46c81989d0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:32.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.175+0000 7f46cda45640 1 --2- 172.21.15.38:0/1400130706 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f46c8100e20 0x7f46c8198490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:32.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.175+0000 7f46cda45640 1 -- 172.21.15.38:0/1400130706 >> 172.21.15.38:0/1400130706 conn(0x7f46c80fc7d0 msgr2=0x7f46c80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:32.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.175+0000 7f46cda45640 1 -- 172.21.15.38:0/1400130706 shutdown_connections 2024-09-16T00:34:32.194 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:32.176+0000 7f46cda45640 1 -- 172.21.15.38:0/1400130706 wait complete. 2024-09-16T00:34:32.641 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4195328 len 8192 2024-09-16T00:34:32.641 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4195328 len 8192 2024-09-16T00:34:32.642 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4195328 len 8192 2024-09-16T00:34:32.642 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4195328 len 8192 2024-09-16T00:34:32.642 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4195328 len 16384 2024-09-16T00:34:32.642 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4195328 len 16384 2024-09-16T00:34:32.642 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4195328 len 16384 2024-09-16T00:34:32.642 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4195328 len 16384 2024-09-16T00:34:32.642 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4195840 len 1024 2024-09-16T00:34:32.642 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4195840 len 1024 2024-09-16T00:34:32.642 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4195840 len 1024 2024-09-16T00:34:32.642 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4195840 len 1024 2024-09-16T00:34:32.642 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4195840 len 2048 2024-09-16T00:34:32.642 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4195840 len 2048 2024-09-16T00:34:32.642 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4195840 len 2048 2024-09-16T00:34:32.642 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4195840 len 2048 2024-09-16T00:34:32.643 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4195840 len 4096 2024-09-16T00:34:32.643 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4195840 len 4096 2024-09-16T00:34:32.643 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4195840 len 4096 2024-09-16T00:34:32.643 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4195840 len 4096 2024-09-16T00:34:32.643 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4195840 len 8192 2024-09-16T00:34:32.643 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4195840 len 8192 2024-09-16T00:34:32.643 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4195840 len 8192 2024-09-16T00:34:32.643 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4195840 len 8192 2024-09-16T00:34:32.643 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4195840 len 16384 2024-09-16T00:34:32.643 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4195840 len 16384 2024-09-16T00:34:32.643 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4195840 len 16384 2024-09-16T00:34:32.643 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4195840 len 16384 2024-09-16T00:34:32.643 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4196352 len 1024 2024-09-16T00:34:32.643 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4196352 len 1024 2024-09-16T00:34:32.644 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4196352 len 1024 2024-09-16T00:34:32.644 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4196352 len 1024 2024-09-16T00:34:32.644 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4196352 len 2048 2024-09-16T00:34:32.644 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4196352 len 2048 2024-09-16T00:34:32.644 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4196352 len 2048 2024-09-16T00:34:32.644 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4196352 len 2048 2024-09-16T00:34:32.644 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4196352 len 4096 2024-09-16T00:34:32.644 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4196352 len 4096 2024-09-16T00:34:32.644 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4196352 len 4096 2024-09-16T00:34:32.644 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4196352 len 4096 2024-09-16T00:34:32.644 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4196352 len 8192 2024-09-16T00:34:32.644 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4196352 len 8192 2024-09-16T00:34:32.644 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4196352 len 8192 2024-09-16T00:34:32.645 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4196352 len 8192 2024-09-16T00:34:32.645 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4196352 len 16384 2024-09-16T00:34:32.645 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4196352 len 16384 2024-09-16T00:34:32.645 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4196352 len 16384 2024-09-16T00:34:32.645 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4196352 len 16384 2024-09-16T00:34:32.645 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4196864 len 1024 2024-09-16T00:34:32.645 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4196864 len 1024 2024-09-16T00:34:32.645 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4196864 len 1024 2024-09-16T00:34:32.645 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4196864 len 1024 2024-09-16T00:34:32.645 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4196864 len 2048 2024-09-16T00:34:32.645 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4196864 len 2048 2024-09-16T00:34:32.645 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4196864 len 2048 2024-09-16T00:34:32.645 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4196864 len 2048 2024-09-16T00:34:32.645 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4196864 len 4096 2024-09-16T00:34:32.646 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4196864 len 4096 2024-09-16T00:34:32.646 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4196864 len 4096 2024-09-16T00:34:32.646 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4196864 len 4096 2024-09-16T00:34:32.646 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4196864 len 8192 2024-09-16T00:34:32.646 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4196864 len 8192 2024-09-16T00:34:32.646 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4196864 len 8192 2024-09-16T00:34:32.646 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4196864 len 8192 2024-09-16T00:34:32.646 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4196864 len 16384 2024-09-16T00:34:32.646 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4196864 len 16384 2024-09-16T00:34:32.646 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4196864 len 16384 2024-09-16T00:34:32.646 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4196864 len 16384 2024-09-16T00:34:32.646 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4197376 len 1024 2024-09-16T00:34:32.646 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4197376 len 1024 2024-09-16T00:34:32.646 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4197376 len 1024 2024-09-16T00:34:32.647 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4197376 len 1024 2024-09-16T00:34:32.647 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4197376 len 2048 2024-09-16T00:34:32.647 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4197376 len 2048 2024-09-16T00:34:32.647 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4197376 len 2048 2024-09-16T00:34:32.647 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4197376 len 2048 2024-09-16T00:34:32.647 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4197376 len 4096 2024-09-16T00:34:32.647 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4197376 len 4096 2024-09-16T00:34:32.647 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4197376 len 4096 2024-09-16T00:34:32.647 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4197376 len 4096 2024-09-16T00:34:32.647 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4197376 len 8192 2024-09-16T00:34:32.647 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4197376 len 8192 2024-09-16T00:34:32.647 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4197376 len 8192 2024-09-16T00:34:32.647 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4197376 len 8192 2024-09-16T00:34:32.647 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4197376 len 16384 2024-09-16T00:34:32.648 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4197376 len 16384 2024-09-16T00:34:33.110 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:34:33.426 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:34:33.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:33 smithi038 ceph-mon[30945]: pgmap v417: 129 pgs: 129 active+clean; 53 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 355 KiB/s wr, 102 op/s 2024-09-16T00:34:33.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:33 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:34:33.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:33 smithi122 ceph-mon[35412]: pgmap v417: 129 pgs: 129 active+clean; 53 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 355 KiB/s wr, 102 op/s 2024-09-16T00:34:33.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:33 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:34:33.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:33 smithi073 ceph-mon[35254]: pgmap v417: 129 pgs: 129 active+clean; 53 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 MiB/s rd, 355 KiB/s wr, 102 op/s 2024-09-16T00:34:33.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:33 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:34:33.663 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4197376 len 16384 2024-09-16T00:34:33.663 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4197376 len 16384 2024-09-16T00:34:33.663 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4197888 len 1024 2024-09-16T00:34:33.663 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4197888 len 1024 2024-09-16T00:34:33.663 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4197888 len 1024 2024-09-16T00:34:33.663 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4197888 len 1024 2024-09-16T00:34:33.663 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4197888 len 2048 2024-09-16T00:34:33.664 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4197888 len 2048 2024-09-16T00:34:33.664 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4197888 len 2048 2024-09-16T00:34:33.664 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4197888 len 2048 2024-09-16T00:34:33.664 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4197888 len 4096 2024-09-16T00:34:33.664 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4197888 len 4096 2024-09-16T00:34:33.664 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4197888 len 4096 2024-09-16T00:34:33.664 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4197888 len 4096 2024-09-16T00:34:33.664 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4197888 len 8192 2024-09-16T00:34:33.664 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4197888 len 8192 2024-09-16T00:34:33.665 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4197888 len 8192 2024-09-16T00:34:33.665 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4197888 len 8192 2024-09-16T00:34:33.665 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 0 offset 4197888 len 16384 2024-09-16T00:34:33.665 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4197888 len 16384 2024-09-16T00:34:33.665 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 0 offset 4197888 len 16384 2024-09-16T00:34:33.665 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 0 offset 4197888 len 16384 2024-09-16T00:34:33.665 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4190720 len 1024 2024-09-16T00:34:33.665 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4190720 len 1024 2024-09-16T00:34:33.666 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4190720 len 1024 2024-09-16T00:34:33.666 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4190720 len 1024 2024-09-16T00:34:33.666 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4190720 len 2048 2024-09-16T00:34:33.666 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4190720 len 2048 2024-09-16T00:34:33.666 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4190720 len 2048 2024-09-16T00:34:33.666 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4190720 len 2048 2024-09-16T00:34:33.666 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4190720 len 4096 2024-09-16T00:34:33.666 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4190720 len 4096 2024-09-16T00:34:33.667 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4190720 len 4096 2024-09-16T00:34:33.667 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4190720 len 4096 2024-09-16T00:34:33.667 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4190720 len 8192 2024-09-16T00:34:33.667 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4190720 len 8192 2024-09-16T00:34:33.667 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4190720 len 8192 2024-09-16T00:34:33.667 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4190720 len 8192 2024-09-16T00:34:33.667 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4190720 len 16384 2024-09-16T00:34:33.667 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4190720 len 16384 2024-09-16T00:34:33.667 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4190720 len 16384 2024-09-16T00:34:33.667 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4190720 len 16384 2024-09-16T00:34:33.667 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4191232 len 1024 2024-09-16T00:34:33.667 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4191232 len 1024 2024-09-16T00:34:33.667 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4191232 len 1024 2024-09-16T00:34:33.668 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4191232 len 1024 2024-09-16T00:34:33.668 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4191232 len 2048 2024-09-16T00:34:33.668 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4191232 len 2048 2024-09-16T00:34:33.668 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4191232 len 2048 2024-09-16T00:34:33.669 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4191232 len 2048 2024-09-16T00:34:33.669 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4191232 len 4096 2024-09-16T00:34:33.669 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4191232 len 4096 2024-09-16T00:34:33.669 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4191232 len 4096 2024-09-16T00:34:33.669 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4191232 len 4096 2024-09-16T00:34:33.669 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4191232 len 8192 2024-09-16T00:34:33.669 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4191232 len 8192 2024-09-16T00:34:33.669 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4191232 len 8192 2024-09-16T00:34:33.669 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4191232 len 8192 2024-09-16T00:34:33.669 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4191232 len 16384 2024-09-16T00:34:33.669 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4191232 len 16384 2024-09-16T00:34:33.669 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4191232 len 16384 2024-09-16T00:34:33.669 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4191232 len 16384 2024-09-16T00:34:33.670 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4191744 len 1024 2024-09-16T00:34:33.670 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4191744 len 1024 2024-09-16T00:34:33.670 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4191744 len 1024 2024-09-16T00:34:33.670 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4191744 len 1024 2024-09-16T00:34:33.670 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4191744 len 2048 2024-09-16T00:34:33.670 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4191744 len 2048 2024-09-16T00:34:33.670 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4191744 len 2048 2024-09-16T00:34:33.670 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4191744 len 2048 2024-09-16T00:34:33.670 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4191744 len 4096 2024-09-16T00:34:33.670 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4191744 len 4096 2024-09-16T00:34:33.671 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4191744 len 4096 2024-09-16T00:34:33.671 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4191744 len 4096 2024-09-16T00:34:33.671 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4191744 len 8192 2024-09-16T00:34:33.671 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4191744 len 8192 2024-09-16T00:34:33.671 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4191744 len 8192 2024-09-16T00:34:33.671 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4191744 len 8192 2024-09-16T00:34:33.671 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4191744 len 16384 2024-09-16T00:34:33.671 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4191744 len 16384 2024-09-16T00:34:33.671 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4191744 len 16384 2024-09-16T00:34:33.671 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4191744 len 16384 2024-09-16T00:34:33.671 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4192256 len 1024 2024-09-16T00:34:33.671 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4192256 len 1024 2024-09-16T00:34:33.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.973+0000 7fec4ae6e640 1 -- 172.21.15.38:0/2396086291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec44106f10 msgr2=0x7fec4410ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:33.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.973+0000 7fec4ae6e640 1 --2- 172.21.15.38:0/2396086291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec44106f10 0x7fec4410ea90 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fec3c009ca0 tx=0x7fec3c02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.974+0000 7fec4ae6e640 1 -- 172.21.15.38:0/2396086291 shutdown_connections 2024-09-16T00:34:33.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.974+0000 7fec4ae6e640 1 --2- 172.21.15.38:0/2396086291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec44106f10 0x7fec4410ea90 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.986 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.974+0000 7fec4ae6e640 1 --2- 172.21.15.38:0/2396086291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec44102790 0x7fec441067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.974+0000 7fec4ae6e640 1 --2- 172.21.15.38:0/2396086291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec44101de0 0x7fec441021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.974+0000 7fec4ae6e640 1 -- 172.21.15.38:0/2396086291 >> 172.21.15.38:0/2396086291 conn(0x7fec440fd8c0 msgr2=0x7fec440ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:33.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.974+0000 7fec4ae6e640 1 -- 172.21.15.38:0/2396086291 shutdown_connections 2024-09-16T00:34:33.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.974+0000 7fec4ae6e640 1 -- 172.21.15.38:0/2396086291 wait complete. 2024-09-16T00:34:33.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.974+0000 7fec4ae6e640 1 Processor -- start 2024-09-16T00:34:33.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.974+0000 7fec4ae6e640 1 -- start start 2024-09-16T00:34:33.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.974+0000 7fec4ae6e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec44101de0 0x7fec441983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:33.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.974+0000 7fec4ae6e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec44102790 0x7fec441988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:33.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.974+0000 7fec4ae6e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec44106f10 0x7fec4419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:33.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.974+0000 7fec4ae6e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fec44111f70 con 0x7fec44101de0 2024-09-16T00:34:33.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.974+0000 7fec4ae6e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fec44111df0 con 0x7fec44106f10 2024-09-16T00:34:33.987 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.974+0000 7fec4ae6e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fec441120f0 con 0x7fec44102790 2024-09-16T00:34:33.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.975+0000 7fec48c0a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec44101de0 0x7fec441983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:33.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.975+0000 7fec43fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec44102790 0x7fec441988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:33.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.975+0000 7fec48c0a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec44101de0 0x7fec441983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51312/0 (socket says 172.21.15.38:51312) 2024-09-16T00:34:33.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.975+0000 7fec48c0a640 1 -- 172.21.15.38:0/3546228181 learned_addr learned my addr 172.21.15.38:0/3546228181 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:34:33.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.975+0000 7fec4940b640 1 --2- 172.21.15.38:0/3546228181 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec44106f10 0x7fec4419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:33.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.975+0000 7fec43fff640 1 -- 172.21.15.38:0/3546228181 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec44106f10 msgr2=0x7fec4419cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:33.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.975+0000 7fec43fff640 1 --2- 172.21.15.38:0/3546228181 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec44106f10 0x7fec4419cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.975+0000 7fec43fff640 1 -- 172.21.15.38:0/3546228181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec44101de0 msgr2=0x7fec441983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:33.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.975+0000 7fec43fff640 1 --2- 172.21.15.38:0/3546228181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec44101de0 0x7fec441983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.975+0000 7fec43fff640 1 -- 172.21.15.38:0/3546228181 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec4419d360 con 0x7fec44102790 2024-09-16T00:34:33.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.975+0000 7fec48c0a640 1 --2- 172.21.15.38:0/3546228181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec44101de0 0x7fec441983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:34:33.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.975+0000 7fec43fff640 1 --2- 172.21.15.38:0/3546228181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec44102790 0x7fec441988f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fec3000e9f0 tx=0x7fec3000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:33.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.975+0000 7fec41ffb640 1 -- 172.21.15.38:0/3546228181 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3000ccc0 con 0x7fec44102790 2024-09-16T00:34:33.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.975+0000 7fec41ffb640 1 -- 172.21.15.38:0/3546228181 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec30010910 con 0x7fec44102790 2024-09-16T00:34:33.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.975+0000 7fec4ae6e640 1 -- 172.21.15.38:0/3546228181 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec4419d650 con 0x7fec44102790 2024-09-16T00:34:33.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.975+0000 7fec41ffb640 1 -- 172.21.15.38:0/3546228181 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec30018960 con 0x7fec44102790 2024-09-16T00:34:33.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.976+0000 7fec4ae6e640 1 -- 172.21.15.38:0/3546228181 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec441a4f30 con 0x7fec44102790 2024-09-16T00:34:33.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.978+0000 7fec41ffb640 1 -- 172.21.15.38:0/3546228181 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fec30010ac0 con 0x7fec44102790 2024-09-16T00:34:33.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.978+0000 7fec4d247740 1 Processor -- start 2024-09-16T00:34:33.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.978+0000 7fec4d247740 1 -- start start 2024-09-16T00:34:33.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.979+0000 7fec4d247740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596bccaeb40 0x5596bccad090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:33.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.979+0000 7fec4d247740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596bccad5d0 0x5596bccada10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:33.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.979+0000 7fec4d247740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596bccbc780 0x5596bccbeb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:33.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.979+0000 7fec41ffb640 1 --2- 172.21.15.38:0/3546228181 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fec340a3240 0x7fec340a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:33.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.979+0000 7fec4d247740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5596bcc132f0 con 0x5596bccad5d0 2024-09-16T00:34:33.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.979+0000 7fec4d247740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5596bcbbacf0 con 0x5596bccbc780 2024-09-16T00:34:33.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.979+0000 7fec4d247740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5596bcc0f2b0 con 0x5596bccaeb40 2024-09-16T00:34:33.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.979+0000 7fec48c0a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596bccad5d0 0x5596bccada10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:33.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.979+0000 7fec48c0a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596bccad5d0 0x5596bccada10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51314/0 (socket says 172.21.15.38:51314) 2024-09-16T00:34:33.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.979+0000 7fec48c0a640 1 -- 172.21.15.38:0/56096045 learned_addr learned my addr 172.21.15.38:0/56096045 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:34:33.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.979+0000 7fec4940b640 1 --2- 172.21.15.38:0/3546228181 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fec340a3240 0x7fec340a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:33.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.979+0000 7fec41ffb640 1 -- 172.21.15.38:0/3546228181 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7fec300cfff0 con 0x7fec44102790 2024-09-16T00:34:33.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.979+0000 7fec43fff640 1 --2- 172.21.15.38:0/56096045 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596bccbc780 0x5596bccbeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:33.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.979+0000 7fec4940b640 1 --2- 172.21.15.38:0/56096045 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596bccaeb40 0x5596bccad090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:33.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.979+0000 7fec48c0a640 1 -- 172.21.15.38:0/56096045 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596bccaeb40 msgr2=0x5596bccad090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:33.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.979+0000 7fec48c0a640 1 --2- 172.21.15.38:0/56096045 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596bccaeb40 0x5596bccad090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.979+0000 7fec48c0a640 1 -- 172.21.15.38:0/56096045 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596bccbc780 msgr2=0x5596bccbeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:33.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.979+0000 7fec48c0a640 1 --2- 172.21.15.38:0/56096045 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596bccbc780 0x5596bccbeb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.979+0000 7fec48c0a640 1 -- 172.21.15.38:0/56096045 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596bcb9f3a0 con 0x5596bccad5d0 2024-09-16T00:34:33.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.979+0000 7fec48c0a640 1 --2- 172.21.15.38:0/56096045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596bccad5d0 0x5596bccada10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fec38011630 tx=0x7fec38011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e7ebbb00ad8f80 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:33.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.979+0000 7fec4940b640 1 --2- 172.21.15.38:0/3546228181 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fec340a3240 0x7fec340a5700 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fec3c004af0 tx=0x7fec3c004a80 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:33.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.980+0000 7fec2effd640 1 -- 172.21.15.38:0/56096045 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3801e070 con 0x5596bccad5d0 2024-09-16T00:34:33.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.980+0000 7fec2effd640 1 -- 172.21.15.38:0/56096045 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec3800fbd0 con 0x5596bccad5d0 2024-09-16T00:34:33.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.980+0000 7fec2effd640 1 -- 172.21.15.38:0/56096045 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec38019d00 con 0x5596bccad5d0 2024-09-16T00:34:33.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.980+0000 7fec4d247740 1 -- 172.21.15.38:0/56096045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596bccad5d0 msgr2=0x5596bccada10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:33.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.980+0000 7fec4d247740 1 --2- 172.21.15.38:0/56096045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596bccad5d0 0x5596bccada10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fec38011630 tx=0x7fec38011af0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.980+0000 7fec4d247740 1 -- 172.21.15.38:0/56096045 shutdown_connections 2024-09-16T00:34:33.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.981+0000 7fec4d247740 1 --2- 172.21.15.38:0/56096045 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596bccbc780 0x5596bccbeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.981+0000 7fec4d247740 1 --2- 172.21.15.38:0/56096045 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596bccad5d0 0x5596bccada10 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.981+0000 7fec4d247740 1 --2- 172.21.15.38:0/56096045 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596bccaeb40 0x5596bccad090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.981+0000 7fec4d247740 1 -- 172.21.15.38:0/56096045 >> 172.21.15.38:0/56096045 conn(0x5596bcc76310 msgr2=0x5596bcca9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:33.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.981+0000 7fec4d247740 1 -- 172.21.15.38:0/56096045 shutdown_connections 2024-09-16T00:34:33.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.981+0000 7fec4d247740 1 -- 172.21.15.38:0/56096045 wait complete. 2024-09-16T00:34:33.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.981+0000 7fec4d247740 1 -- start start 2024-09-16T00:34:33.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.981+0000 7fec4d247740 1 Processor -- start 2024-09-16T00:34:33.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.981+0000 7fec4d247740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596bccad5d0 0x5596bccbc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:33.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.981+0000 7fec4d247740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596bccaeb40 0x5596bccb9440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:33.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.981+0000 7fec4d247740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596bccbc780 0x5596bccb9a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:33.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.981+0000 7fec4d247740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5596bccbf480 con 0x5596bccbc780 2024-09-16T00:34:33.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.981+0000 7fec4d247740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5596bcbce060 con 0x5596bccad5d0 2024-09-16T00:34:33.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.981+0000 7fec4d247740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5596bccbf600 con 0x5596bccaeb40 2024-09-16T00:34:33.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.982+0000 7fec4d247740 10 client.0 _ll_register_callbacks cb 0x5596bc9218c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:33.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.982+0000 7fec4940b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596bccbc780 0x5596bccb9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:33.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.982+0000 7fec48c0a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596bccad5d0 0x5596bccbc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:33.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.982+0000 7fec48c0a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596bccad5d0 0x5596bccbc190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.38:58020/0 (socket says 172.21.15.38:58020) 2024-09-16T00:34:33.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.982+0000 7fec4940b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596bccbc780 0x5596bccb9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51320/0 (socket says 172.21.15.38:51320) 2024-09-16T00:34:33.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.982+0000 7fec48c0a640 1 -- 172.21.15.38:0/3532356001 learned_addr learned my addr 172.21.15.38:0/3532356001 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:34:33.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.982+0000 7fec43fff640 1 --2- 172.21.15.38:0/3532356001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596bccaeb40 0x5596bccb9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:33.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.982+0000 7fec48c0a640 1 -- 172.21.15.38:0/3532356001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596bccaeb40 msgr2=0x5596bccb9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:33.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.982+0000 7fec48c0a640 1 --2- 172.21.15.38:0/3532356001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596bccaeb40 0x5596bccb9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.982+0000 7fec48c0a640 1 -- 172.21.15.38:0/3532356001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596bccbc780 msgr2=0x5596bccb9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:33.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.982+0000 7fec48c0a640 1 --2- 172.21.15.38:0/3532356001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596bccbc780 0x5596bccb9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.982+0000 7fec48c0a640 1 -- 172.21.15.38:0/3532356001 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596bc9843c0 con 0x5596bccad5d0 2024-09-16T00:34:33.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.982+0000 7fec48c0a640 1 --2- 172.21.15.38:0/3532356001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596bccad5d0 0x5596bccbc190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fec3801c040 tx=0x7fec380196a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3669a5b7ca5b617 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:33.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.982+0000 7fec23fff640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-09-16T00:34:33.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.982+0000 7fec23fff640 1 -- 172.21.15.38:0/3532356001 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec38026020 con 0x5596bccad5d0 2024-09-16T00:34:33.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.983+0000 7fec4d247740 1 -- 172.21.15.38:0/3532356001 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec3800fbd0 con 0x5596bccad5d0 2024-09-16T00:34:33.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.983+0000 7fec23fff640 1 -- 172.21.15.38:0/3532356001 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7fec3800fd80 con 0x5596bccad5d0 2024-09-16T00:34:33.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.983+0000 7fec23fff640 1 -- 172.21.15.38:0/3532356001 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fec38031050 con 0x5596bccad5d0 2024-09-16T00:34:33.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.983+0000 7fec4d247740 10 client.25370 fetch_fsmap learned FSMap version 26 2024-09-16T00:34:33.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.983+0000 7fec4d247740 1 -- 172.21.15.38:0/3532356001 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5596bccb1d40 con 0x5596bccad5d0 2024-09-16T00:34:33.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.983+0000 7fec23fff640 1 -- 172.21.15.38:0/3532356001 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec3800fbd0 con 0x5596bccad5d0 2024-09-16T00:34:33.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.983+0000 7fec23fff640 1 -- 172.21.15.38:0/3532356001 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fec38021050 con 0x5596bccad5d0 2024-09-16T00:34:33.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.983+0000 7fec4d247740 10 client.25370 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:34:33.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.983+0000 7fec4d247740 10 client.25370 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:34:33.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.983+0000 7fec4d247740 20 client.25370 populate_metadata read hostname 'smithi038' 2024-09-16T00:34:33.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.983+0000 7fec4d247740 1 --2- 172.21.15.38:0/3532356001 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5596bccb30f0 0x5596bccb3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:33.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.983+0000 7fec4d247740 4 client.25370 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:34:33.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.983+0000 7fec4d247740 1 -- 172.21.15.38:0/3532356001 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5596bcc20860 con 0x5596bccb30f0 2024-09-16T00:34:33.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.983+0000 7fec43fff640 1 --2- 172.21.15.38:0/3532356001 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5596bccb30f0 0x5596bccb3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:33.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.984+0000 7fec43fff640 1 --2- 172.21.15.38:0/3532356001 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5596bccb30f0 0x5596bccb3510 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb3711f54c1fa549 server_cookie=7b363648db74d2bb in_seq=0 out_seq=0 2024-09-16T00:34:33.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.984+0000 7fec23fff640 10 client.25370 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:34:33.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.985+0000 7fec23fff640 1 -- 172.21.15.38:0/3532356001 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5596bcc20860 con 0x5596bccb30f0 2024-09-16T00:34:33.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.985+0000 7fec23fff640 10 client.25370 handle_command_reply: tid=0 2024-09-16T00:34:33.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.985+0000 7fec4d247740 1 client.25370 shutdown 2024-09-16T00:34:33.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.986+0000 7fec4d247740 1 -- 172.21.15.38:0/3532356001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596bccad5d0 msgr2=0x5596bccbc190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:33.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.986+0000 7fec4d247740 1 --2- 172.21.15.38:0/3532356001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596bccad5d0 0x5596bccbc190 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fec3801c040 tx=0x7fec380196a0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.986+0000 7fec4d247740 1 -- 172.21.15.38:0/3532356001 shutdown_connections 2024-09-16T00:34:33.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.986+0000 7fec4d247740 1 --2- 172.21.15.38:0/3532356001 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5596bccb30f0 0x5596bccb3510 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.986+0000 7fec4d247740 1 --2- 172.21.15.38:0/3532356001 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596bccbc780 0x5596bccb9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.986+0000 7fec4d247740 1 --2- 172.21.15.38:0/3532356001 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596bccaeb40 0x5596bccb9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.986+0000 7fec23fff640 0 client.25370 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:34:33.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.986+0000 7fec4d247740 1 --2- 172.21.15.38:0/3532356001 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596bccad5d0 0x5596bccbc190 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:33.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.986+0000 7fec4d247740 1 -- 172.21.15.38:0/3532356001 >> 172.21.15.38:0/3532356001 conn(0x5596bcc76310 msgr2=0x5596bccaf960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:33.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.986+0000 7fec4d247740 1 -- 172.21.15.38:0/3532356001 shutdown_connections 2024-09-16T00:34:33.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.986+0000 7fec4d247740 1 -- 172.21.15.38:0/3532356001 wait complete. 2024-09-16T00:34:33.997 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:33.986+0000 7fec4d247740 20 client.25370 trim_cache size 0 max 16384 2024-09-16T00:34:34.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.002+0000 7fec4d247740 1 Processor -- start 2024-09-16T00:34:34.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.002+0000 7fec4d247740 1 -- start start 2024-09-16T00:34:34.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.002+0000 7fec4d247740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596bccad5d0 0x5596bccc88a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:34.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.002+0000 7fec4d247740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596bccaeb40 0x5596bccb4c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:34.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.002+0000 7fec4d247740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596bccbc780 0x5596bccb66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:34.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.002+0000 7fec4d247740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fec441af350 con 0x5596bccad5d0 2024-09-16T00:34:34.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.002+0000 7fec4d247740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fec441acc20 con 0x5596bccbc780 2024-09-16T00:34:34.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.002+0000 7fec4d247740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fec441aaa70 con 0x5596bccaeb40 2024-09-16T00:34:34.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.002+0000 7fec48c0a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596bccad5d0 0x5596bccc88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:34.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.002+0000 7fec4940b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596bccbc780 0x5596bccb66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:34.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.002+0000 7fec48c0a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596bccad5d0 0x5596bccc88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51330/0 (socket says 172.21.15.38:51330) 2024-09-16T00:34:34.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.002+0000 7fec4940b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596bccbc780 0x5596bccb66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.38:58034/0 (socket says 172.21.15.38:58034) 2024-09-16T00:34:34.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.002+0000 7fec4940b640 1 -- 172.21.15.38:0/2000392981 learned_addr learned my addr 172.21.15.38:0/2000392981 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:34:34.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.002+0000 7fec43fff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596bccaeb40 0x5596bccb4c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:34.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.003+0000 7fec4940b640 1 -- 172.21.15.38:0/2000392981 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596bccaeb40 msgr2=0x5596bccb4c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:34.007 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.003+0000 7fec4940b640 1 --2- 172.21.15.38:0/2000392981 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596bccaeb40 0x5596bccb4c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.003+0000 7fec4940b640 1 -- 172.21.15.38:0/2000392981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596bccad5d0 msgr2=0x5596bccc88a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:34.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.003+0000 7fec4940b640 1 --2- 172.21.15.38:0/2000392981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596bccad5d0 0x5596bccc88a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.003+0000 7fec4940b640 1 -- 172.21.15.38:0/2000392981 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5596bccb9dc0 con 0x5596bccbc780 2024-09-16T00:34:34.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.003+0000 7fec48c0a640 1 --2- 172.21.15.38:0/2000392981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596bccad5d0 0x5596bccc88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:34:34.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.003+0000 7fec4940b640 1 --2- 172.21.15.38:0/2000392981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596bccbc780 0x5596bccb66a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fec3c009ca0 tx=0x7fec3c040040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d487d50cf14ca740 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:34.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.003+0000 7fec2e7fc640 1 -- 172.21.15.38:0/2000392981 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c003d40 con 0x5596bccbc780 2024-09-16T00:34:34.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.003+0000 7fec2e7fc640 1 -- 172.21.15.38:0/2000392981 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec3c036040 con 0x5596bccbc780 2024-09-16T00:34:34.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.003+0000 7fec2e7fc640 1 -- 172.21.15.38:0/2000392981 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec3c02fa20 con 0x5596bccbc780 2024-09-16T00:34:34.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.004+0000 7fec4d247740 1 -- 172.21.15.38:0/2000392981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596bccbc780 msgr2=0x5596bccb66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:34.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.004+0000 7fec4d247740 1 --2- 172.21.15.38:0/2000392981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596bccbc780 0x5596bccb66a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fec3c009ca0 tx=0x7fec3c040040 comp rx=0 tx=0).stop 2024-09-16T00:34:34.008 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.004+0000 7fec4d247740 1 -- 172.21.15.38:0/2000392981 shutdown_connections 2024-09-16T00:34:34.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.004+0000 7fec4d247740 1 --2- 172.21.15.38:0/2000392981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596bccbc780 0x5596bccb66a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.004+0000 7fec4d247740 1 --2- 172.21.15.38:0/2000392981 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596bccaeb40 0x5596bccb4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.004+0000 7fec4d247740 1 --2- 172.21.15.38:0/2000392981 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596bccad5d0 0x5596bccc88a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.004+0000 7fec4d247740 1 -- 172.21.15.38:0/2000392981 >> 172.21.15.38:0/2000392981 conn(0x5596bcc76310 msgr2=0x5596bccb8400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:34.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.004+0000 7fec4d247740 1 -- 172.21.15.38:0/2000392981 shutdown_connections 2024-09-16T00:34:34.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.004+0000 7fec4d247740 1 -- 172.21.15.38:0/2000392981 wait complete. 2024-09-16T00:34:34.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.004+0000 7fec4d247740 1 -- start start 2024-09-16T00:34:34.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.005+0000 7fec4d247740 1 Processor -- start 2024-09-16T00:34:34.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.005+0000 7fec4d247740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596bccad5d0 0x5596bccd39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:34.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.005+0000 7fec4d247740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596bccaeb40 0x5596bccd3f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:34.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.005+0000 7fec4d247740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596bccbc780 0x5596bccd4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:34.009 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.005+0000 7fec4d247740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fec441ab0e0 con 0x5596bccaeb40 2024-09-16T00:34:34.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.005+0000 7fec4d247740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fec441ac550 con 0x5596bccbc780 2024-09-16T00:34:34.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.005+0000 7fec4d247740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fec441ab7b0 con 0x5596bccad5d0 2024-09-16T00:34:34.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.005+0000 7fec43fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596bccaeb40 0x5596bccd3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:34.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.005+0000 7fec4d247740 10 client.0 _ll_register_callbacks cb 0x7fec3801c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:34.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.005+0000 7fec48c0a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596bccad5d0 0x5596bccd39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:34.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.005+0000 7fec4940b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596bccbc780 0x5596bccd4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:34.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.005+0000 7fec48c0a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596bccad5d0 0x5596bccd39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.38:59660/0 (socket says 172.21.15.38:59660) 2024-09-16T00:34:34.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.005+0000 7fec43fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596bccaeb40 0x5596bccd3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:51340/0 (socket says 172.21.15.38:51340) 2024-09-16T00:34:34.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.005+0000 7fec48c0a640 1 -- 172.21.15.38:0/840860218 learned_addr learned my addr 172.21.15.38:0/840860218 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:34:34.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.006+0000 7fec48c0a640 1 -- 172.21.15.38:0/840860218 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596bccbc780 msgr2=0x5596bccd4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:34.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.006+0000 7fec48c0a640 1 --2- 172.21.15.38:0/840860218 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596bccbc780 0x5596bccd4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.006+0000 7fec48c0a640 1 -- 172.21.15.38:0/840860218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596bccaeb40 msgr2=0x5596bccd3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:34.010 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.006+0000 7fec48c0a640 1 --2- 172.21.15.38:0/840860218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596bccaeb40 0x5596bccd3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.006+0000 7fec48c0a640 1 -- 172.21.15.38:0/840860218 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5596bcd39d30 con 0x5596bccad5d0 2024-09-16T00:34:34.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.006+0000 7fec48c0a640 1 --2- 172.21.15.38:0/840860218 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596bccad5d0 0x5596bccd39c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fec38022370 tx=0x7fec380223a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c13653282644d83d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:34.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.006+0000 7fec23fff640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-16T00:34:34.011 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.006+0000 7fec23fff640 1 -- 172.21.15.38:0/840860218 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec38026040 con 0x5596bccad5d0 2024-09-16T00:34:34.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.006+0000 7fec23fff640 1 -- 172.21.15.38:0/840860218 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7fec38009e30 con 0x5596bccad5d0 2024-09-16T00:34:34.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.006+0000 7fec4d247740 1 -- 172.21.15.38:0/840860218 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec3c036040 con 0x5596bccad5d0 2024-09-16T00:34:34.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.007+0000 7fec23fff640 1 -- 172.21.15.38:0/840860218 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fec3801e040 con 0x5596bccad5d0 2024-09-16T00:34:34.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.007+0000 7fec4d247740 10 client.25306 fetch_fsmap learned FSMap version 26 2024-09-16T00:34:34.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.007+0000 7fec4d247740 1 -- 172.21.15.38:0/840860218 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5596bcd3af10 con 0x5596bccad5d0 2024-09-16T00:34:34.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.007+0000 7fec23fff640 1 -- 172.21.15.38:0/840860218 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fec3c036040 con 0x5596bccad5d0 2024-09-16T00:34:34.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.007+0000 7fec23fff640 1 -- 172.21.15.38:0/840860218 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fec38021070 con 0x5596bccad5d0 2024-09-16T00:34:34.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.007+0000 7fec4d247740 10 client.25306 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:34:34.013 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.007+0000 7fec4d247740 10 client.25306 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:34:34.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.007+0000 7fec4d247740 20 client.25306 populate_metadata read hostname 'smithi038' 2024-09-16T00:34:34.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.007+0000 7fec4d247740 1 --2- 172.21.15.38:0/840860218 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5596bcd3b5c0 0x5596bcd3d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:34.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.007+0000 7fec4d247740 4 client.25306 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:34:34.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.007+0000 7fec4d247740 1 -- 172.21.15.38:0/840860218 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5596bcc1ad70 con 0x5596bcd3b5c0 2024-09-16T00:34:34.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.008+0000 7fec43fff640 1 --2- 172.21.15.38:0/840860218 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5596bcd3b5c0 0x5596bcd3d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:34.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.008+0000 7fec23fff640 10 client.25306 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:34:34.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.008+0000 7fec43fff640 1 --2- 172.21.15.38:0/840860218 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5596bcd3b5c0 0x5596bcd3d9e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=299e11136a6e9820 server_cookie=da80b69586a6b855 in_seq=0 out_seq=0 2024-09-16T00:34:34.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.008+0000 7fec23fff640 1 -- 172.21.15.38:0/840860218 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5596bcc1ad70 con 0x5596bcd3b5c0 2024-09-16T00:34:34.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.008+0000 7fec23fff640 10 client.25306 handle_command_reply: tid=0 2024-09-16T00:34:34.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.008+0000 7fec4d247740 1 client.25306 shutdown 2024-09-16T00:34:34.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.008+0000 7fec4d247740 1 -- 172.21.15.38:0/840860218 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596bccad5d0 msgr2=0x5596bccd39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:34.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.008+0000 7fec4d247740 1 --2- 172.21.15.38:0/840860218 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596bccad5d0 0x5596bccd39c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fec38022370 tx=0x7fec380223a0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.008+0000 7fec4d247740 1 -- 172.21.15.38:0/840860218 shutdown_connections 2024-09-16T00:34:34.014 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.008+0000 7fec4d247740 1 --2- 172.21.15.38:0/840860218 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5596bccbc780 0x5596bccd4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.008+0000 7fec4d247740 1 --2- 172.21.15.38:0/840860218 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5596bcd3b5c0 0x5596bcd3d9e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.008+0000 7fec4d247740 1 --2- 172.21.15.38:0/840860218 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5596bccaeb40 0x5596bccd3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.008+0000 7fec23fff640 0 client.25306 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:34:34.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.008+0000 7fec4d247740 1 --2- 172.21.15.38:0/840860218 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5596bccad5d0 0x5596bccd39c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.008+0000 7fec4d247740 1 -- 172.21.15.38:0/840860218 >> 172.21.15.38:0/840860218 conn(0x5596bcc76310 msgr2=0x5596bcca84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:34.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.009+0000 7fec4d247740 1 -- 172.21.15.38:0/840860218 shutdown_connections 2024-09-16T00:34:34.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.009+0000 7fec4d247740 1 -- 172.21.15.38:0/840860218 wait complete. 2024-09-16T00:34:34.015 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.009+0000 7fec4d247740 20 client.25306 trim_cache size 0 max 16384 2024-09-16T00:34:34.015 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-09-16T00:34:34.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.009+0000 7fec4ae6e640 1 -- 172.21.15.38:0/3546228181 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fec340a3240 msgr2=0x7fec340a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:34.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.009+0000 7fec4ae6e640 1 --2- 172.21.15.38:0/3546228181 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fec340a3240 0x7fec340a5700 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fec3c004af0 tx=0x7fec3c004a80 comp rx=0 tx=0).stop 2024-09-16T00:34:34.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.009+0000 7fec4ae6e640 1 -- 172.21.15.38:0/3546228181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec44102790 msgr2=0x7fec441988f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:34.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.009+0000 7fec4ae6e640 1 --2- 172.21.15.38:0/3546228181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec44102790 0x7fec441988f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fec3000e9f0 tx=0x7fec3000eeb0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.009+0000 7fec4ae6e640 1 -- 172.21.15.38:0/3546228181 shutdown_connections 2024-09-16T00:34:34.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.009+0000 7fec4ae6e640 1 --2- 172.21.15.38:0/3546228181 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fec340a3240 0x7fec340a5700 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.016 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.009+0000 7fec4ae6e640 1 --2- 172.21.15.38:0/3546228181 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec44106f10 0x7fec4419cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.009+0000 7fec4ae6e640 1 --2- 172.21.15.38:0/3546228181 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fec44102790 0x7fec441988f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.009+0000 7fec4ae6e640 1 --2- 172.21.15.38:0/3546228181 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fec44101de0 0x7fec441983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:34.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.009+0000 7fec4ae6e640 1 -- 172.21.15.38:0/3546228181 >> 172.21.15.38:0/3546228181 conn(0x7fec440fd8c0 msgr2=0x7fec440fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:34.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.010+0000 7fec4ae6e640 1 -- 172.21.15.38:0/3546228181 shutdown_connections 2024-09-16T00:34:34.017 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:34.010+0000 7fec4ae6e640 1 -- 172.21.15.38:0/3546228181 wait complete. 2024-09-16T00:34:34.667 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:34 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:34:34.667 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:34 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:34:34.667 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:34 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:34:34.667 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:34 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:34:34.667 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:34 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:34:34.667 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:34 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:34:34.668 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:34 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:34:34.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:34 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:34:34.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:34 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:34:34.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:34 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:34:34.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:34 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:34:34.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:34 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:34:34.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:34 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:34:34.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:34 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:34:34.793 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:34 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:34:34.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:34 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:34:34.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:34 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:34:34.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:34 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:34:34.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:34 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:34:34.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:34 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:34:34.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:34 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:34:34.929 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4192256 len 1024 2024-09-16T00:34:34.930 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4192256 len 1024 2024-09-16T00:34:34.930 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4192256 len 2048 2024-09-16T00:34:34.930 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4192256 len 2048 2024-09-16T00:34:34.930 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4192256 len 2048 2024-09-16T00:34:34.930 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4192256 len 2048 2024-09-16T00:34:34.930 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4192256 len 4096 2024-09-16T00:34:34.930 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4192256 len 4096 2024-09-16T00:34:34.930 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4192256 len 4096 2024-09-16T00:34:34.930 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4192256 len 4096 2024-09-16T00:34:34.930 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4192256 len 8192 2024-09-16T00:34:34.930 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4192256 len 8192 2024-09-16T00:34:34.931 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4192256 len 8192 2024-09-16T00:34:34.931 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4192256 len 8192 2024-09-16T00:34:34.931 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4192256 len 16384 2024-09-16T00:34:34.931 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4192256 len 16384 2024-09-16T00:34:34.931 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4192256 len 16384 2024-09-16T00:34:34.931 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4192256 len 16384 2024-09-16T00:34:34.931 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4192768 len 1024 2024-09-16T00:34:34.931 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4192768 len 1024 2024-09-16T00:34:34.931 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4192768 len 1024 2024-09-16T00:34:34.931 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4192768 len 1024 2024-09-16T00:34:34.931 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4192768 len 2048 2024-09-16T00:34:34.931 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4192768 len 2048 2024-09-16T00:34:34.931 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4192768 len 2048 2024-09-16T00:34:34.932 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4192768 len 2048 2024-09-16T00:34:34.932 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4192768 len 4096 2024-09-16T00:34:34.932 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4192768 len 4096 2024-09-16T00:34:34.932 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4192768 len 4096 2024-09-16T00:34:34.932 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4192768 len 4096 2024-09-16T00:34:34.932 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4192768 len 8192 2024-09-16T00:34:34.932 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4192768 len 8192 2024-09-16T00:34:34.932 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4192768 len 8192 2024-09-16T00:34:34.932 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4192768 len 8192 2024-09-16T00:34:34.932 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4192768 len 16384 2024-09-16T00:34:34.932 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4192768 len 16384 2024-09-16T00:34:34.932 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4192768 len 16384 2024-09-16T00:34:34.932 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4192768 len 16384 2024-09-16T00:34:34.933 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4193280 len 1024 2024-09-16T00:34:34.933 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4193280 len 1024 2024-09-16T00:34:34.933 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4193280 len 1024 2024-09-16T00:34:34.933 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4193280 len 1024 2024-09-16T00:34:34.933 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4193280 len 2048 2024-09-16T00:34:34.933 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4193280 len 2048 2024-09-16T00:34:34.933 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4193280 len 2048 2024-09-16T00:34:34.933 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4193280 len 2048 2024-09-16T00:34:34.933 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4193280 len 4096 2024-09-16T00:34:34.933 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4193280 len 4096 2024-09-16T00:34:34.933 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4193280 len 4096 2024-09-16T00:34:34.933 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4193280 len 4096 2024-09-16T00:34:34.933 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4193280 len 8192 2024-09-16T00:34:34.934 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4193280 len 8192 2024-09-16T00:34:34.934 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4193280 len 8192 2024-09-16T00:34:34.934 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4193280 len 8192 2024-09-16T00:34:34.934 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4193280 len 16384 2024-09-16T00:34:34.934 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4193280 len 16384 2024-09-16T00:34:34.934 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4193280 len 16384 2024-09-16T00:34:34.934 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4193280 len 16384 2024-09-16T00:34:34.934 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4193792 len 1024 2024-09-16T00:34:34.934 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4193792 len 1024 2024-09-16T00:34:34.934 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4193792 len 1024 2024-09-16T00:34:34.934 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4193792 len 1024 2024-09-16T00:34:34.934 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4193792 len 2048 2024-09-16T00:34:34.935 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4193792 len 2048 2024-09-16T00:34:34.935 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4193792 len 2048 2024-09-16T00:34:34.935 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4193792 len 2048 2024-09-16T00:34:34.935 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4193792 len 4096 2024-09-16T00:34:34.935 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4193792 len 4096 2024-09-16T00:34:34.935 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4193792 len 4096 2024-09-16T00:34:34.935 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4193792 len 4096 2024-09-16T00:34:34.935 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4193792 len 8192 2024-09-16T00:34:34.935 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4193792 len 8192 2024-09-16T00:34:34.935 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4193792 len 8192 2024-09-16T00:34:34.936 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4193792 len 8192 2024-09-16T00:34:34.936 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4193792 len 16384 2024-09-16T00:34:34.936 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4193792 len 16384 2024-09-16T00:34:34.936 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4193792 len 16384 2024-09-16T00:34:34.936 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4193792 len 16384 2024-09-16T00:34:34.936 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4194304 len 1024 2024-09-16T00:34:34.936 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4194304 len 1024 2024-09-16T00:34:34.936 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4194304 len 1024 2024-09-16T00:34:34.936 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4194304 len 1024 2024-09-16T00:34:35.346 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4194304 len 2048 2024-09-16T00:34:35.346 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4194304 len 2048 2024-09-16T00:34:35.346 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4194304 len 2048 2024-09-16T00:34:35.346 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4194304 len 2048 2024-09-16T00:34:35.346 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4194304 len 4096 2024-09-16T00:34:35.346 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4194304 len 4096 2024-09-16T00:34:35.346 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4194304 len 4096 2024-09-16T00:34:35.346 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4194304 len 4096 2024-09-16T00:34:35.346 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4194304 len 8192 2024-09-16T00:34:35.346 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4194304 len 8192 2024-09-16T00:34:35.347 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4194304 len 8192 2024-09-16T00:34:35.347 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4194304 len 8192 2024-09-16T00:34:35.347 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4194304 len 16384 2024-09-16T00:34:35.347 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4194304 len 16384 2024-09-16T00:34:35.347 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4194304 len 16384 2024-09-16T00:34:35.347 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4194304 len 16384 2024-09-16T00:34:35.347 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4194816 len 1024 2024-09-16T00:34:35.347 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4194816 len 1024 2024-09-16T00:34:35.347 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4194816 len 1024 2024-09-16T00:34:35.347 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4194816 len 1024 2024-09-16T00:34:35.347 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4194816 len 2048 2024-09-16T00:34:35.347 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4194816 len 2048 2024-09-16T00:34:35.347 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4194816 len 2048 2024-09-16T00:34:35.348 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4194816 len 2048 2024-09-16T00:34:35.348 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4194816 len 4096 2024-09-16T00:34:35.348 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4194816 len 4096 2024-09-16T00:34:35.348 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4194816 len 4096 2024-09-16T00:34:35.348 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4194816 len 4096 2024-09-16T00:34:35.348 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4194816 len 8192 2024-09-16T00:34:35.348 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4194816 len 8192 2024-09-16T00:34:35.348 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4194816 len 8192 2024-09-16T00:34:35.348 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4194816 len 8192 2024-09-16T00:34:35.348 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4194816 len 16384 2024-09-16T00:34:35.348 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4194816 len 16384 2024-09-16T00:34:35.348 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4194816 len 16384 2024-09-16T00:34:35.348 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4194816 len 16384 2024-09-16T00:34:35.349 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4195328 len 1024 2024-09-16T00:34:35.349 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4195328 len 1024 2024-09-16T00:34:35.349 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4195328 len 1024 2024-09-16T00:34:35.349 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4195328 len 1024 2024-09-16T00:34:35.349 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4195328 len 2048 2024-09-16T00:34:35.349 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4195328 len 2048 2024-09-16T00:34:35.349 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4195328 len 2048 2024-09-16T00:34:35.349 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4195328 len 2048 2024-09-16T00:34:35.349 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4195328 len 4096 2024-09-16T00:34:35.349 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4195328 len 4096 2024-09-16T00:34:35.349 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4195328 len 4096 2024-09-16T00:34:35.349 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4195328 len 4096 2024-09-16T00:34:35.349 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4195328 len 8192 2024-09-16T00:34:35.349 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4195328 len 8192 2024-09-16T00:34:35.350 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4195328 len 8192 2024-09-16T00:34:35.350 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4195328 len 8192 2024-09-16T00:34:35.350 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4195328 len 16384 2024-09-16T00:34:35.350 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4195328 len 16384 2024-09-16T00:34:35.350 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4195328 len 16384 2024-09-16T00:34:35.350 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4195328 len 16384 2024-09-16T00:34:35.350 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4195840 len 1024 2024-09-16T00:34:35.350 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4195840 len 1024 2024-09-16T00:34:35.350 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4195840 len 1024 2024-09-16T00:34:35.350 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4195840 len 1024 2024-09-16T00:34:35.350 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4195840 len 2048 2024-09-16T00:34:35.350 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4195840 len 2048 2024-09-16T00:34:35.350 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4195840 len 2048 2024-09-16T00:34:35.351 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4195840 len 2048 2024-09-16T00:34:35.351 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4195840 len 4096 2024-09-16T00:34:35.351 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4195840 len 4096 2024-09-16T00:34:35.351 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4195840 len 4096 2024-09-16T00:34:35.351 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4195840 len 4096 2024-09-16T00:34:35.351 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4195840 len 8192 2024-09-16T00:34:35.351 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4195840 len 8192 2024-09-16T00:34:35.351 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4195840 len 8192 2024-09-16T00:34:35.351 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4195840 len 8192 2024-09-16T00:34:35.351 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4195840 len 16384 2024-09-16T00:34:35.351 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4195840 len 16384 2024-09-16T00:34:35.351 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4195840 len 16384 2024-09-16T00:34:35.351 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4195840 len 16384 2024-09-16T00:34:35.351 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4196352 len 1024 2024-09-16T00:34:35.351 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4196352 len 1024 2024-09-16T00:34:35.352 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4196352 len 1024 2024-09-16T00:34:35.352 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4196352 len 1024 2024-09-16T00:34:35.352 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4196352 len 2048 2024-09-16T00:34:35.352 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4196352 len 2048 2024-09-16T00:34:35.697 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-16T00:34:35.697 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-16T00:34:35.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:35 smithi038 ceph-mon[30945]: pgmap v418: 129 pgs: 129 active+clean; 185 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 735 KiB/s wr, 174 op/s 2024-09-16T00:34:35.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:35 smithi122 ceph-mon[35412]: pgmap v418: 129 pgs: 129 active+clean; 185 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 735 KiB/s wr, 174 op/s 2024-09-16T00:34:35.793 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:35 smithi073 ceph-mon[35254]: pgmap v418: 129 pgs: 129 active+clean; 185 MiB data, 605 MiB used, 1.0 TiB / 1.0 TiB avail; 19 MiB/s rd, 735 KiB/s wr, 174 op/s 2024-09-16T00:34:35.896 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4196352 len 2048 2024-09-16T00:34:35.897 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4196352 len 2048 2024-09-16T00:34:35.897 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4196352 len 4096 2024-09-16T00:34:35.897 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4196352 len 4096 2024-09-16T00:34:35.897 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4196352 len 4096 2024-09-16T00:34:35.897 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4196352 len 4096 2024-09-16T00:34:35.897 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4196352 len 8192 2024-09-16T00:34:35.897 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4196352 len 8192 2024-09-16T00:34:35.897 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4196352 len 8192 2024-09-16T00:34:35.897 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4196352 len 8192 2024-09-16T00:34:35.897 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4196352 len 16384 2024-09-16T00:34:35.897 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4196352 len 16384 2024-09-16T00:34:35.897 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4196352 len 16384 2024-09-16T00:34:35.898 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4196352 len 16384 2024-09-16T00:34:35.898 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4196864 len 1024 2024-09-16T00:34:35.898 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4196864 len 1024 2024-09-16T00:34:35.898 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4196864 len 1024 2024-09-16T00:34:35.898 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4196864 len 1024 2024-09-16T00:34:35.898 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4196864 len 2048 2024-09-16T00:34:35.898 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4196864 len 2048 2024-09-16T00:34:35.898 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4196864 len 2048 2024-09-16T00:34:35.898 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4196864 len 2048 2024-09-16T00:34:35.898 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4196864 len 4096 2024-09-16T00:34:35.898 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4196864 len 4096 2024-09-16T00:34:35.898 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4196864 len 4096 2024-09-16T00:34:35.899 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4196864 len 4096 2024-09-16T00:34:35.899 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4196864 len 8192 2024-09-16T00:34:35.899 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4196864 len 8192 2024-09-16T00:34:35.899 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4196864 len 8192 2024-09-16T00:34:35.899 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4196864 len 8192 2024-09-16T00:34:35.899 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4196864 len 16384 2024-09-16T00:34:35.899 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4196864 len 16384 2024-09-16T00:34:35.899 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4196864 len 16384 2024-09-16T00:34:35.899 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4196864 len 16384 2024-09-16T00:34:35.899 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4197376 len 1024 2024-09-16T00:34:35.899 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4197376 len 1024 2024-09-16T00:34:35.899 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4197376 len 1024 2024-09-16T00:34:35.899 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4197376 len 1024 2024-09-16T00:34:35.899 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4197376 len 2048 2024-09-16T00:34:35.899 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4197376 len 2048 2024-09-16T00:34:35.900 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4197376 len 2048 2024-09-16T00:34:35.900 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4197376 len 2048 2024-09-16T00:34:35.900 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4197376 len 4096 2024-09-16T00:34:35.900 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4197376 len 4096 2024-09-16T00:34:35.900 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4197376 len 4096 2024-09-16T00:34:35.900 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4197376 len 4096 2024-09-16T00:34:35.900 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4197376 len 8192 2024-09-16T00:34:35.900 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4197376 len 8192 2024-09-16T00:34:35.900 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4197376 len 8192 2024-09-16T00:34:35.900 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4197376 len 8192 2024-09-16T00:34:35.900 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4197376 len 16384 2024-09-16T00:34:35.900 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4197376 len 16384 2024-09-16T00:34:35.900 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4197376 len 16384 2024-09-16T00:34:35.900 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4197376 len 16384 2024-09-16T00:34:35.901 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4197888 len 1024 2024-09-16T00:34:35.901 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4197888 len 1024 2024-09-16T00:34:35.901 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4197888 len 1024 2024-09-16T00:34:35.901 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4197888 len 1024 2024-09-16T00:34:35.901 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4197888 len 2048 2024-09-16T00:34:35.901 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4197888 len 2048 2024-09-16T00:34:35.901 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4197888 len 2048 2024-09-16T00:34:35.901 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4197888 len 2048 2024-09-16T00:34:35.901 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4197888 len 4096 2024-09-16T00:34:35.901 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4197888 len 4096 2024-09-16T00:34:35.901 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4197888 len 4096 2024-09-16T00:34:35.901 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4197888 len 4096 2024-09-16T00:34:35.901 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4197888 len 8192 2024-09-16T00:34:35.902 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4197888 len 8192 2024-09-16T00:34:35.902 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4197888 len 8192 2024-09-16T00:34:35.902 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4197888 len 8192 2024-09-16T00:34:35.902 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 512 offset 4197888 len 16384 2024-09-16T00:34:35.902 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4197888 len 16384 2024-09-16T00:34:35.902 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 512 offset 4197888 len 16384 2024-09-16T00:34:35.902 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 512 offset 4197888 len 16384 2024-09-16T00:34:35.902 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4190720 len 1024 2024-09-16T00:34:35.902 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4190720 len 1024 2024-09-16T00:34:35.902 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4190720 len 1024 2024-09-16T00:34:35.902 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4190720 len 1024 2024-09-16T00:34:35.902 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4190720 len 2048 2024-09-16T00:34:35.902 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4190720 len 2048 2024-09-16T00:34:35.902 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4190720 len 2048 2024-09-16T00:34:35.902 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4190720 len 2048 2024-09-16T00:34:35.903 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4190720 len 4096 2024-09-16T00:34:35.976 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:34:37.016 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4190720 len 4096 2024-09-16T00:34:37.017 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4190720 len 4096 2024-09-16T00:34:37.017 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4190720 len 4096 2024-09-16T00:34:37.017 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4190720 len 8192 2024-09-16T00:34:37.017 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4190720 len 8192 2024-09-16T00:34:37.017 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4190720 len 8192 2024-09-16T00:34:37.017 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4190720 len 8192 2024-09-16T00:34:37.017 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4190720 len 16384 2024-09-16T00:34:37.017 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4190720 len 16384 2024-09-16T00:34:37.017 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4190720 len 16384 2024-09-16T00:34:37.017 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4190720 len 16384 2024-09-16T00:34:37.017 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4191232 len 1024 2024-09-16T00:34:37.017 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4191232 len 1024 2024-09-16T00:34:37.018 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4191232 len 1024 2024-09-16T00:34:37.018 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4191232 len 1024 2024-09-16T00:34:37.018 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4191232 len 2048 2024-09-16T00:34:37.018 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4191232 len 2048 2024-09-16T00:34:37.018 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4191232 len 2048 2024-09-16T00:34:37.018 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4191232 len 2048 2024-09-16T00:34:37.018 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4191232 len 4096 2024-09-16T00:34:37.018 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4191232 len 4096 2024-09-16T00:34:37.018 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4191232 len 4096 2024-09-16T00:34:37.018 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4191232 len 4096 2024-09-16T00:34:37.018 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4191232 len 8192 2024-09-16T00:34:37.018 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4191232 len 8192 2024-09-16T00:34:37.018 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4191232 len 8192 2024-09-16T00:34:37.018 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4191232 len 8192 2024-09-16T00:34:37.019 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4191232 len 16384 2024-09-16T00:34:37.019 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4191232 len 16384 2024-09-16T00:34:37.019 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4191232 len 16384 2024-09-16T00:34:37.019 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4191232 len 16384 2024-09-16T00:34:37.019 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4191744 len 1024 2024-09-16T00:34:37.019 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4191744 len 1024 2024-09-16T00:34:37.019 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4191744 len 1024 2024-09-16T00:34:37.019 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4191744 len 1024 2024-09-16T00:34:37.019 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4191744 len 2048 2024-09-16T00:34:37.019 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4191744 len 2048 2024-09-16T00:34:37.019 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4191744 len 2048 2024-09-16T00:34:37.019 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4191744 len 2048 2024-09-16T00:34:37.019 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4191744 len 4096 2024-09-16T00:34:37.019 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4191744 len 4096 2024-09-16T00:34:37.020 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4191744 len 4096 2024-09-16T00:34:37.020 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4191744 len 4096 2024-09-16T00:34:37.020 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4191744 len 8192 2024-09-16T00:34:37.020 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4191744 len 8192 2024-09-16T00:34:37.020 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4191744 len 8192 2024-09-16T00:34:37.020 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4191744 len 8192 2024-09-16T00:34:37.020 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4191744 len 16384 2024-09-16T00:34:37.020 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4191744 len 16384 2024-09-16T00:34:37.020 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4191744 len 16384 2024-09-16T00:34:37.021 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4191744 len 16384 2024-09-16T00:34:37.021 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4192256 len 1024 2024-09-16T00:34:37.021 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4192256 len 1024 2024-09-16T00:34:37.021 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4192256 len 1024 2024-09-16T00:34:37.021 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4192256 len 1024 2024-09-16T00:34:37.021 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4192256 len 2048 2024-09-16T00:34:37.021 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4192256 len 2048 2024-09-16T00:34:37.021 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4192256 len 2048 2024-09-16T00:34:37.021 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4192256 len 2048 2024-09-16T00:34:37.021 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4192256 len 4096 2024-09-16T00:34:37.022 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4192256 len 4096 2024-09-16T00:34:37.022 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4192256 len 4096 2024-09-16T00:34:37.022 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4192256 len 4096 2024-09-16T00:34:37.022 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4192256 len 8192 2024-09-16T00:34:37.022 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4192256 len 8192 2024-09-16T00:34:37.022 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4192256 len 8192 2024-09-16T00:34:37.022 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4192256 len 8192 2024-09-16T00:34:37.022 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4192256 len 16384 2024-09-16T00:34:37.022 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4192256 len 16384 2024-09-16T00:34:37.022 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4192256 len 16384 2024-09-16T00:34:37.023 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4192256 len 16384 2024-09-16T00:34:37.023 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4192768 len 1024 2024-09-16T00:34:37.023 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4192768 len 1024 2024-09-16T00:34:37.023 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4192768 len 1024 2024-09-16T00:34:37.023 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4192768 len 1024 2024-09-16T00:34:37.023 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4192768 len 2048 2024-09-16T00:34:37.023 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4192768 len 2048 2024-09-16T00:34:37.023 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4192768 len 2048 2024-09-16T00:34:37.023 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4192768 len 2048 2024-09-16T00:34:37.023 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4192768 len 4096 2024-09-16T00:34:37.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:37 smithi038 ceph-mon[30945]: pgmap v419: 129 pgs: 129 active+clean; 203 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 35 MiB/s rd, 948 KiB/s wr, 234 op/s 2024-09-16T00:34:37.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:37 smithi122 ceph-mon[35412]: pgmap v419: 129 pgs: 129 active+clean; 203 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 35 MiB/s rd, 948 KiB/s wr, 234 op/s 2024-09-16T00:34:37.793 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:37 smithi073 ceph-mon[35254]: pgmap v419: 129 pgs: 129 active+clean; 203 MiB data, 701 MiB used, 1.0 TiB / 1.0 TiB avail; 35 MiB/s rd, 948 KiB/s wr, 234 op/s 2024-09-16T00:34:38.014 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4192768 len 4096 2024-09-16T00:34:38.014 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4192768 len 4096 2024-09-16T00:34:38.014 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4192768 len 4096 2024-09-16T00:34:38.014 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4192768 len 8192 2024-09-16T00:34:38.014 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4192768 len 8192 2024-09-16T00:34:38.014 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4192768 len 8192 2024-09-16T00:34:38.015 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4192768 len 8192 2024-09-16T00:34:38.015 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4192768 len 16384 2024-09-16T00:34:38.015 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4192768 len 16384 2024-09-16T00:34:38.015 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4192768 len 16384 2024-09-16T00:34:38.015 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4192768 len 16384 2024-09-16T00:34:38.015 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4193280 len 1024 2024-09-16T00:34:38.015 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4193280 len 1024 2024-09-16T00:34:38.015 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4193280 len 1024 2024-09-16T00:34:38.015 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4193280 len 1024 2024-09-16T00:34:38.015 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4193280 len 2048 2024-09-16T00:34:38.015 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4193280 len 2048 2024-09-16T00:34:38.015 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4193280 len 2048 2024-09-16T00:34:38.015 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4193280 len 2048 2024-09-16T00:34:38.016 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4193280 len 4096 2024-09-16T00:34:38.016 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4193280 len 4096 2024-09-16T00:34:38.016 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4193280 len 4096 2024-09-16T00:34:38.016 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4193280 len 4096 2024-09-16T00:34:38.016 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4193280 len 8192 2024-09-16T00:34:38.016 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4193280 len 8192 2024-09-16T00:34:38.016 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4193280 len 8192 2024-09-16T00:34:38.016 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4193280 len 8192 2024-09-16T00:34:38.016 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4193280 len 16384 2024-09-16T00:34:38.016 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4193280 len 16384 2024-09-16T00:34:38.016 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4193280 len 16384 2024-09-16T00:34:38.016 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4193280 len 16384 2024-09-16T00:34:38.016 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4193792 len 1024 2024-09-16T00:34:38.016 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4193792 len 1024 2024-09-16T00:34:38.017 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4193792 len 1024 2024-09-16T00:34:38.017 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4193792 len 1024 2024-09-16T00:34:38.017 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4193792 len 2048 2024-09-16T00:34:38.017 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4193792 len 2048 2024-09-16T00:34:38.017 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4193792 len 2048 2024-09-16T00:34:38.017 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4193792 len 2048 2024-09-16T00:34:38.017 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4193792 len 4096 2024-09-16T00:34:38.017 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4193792 len 4096 2024-09-16T00:34:38.017 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4193792 len 4096 2024-09-16T00:34:38.017 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4193792 len 4096 2024-09-16T00:34:38.017 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4193792 len 8192 2024-09-16T00:34:38.017 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4193792 len 8192 2024-09-16T00:34:38.017 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4193792 len 8192 2024-09-16T00:34:38.017 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4193792 len 8192 2024-09-16T00:34:38.018 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4193792 len 16384 2024-09-16T00:34:38.018 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4193792 len 16384 2024-09-16T00:34:38.018 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4193792 len 16384 2024-09-16T00:34:38.018 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4193792 len 16384 2024-09-16T00:34:38.018 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4194304 len 1024 2024-09-16T00:34:38.018 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4194304 len 1024 2024-09-16T00:34:38.018 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4194304 len 1024 2024-09-16T00:34:38.018 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4194304 len 1024 2024-09-16T00:34:38.018 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4194304 len 2048 2024-09-16T00:34:38.018 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4194304 len 2048 2024-09-16T00:34:38.018 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4194304 len 2048 2024-09-16T00:34:38.018 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4194304 len 2048 2024-09-16T00:34:38.018 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4194304 len 4096 2024-09-16T00:34:38.018 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4194304 len 4096 2024-09-16T00:34:38.018 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4194304 len 4096 2024-09-16T00:34:38.019 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4194304 len 4096 2024-09-16T00:34:38.019 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4194304 len 8192 2024-09-16T00:34:38.019 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4194304 len 8192 2024-09-16T00:34:38.019 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4194304 len 8192 2024-09-16T00:34:38.019 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4194304 len 8192 2024-09-16T00:34:38.019 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4194304 len 16384 2024-09-16T00:34:38.019 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4194304 len 16384 2024-09-16T00:34:38.019 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4194304 len 16384 2024-09-16T00:34:38.019 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4194304 len 16384 2024-09-16T00:34:38.019 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4194816 len 1024 2024-09-16T00:34:38.020 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4194816 len 1024 2024-09-16T00:34:38.020 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4194816 len 1024 2024-09-16T00:34:38.020 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4194816 len 1024 2024-09-16T00:34:38.020 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4194816 len 2048 2024-09-16T00:34:38.020 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4194816 len 2048 2024-09-16T00:34:38.020 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4194816 len 2048 2024-09-16T00:34:38.020 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4194816 len 2048 2024-09-16T00:34:38.020 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4194816 len 4096 2024-09-16T00:34:38.020 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4194816 len 4096 2024-09-16T00:34:38.436 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4194816 len 4096 2024-09-16T00:34:38.436 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4194816 len 4096 2024-09-16T00:34:38.436 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4194816 len 8192 2024-09-16T00:34:38.436 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4194816 len 8192 2024-09-16T00:34:38.437 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4194816 len 8192 2024-09-16T00:34:38.437 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4194816 len 8192 2024-09-16T00:34:38.437 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4194816 len 16384 2024-09-16T00:34:38.437 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4194816 len 16384 2024-09-16T00:34:38.437 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4194816 len 16384 2024-09-16T00:34:38.437 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4194816 len 16384 2024-09-16T00:34:38.437 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4195328 len 1024 2024-09-16T00:34:38.437 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4195328 len 1024 2024-09-16T00:34:38.437 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4195328 len 1024 2024-09-16T00:34:38.437 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4195328 len 1024 2024-09-16T00:34:38.437 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4195328 len 2048 2024-09-16T00:34:38.437 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4195328 len 2048 2024-09-16T00:34:38.437 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4195328 len 2048 2024-09-16T00:34:38.438 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4195328 len 2048 2024-09-16T00:34:38.438 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4195328 len 4096 2024-09-16T00:34:38.438 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4195328 len 4096 2024-09-16T00:34:38.438 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4195328 len 4096 2024-09-16T00:34:38.438 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4195328 len 4096 2024-09-16T00:34:38.438 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4195328 len 8192 2024-09-16T00:34:38.438 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4195328 len 8192 2024-09-16T00:34:38.438 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4195328 len 8192 2024-09-16T00:34:38.438 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4195328 len 8192 2024-09-16T00:34:38.438 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4195328 len 16384 2024-09-16T00:34:38.438 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4195328 len 16384 2024-09-16T00:34:38.438 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4195328 len 16384 2024-09-16T00:34:38.438 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4195328 len 16384 2024-09-16T00:34:38.438 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4195840 len 1024 2024-09-16T00:34:38.439 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4195840 len 1024 2024-09-16T00:34:38.439 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4195840 len 1024 2024-09-16T00:34:38.439 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4195840 len 1024 2024-09-16T00:34:38.439 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4195840 len 2048 2024-09-16T00:34:38.439 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4195840 len 2048 2024-09-16T00:34:38.439 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4195840 len 2048 2024-09-16T00:34:38.439 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4195840 len 2048 2024-09-16T00:34:38.439 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4195840 len 4096 2024-09-16T00:34:38.439 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4195840 len 4096 2024-09-16T00:34:38.439 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4195840 len 4096 2024-09-16T00:34:38.439 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4195840 len 4096 2024-09-16T00:34:38.439 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4195840 len 8192 2024-09-16T00:34:38.439 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4195840 len 8192 2024-09-16T00:34:38.439 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4195840 len 8192 2024-09-16T00:34:38.440 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4195840 len 8192 2024-09-16T00:34:38.440 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4195840 len 16384 2024-09-16T00:34:38.440 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4195840 len 16384 2024-09-16T00:34:38.440 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4195840 len 16384 2024-09-16T00:34:38.440 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4195840 len 16384 2024-09-16T00:34:38.440 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4196352 len 1024 2024-09-16T00:34:38.440 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4196352 len 1024 2024-09-16T00:34:38.440 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4196352 len 1024 2024-09-16T00:34:38.440 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4196352 len 1024 2024-09-16T00:34:38.440 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4196352 len 2048 2024-09-16T00:34:38.440 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4196352 len 2048 2024-09-16T00:34:38.440 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4196352 len 2048 2024-09-16T00:34:38.440 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4196352 len 2048 2024-09-16T00:34:38.441 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4196352 len 4096 2024-09-16T00:34:38.441 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4196352 len 4096 2024-09-16T00:34:38.441 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4196352 len 4096 2024-09-16T00:34:38.441 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4196352 len 4096 2024-09-16T00:34:38.442 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4196352 len 8192 2024-09-16T00:34:38.442 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4196352 len 8192 2024-09-16T00:34:38.442 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4196352 len 8192 2024-09-16T00:34:38.442 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4196352 len 8192 2024-09-16T00:34:38.442 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4196352 len 16384 2024-09-16T00:34:38.442 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4196352 len 16384 2024-09-16T00:34:38.442 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4196352 len 16384 2024-09-16T00:34:38.443 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4196352 len 16384 2024-09-16T00:34:38.443 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4196864 len 1024 2024-09-16T00:34:38.443 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4196864 len 1024 2024-09-16T00:34:38.443 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4196864 len 1024 2024-09-16T00:34:38.443 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4196864 len 1024 2024-09-16T00:34:38.443 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4196864 len 2048 2024-09-16T00:34:38.443 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4196864 len 2048 2024-09-16T00:34:38.443 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4196864 len 2048 2024-09-16T00:34:38.443 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4196864 len 2048 2024-09-16T00:34:38.444 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4196864 len 4096 2024-09-16T00:34:38.444 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4196864 len 4096 2024-09-16T00:34:38.444 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4196864 len 4096 2024-09-16T00:34:38.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.167+0000 7f5e8660b640 1 -- 172.21.15.38:0/2337056636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e80102780 msgr2=0x7f5e801067d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:38.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.167+0000 7f5e8660b640 1 --2- 172.21.15.38:0/2337056636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e80102780 0x7f5e801067d0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f5e74009900 tx=0x7f5e7402a410 comp rx=0 tx=0).stop 2024-09-16T00:34:38.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.167+0000 7f5e8660b640 1 -- 172.21.15.38:0/2337056636 shutdown_connections 2024-09-16T00:34:38.565 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.167+0000 7f5e8660b640 1 --2- 172.21.15.38:0/2337056636 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e80106f00 0x7f5e8010ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.167+0000 7f5e8660b640 1 --2- 172.21.15.38:0/2337056636 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e80102780 0x7f5e801067d0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.167+0000 7f5e8660b640 1 --2- 172.21.15.38:0/2337056636 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e80101dd0 0x7f5e801021b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.167+0000 7f5e8660b640 1 -- 172.21.15.38:0/2337056636 >> 172.21.15.38:0/2337056636 conn(0x7f5e800fd890 msgr2=0x7f5e800ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:38.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.167+0000 7f5e8660b640 1 -- 172.21.15.38:0/2337056636 shutdown_connections 2024-09-16T00:34:38.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.167+0000 7f5e8660b640 1 -- 172.21.15.38:0/2337056636 wait complete. 2024-09-16T00:34:38.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.168+0000 7f5e8660b640 1 Processor -- start 2024-09-16T00:34:38.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.168+0000 7f5e8660b640 1 -- start start 2024-09-16T00:34:38.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.168+0000 7f5e8660b640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e80101dd0 0x7f5e800744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:38.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.168+0000 7f5e8660b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e80102780 0x7f5e80074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:38.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.168+0000 7f5e8660b640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e80106f00 0x7f5e80074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:38.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.168+0000 7f5e8660b640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5e80111f60 con 0x7f5e80102780 2024-09-16T00:34:38.566 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.168+0000 7f5e8660b640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5e80111de0 con 0x7f5e80106f00 2024-09-16T00:34:38.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.168+0000 7f5e8660b640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5e801120e0 con 0x7f5e80101dd0 2024-09-16T00:34:38.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.168+0000 7f5e7f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e80102780 0x7f5e80074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:38.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.168+0000 7f5e7f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e80102780 0x7f5e80074a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41040/0 (socket says 172.21.15.38:41040) 2024-09-16T00:34:38.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.168+0000 7f5e7ffff640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e80101dd0 0x7f5e800744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:38.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.168+0000 7f5e7f7fe640 1 -- 172.21.15.38:0/2407676946 learned_addr learned my addr 172.21.15.38:0/2407676946 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:34:38.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.168+0000 7f5e84ba8640 1 --2- 172.21.15.38:0/2407676946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e80106f00 0x7f5e80074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:38.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.169+0000 7f5e7f7fe640 1 -- 172.21.15.38:0/2407676946 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e80101dd0 msgr2=0x7f5e800744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:38.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.169+0000 7f5e7f7fe640 1 --2- 172.21.15.38:0/2407676946 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e80101dd0 0x7f5e800744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.169+0000 7f5e7f7fe640 1 -- 172.21.15.38:0/2407676946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e80106f00 msgr2=0x7f5e80074f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:38.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.169+0000 7f5e7f7fe640 1 --2- 172.21.15.38:0/2407676946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e80106f00 0x7f5e80074f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.169+0000 7f5e7f7fe640 1 -- 172.21.15.38:0/2407676946 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e801a69d0 con 0x7f5e80102780 2024-09-16T00:34:38.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.169+0000 7f5e84ba8640 1 --2- 172.21.15.38:0/2407676946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e80106f00 0x7f5e80074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:34:38.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.169+0000 7f5e7f7fe640 1 --2- 172.21.15.38:0/2407676946 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e80102780 0x7f5e80074a30 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5e74006010 tx=0x7f5e740042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:38.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.169+0000 7f5e7d7fa640 1 -- 172.21.15.38:0/2407676946 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7403f030 con 0x7f5e80102780 2024-09-16T00:34:38.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.169+0000 7f5e8660b640 1 -- 172.21.15.38:0/2407676946 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e801a6c60 con 0x7f5e80102780 2024-09-16T00:34:38.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.169+0000 7f5e7d7fa640 1 -- 172.21.15.38:0/2407676946 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e74003ce0 con 0x7f5e80102780 2024-09-16T00:34:38.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.169+0000 7f5e7d7fa640 1 -- 172.21.15.38:0/2407676946 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e74035070 con 0x7f5e80102780 2024-09-16T00:34:38.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.169+0000 7f5e8660b640 1 -- 172.21.15.38:0/2407676946 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e801a7190 con 0x7f5e80102780 2024-09-16T00:34:38.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.172+0000 7f5e7d7fa640 1 -- 172.21.15.38:0/2407676946 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f5e74042030 con 0x7f5e80102780 2024-09-16T00:34:38.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.172+0000 7f5e889e4740 1 Processor -- start 2024-09-16T00:34:38.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.172+0000 7f5e889e4740 1 -- start start 2024-09-16T00:34:38.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.172+0000 7f5e889e4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c889ddb60 0x562c889ddf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:38.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.172+0000 7f5e889e4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c889de480 0x562c889e8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:38.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.172+0000 7f5e889e4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c889e8d40 0x562c889eb130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:38.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.172+0000 7f5e889e4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562c88943280 con 0x562c889e8d40 2024-09-16T00:34:38.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.172+0000 7f5e889e4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562c888eab10 con 0x562c889de480 2024-09-16T00:34:38.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.172+0000 7f5e889e4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562c8893f240 con 0x562c889ddb60 2024-09-16T00:34:38.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.172+0000 7f5e7ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c889de480 0x562c889e8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:38.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.173+0000 7f5e7ffff640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c889de480 0x562c889e8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.38:47096/0 (socket says 172.21.15.38:47096) 2024-09-16T00:34:38.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.173+0000 7f5e7ffff640 1 -- 172.21.15.38:0/708136460 learned_addr learned my addr 172.21.15.38:0/708136460 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:34:38.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.173+0000 7f5e7f7fe640 1 --2- 172.21.15.38:0/708136460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c889e8d40 0x562c889eb130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:38.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.173+0000 7f5e84ba8640 1 --2- 172.21.15.38:0/708136460 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c889ddb60 0x562c889ddf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:38.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.173+0000 7f5e7ffff640 1 -- 172.21.15.38:0/708136460 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c889ddb60 msgr2=0x562c889ddf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:38.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.173+0000 7f5e7ffff640 1 --2- 172.21.15.38:0/708136460 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c889ddb60 0x562c889ddf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.173+0000 7f5e7ffff640 1 -- 172.21.15.38:0/708136460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c889e8d40 msgr2=0x562c889eb130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:38.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.173+0000 7f5e7ffff640 1 --2- 172.21.15.38:0/708136460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c889e8d40 0x562c889eb130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.173+0000 7f5e7ffff640 1 -- 172.21.15.38:0/708136460 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c888f71f0 con 0x562c889de480 2024-09-16T00:34:38.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.173+0000 7f5e7d7fa640 1 --2- 172.21.15.38:0/2407676946 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f5e680a32c0 0x7f5e680a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:38.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.173+0000 7f5e7ffff640 1 --2- 172.21.15.38:0/708136460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c889de480 0x562c889e8800 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5e7000bcd0 tx=0x7f5e700107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf7594e1232f9154 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:38.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.173+0000 7f5e84ba8640 1 --2- 172.21.15.38:0/2407676946 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f5e680a32c0 0x7f5e680a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:38.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.173+0000 7f5e7d7fa640 1 -- 172.21.15.38:0/2407676946 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f5e740efea0 con 0x7f5e80102780 2024-09-16T00:34:38.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.173+0000 7f5e667fc640 1 -- 172.21.15.38:0/708136460 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e70005220 con 0x562c889de480 2024-09-16T00:34:38.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.173+0000 7f5e667fc640 1 -- 172.21.15.38:0/708136460 <== mon.2 v2:172.21.15.73:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e70004c80 con 0x562c889de480 2024-09-16T00:34:38.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.173+0000 7f5e667fc640 1 -- 172.21.15.38:0/708136460 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e700117e0 con 0x562c889de480 2024-09-16T00:34:38.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.174+0000 7f5e889e4740 1 -- 172.21.15.38:0/708136460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c889de480 msgr2=0x562c889e8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:38.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.174+0000 7f5e889e4740 1 --2- 172.21.15.38:0/708136460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c889de480 0x562c889e8800 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f5e7000bcd0 tx=0x7f5e700107a0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.174+0000 7f5e84ba8640 1 --2- 172.21.15.38:0/2407676946 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f5e680a32c0 0x7f5e680a5780 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5e78005fd0 tx=0x7f5e78009500 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:38.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.174+0000 7f5e889e4740 1 -- 172.21.15.38:0/708136460 shutdown_connections 2024-09-16T00:34:38.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.174+0000 7f5e889e4740 1 --2- 172.21.15.38:0/708136460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c889e8d40 0x562c889eb130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.174+0000 7f5e889e4740 1 --2- 172.21.15.38:0/708136460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c889de480 0x562c889e8800 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.174+0000 7f5e889e4740 1 --2- 172.21.15.38:0/708136460 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c889ddb60 0x562c889ddf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.174+0000 7f5e889e4740 1 -- 172.21.15.38:0/708136460 >> 172.21.15.38:0/708136460 conn(0x562c889da0f0 msgr2=0x562c889da4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:38.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.174+0000 7f5e889e4740 1 -- 172.21.15.38:0/708136460 shutdown_connections 2024-09-16T00:34:38.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.174+0000 7f5e889e4740 1 -- 172.21.15.38:0/708136460 wait complete. 2024-09-16T00:34:38.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.175+0000 7f5e889e4740 1 -- start start 2024-09-16T00:34:38.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.175+0000 7f5e889e4740 1 Processor -- start 2024-09-16T00:34:38.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.175+0000 7f5e889e4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c889ddb60 0x562c889e5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:38.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.175+0000 7f5e889e4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c889de480 0x562c889e6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:38.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.175+0000 7f5e889e4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c889e8d40 0x562c889e6640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:38.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.175+0000 7f5e889e4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x562c889eb910 con 0x562c889ddb60 2024-09-16T00:34:38.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.175+0000 7f5e889e4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x562c888fdf10 con 0x562c889de480 2024-09-16T00:34:38.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.175+0000 7f5e889e4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x562c889eba90 con 0x562c889e8d40 2024-09-16T00:34:38.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.175+0000 7f5e889e4740 10 client.0 _ll_register_callbacks cb 0x562c888c6a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:38.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.175+0000 7f5e7ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c889ddb60 0x562c889e5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:38.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.175+0000 7f5e7f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c889de480 0x562c889e6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:38.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.175+0000 7f5e7ffff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c889ddb60 0x562c889e5bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41062/0 (socket says 172.21.15.38:41062) 2024-09-16T00:34:38.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.175+0000 7f5e7ffff640 1 -- 172.21.15.38:0/3672471788 learned_addr learned my addr 172.21.15.38:0/3672471788 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:34:38.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.175+0000 7f5e84ba8640 1 --2- 172.21.15.38:0/3672471788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c889e8d40 0x562c889e6640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:38.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.176+0000 7f5e7f7fe640 1 -- 172.21.15.38:0/3672471788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c889e8d40 msgr2=0x562c889e6640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:38.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.176+0000 7f5e7f7fe640 1 --2- 172.21.15.38:0/3672471788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c889e8d40 0x562c889e6640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.176+0000 7f5e7f7fe640 1 -- 172.21.15.38:0/3672471788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c889ddb60 msgr2=0x562c889e5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:38.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.176+0000 7f5e7f7fe640 1 --2- 172.21.15.38:0/3672471788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c889ddb60 0x562c889e5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.176+0000 7f5e7f7fe640 1 -- 172.21.15.38:0/3672471788 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c886b46a0 con 0x562c889de480 2024-09-16T00:34:38.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.176+0000 7f5e7ffff640 1 --2- 172.21.15.38:0/3672471788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c889ddb60 0x562c889e5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:34:38.572 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.176+0000 7f5e84ba8640 1 --2- 172.21.15.38:0/3672471788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c889e8d40 0x562c889e6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:34:38.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.176+0000 7f5e7f7fe640 1 --2- 172.21.15.38:0/3672471788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c889de480 0x562c889e6100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5e740ed670 tx=0x7f5e74031a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3c4349b3da53afa server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:38.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.176+0000 7f5e5f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-09-16T00:34:38.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.176+0000 7f5e5f7fe640 1 -- 172.21.15.38:0/3672471788 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e74049070 con 0x562c889de480 2024-09-16T00:34:38.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.176+0000 7f5e5f7fe640 1 -- 172.21.15.38:0/3672471788 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f5e740b87b0 con 0x562c889de480 2024-09-16T00:34:38.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.176+0000 7f5e889e4740 1 -- 172.21.15.38:0/3672471788 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e70004c80 con 0x562c889de480 2024-09-16T00:34:38.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.176+0000 7f5e5f7fe640 1 -- 172.21.15.38:0/3672471788 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5e740b83e0 con 0x562c889de480 2024-09-16T00:34:38.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.176+0000 7f5e889e4740 10 client.25388 fetch_fsmap learned FSMap version 26 2024-09-16T00:34:38.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.176+0000 7f5e889e4740 1 -- 172.21.15.38:0/3672471788 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562c889f74f0 con 0x562c889de480 2024-09-16T00:34:38.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.176+0000 7f5e5f7fe640 1 -- 172.21.15.38:0/3672471788 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e70004c80 con 0x562c889de480 2024-09-16T00:34:38.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.177+0000 7f5e5f7fe640 1 -- 172.21.15.38:0/3672471788 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5e740c0120 con 0x562c889de480 2024-09-16T00:34:38.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.177+0000 7f5e889e4740 10 client.25388 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:34:38.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.177+0000 7f5e889e4740 10 client.25388 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:34:38.573 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.177+0000 7f5e889e4740 20 client.25388 populate_metadata read hostname 'smithi038' 2024-09-16T00:34:38.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.177+0000 7f5e889e4740 1 --2- 172.21.15.38:0/3672471788 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x562c889f8890 0x562c889fad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:38.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.177+0000 7f5e889e4740 4 client.25388 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:34:38.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.177+0000 7f5e889e4740 1 -- 172.21.15.38:0/3672471788 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562c889507f0 con 0x562c889f8890 2024-09-16T00:34:38.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.177+0000 7f5e7ffff640 1 --2- 172.21.15.38:0/3672471788 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x562c889f8890 0x562c889fad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:38.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.177+0000 7f5e7ffff640 1 --2- 172.21.15.38:0/3672471788 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x562c889f8890 0x562c889fad00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7790457963764e4 server_cookie=f562aff86e97dbd in_seq=0 out_seq=0 2024-09-16T00:34:38.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.177+0000 7f5e5f7fe640 10 client.25388 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:34:38.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.178+0000 7f5e5f7fe640 1 -- 172.21.15.38:0/3672471788 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x562c889507f0 con 0x562c889f8890 2024-09-16T00:34:38.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.178+0000 7f5e5f7fe640 10 client.25388 handle_command_reply: tid=0 2024-09-16T00:34:38.574 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.178+0000 7f5e889e4740 1 client.25388 shutdown 2024-09-16T00:34:38.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.178+0000 7f5e889e4740 1 -- 172.21.15.38:0/3672471788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c889de480 msgr2=0x562c889e6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:38.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.178+0000 7f5e889e4740 1 --2- 172.21.15.38:0/3672471788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c889de480 0x562c889e6100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5e740ed670 tx=0x7f5e74031a60 comp rx=0 tx=0).stop 2024-09-16T00:34:38.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.178+0000 7f5e889e4740 1 -- 172.21.15.38:0/3672471788 shutdown_connections 2024-09-16T00:34:38.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.178+0000 7f5e889e4740 1 --2- 172.21.15.38:0/3672471788 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c889e8d40 0x562c889e6640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.178+0000 7f5e889e4740 1 --2- 172.21.15.38:0/3672471788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c889de480 0x562c889e6100 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.178+0000 7f5e889e4740 1 --2- 172.21.15.38:0/3672471788 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x562c889f8890 0x562c889fad00 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.178+0000 7f5e889e4740 1 --2- 172.21.15.38:0/3672471788 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c889ddb60 0x562c889e5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.178+0000 7f5e5f7fe640 0 client.25388 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:34:38.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.178+0000 7f5e889e4740 1 -- 172.21.15.38:0/3672471788 >> 172.21.15.38:0/3672471788 conn(0x562c889da0f0 msgr2=0x562c889dfdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:38.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.178+0000 7f5e889e4740 1 -- 172.21.15.38:0/3672471788 shutdown_connections 2024-09-16T00:34:38.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.178+0000 7f5e889e4740 1 -- 172.21.15.38:0/3672471788 wait complete. 2024-09-16T00:34:38.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.178+0000 7f5e889e4740 20 client.25388 trim_cache size 0 max 16384 2024-09-16T00:34:38.575 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.196+0000 7f5e889e4740 1 Processor -- start 2024-09-16T00:34:38.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.196+0000 7f5e889e4740 1 -- start start 2024-09-16T00:34:38.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.196+0000 7f5e889e4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c889ddb60 0x562c889dfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:38.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.196+0000 7f5e889e4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c889de480 0x562c88a00570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:38.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.196+0000 7f5e889e4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c889e8d40 0x562c88a00ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:38.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.197+0000 7f5e889e4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5e801b3730 con 0x562c889de480 2024-09-16T00:34:38.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.197+0000 7f5e889e4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5e801b1000 con 0x562c889ddb60 2024-09-16T00:34:38.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.197+0000 7f5e889e4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5e801aee50 con 0x562c889e8d40 2024-09-16T00:34:38.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.197+0000 7f5e7f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c889de480 0x562c88a00570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:38.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.197+0000 7f5e7f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c889de480 0x562c88a00570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41064/0 (socket says 172.21.15.38:41064) 2024-09-16T00:34:38.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.197+0000 7f5e7f7fe640 1 -- 172.21.15.38:0/2818038044 learned_addr learned my addr 172.21.15.38:0/2818038044 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:34:38.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.197+0000 7f5e7ffff640 1 --2- 172.21.15.38:0/2818038044 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c889ddb60 0x562c889dfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:38.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.197+0000 7f5e84ba8640 1 --2- 172.21.15.38:0/2818038044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c889e8d40 0x562c88a00ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:38.576 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.197+0000 7f5e7f7fe640 1 -- 172.21.15.38:0/2818038044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c889e8d40 msgr2=0x562c88a00ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:38.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.197+0000 7f5e7f7fe640 1 --2- 172.21.15.38:0/2818038044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c889e8d40 0x562c88a00ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.197+0000 7f5e7f7fe640 1 -- 172.21.15.38:0/2818038044 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c889ddb60 msgr2=0x562c889dfcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:38.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.197+0000 7f5e7f7fe640 1 --2- 172.21.15.38:0/2818038044 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c889ddb60 0x562c889dfcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.197+0000 7f5e7f7fe640 1 -- 172.21.15.38:0/2818038044 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562c889e09d0 con 0x562c889de480 2024-09-16T00:34:38.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.197+0000 7f5e7f7fe640 1 --2- 172.21.15.38:0/2818038044 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c889de480 0x562c88a00570 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f5e740f00f0 tx=0x7f5e740ba650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49ae32d4f05f02bf server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:38.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.197+0000 7f5e65ffb640 1 -- 172.21.15.38:0/2818038044 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e74049070 con 0x562c889de480 2024-09-16T00:34:38.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.197+0000 7f5e65ffb640 1 -- 172.21.15.38:0/2818038044 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e740bc840 con 0x562c889de480 2024-09-16T00:34:38.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.197+0000 7f5e65ffb640 1 -- 172.21.15.38:0/2818038044 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e740bbaa0 con 0x562c889de480 2024-09-16T00:34:38.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.198+0000 7f5e889e4740 1 -- 172.21.15.38:0/2818038044 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c889de480 msgr2=0x562c88a00570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:38.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.198+0000 7f5e889e4740 1 --2- 172.21.15.38:0/2818038044 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c889de480 0x562c88a00570 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f5e740f00f0 tx=0x7f5e740ba650 comp rx=0 tx=0).stop 2024-09-16T00:34:38.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.198+0000 7f5e889e4740 1 -- 172.21.15.38:0/2818038044 shutdown_connections 2024-09-16T00:34:38.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.198+0000 7f5e889e4740 1 --2- 172.21.15.38:0/2818038044 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c889e8d40 0x562c88a00ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.577 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.198+0000 7f5e889e4740 1 --2- 172.21.15.38:0/2818038044 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c889de480 0x562c88a00570 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.198+0000 7f5e889e4740 1 --2- 172.21.15.38:0/2818038044 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c889ddb60 0x562c889dfcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.198+0000 7f5e889e4740 1 -- 172.21.15.38:0/2818038044 >> 172.21.15.38:0/2818038044 conn(0x562c889da0f0 msgr2=0x562c889df5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:38.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.198+0000 7f5e889e4740 1 -- 172.21.15.38:0/2818038044 shutdown_connections 2024-09-16T00:34:38.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.198+0000 7f5e889e4740 1 -- 172.21.15.38:0/2818038044 wait complete. 2024-09-16T00:34:38.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.198+0000 7f5e889e4740 1 -- start start 2024-09-16T00:34:38.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.198+0000 7f5e889e4740 1 Processor -- start 2024-09-16T00:34:38.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.199+0000 7f5e889e4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c889ddb60 0x562c889ecbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:38.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.199+0000 7f5e889e4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c889de480 0x562c889ed0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:38.578 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.199+0000 7f5e889e4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c889e8d40 0x562c889ed630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:38.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.199+0000 7f5e889e4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f5e801af4c0 con 0x562c889ddb60 2024-09-16T00:34:38.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.199+0000 7f5e889e4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f5e801b0930 con 0x562c889de480 2024-09-16T00:34:38.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.199+0000 7f5e889e4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f5e801afb90 con 0x562c889e8d40 2024-09-16T00:34:38.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.199+0000 7f5e889e4740 10 client.0 _ll_register_callbacks cb 0x7f5e740314b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:38.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.199+0000 7f5e7f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c889de480 0x562c889ed0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:38.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.199+0000 7f5e7f7fe640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c889de480 0x562c889ed0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.38:47110/0 (socket says 172.21.15.38:47110) 2024-09-16T00:34:38.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.199+0000 7f5e7f7fe640 1 -- 172.21.15.38:0/2372531497 learned_addr learned my addr 172.21.15.38:0/2372531497 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:34:38.579 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.199+0000 7f5e84ba8640 1 --2- 172.21.15.38:0/2372531497 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c889e8d40 0x562c889ed630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:38.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.199+0000 7f5e7ffff640 1 --2- 172.21.15.38:0/2372531497 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c889ddb60 0x562c889ecbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:38.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.199+0000 7f5e7f7fe640 1 -- 172.21.15.38:0/2372531497 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c889e8d40 msgr2=0x562c889ed630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:38.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.199+0000 7f5e7f7fe640 1 --2- 172.21.15.38:0/2372531497 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c889e8d40 0x562c889ed630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.199+0000 7f5e7f7fe640 1 -- 172.21.15.38:0/2372531497 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c889ddb60 msgr2=0x562c889ecbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:38.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.199+0000 7f5e7f7fe640 1 --2- 172.21.15.38:0/2372531497 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c889ddb60 0x562c889ecbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.199+0000 7f5e7f7fe640 1 -- 172.21.15.38:0/2372531497 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562c88a626a0 con 0x562c889de480 2024-09-16T00:34:38.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.200+0000 7f5e7f7fe640 1 --2- 172.21.15.38:0/2372531497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c889de480 0x562c889ed0f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5e740bc0d0 tx=0x7f5e740b8a60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec8edb29fb759ced server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:38.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.200+0000 7f5e5f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-09-16T00:34:38.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.200+0000 7f5e5f7fe640 1 -- 172.21.15.38:0/2372531497 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e74049070 con 0x562c889de480 2024-09-16T00:34:38.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.200+0000 7f5e5f7fe640 1 -- 172.21.15.38:0/2372531497 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f5e740bc5a0 con 0x562c889de480 2024-09-16T00:34:38.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.200+0000 7f5e889e4740 1 -- 172.21.15.38:0/2372531497 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e740bc840 con 0x562c889de480 2024-09-16T00:34:38.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.200+0000 7f5e5f7fe640 1 -- 172.21.15.38:0/2372531497 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f5e740bbc80 con 0x562c889de480 2024-09-16T00:34:38.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.200+0000 7f5e889e4740 10 client.25394 fetch_fsmap learned FSMap version 26 2024-09-16T00:34:38.580 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.200+0000 7f5e889e4740 1 -- 172.21.15.38:0/2372531497 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x562c88a626a0 con 0x562c889de480 2024-09-16T00:34:38.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.200+0000 7f5e5f7fe640 1 -- 172.21.15.38:0/2372531497 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f5e740bc840 con 0x562c889de480 2024-09-16T00:34:38.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.200+0000 7f5e5f7fe640 1 -- 172.21.15.38:0/2372531497 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f5e740f30d0 con 0x562c889de480 2024-09-16T00:34:38.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.201+0000 7f5e889e4740 10 client.25394 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:34:38.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.201+0000 7f5e889e4740 10 client.25394 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:34:38.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.201+0000 7f5e889e4740 20 client.25394 populate_metadata read hostname 'smithi038' 2024-09-16T00:34:38.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.201+0000 7f5e889e4740 1 --2- 172.21.15.38:0/2372531497 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x562c88a63c40 0x562c88a66030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:38.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.201+0000 7f5e889e4740 4 client.25394 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-16T00:34:38.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.201+0000 7f5e889e4740 1 -- 172.21.15.38:0/2372531497 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562c8894ad00 con 0x562c88a63c40 2024-09-16T00:34:38.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.201+0000 7f5e7ffff640 1 --2- 172.21.15.38:0/2372531497 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x562c88a63c40 0x562c88a66030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:38.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.201+0000 7f5e7ffff640 1 --2- 172.21.15.38:0/2372531497 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x562c88a63c40 0x562c88a66030 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d0942b981d26c860 server_cookie=ed851520424b4105 in_seq=0 out_seq=0 2024-09-16T00:34:38.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.201+0000 7f5e5f7fe640 10 client.25394 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:34:38.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.202+0000 7f5e5f7fe640 1 -- 172.21.15.38:0/2372531497 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562c8894ad00 con 0x562c88a63c40 2024-09-16T00:34:38.581 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.202+0000 7f5e5f7fe640 10 client.25394 handle_command_reply: tid=0 2024-09-16T00:34:38.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.202+0000 7f5e889e4740 1 client.25394 shutdown 2024-09-16T00:34:38.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.203+0000 7f5e889e4740 1 -- 172.21.15.38:0/2372531497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c889de480 msgr2=0x562c889ed0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:38.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.203+0000 7f5e889e4740 1 --2- 172.21.15.38:0/2372531497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c889de480 0x562c889ed0f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5e740bc0d0 tx=0x7f5e740b8a60 comp rx=0 tx=0).stop 2024-09-16T00:34:38.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.203+0000 7f5e889e4740 1 -- 172.21.15.38:0/2372531497 shutdown_connections 2024-09-16T00:34:38.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.203+0000 7f5e889e4740 1 --2- 172.21.15.38:0/2372531497 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x562c889e8d40 0x562c889ed630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.203+0000 7f5e889e4740 1 --2- 172.21.15.38:0/2372531497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562c889de480 0x562c889ed0f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.203+0000 7f5e889e4740 1 --2- 172.21.15.38:0/2372531497 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x562c88a63c40 0x562c88a66030 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.203+0000 7f5e889e4740 1 --2- 172.21.15.38:0/2372531497 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x562c889ddb60 0x562c889ecbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.203+0000 7f5e889e4740 1 -- 172.21.15.38:0/2372531497 >> 172.21.15.38:0/2372531497 conn(0x562c889da0f0 msgr2=0x562c889d9270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:38.582 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.203+0000 7f5e5f7fe640 0 client.25394 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:34:38.583 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-16T00:34:38.583 INFO:teuthology.orchestra.run.smithi038.stdout: "return_code": 0, 2024-09-16T00:34:38.583 INFO:teuthology.orchestra.run.smithi038.stdout: "scrub_tag": "a503e155-5582-4fc4-a951-029d376a5a1e", 2024-09-16T00:34:38.583 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": "asynchronous" 2024-09-16T00:34:38.583 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-16T00:34:38.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.203+0000 7f5e889e4740 1 -- 172.21.15.38:0/2372531497 shutdown_connections 2024-09-16T00:34:38.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.203+0000 7f5e889e4740 1 -- 172.21.15.38:0/2372531497 wait complete. 2024-09-16T00:34:38.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.203+0000 7f5e889e4740 20 client.25394 trim_cache size 0 max 16384 2024-09-16T00:34:38.583 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.203+0000 7f5e8660b640 1 -- 172.21.15.38:0/2407676946 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f5e680a32c0 msgr2=0x7f5e680a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:38.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.203+0000 7f5e8660b640 1 --2- 172.21.15.38:0/2407676946 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f5e680a32c0 0x7f5e680a5780 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f5e78005fd0 tx=0x7f5e78009500 comp rx=0 tx=0).stop 2024-09-16T00:34:38.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.203+0000 7f5e8660b640 1 -- 172.21.15.38:0/2407676946 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e80102780 msgr2=0x7f5e80074a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:38.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.203+0000 7f5e8660b640 1 --2- 172.21.15.38:0/2407676946 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e80102780 0x7f5e80074a30 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5e74006010 tx=0x7f5e740042f0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.204+0000 7f5e8660b640 1 -- 172.21.15.38:0/2407676946 shutdown_connections 2024-09-16T00:34:38.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.204+0000 7f5e8660b640 1 --2- 172.21.15.38:0/2407676946 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f5e680a32c0 0x7f5e680a5780 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.204+0000 7f5e8660b640 1 --2- 172.21.15.38:0/2407676946 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e80106f00 0x7f5e80074f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.204+0000 7f5e8660b640 1 --2- 172.21.15.38:0/2407676946 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f5e80102780 0x7f5e80074a30 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.204+0000 7f5e8660b640 1 --2- 172.21.15.38:0/2407676946 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f5e80101dd0 0x7f5e800744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:38.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.204+0000 7f5e8660b640 1 -- 172.21.15.38:0/2407676946 >> 172.21.15.38:0/2407676946 conn(0x7f5e800fd890 msgr2=0x7f5e800fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:38.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.204+0000 7f5e8660b640 1 -- 172.21.15.38:0/2407676946 shutdown_connections 2024-09-16T00:34:38.584 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:38.204+0000 7f5e8660b640 1 -- 172.21.15.38:0/2407676946 wait complete. 2024-09-16T00:34:39.162 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4196864 len 4096 2024-09-16T00:34:39.162 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4196864 len 8192 2024-09-16T00:34:39.162 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4196864 len 8192 2024-09-16T00:34:39.162 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4196864 len 8192 2024-09-16T00:34:39.162 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4196864 len 8192 2024-09-16T00:34:39.162 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4196864 len 16384 2024-09-16T00:34:39.162 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4196864 len 16384 2024-09-16T00:34:39.162 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4196864 len 16384 2024-09-16T00:34:39.162 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4196864 len 16384 2024-09-16T00:34:39.162 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4197376 len 1024 2024-09-16T00:34:39.163 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4197376 len 1024 2024-09-16T00:34:39.163 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4197376 len 1024 2024-09-16T00:34:39.163 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4197376 len 1024 2024-09-16T00:34:39.163 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4197376 len 2048 2024-09-16T00:34:39.163 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4197376 len 2048 2024-09-16T00:34:39.163 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4197376 len 2048 2024-09-16T00:34:39.163 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4197376 len 2048 2024-09-16T00:34:39.163 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4197376 len 4096 2024-09-16T00:34:39.163 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4197376 len 4096 2024-09-16T00:34:39.163 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4197376 len 4096 2024-09-16T00:34:39.163 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4197376 len 4096 2024-09-16T00:34:39.163 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4197376 len 8192 2024-09-16T00:34:39.163 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4197376 len 8192 2024-09-16T00:34:39.163 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4197376 len 8192 2024-09-16T00:34:39.164 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4197376 len 8192 2024-09-16T00:34:39.164 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4197376 len 16384 2024-09-16T00:34:39.164 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4197376 len 16384 2024-09-16T00:34:39.164 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4197376 len 16384 2024-09-16T00:34:39.164 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4197376 len 16384 2024-09-16T00:34:39.164 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4197888 len 1024 2024-09-16T00:34:39.164 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4197888 len 1024 2024-09-16T00:34:39.164 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4197888 len 1024 2024-09-16T00:34:39.164 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4197888 len 1024 2024-09-16T00:34:39.164 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4197888 len 2048 2024-09-16T00:34:39.164 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4197888 len 2048 2024-09-16T00:34:39.164 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4197888 len 2048 2024-09-16T00:34:39.164 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4197888 len 2048 2024-09-16T00:34:39.164 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4197888 len 4096 2024-09-16T00:34:39.165 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4197888 len 4096 2024-09-16T00:34:39.165 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4197888 len 4096 2024-09-16T00:34:39.165 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4197888 len 4096 2024-09-16T00:34:39.165 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4197888 len 8192 2024-09-16T00:34:39.165 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4197888 len 8192 2024-09-16T00:34:39.165 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4197888 len 8192 2024-09-16T00:34:39.165 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4197888 len 8192 2024-09-16T00:34:39.165 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1024 offset 4197888 len 16384 2024-09-16T00:34:39.165 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4197888 len 16384 2024-09-16T00:34:39.165 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1024 offset 4197888 len 16384 2024-09-16T00:34:39.165 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1024 offset 4197888 len 16384 2024-09-16T00:34:39.165 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4190720 len 1024 2024-09-16T00:34:39.165 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4190720 len 1024 2024-09-16T00:34:39.165 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4190720 len 1024 2024-09-16T00:34:39.166 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4190720 len 1024 2024-09-16T00:34:39.166 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4190720 len 2048 2024-09-16T00:34:39.166 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4190720 len 2048 2024-09-16T00:34:39.166 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4190720 len 2048 2024-09-16T00:34:39.166 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4190720 len 2048 2024-09-16T00:34:39.166 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4190720 len 4096 2024-09-16T00:34:39.166 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4190720 len 4096 2024-09-16T00:34:39.166 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4190720 len 4096 2024-09-16T00:34:39.166 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4190720 len 4096 2024-09-16T00:34:39.166 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4190720 len 8192 2024-09-16T00:34:39.166 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4190720 len 8192 2024-09-16T00:34:39.166 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4190720 len 8192 2024-09-16T00:34:39.166 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4190720 len 8192 2024-09-16T00:34:39.166 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4190720 len 16384 2024-09-16T00:34:39.167 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4190720 len 16384 2024-09-16T00:34:39.167 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4190720 len 16384 2024-09-16T00:34:39.167 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4190720 len 16384 2024-09-16T00:34:39.167 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4191232 len 1024 2024-09-16T00:34:39.167 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4191232 len 1024 2024-09-16T00:34:39.167 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4191232 len 1024 2024-09-16T00:34:39.167 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4191232 len 1024 2024-09-16T00:34:39.167 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4191232 len 2048 2024-09-16T00:34:39.167 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4191232 len 2048 2024-09-16T00:34:39.167 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4191232 len 2048 2024-09-16T00:34:39.167 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4191232 len 2048 2024-09-16T00:34:39.167 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4191232 len 4096 2024-09-16T00:34:39.167 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4191232 len 4096 2024-09-16T00:34:39.167 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4191232 len 4096 2024-09-16T00:34:39.167 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4191232 len 4096 2024-09-16T00:34:39.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:39 smithi038 ceph-mon[30945]: pgmap v420: 129 pgs: 129 active+clean; 239 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 43 MiB/s rd, 980 KiB/s wr, 247 op/s 2024-09-16T00:34:39.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:39 smithi038 ceph-mon[30945]: scrub queued for path: / 2024-09-16T00:34:39.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:39 smithi038 ceph-mon[30945]: scrub summary: idle+waiting paths [/] 2024-09-16T00:34:39.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:39 smithi038 ceph-mon[30945]: scrub summary: active paths [/] 2024-09-16T00:34:39.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:39 smithi038 ceph-mon[30945]: scrub summary: idle+waiting paths [/] 2024-09-16T00:34:39.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:39 smithi122 ceph-mon[35412]: pgmap v420: 129 pgs: 129 active+clean; 239 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 43 MiB/s rd, 980 KiB/s wr, 247 op/s 2024-09-16T00:34:39.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:39 smithi122 ceph-mon[35412]: scrub queued for path: / 2024-09-16T00:34:39.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:39 smithi122 ceph-mon[35412]: scrub summary: idle+waiting paths [/] 2024-09-16T00:34:39.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:39 smithi122 ceph-mon[35412]: scrub summary: active paths [/] 2024-09-16T00:34:39.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:39 smithi122 ceph-mon[35412]: scrub summary: idle+waiting paths [/] 2024-09-16T00:34:39.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:39 smithi073 ceph-mon[35254]: pgmap v420: 129 pgs: 129 active+clean; 239 MiB data, 728 MiB used, 1.0 TiB / 1.0 TiB avail; 43 MiB/s rd, 980 KiB/s wr, 247 op/s 2024-09-16T00:34:39.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:39 smithi073 ceph-mon[35254]: scrub queued for path: / 2024-09-16T00:34:39.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:39 smithi073 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-09-16T00:34:39.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:39 smithi073 ceph-mon[35254]: scrub summary: active paths [/] 2024-09-16T00:34:39.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:39 smithi073 ceph-mon[35254]: scrub summary: idle+waiting paths [/] 2024-09-16T00:34:40.342 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:34:40.469 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4191232 len 8192 2024-09-16T00:34:40.469 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4191232 len 8192 2024-09-16T00:34:40.469 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4191232 len 8192 2024-09-16T00:34:40.469 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4191232 len 8192 2024-09-16T00:34:40.469 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4191232 len 16384 2024-09-16T00:34:40.469 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4191232 len 16384 2024-09-16T00:34:40.470 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4191232 len 16384 2024-09-16T00:34:40.470 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4191232 len 16384 2024-09-16T00:34:40.470 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4191744 len 1024 2024-09-16T00:34:40.470 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4191744 len 1024 2024-09-16T00:34:40.470 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4191744 len 1024 2024-09-16T00:34:40.470 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4191744 len 1024 2024-09-16T00:34:40.470 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4191744 len 2048 2024-09-16T00:34:40.470 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4191744 len 2048 2024-09-16T00:34:40.470 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4191744 len 2048 2024-09-16T00:34:40.470 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4191744 len 2048 2024-09-16T00:34:40.470 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4191744 len 4096 2024-09-16T00:34:40.470 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4191744 len 4096 2024-09-16T00:34:40.470 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4191744 len 4096 2024-09-16T00:34:40.470 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4191744 len 4096 2024-09-16T00:34:40.471 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4191744 len 8192 2024-09-16T00:34:40.471 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4191744 len 8192 2024-09-16T00:34:40.471 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4191744 len 8192 2024-09-16T00:34:40.471 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4191744 len 8192 2024-09-16T00:34:40.471 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4191744 len 16384 2024-09-16T00:34:40.471 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4191744 len 16384 2024-09-16T00:34:40.471 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4191744 len 16384 2024-09-16T00:34:40.471 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4191744 len 16384 2024-09-16T00:34:40.471 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4192256 len 1024 2024-09-16T00:34:40.471 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4192256 len 1024 2024-09-16T00:34:40.471 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4192256 len 1024 2024-09-16T00:34:40.471 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4192256 len 1024 2024-09-16T00:34:40.471 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4192256 len 2048 2024-09-16T00:34:40.471 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4192256 len 2048 2024-09-16T00:34:40.472 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4192256 len 2048 2024-09-16T00:34:40.472 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4192256 len 2048 2024-09-16T00:34:40.472 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4192256 len 4096 2024-09-16T00:34:40.472 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4192256 len 4096 2024-09-16T00:34:40.472 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4192256 len 4096 2024-09-16T00:34:40.472 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4192256 len 4096 2024-09-16T00:34:40.472 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4192256 len 8192 2024-09-16T00:34:40.472 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4192256 len 8192 2024-09-16T00:34:40.472 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4192256 len 8192 2024-09-16T00:34:40.472 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4192256 len 8192 2024-09-16T00:34:40.472 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4192256 len 16384 2024-09-16T00:34:40.472 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4192256 len 16384 2024-09-16T00:34:40.472 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4192256 len 16384 2024-09-16T00:34:40.473 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4192256 len 16384 2024-09-16T00:34:40.473 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4192768 len 1024 2024-09-16T00:34:40.473 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4192768 len 1024 2024-09-16T00:34:40.473 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4192768 len 1024 2024-09-16T00:34:40.473 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4192768 len 1024 2024-09-16T00:34:40.473 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4192768 len 2048 2024-09-16T00:34:40.473 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4192768 len 2048 2024-09-16T00:34:40.473 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4192768 len 2048 2024-09-16T00:34:40.473 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4192768 len 2048 2024-09-16T00:34:40.473 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4192768 len 4096 2024-09-16T00:34:40.473 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4192768 len 4096 2024-09-16T00:34:40.474 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4192768 len 4096 2024-09-16T00:34:40.474 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4192768 len 4096 2024-09-16T00:34:40.474 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4192768 len 8192 2024-09-16T00:34:40.474 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4192768 len 8192 2024-09-16T00:34:40.474 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4192768 len 8192 2024-09-16T00:34:40.474 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4192768 len 8192 2024-09-16T00:34:40.474 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4192768 len 16384 2024-09-16T00:34:40.474 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4192768 len 16384 2024-09-16T00:34:40.474 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4192768 len 16384 2024-09-16T00:34:40.474 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4192768 len 16384 2024-09-16T00:34:40.474 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4193280 len 1024 2024-09-16T00:34:40.474 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4193280 len 1024 2024-09-16T00:34:40.474 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4193280 len 1024 2024-09-16T00:34:40.475 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4193280 len 1024 2024-09-16T00:34:40.475 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4193280 len 2048 2024-09-16T00:34:40.475 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4193280 len 2048 2024-09-16T00:34:40.475 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4193280 len 2048 2024-09-16T00:34:40.475 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4193280 len 2048 2024-09-16T00:34:40.475 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4193280 len 4096 2024-09-16T00:34:40.475 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4193280 len 4096 2024-09-16T00:34:40.475 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4193280 len 4096 2024-09-16T00:34:40.475 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4193280 len 4096 2024-09-16T00:34:40.475 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4193280 len 8192 2024-09-16T00:34:40.686 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:34:41.162 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4193280 len 8192 2024-09-16T00:34:41.162 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4193280 len 8192 2024-09-16T00:34:41.162 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4193280 len 8192 2024-09-16T00:34:41.162 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4193280 len 16384 2024-09-16T00:34:41.162 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4193280 len 16384 2024-09-16T00:34:41.162 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4193280 len 16384 2024-09-16T00:34:41.162 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4193280 len 16384 2024-09-16T00:34:41.162 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4193792 len 1024 2024-09-16T00:34:41.162 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4193792 len 1024 2024-09-16T00:34:41.162 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4193792 len 1024 2024-09-16T00:34:41.162 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4193792 len 1024 2024-09-16T00:34:41.163 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4193792 len 2048 2024-09-16T00:34:41.163 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4193792 len 2048 2024-09-16T00:34:41.163 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4193792 len 2048 2024-09-16T00:34:41.163 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4193792 len 2048 2024-09-16T00:34:41.163 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4193792 len 4096 2024-09-16T00:34:41.163 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4193792 len 4096 2024-09-16T00:34:41.163 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4193792 len 4096 2024-09-16T00:34:41.163 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4193792 len 4096 2024-09-16T00:34:41.163 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4193792 len 8192 2024-09-16T00:34:41.163 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4193792 len 8192 2024-09-16T00:34:41.163 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4193792 len 8192 2024-09-16T00:34:41.164 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4193792 len 8192 2024-09-16T00:34:41.164 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4193792 len 16384 2024-09-16T00:34:41.164 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4193792 len 16384 2024-09-16T00:34:41.164 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4193792 len 16384 2024-09-16T00:34:41.164 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4193792 len 16384 2024-09-16T00:34:41.165 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4194304 len 1024 2024-09-16T00:34:41.165 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4194304 len 1024 2024-09-16T00:34:41.165 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4194304 len 1024 2024-09-16T00:34:41.165 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4194304 len 1024 2024-09-16T00:34:41.166 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4194304 len 2048 2024-09-16T00:34:41.166 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4194304 len 2048 2024-09-16T00:34:41.166 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4194304 len 2048 2024-09-16T00:34:41.166 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4194304 len 2048 2024-09-16T00:34:41.166 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4194304 len 4096 2024-09-16T00:34:41.166 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4194304 len 4096 2024-09-16T00:34:41.166 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4194304 len 4096 2024-09-16T00:34:41.167 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4194304 len 4096 2024-09-16T00:34:41.167 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4194304 len 8192 2024-09-16T00:34:41.167 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4194304 len 8192 2024-09-16T00:34:41.167 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4194304 len 8192 2024-09-16T00:34:41.167 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4194304 len 8192 2024-09-16T00:34:41.167 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4194304 len 16384 2024-09-16T00:34:41.167 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4194304 len 16384 2024-09-16T00:34:41.167 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4194304 len 16384 2024-09-16T00:34:41.167 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4194304 len 16384 2024-09-16T00:34:41.167 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4194816 len 1024 2024-09-16T00:34:41.167 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4194816 len 1024 2024-09-16T00:34:41.168 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4194816 len 1024 2024-09-16T00:34:41.168 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4194816 len 1024 2024-09-16T00:34:41.168 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4194816 len 2048 2024-09-16T00:34:41.168 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4194816 len 2048 2024-09-16T00:34:41.168 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4194816 len 2048 2024-09-16T00:34:41.168 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4194816 len 2048 2024-09-16T00:34:41.169 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4194816 len 4096 2024-09-16T00:34:41.169 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4194816 len 4096 2024-09-16T00:34:41.169 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4194816 len 4096 2024-09-16T00:34:41.169 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4194816 len 4096 2024-09-16T00:34:41.169 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4194816 len 8192 2024-09-16T00:34:41.169 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4194816 len 8192 2024-09-16T00:34:41.169 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4194816 len 8192 2024-09-16T00:34:41.169 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4194816 len 8192 2024-09-16T00:34:41.169 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4194816 len 16384 2024-09-16T00:34:41.169 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4194816 len 16384 2024-09-16T00:34:41.169 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4194816 len 16384 2024-09-16T00:34:41.169 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4194816 len 16384 2024-09-16T00:34:41.170 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4195328 len 1024 2024-09-16T00:34:41.170 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4195328 len 1024 2024-09-16T00:34:41.170 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4195328 len 1024 2024-09-16T00:34:41.170 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4195328 len 1024 2024-09-16T00:34:41.170 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4195328 len 2048 2024-09-16T00:34:41.170 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4195328 len 2048 2024-09-16T00:34:41.170 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4195328 len 2048 2024-09-16T00:34:41.170 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4195328 len 2048 2024-09-16T00:34:41.170 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4195328 len 4096 2024-09-16T00:34:41.170 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4195328 len 4096 2024-09-16T00:34:41.170 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4195328 len 4096 2024-09-16T00:34:41.171 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4195328 len 4096 2024-09-16T00:34:41.171 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4195328 len 8192 2024-09-16T00:34:41.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.221+0000 7fd7a5891640 1 -- 172.21.15.38:0/2254984288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7a01037a0 msgr2=0x7fd7a010bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:41.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.221+0000 7fd7a5891640 1 --2- 172.21.15.38:0/2254984288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7a01037a0 0x7fd7a010bc00 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fd794009900 tx=0x7fd79402a3f0 comp rx=0 tx=0).stop 2024-09-16T00:34:41.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.221+0000 7fd7a5891640 1 -- 172.21.15.38:0/2254984288 shutdown_connections 2024-09-16T00:34:41.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.221+0000 7fd7a5891640 1 --2- 172.21.15.38:0/2254984288 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7a010c2f0 0x7fd7a010e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:41.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.221+0000 7fd7a5891640 1 --2- 172.21.15.38:0/2254984288 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7a01037a0 0x7fd7a010bc00 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:41.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.221+0000 7fd7a5891640 1 --2- 172.21.15.38:0/2254984288 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7a0102df0 0x7fd7a01031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:41.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.221+0000 7fd7a5891640 1 -- 172.21.15.38:0/2254984288 >> 172.21.15.38:0/2254984288 conn(0x7fd7a00fc7d0 msgr2=0x7fd7a00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:41.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.221+0000 7fd7a5891640 1 -- 172.21.15.38:0/2254984288 shutdown_connections 2024-09-16T00:34:41.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.221+0000 7fd7a5891640 1 -- 172.21.15.38:0/2254984288 wait complete. 2024-09-16T00:34:41.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.221+0000 7fd7a5891640 1 Processor -- start 2024-09-16T00:34:41.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.222+0000 7fd7a5891640 1 -- start start 2024-09-16T00:34:41.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.222+0000 7fd7a5891640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7a0102df0 0x7fd7a01a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:41.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.222+0000 7fd7a5891640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7a01037a0 0x7fd7a01a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:41.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.222+0000 7fd7a5891640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7a010c2f0 0x7fd7a010ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:41.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.222+0000 7fd7a5891640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a01120d0 con 0x7fd7a010c2f0 2024-09-16T00:34:41.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.222+0000 7fd7a5891640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a0111f50 con 0x7fd7a01037a0 2024-09-16T00:34:41.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.222+0000 7fd7a5891640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a0112250 con 0x7fd7a0102df0 2024-09-16T00:34:41.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.222+0000 7fd79f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7a010c2f0 0x7fd7a010ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:41.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.222+0000 7fd79e7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7a01037a0 0x7fd7a01a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:41.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.222+0000 7fd79effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7a0102df0 0x7fd7a01a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:41.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.222+0000 7fd79e7fc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7a01037a0 0x7fd7a01a3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.38:47124/0 (socket says 172.21.15.38:47124) 2024-09-16T00:34:41.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.222+0000 7fd79f7fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7a010c2f0 0x7fd7a010ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41074/0 (socket says 172.21.15.38:41074) 2024-09-16T00:34:41.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.222+0000 7fd79e7fc640 1 -- 172.21.15.38:0/455613645 learned_addr learned my addr 172.21.15.38:0/455613645 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:34:41.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.222+0000 7fd79effd640 1 -- 172.21.15.38:0/455613645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7a01037a0 msgr2=0x7fd7a01a3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:41.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.222+0000 7fd79effd640 1 --2- 172.21.15.38:0/455613645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7a01037a0 0x7fd7a01a3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:41.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.222+0000 7fd79effd640 1 -- 172.21.15.38:0/455613645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7a010c2f0 msgr2=0x7fd7a010ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:41.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.222+0000 7fd79effd640 1 --2- 172.21.15.38:0/455613645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7a010c2f0 0x7fd7a010ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:41.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.222+0000 7fd79effd640 1 -- 172.21.15.38:0/455613645 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7a010b550 con 0x7fd7a0102df0 2024-09-16T00:34:41.243 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.222+0000 7fd79f7fe640 1 --2- 172.21.15.38:0/455613645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7a010c2f0 0x7fd7a010ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:34:41.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.222+0000 7fd79e7fc640 1 --2- 172.21.15.38:0/455613645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7a01037a0 0x7fd7a01a3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:34:41.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.223+0000 7fd79effd640 1 --2- 172.21.15.38:0/455613645 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7a0102df0 0x7fd7a01a3860 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd79000e9f0 tx=0x7fd79000eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:41.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.223+0000 7fd7a488f640 1 -- 172.21.15.38:0/455613645 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd79000ccc0 con 0x7fd7a0102df0 2024-09-16T00:34:41.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.223+0000 7fd7a488f640 1 -- 172.21.15.38:0/455613645 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd790005020 con 0x7fd7a0102df0 2024-09-16T00:34:41.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.223+0000 7fd7a5891640 1 -- 172.21.15.38:0/455613645 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7a010b840 con 0x7fd7a0102df0 2024-09-16T00:34:41.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.223+0000 7fd7a488f640 1 -- 172.21.15.38:0/455613645 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7900049e0 con 0x7fd7a0102df0 2024-09-16T00:34:41.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.223+0000 7fd7a5891640 1 -- 172.21.15.38:0/455613645 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7a01ab790 con 0x7fd7a0102df0 2024-09-16T00:34:41.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.225+0000 7fd7a488f640 1 -- 172.21.15.38:0/455613645 <== mon.1 v2:172.21.15.122:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fd790005560 con 0x7fd7a0102df0 2024-09-16T00:34:41.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.226+0000 7fd7a7c6a740 1 Processor -- start 2024-09-16T00:34:41.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.226+0000 7fd7a7c6a740 1 -- start start 2024-09-16T00:34:41.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.226+0000 7fd7a488f640 1 --2- 172.21.15.38:0/455613645 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd7880a32c0 0x7fd7880a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:41.244 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.226+0000 7fd7a7c6a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc847b44a0 0x55dc847b4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:41.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.226+0000 7fd7a7c6a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc847b4dc0 0x55dc847bf140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:41.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.226+0000 7fd7a7c6a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc847bf680 0x55dc847c1a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:41.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.226+0000 7fd7a7c6a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55dc8471a2e0 con 0x55dc847b4dc0 2024-09-16T00:34:41.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.226+0000 7fd7a7c6a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55dc846c1ce0 con 0x55dc847bf680 2024-09-16T00:34:41.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.226+0000 7fd7a7c6a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55dc847162a0 con 0x55dc847b44a0 2024-09-16T00:34:41.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.226+0000 7fd79f7fe640 1 --2- 172.21.15.38:0/455613645 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd7880a32c0 0x7fd7880a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:41.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.226+0000 7fd79e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc847b4dc0 0x55dc847bf140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:41.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.226+0000 7fd79effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc847b44a0 0x55dc847b4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:41.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.226+0000 7fd79e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc847b4dc0 0x55dc847bf140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41088/0 (socket says 172.21.15.38:41088) 2024-09-16T00:34:41.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.226+0000 7fd79effd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc847b44a0 0x55dc847b4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.38:59792/0 (socket says 172.21.15.38:59792) 2024-09-16T00:34:41.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.226+0000 7fd79effd640 1 -- 172.21.15.38:0/2172290291 learned_addr learned my addr 172.21.15.38:0/2172290291 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:34:41.245 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.226+0000 7fd7a488f640 1 -- 172.21.15.38:0/455613645 <== mon.1 v2:172.21.15.122:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7fd7900cfbe0 con 0x7fd7a0102df0 2024-09-16T00:34:41.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.226+0000 7fd79f7fe640 1 --2- 172.21.15.38:0/2172290291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc847bf680 0x55dc847c1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:41.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.226+0000 7fd79effd640 1 -- 172.21.15.38:0/2172290291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc847bf680 msgr2=0x55dc847c1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:41.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.226+0000 7fd79effd640 1 --2- 172.21.15.38:0/2172290291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc847bf680 0x55dc847c1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:41.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.226+0000 7fd79effd640 1 -- 172.21.15.38:0/2172290291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc847b4dc0 msgr2=0x55dc847bf140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:41.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.226+0000 7fd79effd640 1 --2- 172.21.15.38:0/2172290291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc847b4dc0 0x55dc847bf140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:41.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.226+0000 7fd79effd640 1 -- 172.21.15.38:0/2172290291 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc846a6440 con 0x55dc847b44a0 2024-09-16T00:34:41.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.227+0000 7fd79effd640 1 --2- 172.21.15.38:0/2172290291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc847b44a0 0x55dc847b4880 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd79009bdb0 tx=0x7fd79001b7d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27d3a2704f4cf086 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:41.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.227+0000 7fd79f7fe640 1 --2- 172.21.15.38:0/455613645 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd7880a32c0 0x7fd7880a5780 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd7980099e0 tx=0x7fd7980135e0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:41.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.227+0000 7fd785ffb640 1 -- 172.21.15.38:0/2172290291 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd790086040 con 0x55dc847b44a0 2024-09-16T00:34:41.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.227+0000 7fd785ffb640 1 -- 172.21.15.38:0/2172290291 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd79009c520 con 0x55dc847b44a0 2024-09-16T00:34:41.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.227+0000 7fd785ffb640 1 -- 172.21.15.38:0/2172290291 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7900a01d0 con 0x55dc847b44a0 2024-09-16T00:34:41.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.228+0000 7fd7a7c6a740 1 -- 172.21.15.38:0/2172290291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc847b44a0 msgr2=0x55dc847b4880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:41.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.228+0000 7fd7a7c6a740 1 --2- 172.21.15.38:0/2172290291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc847b44a0 0x55dc847b4880 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd79009bdb0 tx=0x7fd79001b7d0 comp rx=0 tx=0).stop 2024-09-16T00:34:41.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.228+0000 7fd7a7c6a740 1 -- 172.21.15.38:0/2172290291 shutdown_connections 2024-09-16T00:34:41.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.228+0000 7fd7a7c6a740 1 --2- 172.21.15.38:0/2172290291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc847bf680 0x55dc847c1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:41.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.228+0000 7fd7a7c6a740 1 --2- 172.21.15.38:0/2172290291 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc847b4dc0 0x55dc847bf140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:41.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.228+0000 7fd7a7c6a740 1 --2- 172.21.15.38:0/2172290291 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc847b44a0 0x55dc847b4880 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:41.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.228+0000 7fd7a7c6a740 1 -- 172.21.15.38:0/2172290291 >> 172.21.15.38:0/2172290291 conn(0x55dc847b0a30 msgr2=0x55dc847b0e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:41.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.228+0000 7fd7a7c6a740 1 -- 172.21.15.38:0/2172290291 shutdown_connections 2024-09-16T00:34:41.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.228+0000 7fd7a7c6a740 1 -- 172.21.15.38:0/2172290291 wait complete. 2024-09-16T00:34:41.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.228+0000 7fd7a7c6a740 1 -- start start 2024-09-16T00:34:41.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.228+0000 7fd7a7c6a740 1 Processor -- start 2024-09-16T00:34:41.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.228+0000 7fd7a7c6a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc847b44a0 0x55dc847bf020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:41.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.229+0000 7fd7a7c6a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc847b4dc0 0x55dc847bc2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:41.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.229+0000 7fd7a7c6a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc847bf680 0x55dc847bc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:41.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.229+0000 7fd7a7c6a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55dc847c2250 con 0x55dc847b44a0 2024-09-16T00:34:41.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.229+0000 7fd7a7c6a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55dc846d5050 con 0x55dc847b4dc0 2024-09-16T00:34:41.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.229+0000 7fd7a7c6a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55dc847c23d0 con 0x55dc847bf680 2024-09-16T00:34:41.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.229+0000 7fd7a7c6a740 10 client.0 _ll_register_callbacks cb 0x55dc84715160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:41.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.229+0000 7fd79effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc847b44a0 0x55dc847bf020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:41.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.229+0000 7fd79effd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc847b44a0 0x55dc847bf020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41098/0 (socket says 172.21.15.38:41098) 2024-09-16T00:34:41.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.229+0000 7fd79effd640 1 -- 172.21.15.38:0/4076176632 learned_addr learned my addr 172.21.15.38:0/4076176632 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:34:41.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.229+0000 7fd79e7fc640 1 --2- 172.21.15.38:0/4076176632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc847b4dc0 0x55dc847bc2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:41.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.229+0000 7fd79f7fe640 1 --2- 172.21.15.38:0/4076176632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc847bf680 0x55dc847bc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:41.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.229+0000 7fd79effd640 1 -- 172.21.15.38:0/4076176632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc847bf680 msgr2=0x55dc847bc870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:41.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.229+0000 7fd79effd640 1 --2- 172.21.15.38:0/4076176632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc847bf680 0x55dc847bc870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:41.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.229+0000 7fd79effd640 1 -- 172.21.15.38:0/4076176632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc847b4dc0 msgr2=0x55dc847bc2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:41.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.229+0000 7fd79effd640 1 --2- 172.21.15.38:0/4076176632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc847b4dc0 0x55dc847bc2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:41.248 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.229+0000 7fd79effd640 1 -- 172.21.15.38:0/4076176632 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc8448b3c0 con 0x55dc847b44a0 2024-09-16T00:34:41.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.229+0000 7fd79effd640 1 --2- 172.21.15.38:0/4076176632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc847b44a0 0x55dc847bf020 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd79009bd60 tx=0x7fd7900a3880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54cf99fbbfcfd470 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:41.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.229+0000 7fd77effd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:34:41.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.229+0000 7fd77effd640 1 -- 172.21.15.38:0/4076176632 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd790086050 con 0x55dc847b44a0 2024-09-16T00:34:41.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.229+0000 7fd77effd640 1 -- 172.21.15.38:0/4076176632 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7fd7900a29c0 con 0x55dc847b44a0 2024-09-16T00:34:41.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.229+0000 7fd7a7c6a740 1 -- 172.21.15.38:0/4076176632 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd79009c520 con 0x55dc847b44a0 2024-09-16T00:34:41.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.229+0000 7fd77effd640 1 -- 172.21.15.38:0/4076176632 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd7900a2bc0 con 0x55dc847b44a0 2024-09-16T00:34:41.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.229+0000 7fd7a7c6a740 10 client.15801 fetch_fsmap learned FSMap version 26 2024-09-16T00:34:41.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.230+0000 7fd7a7c6a740 1 -- 172.21.15.38:0/4076176632 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dc847cdc60 con 0x55dc847b44a0 2024-09-16T00:34:41.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.230+0000 7fd77effd640 1 -- 172.21.15.38:0/4076176632 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd79009c520 con 0x55dc847b44a0 2024-09-16T00:34:41.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.230+0000 7fd77effd640 1 -- 172.21.15.38:0/4076176632 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd79009c080 con 0x55dc847b44a0 2024-09-16T00:34:41.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.230+0000 7fd7a7c6a740 10 client.15801 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:34:41.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.230+0000 7fd7a7c6a740 10 client.15801 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:34:41.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.230+0000 7fd7a7c6a740 20 client.15801 populate_metadata read hostname 'smithi038' 2024-09-16T00:34:41.249 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.230+0000 7fd7a7c6a740 1 --2- 172.21.15.38:0/4076176632 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55dc847cede0 0x55dc847d11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:41.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.230+0000 7fd7a7c6a740 4 client.15801 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:34:41.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.230+0000 7fd7a7c6a740 1 -- 172.21.15.38:0/4076176632 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dc84727850 con 0x55dc847cede0 2024-09-16T00:34:41.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.230+0000 7fd79e7fc640 1 --2- 172.21.15.38:0/4076176632 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55dc847cede0 0x55dc847d11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:41.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.230+0000 7fd79e7fc640 1 --2- 172.21.15.38:0/4076176632 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55dc847cede0 0x55dc847d11d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=588271f0d604a40d server_cookie=a14b43a3a360ea66 in_seq=0 out_seq=0 2024-09-16T00:34:41.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.230+0000 7fd77effd640 10 client.15801 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:34:41.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.231+0000 7fd77effd640 1 -- 172.21.15.38:0/4076176632 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55dc84727850 con 0x55dc847cede0 2024-09-16T00:34:41.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.231+0000 7fd77effd640 10 client.15801 handle_command_reply: tid=0 2024-09-16T00:34:41.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.231+0000 7fd7a7c6a740 1 client.15801 shutdown 2024-09-16T00:34:41.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.231+0000 7fd7a7c6a740 1 -- 172.21.15.38:0/4076176632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc847b44a0 msgr2=0x55dc847bf020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:41.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.231+0000 7fd7a7c6a740 1 --2- 172.21.15.38:0/4076176632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc847b44a0 0x55dc847bf020 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fd79009bd60 tx=0x7fd7900a3880 comp rx=0 tx=0).stop 2024-09-16T00:34:41.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.231+0000 7fd7a7c6a740 1 -- 172.21.15.38:0/4076176632 shutdown_connections 2024-09-16T00:34:41.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.231+0000 7fd7a7c6a740 1 --2- 172.21.15.38:0/4076176632 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc847bf680 0x55dc847bc870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:41.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.231+0000 7fd7a7c6a740 1 --2- 172.21.15.38:0/4076176632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc847b4dc0 0x55dc847bc2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:41.250 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.231+0000 7fd7a7c6a740 1 --2- 172.21.15.38:0/4076176632 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55dc847cede0 0x55dc847d11d0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:41.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.231+0000 7fd7a7c6a740 1 --2- 172.21.15.38:0/4076176632 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc847b44a0 0x55dc847bf020 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:41.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.231+0000 7fd7a7c6a740 1 -- 172.21.15.38:0/4076176632 >> 172.21.15.38:0/4076176632 conn(0x55dc847b0a30 msgr2=0x55dc847b6760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:41.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.231+0000 7fd77effd640 0 client.15801 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:34:41.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.232+0000 7fd7a7c6a740 1 -- 172.21.15.38:0/4076176632 shutdown_connections 2024-09-16T00:34:41.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.232+0000 7fd7a7c6a740 1 -- 172.21.15.38:0/4076176632 wait complete. 2024-09-16T00:34:41.251 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.232+0000 7fd7a7c6a740 20 client.15801 trim_cache size 0 max 16384 2024-09-16T00:34:41.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.250+0000 7fd7a7c6a740 1 Processor -- start 2024-09-16T00:34:41.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.250+0000 7fd7a7c6a740 1 -- start start 2024-09-16T00:34:41.253 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.251+0000 7fd7a7c6a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc847b44a0 0x55dc847afa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:41.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.251+0000 7fd7a7c6a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc847b4dc0 0x55dc847d4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:41.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.251+0000 7fd7a7c6a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc847bf680 0x55dc847d4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:41.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.251+0000 7fd7a7c6a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a01b7bb0 con 0x55dc847bf680 2024-09-16T00:34:41.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.251+0000 7fd79effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc847b44a0 0x55dc847afa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:41.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.251+0000 7fd79effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc847b44a0 0x55dc847afa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.38:47140/0 (socket says 172.21.15.38:47140) 2024-09-16T00:34:41.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.251+0000 7fd79effd640 1 -- 172.21.15.38:0/370532053 learned_addr learned my addr 172.21.15.38:0/370532053 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:34:41.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.251+0000 7fd7a7c6a740 1 -- 172.21.15.38:0/370532053 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a01b5480 con 0x55dc847b44a0 2024-09-16T00:34:41.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.251+0000 7fd7a7c6a740 1 -- 172.21.15.38:0/370532053 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a01b32d0 con 0x55dc847b4dc0 2024-09-16T00:34:41.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.251+0000 7fd79f7fe640 1 --2- 172.21.15.38:0/370532053 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc847bf680 0x55dc847d4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:41.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.251+0000 7fd79e7fc640 1 --2- 172.21.15.38:0/370532053 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc847b4dc0 0x55dc847d4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:41.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.251+0000 7fd79e7fc640 1 -- 172.21.15.38:0/370532053 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc847b44a0 msgr2=0x55dc847afa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:41.254 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.251+0000 7fd79e7fc640 1 --2- 172.21.15.38:0/370532053 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc847b44a0 0x55dc847afa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:41.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.251+0000 7fd79e7fc640 1 -- 172.21.15.38:0/370532053 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc847bf680 msgr2=0x55dc847d4740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:41.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.251+0000 7fd79e7fc640 1 --2- 172.21.15.38:0/370532053 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc847bf680 0x55dc847d4740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:41.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.251+0000 7fd79e7fc640 1 -- 172.21.15.38:0/370532053 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dc847b16c0 con 0x55dc847b4dc0 2024-09-16T00:34:41.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.251+0000 7fd79effd640 1 --2- 172.21.15.38:0/370532053 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc847b44a0 0x55dc847afa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:34:41.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.251+0000 7fd79f7fe640 1 --2- 172.21.15.38:0/370532053 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc847bf680 0x55dc847d4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:34:41.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.252+0000 7fd79e7fc640 1 --2- 172.21.15.38:0/370532053 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc847b4dc0 0x55dc847d4200 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd794009230 tx=0x7fd794004310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b06ac21da033f72d server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:41.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.252+0000 7fd7857fa640 1 -- 172.21.15.38:0/370532053 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd794035040 con 0x55dc847b4dc0 2024-09-16T00:34:41.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.252+0000 7fd7857fa640 1 -- 172.21.15.38:0/370532053 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd794031bd0 con 0x55dc847b4dc0 2024-09-16T00:34:41.255 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.252+0000 7fd7857fa640 1 -- 172.21.15.38:0/370532053 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd794003cd0 con 0x55dc847b4dc0 2024-09-16T00:34:41.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.253+0000 7fd7a7c6a740 1 -- 172.21.15.38:0/370532053 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc847b4dc0 msgr2=0x55dc847d4200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:41.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.253+0000 7fd7a7c6a740 1 --2- 172.21.15.38:0/370532053 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc847b4dc0 0x55dc847d4200 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd794009230 tx=0x7fd794004310 comp rx=0 tx=0).stop 2024-09-16T00:34:41.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.253+0000 7fd7a7c6a740 1 -- 172.21.15.38:0/370532053 shutdown_connections 2024-09-16T00:34:41.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.253+0000 7fd7a7c6a740 1 --2- 172.21.15.38:0/370532053 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc847bf680 0x55dc847d4740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:41.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.253+0000 7fd7a7c6a740 1 --2- 172.21.15.38:0/370532053 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc847b4dc0 0x55dc847d4200 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:41.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.253+0000 7fd7a7c6a740 1 --2- 172.21.15.38:0/370532053 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc847b44a0 0x55dc847afa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:41.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.253+0000 7fd7a7c6a740 1 -- 172.21.15.38:0/370532053 >> 172.21.15.38:0/370532053 conn(0x55dc847b0a30 msgr2=0x55dc847af350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:41.256 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.253+0000 7fd7a7c6a740 1 -- 172.21.15.38:0/370532053 shutdown_connections 2024-09-16T00:34:41.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.253+0000 7fd7a7c6a740 1 -- 172.21.15.38:0/370532053 wait complete. 2024-09-16T00:34:41.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.253+0000 7fd7a7c6a740 1 -- start start 2024-09-16T00:34:41.257 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.253+0000 7fd7a7c6a740 1 Processor -- start 2024-09-16T00:34:41.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.254+0000 7fd7a7c6a740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc847b44a0 0x55dc847d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:41.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.254+0000 7fd7a7c6a740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc847b4dc0 0x55dc847d4670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:41.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.254+0000 7fd7a7c6a740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc847bf680 0x55dc84830d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:41.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.254+0000 7fd7a7c6a740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a01b3940 con 0x55dc847b4dc0 2024-09-16T00:34:41.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.254+0000 7fd7a7c6a740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a01b4db0 con 0x55dc847b44a0 2024-09-16T00:34:41.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.254+0000 7fd7a7c6a740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fd7a01b4010 con 0x55dc847bf680 2024-09-16T00:34:41.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.254+0000 7fd7a7c6a740 10 client.0 _ll_register_callbacks cb 0x7fd790005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:41.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.254+0000 7fd79e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc847b4dc0 0x55dc847d4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:41.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.254+0000 7fd79f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc847bf680 0x55dc84830d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:41.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.254+0000 7fd79effd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc847b44a0 0x55dc847d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:41.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.254+0000 7fd79f7fe640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc847bf680 0x55dc84830d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.38:59824/0 (socket says 172.21.15.38:59824) 2024-09-16T00:34:41.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.254+0000 7fd79e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc847b4dc0 0x55dc847d4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41106/0 (socket says 172.21.15.38:41106) 2024-09-16T00:34:41.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.254+0000 7fd79f7fe640 1 -- 172.21.15.38:0/3773172143 learned_addr learned my addr 172.21.15.38:0/3773172143 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:34:41.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.254+0000 7fd79effd640 1 -- 172.21.15.38:0/3773172143 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc847bf680 msgr2=0x55dc84830d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:41.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.254+0000 7fd79effd640 1 --2- 172.21.15.38:0/3773172143 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc847bf680 0x55dc84830d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:41.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.254+0000 7fd79effd640 1 -- 172.21.15.38:0/3773172143 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc847b4dc0 msgr2=0x55dc847d4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:41.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.254+0000 7fd79effd640 1 --2- 172.21.15.38:0/3773172143 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc847b4dc0 0x55dc847d4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:41.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.254+0000 7fd79effd640 1 -- 172.21.15.38:0/3773172143 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dc848313d0 con 0x55dc847b44a0 2024-09-16T00:34:41.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.254+0000 7fd79f7fe640 1 --2- 172.21.15.38:0/3773172143 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc847bf680 0x55dc84830d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:34:41.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.254+0000 7fd79e7fc640 1 --2- 172.21.15.38:0/3773172143 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc847b4dc0 0x55dc847d4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:34:41.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.254+0000 7fd79effd640 1 --2- 172.21.15.38:0/3773172143 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc847b44a0 0x55dc847d4130 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd7900987f0 tx=0x7fd7900a28f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9518c5f6c67bd51 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:41.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.254+0000 7fd77effd640 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2024-09-16T00:34:41.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.254+0000 7fd77effd640 1 -- 172.21.15.38:0/3773172143 <== mon.2 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd790086070 con 0x55dc847b44a0 2024-09-16T00:34:41.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.255+0000 7fd77effd640 1 -- 172.21.15.38:0/3773172143 <== mon.2 v2:172.21.15.73:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7fd7900a4600 con 0x55dc847b44a0 2024-09-16T00:34:41.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.255+0000 7fd7a7c6a740 1 -- 172.21.15.38:0/3773172143 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd794031bd0 con 0x55dc847b44a0 2024-09-16T00:34:41.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.255+0000 7fd77effd640 1 -- 172.21.15.38:0/3773172143 <== mon.2 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fd79001c040 con 0x55dc847b44a0 2024-09-16T00:34:41.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.255+0000 7fd7a7c6a740 10 client.25412 fetch_fsmap learned FSMap version 26 2024-09-16T00:34:41.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.255+0000 7fd7a7c6a740 1 -- 172.21.15.38:0/3773172143 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55dc848313d0 con 0x55dc847b44a0 2024-09-16T00:34:41.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.255+0000 7fd77effd640 1 -- 172.21.15.38:0/3773172143 <== mon.2 v2:172.21.15.73:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fd794031bd0 con 0x55dc847b44a0 2024-09-16T00:34:41.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.255+0000 7fd77effd640 1 -- 172.21.15.38:0/3773172143 <== mon.2 v2:172.21.15.73:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fd7900a4ce0 con 0x55dc847b44a0 2024-09-16T00:34:41.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.255+0000 7fd7a7c6a740 10 client.25412 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:34:41.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.255+0000 7fd7a7c6a740 10 client.25412 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:34:41.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.255+0000 7fd7a7c6a740 20 client.25412 populate_metadata read hostname 'smithi038' 2024-09-16T00:34:41.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.255+0000 7fd7a7c6a740 1 --2- 172.21.15.38:0/3773172143 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55dc84839c80 0x55dc8483c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:41.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.255+0000 7fd7a7c6a740 4 client.25412 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:34:41.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.255+0000 7fd7a7c6a740 1 -- 172.21.15.38:0/3773172143 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55dc84721d60 con 0x55dc84839c80 2024-09-16T00:34:41.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.255+0000 7fd79e7fc640 1 --2- 172.21.15.38:0/3773172143 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55dc84839c80 0x55dc8483c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:41.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.256+0000 7fd79e7fc640 1 --2- 172.21.15.38:0/3773172143 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55dc84839c80 0x55dc8483c070 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb7ab1e9da85ef52 server_cookie=30dd4921dc643f40 in_seq=0 out_seq=0 2024-09-16T00:34:41.262 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.256+0000 7fd77effd640 10 client.25412 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:34:41.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.257+0000 7fd77effd640 1 -- 172.21.15.38:0/3773172143 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55dc84721d60 con 0x55dc84839c80 2024-09-16T00:34:41.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.257+0000 7fd77effd640 10 client.25412 handle_command_reply: tid=0 2024-09-16T00:34:41.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.257+0000 7fd7a7c6a740 1 client.25412 shutdown 2024-09-16T00:34:41.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.257+0000 7fd7a7c6a740 1 -- 172.21.15.38:0/3773172143 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc847b44a0 msgr2=0x55dc847d4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:41.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.257+0000 7fd7a7c6a740 1 --2- 172.21.15.38:0/3773172143 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc847b44a0 0x55dc847d4130 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd7900987f0 tx=0x7fd7900a28f0 comp rx=0 tx=0).stop 2024-09-16T00:34:41.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.257+0000 7fd7a7c6a740 1 -- 172.21.15.38:0/3773172143 shutdown_connections 2024-09-16T00:34:41.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.257+0000 7fd7a7c6a740 1 --2- 172.21.15.38:0/3773172143 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55dc847bf680 0x55dc84830d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:41.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.257+0000 7fd7a7c6a740 1 --2- 172.21.15.38:0/3773172143 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55dc84839c80 0x55dc8483c070 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:41.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.257+0000 7fd7a7c6a740 1 --2- 172.21.15.38:0/3773172143 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55dc847b4dc0 0x55dc847d4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:41.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.257+0000 7fd7a7c6a740 1 --2- 172.21.15.38:0/3773172143 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55dc847b44a0 0x55dc847d4130 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:41.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.257+0000 7fd77effd640 0 client.25412 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:34:41.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.257+0000 7fd7a7c6a740 1 -- 172.21.15.38:0/3773172143 >> 172.21.15.38:0/3773172143 conn(0x55dc847b0a30 msgr2=0x55dc847afb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:41.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.257+0000 7fd7a7c6a740 1 -- 172.21.15.38:0/3773172143 shutdown_connections 2024-09-16T00:34:41.263 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.257+0000 7fd7a7c6a740 1 -- 172.21.15.38:0/3773172143 wait complete. 2024-09-16T00:34:41.264 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.257+0000 7fd7a7c6a740 20 client.25412 trim_cache size 0 max 16384 2024-09-16T00:34:41.264 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-16T00:34:41.264 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-16T00:34:41.264 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": { 2024-09-16T00:34:41.264 INFO:teuthology.orchestra.run.smithi038.stdout: "a503e155-5582-4fc4-a951-029d376a5a1e": { 2024-09-16T00:34:41.264 INFO:teuthology.orchestra.run.smithi038.stdout: "path": "/", 2024-09-16T00:34:41.264 INFO:teuthology.orchestra.run.smithi038.stdout: "tag": "a503e155-5582-4fc4-a951-029d376a5a1e", 2024-09-16T00:34:41.264 INFO:teuthology.orchestra.run.smithi038.stdout: "options": "recursive,force" 2024-09-16T00:34:41.264 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:34:41.265 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:34:41.265 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-16T00:34:41.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.258+0000 7fd7a5891640 1 -- 172.21.15.38:0/455613645 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd7880a32c0 msgr2=0x7fd7880a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:41.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.258+0000 7fd7a5891640 1 --2- 172.21.15.38:0/455613645 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd7880a32c0 0x7fd7880a5780 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fd7980099e0 tx=0x7fd7980135e0 comp rx=0 tx=0).stop 2024-09-16T00:34:41.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.258+0000 7fd7a5891640 1 -- 172.21.15.38:0/455613645 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7a0102df0 msgr2=0x7fd7a01a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:41.265 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.258+0000 7fd7a5891640 1 --2- 172.21.15.38:0/455613645 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7a0102df0 0x7fd7a01a3860 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd79000e9f0 tx=0x7fd79000eeb0 comp rx=0 tx=0).stop 2024-09-16T00:34:41.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.258+0000 7fd7a5891640 1 -- 172.21.15.38:0/455613645 shutdown_connections 2024-09-16T00:34:41.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.258+0000 7fd7a5891640 1 --2- 172.21.15.38:0/455613645 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fd7880a32c0 0x7fd7880a5780 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:41.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.258+0000 7fd7a5891640 1 --2- 172.21.15.38:0/455613645 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fd7a010c2f0 0x7fd7a010ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:41.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.258+0000 7fd7a5891640 1 --2- 172.21.15.38:0/455613645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7a01037a0 0x7fd7a01a3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:41.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.258+0000 7fd7a5891640 1 --2- 172.21.15.38:0/455613645 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fd7a0102df0 0x7fd7a01a3860 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:41.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.258+0000 7fd7a5891640 1 -- 172.21.15.38:0/455613645 >> 172.21.15.38:0/455613645 conn(0x7fd7a00fc7d0 msgr2=0x7fd7a010d610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:41.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.258+0000 7fd7a5891640 1 -- 172.21.15.38:0/455613645 shutdown_connections 2024-09-16T00:34:41.266 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:41.258+0000 7fd7a5891640 1 -- 172.21.15.38:0/455613645 wait complete. 2024-09-16T00:34:41.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:41 smithi038 ceph-mon[30945]: pgmap v421: 129 pgs: 129 active+clean; 193 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 62 MiB/s rd, 1.2 MiB/s wr, 330 op/s 2024-09-16T00:34:41.555 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4195328 len 8192 2024-09-16T00:34:41.555 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4195328 len 8192 2024-09-16T00:34:41.555 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4195328 len 8192 2024-09-16T00:34:41.556 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4195328 len 16384 2024-09-16T00:34:41.556 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4195328 len 16384 2024-09-16T00:34:41.556 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4195328 len 16384 2024-09-16T00:34:41.556 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4195328 len 16384 2024-09-16T00:34:41.556 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4195840 len 1024 2024-09-16T00:34:41.557 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4195840 len 1024 2024-09-16T00:34:41.557 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4195840 len 1024 2024-09-16T00:34:41.557 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4195840 len 1024 2024-09-16T00:34:41.557 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4195840 len 2048 2024-09-16T00:34:41.557 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4195840 len 2048 2024-09-16T00:34:41.557 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4195840 len 2048 2024-09-16T00:34:41.557 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4195840 len 2048 2024-09-16T00:34:41.558 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4195840 len 4096 2024-09-16T00:34:41.558 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4195840 len 4096 2024-09-16T00:34:41.558 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4195840 len 4096 2024-09-16T00:34:41.558 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4195840 len 4096 2024-09-16T00:34:41.558 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4195840 len 8192 2024-09-16T00:34:41.558 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4195840 len 8192 2024-09-16T00:34:41.558 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4195840 len 8192 2024-09-16T00:34:41.559 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4195840 len 8192 2024-09-16T00:34:41.559 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4195840 len 16384 2024-09-16T00:34:41.559 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4195840 len 16384 2024-09-16T00:34:41.559 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4195840 len 16384 2024-09-16T00:34:41.559 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4195840 len 16384 2024-09-16T00:34:41.559 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4196352 len 1024 2024-09-16T00:34:41.559 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4196352 len 1024 2024-09-16T00:34:41.559 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4196352 len 1024 2024-09-16T00:34:41.560 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4196352 len 1024 2024-09-16T00:34:41.560 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4196352 len 2048 2024-09-16T00:34:41.560 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4196352 len 2048 2024-09-16T00:34:41.560 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4196352 len 2048 2024-09-16T00:34:41.560 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4196352 len 2048 2024-09-16T00:34:41.560 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4196352 len 4096 2024-09-16T00:34:41.560 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4196352 len 4096 2024-09-16T00:34:41.561 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4196352 len 4096 2024-09-16T00:34:41.561 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4196352 len 4096 2024-09-16T00:34:41.561 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4196352 len 8192 2024-09-16T00:34:41.561 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4196352 len 8192 2024-09-16T00:34:41.561 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4196352 len 8192 2024-09-16T00:34:41.561 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4196352 len 8192 2024-09-16T00:34:41.561 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4196352 len 16384 2024-09-16T00:34:41.562 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4196352 len 16384 2024-09-16T00:34:41.562 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4196352 len 16384 2024-09-16T00:34:41.562 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4196352 len 16384 2024-09-16T00:34:41.562 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4196864 len 1024 2024-09-16T00:34:41.562 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4196864 len 1024 2024-09-16T00:34:41.562 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4196864 len 1024 2024-09-16T00:34:41.563 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4196864 len 1024 2024-09-16T00:34:41.563 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4196864 len 2048 2024-09-16T00:34:41.563 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4196864 len 2048 2024-09-16T00:34:41.563 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4196864 len 2048 2024-09-16T00:34:41.563 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4196864 len 2048 2024-09-16T00:34:41.563 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4196864 len 4096 2024-09-16T00:34:41.563 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4196864 len 4096 2024-09-16T00:34:41.564 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4196864 len 4096 2024-09-16T00:34:41.564 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4196864 len 4096 2024-09-16T00:34:41.564 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4196864 len 8192 2024-09-16T00:34:41.564 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4196864 len 8192 2024-09-16T00:34:41.564 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4196864 len 8192 2024-09-16T00:34:41.564 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4196864 len 8192 2024-09-16T00:34:41.565 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4196864 len 16384 2024-09-16T00:34:41.565 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4196864 len 16384 2024-09-16T00:34:41.565 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4196864 len 16384 2024-09-16T00:34:41.565 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4196864 len 16384 2024-09-16T00:34:41.565 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4197376 len 1024 2024-09-16T00:34:41.565 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4197376 len 1024 2024-09-16T00:34:41.565 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4197376 len 1024 2024-09-16T00:34:41.566 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4197376 len 1024 2024-09-16T00:34:41.566 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4197376 len 2048 2024-09-16T00:34:41.566 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4197376 len 2048 2024-09-16T00:34:41.566 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4197376 len 2048 2024-09-16T00:34:41.566 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4197376 len 2048 2024-09-16T00:34:41.566 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4197376 len 4096 2024-09-16T00:34:41.567 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4197376 len 4096 2024-09-16T00:34:41.567 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4197376 len 4096 2024-09-16T00:34:41.567 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4197376 len 4096 2024-09-16T00:34:41.567 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4197376 len 8192 2024-09-16T00:34:41.567 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4197376 len 8192 2024-09-16T00:34:41.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:41 smithi122 ceph-mon[35412]: pgmap v421: 129 pgs: 129 active+clean; 193 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 62 MiB/s rd, 1.2 MiB/s wr, 330 op/s 2024-09-16T00:34:41.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:41 smithi073 ceph-mon[35254]: pgmap v421: 129 pgs: 129 active+clean; 193 MiB data, 670 MiB used, 1.0 TiB / 1.0 TiB avail; 62 MiB/s rd, 1.2 MiB/s wr, 330 op/s 2024-09-16T00:34:41.970 INFO:tasks.cephfs.filesystem:scrub status for tag:a503e155-5582-4fc4-a951-029d376a5a1e - {'path': '/', 'tag': 'a503e155-5582-4fc4-a951-029d376a5a1e', 'options': 'recursive,force'} 2024-09-16T00:34:42.426 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4197376 len 8192 2024-09-16T00:34:42.426 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4197376 len 8192 2024-09-16T00:34:42.426 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4197376 len 16384 2024-09-16T00:34:42.426 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4197376 len 16384 2024-09-16T00:34:42.426 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4197376 len 16384 2024-09-16T00:34:42.426 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4197376 len 16384 2024-09-16T00:34:42.426 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4197888 len 1024 2024-09-16T00:34:42.426 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4197888 len 1024 2024-09-16T00:34:42.426 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4197888 len 1024 2024-09-16T00:34:42.426 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4197888 len 1024 2024-09-16T00:34:42.426 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4197888 len 2048 2024-09-16T00:34:42.427 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4197888 len 2048 2024-09-16T00:34:42.427 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4197888 len 2048 2024-09-16T00:34:42.427 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4197888 len 2048 2024-09-16T00:34:42.427 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4197888 len 4096 2024-09-16T00:34:42.427 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4197888 len 4096 2024-09-16T00:34:42.427 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4197888 len 4096 2024-09-16T00:34:42.427 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4197888 len 4096 2024-09-16T00:34:42.427 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4197888 len 8192 2024-09-16T00:34:42.427 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4197888 len 8192 2024-09-16T00:34:42.427 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4197888 len 8192 2024-09-16T00:34:42.427 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4197888 len 8192 2024-09-16T00:34:42.427 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 1536 offset 4197888 len 16384 2024-09-16T00:34:42.427 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4197888 len 16384 2024-09-16T00:34:42.428 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 1536 offset 4197888 len 16384 2024-09-16T00:34:42.428 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 1536 offset 4197888 len 16384 2024-09-16T00:34:42.428 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4190720 len 1024 2024-09-16T00:34:42.428 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4190720 len 1024 2024-09-16T00:34:42.428 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4190720 len 1024 2024-09-16T00:34:42.428 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4190720 len 1024 2024-09-16T00:34:42.428 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4190720 len 2048 2024-09-16T00:34:42.428 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4190720 len 2048 2024-09-16T00:34:42.428 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4190720 len 2048 2024-09-16T00:34:42.428 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4190720 len 2048 2024-09-16T00:34:42.428 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4190720 len 4096 2024-09-16T00:34:42.428 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4190720 len 4096 2024-09-16T00:34:42.428 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4190720 len 4096 2024-09-16T00:34:42.428 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4190720 len 4096 2024-09-16T00:34:42.429 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4190720 len 8192 2024-09-16T00:34:42.429 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4190720 len 8192 2024-09-16T00:34:42.429 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4190720 len 8192 2024-09-16T00:34:42.429 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4190720 len 8192 2024-09-16T00:34:42.429 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4190720 len 16384 2024-09-16T00:34:42.429 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4190720 len 16384 2024-09-16T00:34:42.429 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4190720 len 16384 2024-09-16T00:34:42.429 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4190720 len 16384 2024-09-16T00:34:42.429 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4191232 len 1024 2024-09-16T00:34:42.429 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4191232 len 1024 2024-09-16T00:34:42.429 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4191232 len 1024 2024-09-16T00:34:42.429 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4191232 len 1024 2024-09-16T00:34:42.429 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4191232 len 2048 2024-09-16T00:34:42.429 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4191232 len 2048 2024-09-16T00:34:42.430 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4191232 len 2048 2024-09-16T00:34:42.430 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4191232 len 2048 2024-09-16T00:34:42.430 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4191232 len 4096 2024-09-16T00:34:42.430 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4191232 len 4096 2024-09-16T00:34:42.430 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4191232 len 4096 2024-09-16T00:34:42.430 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4191232 len 4096 2024-09-16T00:34:42.430 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4191232 len 8192 2024-09-16T00:34:42.430 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4191232 len 8192 2024-09-16T00:34:42.430 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4191232 len 8192 2024-09-16T00:34:42.430 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4191232 len 8192 2024-09-16T00:34:42.430 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4191232 len 16384 2024-09-16T00:34:42.430 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4191232 len 16384 2024-09-16T00:34:42.430 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4191232 len 16384 2024-09-16T00:34:42.431 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4191232 len 16384 2024-09-16T00:34:42.431 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4191744 len 1024 2024-09-16T00:34:42.431 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4191744 len 1024 2024-09-16T00:34:42.431 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4191744 len 1024 2024-09-16T00:34:42.431 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4191744 len 1024 2024-09-16T00:34:42.431 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4191744 len 2048 2024-09-16T00:34:42.431 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4191744 len 2048 2024-09-16T00:34:42.431 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4191744 len 2048 2024-09-16T00:34:42.431 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4191744 len 2048 2024-09-16T00:34:42.431 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4191744 len 4096 2024-09-16T00:34:42.431 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4191744 len 4096 2024-09-16T00:34:42.431 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4191744 len 4096 2024-09-16T00:34:42.431 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4191744 len 4096 2024-09-16T00:34:42.431 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4191744 len 8192 2024-09-16T00:34:42.432 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4191744 len 8192 2024-09-16T00:34:42.432 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4191744 len 8192 2024-09-16T00:34:42.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:42 smithi038 ceph-mon[30945]: scrub summary: idle 2024-09-16T00:34:42.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:42 smithi122 ceph-mon[35412]: scrub summary: idle 2024-09-16T00:34:42.793 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:42 smithi073 ceph-mon[35254]: scrub summary: idle 2024-09-16T00:34:43.597 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4191744 len 8192 2024-09-16T00:34:43.598 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4191744 len 16384 2024-09-16T00:34:43.598 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4191744 len 16384 2024-09-16T00:34:43.598 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4191744 len 16384 2024-09-16T00:34:43.598 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4191744 len 16384 2024-09-16T00:34:43.598 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4192256 len 1024 2024-09-16T00:34:43.598 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4192256 len 1024 2024-09-16T00:34:43.599 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4192256 len 1024 2024-09-16T00:34:43.599 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4192256 len 1024 2024-09-16T00:34:43.599 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4192256 len 2048 2024-09-16T00:34:43.599 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4192256 len 2048 2024-09-16T00:34:43.599 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4192256 len 2048 2024-09-16T00:34:43.599 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4192256 len 2048 2024-09-16T00:34:43.599 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4192256 len 4096 2024-09-16T00:34:43.600 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4192256 len 4096 2024-09-16T00:34:43.600 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4192256 len 4096 2024-09-16T00:34:43.600 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4192256 len 4096 2024-09-16T00:34:43.600 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4192256 len 8192 2024-09-16T00:34:43.600 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4192256 len 8192 2024-09-16T00:34:43.600 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4192256 len 8192 2024-09-16T00:34:43.600 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4192256 len 8192 2024-09-16T00:34:43.601 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4192256 len 16384 2024-09-16T00:34:43.601 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4192256 len 16384 2024-09-16T00:34:43.601 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4192256 len 16384 2024-09-16T00:34:43.601 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4192256 len 16384 2024-09-16T00:34:43.601 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4192768 len 1024 2024-09-16T00:34:43.601 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4192768 len 1024 2024-09-16T00:34:43.601 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4192768 len 1024 2024-09-16T00:34:43.601 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4192768 len 1024 2024-09-16T00:34:43.602 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4192768 len 2048 2024-09-16T00:34:43.602 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4192768 len 2048 2024-09-16T00:34:43.602 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4192768 len 2048 2024-09-16T00:34:43.602 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4192768 len 2048 2024-09-16T00:34:43.602 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4192768 len 4096 2024-09-16T00:34:43.602 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4192768 len 4096 2024-09-16T00:34:43.602 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4192768 len 4096 2024-09-16T00:34:43.603 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4192768 len 4096 2024-09-16T00:34:43.603 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4192768 len 8192 2024-09-16T00:34:43.603 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4192768 len 8192 2024-09-16T00:34:43.603 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4192768 len 8192 2024-09-16T00:34:43.603 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4192768 len 8192 2024-09-16T00:34:43.603 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4192768 len 16384 2024-09-16T00:34:43.603 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4192768 len 16384 2024-09-16T00:34:43.604 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4192768 len 16384 2024-09-16T00:34:43.604 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4192768 len 16384 2024-09-16T00:34:43.604 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4193280 len 1024 2024-09-16T00:34:43.604 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4193280 len 1024 2024-09-16T00:34:43.604 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4193280 len 1024 2024-09-16T00:34:43.604 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4193280 len 1024 2024-09-16T00:34:43.605 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4193280 len 2048 2024-09-16T00:34:43.605 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4193280 len 2048 2024-09-16T00:34:43.605 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4193280 len 2048 2024-09-16T00:34:43.605 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4193280 len 2048 2024-09-16T00:34:43.605 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4193280 len 4096 2024-09-16T00:34:43.605 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4193280 len 4096 2024-09-16T00:34:43.606 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4193280 len 4096 2024-09-16T00:34:43.606 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4193280 len 4096 2024-09-16T00:34:43.606 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4193280 len 8192 2024-09-16T00:34:43.606 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4193280 len 8192 2024-09-16T00:34:43.606 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4193280 len 8192 2024-09-16T00:34:43.606 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4193280 len 8192 2024-09-16T00:34:43.606 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4193280 len 16384 2024-09-16T00:34:43.606 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4193280 len 16384 2024-09-16T00:34:43.607 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4193280 len 16384 2024-09-16T00:34:43.607 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4193280 len 16384 2024-09-16T00:34:43.607 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4193792 len 1024 2024-09-16T00:34:43.607 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4193792 len 1024 2024-09-16T00:34:43.607 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4193792 len 1024 2024-09-16T00:34:43.607 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4193792 len 1024 2024-09-16T00:34:43.607 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4193792 len 2048 2024-09-16T00:34:43.607 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4193792 len 2048 2024-09-16T00:34:43.607 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4193792 len 2048 2024-09-16T00:34:43.607 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4193792 len 2048 2024-09-16T00:34:43.607 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4193792 len 4096 2024-09-16T00:34:43.607 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4193792 len 4096 2024-09-16T00:34:43.607 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4193792 len 4096 2024-09-16T00:34:43.608 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4193792 len 4096 2024-09-16T00:34:43.608 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4193792 len 8192 2024-09-16T00:34:43.608 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4193792 len 8192 2024-09-16T00:34:43.608 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4193792 len 8192 2024-09-16T00:34:43.608 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4193792 len 8192 2024-09-16T00:34:43.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:43 smithi038 ceph-mon[30945]: pgmap v422: 129 pgs: 129 active+clean; 197 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s rd, 1.3 MiB/s wr, 365 op/s 2024-09-16T00:34:43.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:43 smithi122 ceph-mon[35412]: pgmap v422: 129 pgs: 129 active+clean; 197 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s rd, 1.3 MiB/s wr, 365 op/s 2024-09-16T00:34:43.793 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:43 smithi073 ceph-mon[35254]: pgmap v422: 129 pgs: 129 active+clean; 197 MiB data, 826 MiB used, 1.0 TiB / 1.0 TiB avail; 78 MiB/s rd, 1.3 MiB/s wr, 365 op/s 2024-09-16T00:34:44.093 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4193792 len 16384 2024-09-16T00:34:44.093 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4193792 len 16384 2024-09-16T00:34:44.093 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4193792 len 16384 2024-09-16T00:34:44.093 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4193792 len 16384 2024-09-16T00:34:44.093 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4194304 len 1024 2024-09-16T00:34:44.093 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4194304 len 1024 2024-09-16T00:34:44.093 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4194304 len 1024 2024-09-16T00:34:44.093 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4194304 len 1024 2024-09-16T00:34:44.093 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4194304 len 2048 2024-09-16T00:34:44.094 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4194304 len 2048 2024-09-16T00:34:44.094 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4194304 len 2048 2024-09-16T00:34:44.094 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4194304 len 2048 2024-09-16T00:34:44.094 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4194304 len 4096 2024-09-16T00:34:44.094 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4194304 len 4096 2024-09-16T00:34:44.094 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4194304 len 4096 2024-09-16T00:34:44.094 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4194304 len 4096 2024-09-16T00:34:44.094 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4194304 len 8192 2024-09-16T00:34:44.094 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4194304 len 8192 2024-09-16T00:34:44.094 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4194304 len 8192 2024-09-16T00:34:44.094 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4194304 len 8192 2024-09-16T00:34:44.094 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4194304 len 16384 2024-09-16T00:34:44.095 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4194304 len 16384 2024-09-16T00:34:44.095 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4194304 len 16384 2024-09-16T00:34:44.095 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4194304 len 16384 2024-09-16T00:34:44.095 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4194816 len 1024 2024-09-16T00:34:44.095 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4194816 len 1024 2024-09-16T00:34:44.095 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4194816 len 1024 2024-09-16T00:34:44.095 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4194816 len 1024 2024-09-16T00:34:44.095 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4194816 len 2048 2024-09-16T00:34:44.095 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4194816 len 2048 2024-09-16T00:34:44.095 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4194816 len 2048 2024-09-16T00:34:44.095 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4194816 len 2048 2024-09-16T00:34:44.096 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4194816 len 4096 2024-09-16T00:34:44.096 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4194816 len 4096 2024-09-16T00:34:44.096 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4194816 len 4096 2024-09-16T00:34:44.096 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4194816 len 4096 2024-09-16T00:34:44.096 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4194816 len 8192 2024-09-16T00:34:44.096 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4194816 len 8192 2024-09-16T00:34:44.096 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4194816 len 8192 2024-09-16T00:34:44.096 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4194816 len 8192 2024-09-16T00:34:44.096 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4194816 len 16384 2024-09-16T00:34:44.096 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4194816 len 16384 2024-09-16T00:34:44.096 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4194816 len 16384 2024-09-16T00:34:44.096 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4194816 len 16384 2024-09-16T00:34:44.096 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4195328 len 1024 2024-09-16T00:34:44.097 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4195328 len 1024 2024-09-16T00:34:44.097 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4195328 len 1024 2024-09-16T00:34:44.097 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4195328 len 1024 2024-09-16T00:34:44.097 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4195328 len 2048 2024-09-16T00:34:44.097 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4195328 len 2048 2024-09-16T00:34:44.097 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4195328 len 2048 2024-09-16T00:34:44.097 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4195328 len 2048 2024-09-16T00:34:44.098 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4195328 len 4096 2024-09-16T00:34:44.098 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4195328 len 4096 2024-09-16T00:34:44.098 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4195328 len 4096 2024-09-16T00:34:44.098 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4195328 len 4096 2024-09-16T00:34:44.098 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4195328 len 8192 2024-09-16T00:34:44.098 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4195328 len 8192 2024-09-16T00:34:44.098 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4195328 len 8192 2024-09-16T00:34:44.098 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4195328 len 8192 2024-09-16T00:34:44.098 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4195328 len 16384 2024-09-16T00:34:44.098 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4195328 len 16384 2024-09-16T00:34:44.098 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4195328 len 16384 2024-09-16T00:34:44.098 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4195328 len 16384 2024-09-16T00:34:44.098 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4195840 len 1024 2024-09-16T00:34:44.098 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4195840 len 1024 2024-09-16T00:34:44.099 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4195840 len 1024 2024-09-16T00:34:44.099 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4195840 len 1024 2024-09-16T00:34:44.099 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4195840 len 2048 2024-09-16T00:34:44.099 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4195840 len 2048 2024-09-16T00:34:44.099 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4195840 len 2048 2024-09-16T00:34:44.099 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4195840 len 2048 2024-09-16T00:34:44.099 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4195840 len 4096 2024-09-16T00:34:44.099 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4195840 len 4096 2024-09-16T00:34:44.099 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4195840 len 4096 2024-09-16T00:34:44.099 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4195840 len 4096 2024-09-16T00:34:44.099 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4195840 len 8192 2024-09-16T00:34:44.099 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4195840 len 8192 2024-09-16T00:34:44.099 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4195840 len 8192 2024-09-16T00:34:44.099 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4195840 len 8192 2024-09-16T00:34:44.478 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4195840 len 16384 2024-09-16T00:34:44.478 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4195840 len 16384 2024-09-16T00:34:44.478 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4195840 len 16384 2024-09-16T00:34:44.478 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4195840 len 16384 2024-09-16T00:34:44.478 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4196352 len 1024 2024-09-16T00:34:44.478 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4196352 len 1024 2024-09-16T00:34:44.478 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4196352 len 1024 2024-09-16T00:34:44.479 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4196352 len 1024 2024-09-16T00:34:44.479 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4196352 len 2048 2024-09-16T00:34:44.479 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4196352 len 2048 2024-09-16T00:34:44.479 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4196352 len 2048 2024-09-16T00:34:44.479 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4196352 len 2048 2024-09-16T00:34:44.479 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4196352 len 4096 2024-09-16T00:34:44.479 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4196352 len 4096 2024-09-16T00:34:44.479 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4196352 len 4096 2024-09-16T00:34:44.480 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4196352 len 4096 2024-09-16T00:34:44.480 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4196352 len 8192 2024-09-16T00:34:44.480 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4196352 len 8192 2024-09-16T00:34:44.480 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4196352 len 8192 2024-09-16T00:34:44.480 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4196352 len 8192 2024-09-16T00:34:44.480 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4196352 len 16384 2024-09-16T00:34:44.480 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4196352 len 16384 2024-09-16T00:34:44.480 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4196352 len 16384 2024-09-16T00:34:44.480 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4196352 len 16384 2024-09-16T00:34:44.480 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4196864 len 1024 2024-09-16T00:34:44.481 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4196864 len 1024 2024-09-16T00:34:44.481 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4196864 len 1024 2024-09-16T00:34:44.481 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4196864 len 1024 2024-09-16T00:34:44.481 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4196864 len 2048 2024-09-16T00:34:44.481 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4196864 len 2048 2024-09-16T00:34:44.481 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4196864 len 2048 2024-09-16T00:34:44.481 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4196864 len 2048 2024-09-16T00:34:44.481 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4196864 len 4096 2024-09-16T00:34:44.481 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4196864 len 4096 2024-09-16T00:34:44.481 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4196864 len 4096 2024-09-16T00:34:44.481 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4196864 len 4096 2024-09-16T00:34:44.481 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4196864 len 8192 2024-09-16T00:34:44.482 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4196864 len 8192 2024-09-16T00:34:44.482 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4196864 len 8192 2024-09-16T00:34:44.482 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4196864 len 8192 2024-09-16T00:34:44.482 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4196864 len 16384 2024-09-16T00:34:44.482 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4196864 len 16384 2024-09-16T00:34:44.482 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4196864 len 16384 2024-09-16T00:34:44.482 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4196864 len 16384 2024-09-16T00:34:44.482 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4197376 len 1024 2024-09-16T00:34:44.482 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4197376 len 1024 2024-09-16T00:34:44.483 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4197376 len 1024 2024-09-16T00:34:44.483 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4197376 len 1024 2024-09-16T00:34:44.483 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4197376 len 2048 2024-09-16T00:34:44.483 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4197376 len 2048 2024-09-16T00:34:44.483 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4197376 len 2048 2024-09-16T00:34:44.483 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4197376 len 2048 2024-09-16T00:34:44.483 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4197376 len 4096 2024-09-16T00:34:44.483 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4197376 len 4096 2024-09-16T00:34:44.483 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4197376 len 4096 2024-09-16T00:34:44.483 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4197376 len 4096 2024-09-16T00:34:44.483 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4197376 len 8192 2024-09-16T00:34:44.483 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4197376 len 8192 2024-09-16T00:34:44.483 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4197376 len 8192 2024-09-16T00:34:44.484 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4197376 len 8192 2024-09-16T00:34:44.484 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4197376 len 16384 2024-09-16T00:34:44.484 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4197376 len 16384 2024-09-16T00:34:44.484 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4197376 len 16384 2024-09-16T00:34:44.484 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4197376 len 16384 2024-09-16T00:34:44.484 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4197888 len 1024 2024-09-16T00:34:44.484 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4197888 len 1024 2024-09-16T00:34:44.484 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4197888 len 1024 2024-09-16T00:34:44.484 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4197888 len 1024 2024-09-16T00:34:44.484 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4197888 len 2048 2024-09-16T00:34:44.485 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4197888 len 2048 2024-09-16T00:34:44.485 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4197888 len 2048 2024-09-16T00:34:44.485 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4197888 len 2048 2024-09-16T00:34:44.485 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4197888 len 4096 2024-09-16T00:34:44.485 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4197888 len 4096 2024-09-16T00:34:44.485 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4197888 len 4096 2024-09-16T00:34:44.485 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4197888 len 4096 2024-09-16T00:34:44.485 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4197888 len 8192 2024-09-16T00:34:44.485 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4197888 len 8192 2024-09-16T00:34:44.486 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4197888 len 8192 2024-09-16T00:34:44.486 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4197888 len 8192 2024-09-16T00:34:44.486 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2048 offset 4197888 len 16384 2024-09-16T00:34:45.578 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4197888 len 16384 2024-09-16T00:34:45.578 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2048 offset 4197888 len 16384 2024-09-16T00:34:45.578 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2048 offset 4197888 len 16384 2024-09-16T00:34:45.578 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4190720 len 1024 2024-09-16T00:34:45.578 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4190720 len 1024 2024-09-16T00:34:45.578 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4190720 len 1024 2024-09-16T00:34:45.579 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4190720 len 1024 2024-09-16T00:34:45.579 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4190720 len 2048 2024-09-16T00:34:45.579 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4190720 len 2048 2024-09-16T00:34:45.579 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4190720 len 2048 2024-09-16T00:34:45.579 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4190720 len 2048 2024-09-16T00:34:45.579 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4190720 len 4096 2024-09-16T00:34:45.579 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4190720 len 4096 2024-09-16T00:34:45.579 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4190720 len 4096 2024-09-16T00:34:45.579 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4190720 len 4096 2024-09-16T00:34:45.579 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4190720 len 8192 2024-09-16T00:34:45.579 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4190720 len 8192 2024-09-16T00:34:45.579 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4190720 len 8192 2024-09-16T00:34:45.579 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4190720 len 8192 2024-09-16T00:34:45.579 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4190720 len 16384 2024-09-16T00:34:45.580 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4190720 len 16384 2024-09-16T00:34:45.580 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4190720 len 16384 2024-09-16T00:34:45.580 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4190720 len 16384 2024-09-16T00:34:45.580 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4191232 len 1024 2024-09-16T00:34:45.580 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4191232 len 1024 2024-09-16T00:34:45.580 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4191232 len 1024 2024-09-16T00:34:45.580 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4191232 len 1024 2024-09-16T00:34:45.580 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4191232 len 2048 2024-09-16T00:34:45.580 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4191232 len 2048 2024-09-16T00:34:45.580 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4191232 len 2048 2024-09-16T00:34:45.580 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4191232 len 2048 2024-09-16T00:34:45.580 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4191232 len 4096 2024-09-16T00:34:45.580 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4191232 len 4096 2024-09-16T00:34:45.581 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4191232 len 4096 2024-09-16T00:34:45.581 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4191232 len 4096 2024-09-16T00:34:45.581 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4191232 len 8192 2024-09-16T00:34:45.581 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4191232 len 8192 2024-09-16T00:34:45.581 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4191232 len 8192 2024-09-16T00:34:45.581 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4191232 len 8192 2024-09-16T00:34:45.581 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4191232 len 16384 2024-09-16T00:34:45.581 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4191232 len 16384 2024-09-16T00:34:45.581 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4191232 len 16384 2024-09-16T00:34:45.581 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4191232 len 16384 2024-09-16T00:34:45.581 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4191744 len 1024 2024-09-16T00:34:45.581 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4191744 len 1024 2024-09-16T00:34:45.581 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4191744 len 1024 2024-09-16T00:34:45.581 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4191744 len 1024 2024-09-16T00:34:45.582 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4191744 len 2048 2024-09-16T00:34:45.582 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4191744 len 2048 2024-09-16T00:34:45.582 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4191744 len 2048 2024-09-16T00:34:45.582 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4191744 len 2048 2024-09-16T00:34:45.582 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4191744 len 4096 2024-09-16T00:34:45.582 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4191744 len 4096 2024-09-16T00:34:45.582 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4191744 len 4096 2024-09-16T00:34:45.582 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4191744 len 4096 2024-09-16T00:34:45.582 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4191744 len 8192 2024-09-16T00:34:45.582 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4191744 len 8192 2024-09-16T00:34:45.582 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4191744 len 8192 2024-09-16T00:34:45.582 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4191744 len 8192 2024-09-16T00:34:45.582 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4191744 len 16384 2024-09-16T00:34:45.583 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4191744 len 16384 2024-09-16T00:34:45.583 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4191744 len 16384 2024-09-16T00:34:45.583 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4191744 len 16384 2024-09-16T00:34:45.583 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4192256 len 1024 2024-09-16T00:34:45.583 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4192256 len 1024 2024-09-16T00:34:45.583 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4192256 len 1024 2024-09-16T00:34:45.583 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4192256 len 1024 2024-09-16T00:34:45.583 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4192256 len 2048 2024-09-16T00:34:45.583 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4192256 len 2048 2024-09-16T00:34:45.583 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4192256 len 2048 2024-09-16T00:34:45.583 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4192256 len 2048 2024-09-16T00:34:45.583 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4192256 len 4096 2024-09-16T00:34:45.583 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4192256 len 4096 2024-09-16T00:34:45.583 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4192256 len 4096 2024-09-16T00:34:45.584 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4192256 len 4096 2024-09-16T00:34:45.584 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4192256 len 8192 2024-09-16T00:34:45.584 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4192256 len 8192 2024-09-16T00:34:45.584 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4192256 len 8192 2024-09-16T00:34:45.584 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4192256 len 8192 2024-09-16T00:34:45.584 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4192256 len 16384 2024-09-16T00:34:45.584 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4192256 len 16384 2024-09-16T00:34:45.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:45 smithi038 ceph-mon[30945]: pgmap v423: 129 pgs: 129 active+clean; 208 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 96 MiB/s rd, 1.3 MiB/s wr, 358 op/s 2024-09-16T00:34:45.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:45 smithi122 ceph-mon[35412]: pgmap v423: 129 pgs: 129 active+clean; 208 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 96 MiB/s rd, 1.3 MiB/s wr, 358 op/s 2024-09-16T00:34:45.793 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:45 smithi073 ceph-mon[35254]: pgmap v423: 129 pgs: 129 active+clean; 208 MiB data, 887 MiB used, 1.0 TiB / 1.0 TiB avail; 96 MiB/s rd, 1.3 MiB/s wr, 358 op/s 2024-09-16T00:34:46.617 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4192256 len 16384 2024-09-16T00:34:46.617 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4192256 len 16384 2024-09-16T00:34:46.617 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4192768 len 1024 2024-09-16T00:34:46.617 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4192768 len 1024 2024-09-16T00:34:46.617 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4192768 len 1024 2024-09-16T00:34:46.617 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4192768 len 1024 2024-09-16T00:34:46.617 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4192768 len 2048 2024-09-16T00:34:46.617 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4192768 len 2048 2024-09-16T00:34:46.617 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4192768 len 2048 2024-09-16T00:34:46.617 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4192768 len 2048 2024-09-16T00:34:46.617 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4192768 len 4096 2024-09-16T00:34:46.618 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4192768 len 4096 2024-09-16T00:34:46.618 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4192768 len 4096 2024-09-16T00:34:46.618 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4192768 len 4096 2024-09-16T00:34:46.618 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4192768 len 8192 2024-09-16T00:34:46.618 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4192768 len 8192 2024-09-16T00:34:46.618 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4192768 len 8192 2024-09-16T00:34:46.618 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4192768 len 8192 2024-09-16T00:34:46.618 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4192768 len 16384 2024-09-16T00:34:46.618 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4192768 len 16384 2024-09-16T00:34:46.618 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4192768 len 16384 2024-09-16T00:34:46.618 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4192768 len 16384 2024-09-16T00:34:46.618 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4193280 len 1024 2024-09-16T00:34:46.619 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4193280 len 1024 2024-09-16T00:34:46.619 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4193280 len 1024 2024-09-16T00:34:46.619 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4193280 len 1024 2024-09-16T00:34:46.619 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4193280 len 2048 2024-09-16T00:34:46.619 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4193280 len 2048 2024-09-16T00:34:46.619 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4193280 len 2048 2024-09-16T00:34:46.619 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4193280 len 2048 2024-09-16T00:34:46.619 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4193280 len 4096 2024-09-16T00:34:46.619 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4193280 len 4096 2024-09-16T00:34:46.619 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4193280 len 4096 2024-09-16T00:34:46.619 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4193280 len 4096 2024-09-16T00:34:46.619 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4193280 len 8192 2024-09-16T00:34:46.620 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4193280 len 8192 2024-09-16T00:34:46.620 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4193280 len 8192 2024-09-16T00:34:46.620 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4193280 len 8192 2024-09-16T00:34:46.620 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4193280 len 16384 2024-09-16T00:34:46.620 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4193280 len 16384 2024-09-16T00:34:46.620 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4193280 len 16384 2024-09-16T00:34:46.620 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4193280 len 16384 2024-09-16T00:34:46.620 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4193792 len 1024 2024-09-16T00:34:46.620 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4193792 len 1024 2024-09-16T00:34:46.620 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4193792 len 1024 2024-09-16T00:34:46.620 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4193792 len 1024 2024-09-16T00:34:46.620 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4193792 len 2048 2024-09-16T00:34:46.621 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4193792 len 2048 2024-09-16T00:34:46.621 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4193792 len 2048 2024-09-16T00:34:46.621 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4193792 len 2048 2024-09-16T00:34:46.621 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4193792 len 4096 2024-09-16T00:34:46.621 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4193792 len 4096 2024-09-16T00:34:46.621 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4193792 len 4096 2024-09-16T00:34:46.621 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4193792 len 4096 2024-09-16T00:34:46.621 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4193792 len 8192 2024-09-16T00:34:46.621 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4193792 len 8192 2024-09-16T00:34:46.621 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4193792 len 8192 2024-09-16T00:34:46.621 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4193792 len 8192 2024-09-16T00:34:46.621 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4193792 len 16384 2024-09-16T00:34:46.621 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4193792 len 16384 2024-09-16T00:34:46.622 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4193792 len 16384 2024-09-16T00:34:46.622 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4193792 len 16384 2024-09-16T00:34:46.622 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4194304 len 1024 2024-09-16T00:34:46.622 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4194304 len 1024 2024-09-16T00:34:46.622 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4194304 len 1024 2024-09-16T00:34:46.622 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4194304 len 1024 2024-09-16T00:34:46.622 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4194304 len 2048 2024-09-16T00:34:46.622 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4194304 len 2048 2024-09-16T00:34:46.622 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4194304 len 2048 2024-09-16T00:34:46.622 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4194304 len 2048 2024-09-16T00:34:46.622 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4194304 len 4096 2024-09-16T00:34:46.622 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4194304 len 4096 2024-09-16T00:34:46.623 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4194304 len 4096 2024-09-16T00:34:46.623 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4194304 len 4096 2024-09-16T00:34:46.623 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4194304 len 8192 2024-09-16T00:34:46.623 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4194304 len 8192 2024-09-16T00:34:46.623 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4194304 len 8192 2024-09-16T00:34:46.623 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4194304 len 8192 2024-09-16T00:34:46.623 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4194304 len 16384 2024-09-16T00:34:46.623 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4194304 len 16384 2024-09-16T00:34:46.623 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4194304 len 16384 2024-09-16T00:34:47.038 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4194304 len 16384 2024-09-16T00:34:47.038 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4194816 len 1024 2024-09-16T00:34:47.038 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4194816 len 1024 2024-09-16T00:34:47.038 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4194816 len 1024 2024-09-16T00:34:47.038 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4194816 len 1024 2024-09-16T00:34:47.038 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4194816 len 2048 2024-09-16T00:34:47.039 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4194816 len 2048 2024-09-16T00:34:47.039 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4194816 len 2048 2024-09-16T00:34:47.039 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4194816 len 2048 2024-09-16T00:34:47.039 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4194816 len 4096 2024-09-16T00:34:47.039 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4194816 len 4096 2024-09-16T00:34:47.039 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4194816 len 4096 2024-09-16T00:34:47.039 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4194816 len 4096 2024-09-16T00:34:47.039 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4194816 len 8192 2024-09-16T00:34:47.039 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4194816 len 8192 2024-09-16T00:34:47.039 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4194816 len 8192 2024-09-16T00:34:47.039 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4194816 len 8192 2024-09-16T00:34:47.039 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4194816 len 16384 2024-09-16T00:34:47.039 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4194816 len 16384 2024-09-16T00:34:47.040 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4194816 len 16384 2024-09-16T00:34:47.040 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4194816 len 16384 2024-09-16T00:34:47.040 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4195328 len 1024 2024-09-16T00:34:47.040 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4195328 len 1024 2024-09-16T00:34:47.040 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4195328 len 1024 2024-09-16T00:34:47.040 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4195328 len 1024 2024-09-16T00:34:47.040 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4195328 len 2048 2024-09-16T00:34:47.040 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4195328 len 2048 2024-09-16T00:34:47.040 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4195328 len 2048 2024-09-16T00:34:47.040 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4195328 len 2048 2024-09-16T00:34:47.040 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4195328 len 4096 2024-09-16T00:34:47.040 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4195328 len 4096 2024-09-16T00:34:47.040 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4195328 len 4096 2024-09-16T00:34:47.041 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4195328 len 4096 2024-09-16T00:34:47.041 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4195328 len 8192 2024-09-16T00:34:47.041 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4195328 len 8192 2024-09-16T00:34:47.041 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4195328 len 8192 2024-09-16T00:34:47.041 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4195328 len 8192 2024-09-16T00:34:47.041 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4195328 len 16384 2024-09-16T00:34:47.041 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4195328 len 16384 2024-09-16T00:34:47.041 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4195328 len 16384 2024-09-16T00:34:47.041 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4195328 len 16384 2024-09-16T00:34:47.041 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4195840 len 1024 2024-09-16T00:34:47.041 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4195840 len 1024 2024-09-16T00:34:47.041 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4195840 len 1024 2024-09-16T00:34:47.041 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4195840 len 1024 2024-09-16T00:34:47.041 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4195840 len 2048 2024-09-16T00:34:47.042 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4195840 len 2048 2024-09-16T00:34:47.042 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4195840 len 2048 2024-09-16T00:34:47.042 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4195840 len 2048 2024-09-16T00:34:47.042 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4195840 len 4096 2024-09-16T00:34:47.042 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4195840 len 4096 2024-09-16T00:34:47.042 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4195840 len 4096 2024-09-16T00:34:47.042 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4195840 len 4096 2024-09-16T00:34:47.042 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4195840 len 8192 2024-09-16T00:34:47.042 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4195840 len 8192 2024-09-16T00:34:47.042 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4195840 len 8192 2024-09-16T00:34:47.042 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4195840 len 8192 2024-09-16T00:34:47.042 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4195840 len 16384 2024-09-16T00:34:47.042 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4195840 len 16384 2024-09-16T00:34:47.042 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4195840 len 16384 2024-09-16T00:34:47.042 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4195840 len 16384 2024-09-16T00:34:47.042 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4196352 len 1024 2024-09-16T00:34:47.043 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4196352 len 1024 2024-09-16T00:34:47.043 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4196352 len 1024 2024-09-16T00:34:47.043 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4196352 len 1024 2024-09-16T00:34:47.043 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4196352 len 2048 2024-09-16T00:34:47.043 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4196352 len 2048 2024-09-16T00:34:47.043 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4196352 len 2048 2024-09-16T00:34:47.043 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4196352 len 2048 2024-09-16T00:34:47.043 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4196352 len 4096 2024-09-16T00:34:47.043 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4196352 len 4096 2024-09-16T00:34:47.043 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4196352 len 4096 2024-09-16T00:34:47.043 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4196352 len 4096 2024-09-16T00:34:47.043 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4196352 len 8192 2024-09-16T00:34:47.043 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4196352 len 8192 2024-09-16T00:34:47.043 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4196352 len 8192 2024-09-16T00:34:47.043 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4196352 len 8192 2024-09-16T00:34:47.044 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4196352 len 16384 2024-09-16T00:34:47.044 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4196352 len 16384 2024-09-16T00:34:47.044 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4196352 len 16384 2024-09-16T00:34:47.044 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4196352 len 16384 2024-09-16T00:34:47.627 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4196864 len 1024 2024-09-16T00:34:47.627 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4196864 len 1024 2024-09-16T00:34:47.627 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4196864 len 1024 2024-09-16T00:34:47.627 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4196864 len 1024 2024-09-16T00:34:47.627 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4196864 len 2048 2024-09-16T00:34:47.627 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4196864 len 2048 2024-09-16T00:34:47.628 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4196864 len 2048 2024-09-16T00:34:47.628 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4196864 len 2048 2024-09-16T00:34:47.628 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4196864 len 4096 2024-09-16T00:34:47.628 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4196864 len 4096 2024-09-16T00:34:47.628 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4196864 len 4096 2024-09-16T00:34:47.628 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4196864 len 4096 2024-09-16T00:34:47.628 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4196864 len 8192 2024-09-16T00:34:47.628 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4196864 len 8192 2024-09-16T00:34:47.628 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4196864 len 8192 2024-09-16T00:34:47.628 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4196864 len 8192 2024-09-16T00:34:47.628 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4196864 len 16384 2024-09-16T00:34:47.628 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4196864 len 16384 2024-09-16T00:34:47.628 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4196864 len 16384 2024-09-16T00:34:47.628 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4196864 len 16384 2024-09-16T00:34:47.629 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4197376 len 1024 2024-09-16T00:34:47.629 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4197376 len 1024 2024-09-16T00:34:47.629 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4197376 len 1024 2024-09-16T00:34:47.629 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4197376 len 1024 2024-09-16T00:34:47.629 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4197376 len 2048 2024-09-16T00:34:47.629 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4197376 len 2048 2024-09-16T00:34:47.629 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4197376 len 2048 2024-09-16T00:34:47.629 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4197376 len 2048 2024-09-16T00:34:47.629 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4197376 len 4096 2024-09-16T00:34:47.629 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4197376 len 4096 2024-09-16T00:34:47.629 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4197376 len 4096 2024-09-16T00:34:47.629 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4197376 len 4096 2024-09-16T00:34:47.629 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4197376 len 8192 2024-09-16T00:34:47.629 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4197376 len 8192 2024-09-16T00:34:47.630 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4197376 len 8192 2024-09-16T00:34:47.630 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4197376 len 8192 2024-09-16T00:34:47.630 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4197376 len 16384 2024-09-16T00:34:47.630 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4197376 len 16384 2024-09-16T00:34:47.630 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4197376 len 16384 2024-09-16T00:34:47.630 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4197376 len 16384 2024-09-16T00:34:47.630 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4197888 len 1024 2024-09-16T00:34:47.630 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4197888 len 1024 2024-09-16T00:34:47.630 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4197888 len 1024 2024-09-16T00:34:47.630 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4197888 len 1024 2024-09-16T00:34:47.630 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4197888 len 2048 2024-09-16T00:34:47.630 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4197888 len 2048 2024-09-16T00:34:47.630 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4197888 len 2048 2024-09-16T00:34:47.630 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4197888 len 2048 2024-09-16T00:34:47.631 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4197888 len 4096 2024-09-16T00:34:47.631 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4197888 len 4096 2024-09-16T00:34:47.631 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4197888 len 4096 2024-09-16T00:34:47.631 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4197888 len 4096 2024-09-16T00:34:47.631 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4197888 len 8192 2024-09-16T00:34:47.631 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4197888 len 8192 2024-09-16T00:34:47.631 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4197888 len 8192 2024-09-16T00:34:47.631 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4197888 len 8192 2024-09-16T00:34:47.631 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 2560 offset 4197888 len 16384 2024-09-16T00:34:47.631 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4197888 len 16384 2024-09-16T00:34:47.631 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 2560 offset 4197888 len 16384 2024-09-16T00:34:47.631 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 2560 offset 4197888 len 16384 2024-09-16T00:34:47.631 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4190720 len 1024 2024-09-16T00:34:47.631 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4190720 len 1024 2024-09-16T00:34:47.631 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4190720 len 1024 2024-09-16T00:34:47.632 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4190720 len 1024 2024-09-16T00:34:47.632 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4190720 len 2048 2024-09-16T00:34:47.632 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4190720 len 2048 2024-09-16T00:34:47.632 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4190720 len 2048 2024-09-16T00:34:47.632 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4190720 len 2048 2024-09-16T00:34:47.632 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4190720 len 4096 2024-09-16T00:34:47.632 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4190720 len 4096 2024-09-16T00:34:47.632 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4190720 len 4096 2024-09-16T00:34:47.632 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4190720 len 4096 2024-09-16T00:34:47.632 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4190720 len 8192 2024-09-16T00:34:47.632 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4190720 len 8192 2024-09-16T00:34:47.632 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4190720 len 8192 2024-09-16T00:34:47.632 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4190720 len 8192 2024-09-16T00:34:47.632 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4190720 len 16384 2024-09-16T00:34:47.633 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4190720 len 16384 2024-09-16T00:34:47.633 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4190720 len 16384 2024-09-16T00:34:47.633 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4190720 len 16384 2024-09-16T00:34:47.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:47 smithi038 ceph-mon[30945]: pgmap v424: 129 pgs: 129 active+clean; 232 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 93 MiB/s rd, 967 KiB/s wr, 315 op/s 2024-09-16T00:34:47.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:47 smithi122 ceph-mon[35412]: pgmap v424: 129 pgs: 129 active+clean; 232 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 93 MiB/s rd, 967 KiB/s wr, 315 op/s 2024-09-16T00:34:47.793 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:47 smithi073 ceph-mon[35254]: pgmap v424: 129 pgs: 129 active+clean; 232 MiB data, 864 MiB used, 1.0 TiB / 1.0 TiB avail; 93 MiB/s rd, 967 KiB/s wr, 315 op/s 2024-09-16T00:34:48.731 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4191232 len 1024 2024-09-16T00:34:48.731 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4191232 len 1024 2024-09-16T00:34:48.731 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4191232 len 1024 2024-09-16T00:34:48.731 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4191232 len 1024 2024-09-16T00:34:48.731 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4191232 len 2048 2024-09-16T00:34:48.732 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4191232 len 2048 2024-09-16T00:34:48.732 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4191232 len 2048 2024-09-16T00:34:48.732 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4191232 len 2048 2024-09-16T00:34:48.732 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4191232 len 4096 2024-09-16T00:34:48.732 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4191232 len 4096 2024-09-16T00:34:48.732 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4191232 len 4096 2024-09-16T00:34:48.732 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4191232 len 4096 2024-09-16T00:34:48.732 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4191232 len 8192 2024-09-16T00:34:48.732 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4191232 len 8192 2024-09-16T00:34:48.732 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4191232 len 8192 2024-09-16T00:34:48.732 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4191232 len 8192 2024-09-16T00:34:48.732 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4191232 len 16384 2024-09-16T00:34:48.732 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4191232 len 16384 2024-09-16T00:34:48.733 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4191232 len 16384 2024-09-16T00:34:48.733 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4191232 len 16384 2024-09-16T00:34:48.733 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4191744 len 1024 2024-09-16T00:34:48.733 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4191744 len 1024 2024-09-16T00:34:48.733 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4191744 len 1024 2024-09-16T00:34:48.733 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4191744 len 1024 2024-09-16T00:34:48.733 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4191744 len 2048 2024-09-16T00:34:48.733 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4191744 len 2048 2024-09-16T00:34:48.733 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4191744 len 2048 2024-09-16T00:34:48.733 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4191744 len 2048 2024-09-16T00:34:48.733 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4191744 len 4096 2024-09-16T00:34:48.733 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4191744 len 4096 2024-09-16T00:34:48.733 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4191744 len 4096 2024-09-16T00:34:48.733 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4191744 len 4096 2024-09-16T00:34:48.734 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4191744 len 8192 2024-09-16T00:34:48.734 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4191744 len 8192 2024-09-16T00:34:48.734 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4191744 len 8192 2024-09-16T00:34:48.734 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4191744 len 8192 2024-09-16T00:34:48.734 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4191744 len 16384 2024-09-16T00:34:48.734 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4191744 len 16384 2024-09-16T00:34:48.734 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4191744 len 16384 2024-09-16T00:34:48.734 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4191744 len 16384 2024-09-16T00:34:48.734 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4192256 len 1024 2024-09-16T00:34:48.734 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4192256 len 1024 2024-09-16T00:34:48.734 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4192256 len 1024 2024-09-16T00:34:48.734 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4192256 len 1024 2024-09-16T00:34:48.734 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4192256 len 2048 2024-09-16T00:34:48.734 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4192256 len 2048 2024-09-16T00:34:48.735 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4192256 len 2048 2024-09-16T00:34:48.735 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4192256 len 2048 2024-09-16T00:34:48.735 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4192256 len 4096 2024-09-16T00:34:48.735 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4192256 len 4096 2024-09-16T00:34:48.735 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4192256 len 4096 2024-09-16T00:34:48.735 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4192256 len 4096 2024-09-16T00:34:48.735 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4192256 len 8192 2024-09-16T00:34:48.735 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4192256 len 8192 2024-09-16T00:34:48.735 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4192256 len 8192 2024-09-16T00:34:48.735 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4192256 len 8192 2024-09-16T00:34:48.735 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4192256 len 16384 2024-09-16T00:34:48.735 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4192256 len 16384 2024-09-16T00:34:48.735 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4192256 len 16384 2024-09-16T00:34:48.735 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4192256 len 16384 2024-09-16T00:34:48.735 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4192768 len 1024 2024-09-16T00:34:48.736 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4192768 len 1024 2024-09-16T00:34:48.736 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4192768 len 1024 2024-09-16T00:34:48.736 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4192768 len 1024 2024-09-16T00:34:48.736 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4192768 len 2048 2024-09-16T00:34:48.736 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4192768 len 2048 2024-09-16T00:34:48.736 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4192768 len 2048 2024-09-16T00:34:48.736 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4192768 len 2048 2024-09-16T00:34:48.736 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4192768 len 4096 2024-09-16T00:34:48.736 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4192768 len 4096 2024-09-16T00:34:48.736 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4192768 len 4096 2024-09-16T00:34:48.736 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4192768 len 4096 2024-09-16T00:34:48.736 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4192768 len 8192 2024-09-16T00:34:48.736 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4192768 len 8192 2024-09-16T00:34:48.736 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4192768 len 8192 2024-09-16T00:34:48.737 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4192768 len 8192 2024-09-16T00:34:48.737 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4192768 len 16384 2024-09-16T00:34:48.737 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4192768 len 16384 2024-09-16T00:34:48.737 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4192768 len 16384 2024-09-16T00:34:48.737 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4192768 len 16384 2024-09-16T00:34:48.737 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4193280 len 1024 2024-09-16T00:34:49.509 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4193280 len 1024 2024-09-16T00:34:49.509 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4193280 len 1024 2024-09-16T00:34:49.509 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4193280 len 1024 2024-09-16T00:34:49.509 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4193280 len 2048 2024-09-16T00:34:49.509 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4193280 len 2048 2024-09-16T00:34:49.509 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4193280 len 2048 2024-09-16T00:34:49.509 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4193280 len 2048 2024-09-16T00:34:49.509 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4193280 len 4096 2024-09-16T00:34:49.509 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4193280 len 4096 2024-09-16T00:34:49.509 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4193280 len 4096 2024-09-16T00:34:49.510 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4193280 len 4096 2024-09-16T00:34:49.510 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4193280 len 8192 2024-09-16T00:34:49.510 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4193280 len 8192 2024-09-16T00:34:49.510 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4193280 len 8192 2024-09-16T00:34:49.510 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4193280 len 8192 2024-09-16T00:34:49.510 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4193280 len 16384 2024-09-16T00:34:49.510 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4193280 len 16384 2024-09-16T00:34:49.510 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4193280 len 16384 2024-09-16T00:34:49.510 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4193280 len 16384 2024-09-16T00:34:49.510 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4193792 len 1024 2024-09-16T00:34:49.510 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4193792 len 1024 2024-09-16T00:34:49.510 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4193792 len 1024 2024-09-16T00:34:49.510 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4193792 len 1024 2024-09-16T00:34:49.510 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4193792 len 2048 2024-09-16T00:34:49.511 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4193792 len 2048 2024-09-16T00:34:49.511 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4193792 len 2048 2024-09-16T00:34:49.511 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4193792 len 2048 2024-09-16T00:34:49.511 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4193792 len 4096 2024-09-16T00:34:49.511 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4193792 len 4096 2024-09-16T00:34:49.511 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4193792 len 4096 2024-09-16T00:34:49.511 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4193792 len 4096 2024-09-16T00:34:49.511 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4193792 len 8192 2024-09-16T00:34:49.511 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4193792 len 8192 2024-09-16T00:34:49.511 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4193792 len 8192 2024-09-16T00:34:49.511 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4193792 len 8192 2024-09-16T00:34:49.511 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4193792 len 16384 2024-09-16T00:34:49.512 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4193792 len 16384 2024-09-16T00:34:49.512 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4193792 len 16384 2024-09-16T00:34:49.512 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4193792 len 16384 2024-09-16T00:34:49.512 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4194304 len 1024 2024-09-16T00:34:49.512 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4194304 len 1024 2024-09-16T00:34:49.512 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4194304 len 1024 2024-09-16T00:34:49.512 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4194304 len 1024 2024-09-16T00:34:49.513 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4194304 len 2048 2024-09-16T00:34:49.513 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4194304 len 2048 2024-09-16T00:34:49.513 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4194304 len 2048 2024-09-16T00:34:49.513 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4194304 len 2048 2024-09-16T00:34:49.513 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4194304 len 4096 2024-09-16T00:34:49.513 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4194304 len 4096 2024-09-16T00:34:49.513 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4194304 len 4096 2024-09-16T00:34:49.513 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4194304 len 4096 2024-09-16T00:34:49.513 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4194304 len 8192 2024-09-16T00:34:49.513 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4194304 len 8192 2024-09-16T00:34:49.513 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4194304 len 8192 2024-09-16T00:34:49.514 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4194304 len 8192 2024-09-16T00:34:49.514 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4194304 len 16384 2024-09-16T00:34:49.514 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4194304 len 16384 2024-09-16T00:34:49.514 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4194304 len 16384 2024-09-16T00:34:49.514 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4194304 len 16384 2024-09-16T00:34:49.514 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4194816 len 1024 2024-09-16T00:34:49.514 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4194816 len 1024 2024-09-16T00:34:49.514 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4194816 len 1024 2024-09-16T00:34:49.514 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4194816 len 1024 2024-09-16T00:34:49.514 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4194816 len 2048 2024-09-16T00:34:49.514 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4194816 len 2048 2024-09-16T00:34:49.514 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4194816 len 2048 2024-09-16T00:34:49.514 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4194816 len 2048 2024-09-16T00:34:49.514 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4194816 len 4096 2024-09-16T00:34:49.515 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4194816 len 4096 2024-09-16T00:34:49.515 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4194816 len 4096 2024-09-16T00:34:49.515 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4194816 len 4096 2024-09-16T00:34:49.515 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4194816 len 8192 2024-09-16T00:34:49.515 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4194816 len 8192 2024-09-16T00:34:49.515 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4194816 len 8192 2024-09-16T00:34:49.515 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4194816 len 8192 2024-09-16T00:34:49.515 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4194816 len 16384 2024-09-16T00:34:49.515 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4194816 len 16384 2024-09-16T00:34:49.515 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4194816 len 16384 2024-09-16T00:34:49.515 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4194816 len 16384 2024-09-16T00:34:49.515 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4195328 len 1024 2024-09-16T00:34:49.515 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4195328 len 1024 2024-09-16T00:34:49.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:49 smithi038 ceph-mon[30945]: pgmap v425: 129 pgs: 129 active+clean; 316 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s rd, 810 KiB/s wr, 278 op/s 2024-09-16T00:34:49.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:49 smithi122 ceph-mon[35412]: pgmap v425: 129 pgs: 129 active+clean; 316 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s rd, 810 KiB/s wr, 278 op/s 2024-09-16T00:34:49.793 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:49 smithi073 ceph-mon[35254]: pgmap v425: 129 pgs: 129 active+clean; 316 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 92 MiB/s rd, 810 KiB/s wr, 278 op/s 2024-09-16T00:34:49.874 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4195328 len 1024 2024-09-16T00:34:49.874 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4195328 len 1024 2024-09-16T00:34:49.874 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4195328 len 2048 2024-09-16T00:34:49.874 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4195328 len 2048 2024-09-16T00:34:49.874 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4195328 len 2048 2024-09-16T00:34:49.875 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4195328 len 2048 2024-09-16T00:34:49.875 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4195328 len 4096 2024-09-16T00:34:49.875 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4195328 len 4096 2024-09-16T00:34:49.875 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4195328 len 4096 2024-09-16T00:34:49.875 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4195328 len 4096 2024-09-16T00:34:49.875 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4195328 len 8192 2024-09-16T00:34:49.875 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4195328 len 8192 2024-09-16T00:34:49.875 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4195328 len 8192 2024-09-16T00:34:49.875 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4195328 len 8192 2024-09-16T00:34:49.875 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4195328 len 16384 2024-09-16T00:34:49.875 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4195328 len 16384 2024-09-16T00:34:49.876 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4195328 len 16384 2024-09-16T00:34:49.876 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4195328 len 16384 2024-09-16T00:34:49.876 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4195840 len 1024 2024-09-16T00:34:49.876 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4195840 len 1024 2024-09-16T00:34:49.876 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4195840 len 1024 2024-09-16T00:34:49.876 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4195840 len 1024 2024-09-16T00:34:49.876 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4195840 len 2048 2024-09-16T00:34:49.876 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4195840 len 2048 2024-09-16T00:34:49.876 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4195840 len 2048 2024-09-16T00:34:49.876 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4195840 len 2048 2024-09-16T00:34:49.876 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4195840 len 4096 2024-09-16T00:34:49.876 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4195840 len 4096 2024-09-16T00:34:49.876 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4195840 len 4096 2024-09-16T00:34:49.877 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4195840 len 4096 2024-09-16T00:34:49.877 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4195840 len 8192 2024-09-16T00:34:49.877 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4195840 len 8192 2024-09-16T00:34:49.877 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4195840 len 8192 2024-09-16T00:34:49.877 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4195840 len 8192 2024-09-16T00:34:49.877 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4195840 len 16384 2024-09-16T00:34:49.877 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4195840 len 16384 2024-09-16T00:34:49.877 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4195840 len 16384 2024-09-16T00:34:49.877 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4195840 len 16384 2024-09-16T00:34:49.877 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4196352 len 1024 2024-09-16T00:34:49.877 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4196352 len 1024 2024-09-16T00:34:49.877 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4196352 len 1024 2024-09-16T00:34:49.877 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4196352 len 1024 2024-09-16T00:34:49.878 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4196352 len 2048 2024-09-16T00:34:49.878 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4196352 len 2048 2024-09-16T00:34:49.878 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4196352 len 2048 2024-09-16T00:34:49.878 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4196352 len 2048 2024-09-16T00:34:49.878 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4196352 len 4096 2024-09-16T00:34:49.878 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4196352 len 4096 2024-09-16T00:34:49.878 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4196352 len 4096 2024-09-16T00:34:49.878 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4196352 len 4096 2024-09-16T00:34:49.878 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4196352 len 8192 2024-09-16T00:34:49.878 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4196352 len 8192 2024-09-16T00:34:49.878 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4196352 len 8192 2024-09-16T00:34:49.879 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4196352 len 8192 2024-09-16T00:34:49.879 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4196352 len 16384 2024-09-16T00:34:49.879 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4196352 len 16384 2024-09-16T00:34:49.879 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4196352 len 16384 2024-09-16T00:34:49.879 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4196352 len 16384 2024-09-16T00:34:49.879 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4196864 len 1024 2024-09-16T00:34:49.879 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4196864 len 1024 2024-09-16T00:34:49.879 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4196864 len 1024 2024-09-16T00:34:49.879 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4196864 len 1024 2024-09-16T00:34:49.879 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4196864 len 2048 2024-09-16T00:34:49.879 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4196864 len 2048 2024-09-16T00:34:49.879 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4196864 len 2048 2024-09-16T00:34:49.879 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4196864 len 2048 2024-09-16T00:34:49.880 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4196864 len 4096 2024-09-16T00:34:49.880 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4196864 len 4096 2024-09-16T00:34:49.880 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4196864 len 4096 2024-09-16T00:34:49.880 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4196864 len 4096 2024-09-16T00:34:49.880 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4196864 len 8192 2024-09-16T00:34:49.880 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4196864 len 8192 2024-09-16T00:34:49.880 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4196864 len 8192 2024-09-16T00:34:49.880 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4196864 len 8192 2024-09-16T00:34:49.881 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4196864 len 16384 2024-09-16T00:34:49.881 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4196864 len 16384 2024-09-16T00:34:49.881 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4196864 len 16384 2024-09-16T00:34:49.881 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4196864 len 16384 2024-09-16T00:34:49.881 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4197376 len 1024 2024-09-16T00:34:49.882 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4197376 len 1024 2024-09-16T00:34:49.882 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4197376 len 1024 2024-09-16T00:34:50.663 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4197376 len 1024 2024-09-16T00:34:50.663 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4197376 len 2048 2024-09-16T00:34:50.663 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4197376 len 2048 2024-09-16T00:34:50.663 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4197376 len 2048 2024-09-16T00:34:50.663 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4197376 len 2048 2024-09-16T00:34:50.663 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4197376 len 4096 2024-09-16T00:34:50.663 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4197376 len 4096 2024-09-16T00:34:50.664 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4197376 len 4096 2024-09-16T00:34:50.664 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4197376 len 4096 2024-09-16T00:34:50.664 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4197376 len 8192 2024-09-16T00:34:50.664 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4197376 len 8192 2024-09-16T00:34:50.664 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4197376 len 8192 2024-09-16T00:34:50.664 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4197376 len 8192 2024-09-16T00:34:50.664 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4197376 len 16384 2024-09-16T00:34:50.664 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4197376 len 16384 2024-09-16T00:34:50.664 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4197376 len 16384 2024-09-16T00:34:50.664 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4197376 len 16384 2024-09-16T00:34:50.664 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4197888 len 1024 2024-09-16T00:34:50.664 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4197888 len 1024 2024-09-16T00:34:50.664 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4197888 len 1024 2024-09-16T00:34:50.664 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4197888 len 1024 2024-09-16T00:34:50.665 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4197888 len 2048 2024-09-16T00:34:50.665 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4197888 len 2048 2024-09-16T00:34:50.665 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4197888 len 2048 2024-09-16T00:34:50.665 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4197888 len 2048 2024-09-16T00:34:50.665 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4197888 len 4096 2024-09-16T00:34:50.665 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4197888 len 4096 2024-09-16T00:34:50.665 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4197888 len 4096 2024-09-16T00:34:50.665 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4197888 len 4096 2024-09-16T00:34:50.665 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4197888 len 8192 2024-09-16T00:34:50.665 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4197888 len 8192 2024-09-16T00:34:50.665 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4197888 len 8192 2024-09-16T00:34:50.665 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4197888 len 8192 2024-09-16T00:34:50.665 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3072 offset 4197888 len 16384 2024-09-16T00:34:50.666 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4197888 len 16384 2024-09-16T00:34:50.666 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3072 offset 4197888 len 16384 2024-09-16T00:34:50.666 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3072 offset 4197888 len 16384 2024-09-16T00:34:50.666 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4190720 len 1024 2024-09-16T00:34:50.666 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4190720 len 1024 2024-09-16T00:34:50.666 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4190720 len 1024 2024-09-16T00:34:50.666 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4190720 len 1024 2024-09-16T00:34:50.666 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4190720 len 2048 2024-09-16T00:34:50.666 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4190720 len 2048 2024-09-16T00:34:50.666 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4190720 len 2048 2024-09-16T00:34:50.666 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4190720 len 2048 2024-09-16T00:34:50.666 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4190720 len 4096 2024-09-16T00:34:50.666 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4190720 len 4096 2024-09-16T00:34:50.666 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4190720 len 4096 2024-09-16T00:34:50.667 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4190720 len 4096 2024-09-16T00:34:50.667 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4190720 len 8192 2024-09-16T00:34:50.667 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4190720 len 8192 2024-09-16T00:34:50.667 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4190720 len 8192 2024-09-16T00:34:50.667 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4190720 len 8192 2024-09-16T00:34:50.667 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4190720 len 16384 2024-09-16T00:34:50.667 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4190720 len 16384 2024-09-16T00:34:50.667 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4190720 len 16384 2024-09-16T00:34:50.667 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4190720 len 16384 2024-09-16T00:34:50.667 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4191232 len 1024 2024-09-16T00:34:50.667 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4191232 len 1024 2024-09-16T00:34:50.667 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4191232 len 1024 2024-09-16T00:34:50.667 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4191232 len 1024 2024-09-16T00:34:50.668 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4191232 len 2048 2024-09-16T00:34:50.668 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4191232 len 2048 2024-09-16T00:34:50.668 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4191232 len 2048 2024-09-16T00:34:50.668 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4191232 len 2048 2024-09-16T00:34:50.668 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4191232 len 4096 2024-09-16T00:34:50.668 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4191232 len 4096 2024-09-16T00:34:50.668 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4191232 len 4096 2024-09-16T00:34:50.668 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4191232 len 4096 2024-09-16T00:34:50.668 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4191232 len 8192 2024-09-16T00:34:50.668 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4191232 len 8192 2024-09-16T00:34:50.668 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4191232 len 8192 2024-09-16T00:34:50.668 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4191232 len 8192 2024-09-16T00:34:50.668 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4191232 len 16384 2024-09-16T00:34:50.669 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4191232 len 16384 2024-09-16T00:34:50.669 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4191232 len 16384 2024-09-16T00:34:50.669 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4191232 len 16384 2024-09-16T00:34:50.669 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4191744 len 1024 2024-09-16T00:34:50.669 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4191744 len 1024 2024-09-16T00:34:50.669 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4191744 len 1024 2024-09-16T00:34:50.669 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4191744 len 1024 2024-09-16T00:34:51.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:51 smithi038 ceph-mon[30945]: pgmap v426: 129 pgs: 129 active+clean; 290 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 100 MiB/s rd, 968 KiB/s wr, 327 op/s 2024-09-16T00:34:51.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:51 smithi122 ceph-mon[35412]: pgmap v426: 129 pgs: 129 active+clean; 290 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 100 MiB/s rd, 968 KiB/s wr, 327 op/s 2024-09-16T00:34:51.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:51 smithi073 ceph-mon[35254]: pgmap v426: 129 pgs: 129 active+clean; 290 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 100 MiB/s rd, 968 KiB/s wr, 327 op/s 2024-09-16T00:34:51.853 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4191744 len 2048 2024-09-16T00:34:51.853 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4191744 len 2048 2024-09-16T00:34:51.853 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4191744 len 2048 2024-09-16T00:34:51.854 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4191744 len 2048 2024-09-16T00:34:51.854 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4191744 len 4096 2024-09-16T00:34:51.854 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4191744 len 4096 2024-09-16T00:34:51.854 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4191744 len 4096 2024-09-16T00:34:51.854 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4191744 len 4096 2024-09-16T00:34:51.854 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4191744 len 8192 2024-09-16T00:34:51.854 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4191744 len 8192 2024-09-16T00:34:51.854 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4191744 len 8192 2024-09-16T00:34:51.854 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4191744 len 8192 2024-09-16T00:34:51.854 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4191744 len 16384 2024-09-16T00:34:51.854 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4191744 len 16384 2024-09-16T00:34:51.854 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4191744 len 16384 2024-09-16T00:34:51.854 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4191744 len 16384 2024-09-16T00:34:51.855 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4192256 len 1024 2024-09-16T00:34:51.855 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4192256 len 1024 2024-09-16T00:34:51.855 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4192256 len 1024 2024-09-16T00:34:51.855 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4192256 len 1024 2024-09-16T00:34:51.855 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4192256 len 2048 2024-09-16T00:34:51.855 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4192256 len 2048 2024-09-16T00:34:51.855 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4192256 len 2048 2024-09-16T00:34:51.855 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4192256 len 2048 2024-09-16T00:34:51.855 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4192256 len 4096 2024-09-16T00:34:51.855 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4192256 len 4096 2024-09-16T00:34:51.855 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4192256 len 4096 2024-09-16T00:34:51.855 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4192256 len 4096 2024-09-16T00:34:51.855 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4192256 len 8192 2024-09-16T00:34:51.855 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4192256 len 8192 2024-09-16T00:34:51.856 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4192256 len 8192 2024-09-16T00:34:51.856 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4192256 len 8192 2024-09-16T00:34:51.856 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4192256 len 16384 2024-09-16T00:34:51.856 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4192256 len 16384 2024-09-16T00:34:51.856 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4192256 len 16384 2024-09-16T00:34:51.856 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4192256 len 16384 2024-09-16T00:34:51.856 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4192768 len 1024 2024-09-16T00:34:51.856 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4192768 len 1024 2024-09-16T00:34:51.856 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4192768 len 1024 2024-09-16T00:34:51.856 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4192768 len 1024 2024-09-16T00:34:51.856 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4192768 len 2048 2024-09-16T00:34:51.856 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4192768 len 2048 2024-09-16T00:34:51.856 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4192768 len 2048 2024-09-16T00:34:51.856 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4192768 len 2048 2024-09-16T00:34:51.857 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4192768 len 4096 2024-09-16T00:34:51.857 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4192768 len 4096 2024-09-16T00:34:51.857 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4192768 len 4096 2024-09-16T00:34:51.857 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4192768 len 4096 2024-09-16T00:34:51.857 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4192768 len 8192 2024-09-16T00:34:51.857 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4192768 len 8192 2024-09-16T00:34:51.857 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4192768 len 8192 2024-09-16T00:34:51.857 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4192768 len 8192 2024-09-16T00:34:51.857 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4192768 len 16384 2024-09-16T00:34:51.857 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4192768 len 16384 2024-09-16T00:34:51.857 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4192768 len 16384 2024-09-16T00:34:51.857 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4192768 len 16384 2024-09-16T00:34:51.857 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4193280 len 1024 2024-09-16T00:34:51.857 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4193280 len 1024 2024-09-16T00:34:51.858 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4193280 len 1024 2024-09-16T00:34:51.858 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4193280 len 1024 2024-09-16T00:34:51.858 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4193280 len 2048 2024-09-16T00:34:51.858 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4193280 len 2048 2024-09-16T00:34:51.858 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4193280 len 2048 2024-09-16T00:34:51.858 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4193280 len 2048 2024-09-16T00:34:51.858 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4193280 len 4096 2024-09-16T00:34:51.858 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4193280 len 4096 2024-09-16T00:34:51.858 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4193280 len 4096 2024-09-16T00:34:51.858 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4193280 len 4096 2024-09-16T00:34:51.858 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4193280 len 8192 2024-09-16T00:34:51.858 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4193280 len 8192 2024-09-16T00:34:51.858 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4193280 len 8192 2024-09-16T00:34:51.859 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4193280 len 8192 2024-09-16T00:34:51.859 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4193280 len 16384 2024-09-16T00:34:51.859 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4193280 len 16384 2024-09-16T00:34:51.859 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4193280 len 16384 2024-09-16T00:34:51.859 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4193280 len 16384 2024-09-16T00:34:51.859 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4193792 len 1024 2024-09-16T00:34:51.859 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4193792 len 1024 2024-09-16T00:34:51.859 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4193792 len 1024 2024-09-16T00:34:51.859 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4193792 len 1024 2024-09-16T00:34:52.394 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4193792 len 2048 2024-09-16T00:34:52.394 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4193792 len 2048 2024-09-16T00:34:52.394 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4193792 len 2048 2024-09-16T00:34:52.394 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4193792 len 2048 2024-09-16T00:34:52.395 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4193792 len 4096 2024-09-16T00:34:52.395 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4193792 len 4096 2024-09-16T00:34:52.395 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4193792 len 4096 2024-09-16T00:34:52.395 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4193792 len 4096 2024-09-16T00:34:52.395 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4193792 len 8192 2024-09-16T00:34:52.395 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4193792 len 8192 2024-09-16T00:34:52.395 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4193792 len 8192 2024-09-16T00:34:52.395 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4193792 len 8192 2024-09-16T00:34:52.395 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4193792 len 16384 2024-09-16T00:34:52.395 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4193792 len 16384 2024-09-16T00:34:52.395 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4193792 len 16384 2024-09-16T00:34:52.395 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4193792 len 16384 2024-09-16T00:34:52.395 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4194304 len 1024 2024-09-16T00:34:52.395 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4194304 len 1024 2024-09-16T00:34:52.396 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4194304 len 1024 2024-09-16T00:34:52.396 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4194304 len 1024 2024-09-16T00:34:52.396 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4194304 len 2048 2024-09-16T00:34:52.396 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4194304 len 2048 2024-09-16T00:34:52.396 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4194304 len 2048 2024-09-16T00:34:52.396 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4194304 len 2048 2024-09-16T00:34:52.396 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4194304 len 4096 2024-09-16T00:34:52.396 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4194304 len 4096 2024-09-16T00:34:52.396 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4194304 len 4096 2024-09-16T00:34:52.396 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4194304 len 4096 2024-09-16T00:34:52.396 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4194304 len 8192 2024-09-16T00:34:52.396 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4194304 len 8192 2024-09-16T00:34:52.396 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4194304 len 8192 2024-09-16T00:34:52.397 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4194304 len 8192 2024-09-16T00:34:52.397 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4194304 len 16384 2024-09-16T00:34:52.397 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4194304 len 16384 2024-09-16T00:34:52.397 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4194304 len 16384 2024-09-16T00:34:52.397 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4194304 len 16384 2024-09-16T00:34:52.397 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4194816 len 1024 2024-09-16T00:34:52.397 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4194816 len 1024 2024-09-16T00:34:52.397 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4194816 len 1024 2024-09-16T00:34:52.397 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4194816 len 1024 2024-09-16T00:34:52.397 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4194816 len 2048 2024-09-16T00:34:52.397 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4194816 len 2048 2024-09-16T00:34:52.397 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4194816 len 2048 2024-09-16T00:34:52.397 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4194816 len 2048 2024-09-16T00:34:52.397 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4194816 len 4096 2024-09-16T00:34:52.398 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4194816 len 4096 2024-09-16T00:34:52.398 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4194816 len 4096 2024-09-16T00:34:52.398 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4194816 len 4096 2024-09-16T00:34:52.398 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4194816 len 8192 2024-09-16T00:34:52.398 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4194816 len 8192 2024-09-16T00:34:52.398 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4194816 len 8192 2024-09-16T00:34:52.398 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4194816 len 8192 2024-09-16T00:34:52.398 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4194816 len 16384 2024-09-16T00:34:52.398 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4194816 len 16384 2024-09-16T00:34:52.398 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4194816 len 16384 2024-09-16T00:34:52.398 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4194816 len 16384 2024-09-16T00:34:52.398 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4195328 len 1024 2024-09-16T00:34:52.398 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4195328 len 1024 2024-09-16T00:34:52.398 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4195328 len 1024 2024-09-16T00:34:52.399 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4195328 len 1024 2024-09-16T00:34:52.399 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4195328 len 2048 2024-09-16T00:34:52.399 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4195328 len 2048 2024-09-16T00:34:52.399 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4195328 len 2048 2024-09-16T00:34:52.399 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4195328 len 2048 2024-09-16T00:34:52.399 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4195328 len 4096 2024-09-16T00:34:52.399 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4195328 len 4096 2024-09-16T00:34:52.399 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4195328 len 4096 2024-09-16T00:34:52.399 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4195328 len 4096 2024-09-16T00:34:52.399 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4195328 len 8192 2024-09-16T00:34:52.399 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4195328 len 8192 2024-09-16T00:34:52.399 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4195328 len 8192 2024-09-16T00:34:52.400 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4195328 len 8192 2024-09-16T00:34:52.400 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4195328 len 16384 2024-09-16T00:34:52.400 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4195328 len 16384 2024-09-16T00:34:52.400 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4195328 len 16384 2024-09-16T00:34:52.400 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4195328 len 16384 2024-09-16T00:34:52.400 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4195840 len 1024 2024-09-16T00:34:52.400 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4195840 len 1024 2024-09-16T00:34:52.400 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4195840 len 1024 2024-09-16T00:34:52.400 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4195840 len 1024 2024-09-16T00:34:52.400 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4195840 len 2048 2024-09-16T00:34:52.837 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4195840 len 2048 2024-09-16T00:34:52.837 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4195840 len 2048 2024-09-16T00:34:52.837 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4195840 len 2048 2024-09-16T00:34:52.837 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4195840 len 4096 2024-09-16T00:34:52.837 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4195840 len 4096 2024-09-16T00:34:52.837 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4195840 len 4096 2024-09-16T00:34:52.837 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4195840 len 4096 2024-09-16T00:34:52.837 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4195840 len 8192 2024-09-16T00:34:52.838 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4195840 len 8192 2024-09-16T00:34:52.838 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4195840 len 8192 2024-09-16T00:34:52.838 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4195840 len 8192 2024-09-16T00:34:52.838 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4195840 len 16384 2024-09-16T00:34:52.838 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4195840 len 16384 2024-09-16T00:34:52.838 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4195840 len 16384 2024-09-16T00:34:52.838 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4195840 len 16384 2024-09-16T00:34:52.838 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4196352 len 1024 2024-09-16T00:34:52.838 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4196352 len 1024 2024-09-16T00:34:52.838 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4196352 len 1024 2024-09-16T00:34:52.839 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4196352 len 1024 2024-09-16T00:34:52.839 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4196352 len 2048 2024-09-16T00:34:52.839 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4196352 len 2048 2024-09-16T00:34:52.839 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4196352 len 2048 2024-09-16T00:34:52.839 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4196352 len 2048 2024-09-16T00:34:52.839 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4196352 len 4096 2024-09-16T00:34:52.839 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4196352 len 4096 2024-09-16T00:34:52.839 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4196352 len 4096 2024-09-16T00:34:52.839 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4196352 len 4096 2024-09-16T00:34:52.839 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4196352 len 8192 2024-09-16T00:34:52.839 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4196352 len 8192 2024-09-16T00:34:52.839 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4196352 len 8192 2024-09-16T00:34:52.839 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4196352 len 8192 2024-09-16T00:34:52.839 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4196352 len 16384 2024-09-16T00:34:52.840 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4196352 len 16384 2024-09-16T00:34:52.840 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4196352 len 16384 2024-09-16T00:34:52.840 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4196352 len 16384 2024-09-16T00:34:52.840 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4196864 len 1024 2024-09-16T00:34:52.840 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4196864 len 1024 2024-09-16T00:34:52.840 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4196864 len 1024 2024-09-16T00:34:52.840 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4196864 len 1024 2024-09-16T00:34:52.840 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4196864 len 2048 2024-09-16T00:34:52.840 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4196864 len 2048 2024-09-16T00:34:52.840 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4196864 len 2048 2024-09-16T00:34:52.840 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4196864 len 2048 2024-09-16T00:34:52.840 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4196864 len 4096 2024-09-16T00:34:52.840 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4196864 len 4096 2024-09-16T00:34:52.841 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4196864 len 4096 2024-09-16T00:34:52.841 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4196864 len 4096 2024-09-16T00:34:52.841 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4196864 len 8192 2024-09-16T00:34:52.841 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4196864 len 8192 2024-09-16T00:34:52.841 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4196864 len 8192 2024-09-16T00:34:52.841 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4196864 len 8192 2024-09-16T00:34:52.841 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4196864 len 16384 2024-09-16T00:34:52.841 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4196864 len 16384 2024-09-16T00:34:52.841 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4196864 len 16384 2024-09-16T00:34:52.841 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4196864 len 16384 2024-09-16T00:34:52.841 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4197376 len 1024 2024-09-16T00:34:52.841 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4197376 len 1024 2024-09-16T00:34:52.841 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4197376 len 1024 2024-09-16T00:34:52.841 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4197376 len 1024 2024-09-16T00:34:52.842 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4197376 len 2048 2024-09-16T00:34:52.842 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4197376 len 2048 2024-09-16T00:34:52.842 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4197376 len 2048 2024-09-16T00:34:52.842 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4197376 len 2048 2024-09-16T00:34:52.842 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4197376 len 4096 2024-09-16T00:34:52.842 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4197376 len 4096 2024-09-16T00:34:52.842 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4197376 len 4096 2024-09-16T00:34:52.842 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4197376 len 4096 2024-09-16T00:34:52.842 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4197376 len 8192 2024-09-16T00:34:52.842 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4197376 len 8192 2024-09-16T00:34:52.843 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4197376 len 8192 2024-09-16T00:34:52.843 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4197376 len 8192 2024-09-16T00:34:52.843 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4197376 len 16384 2024-09-16T00:34:52.843 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4197376 len 16384 2024-09-16T00:34:52.843 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4197376 len 16384 2024-09-16T00:34:52.843 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4197376 len 16384 2024-09-16T00:34:52.843 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4197888 len 1024 2024-09-16T00:34:52.843 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4197888 len 1024 2024-09-16T00:34:52.843 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4197888 len 1024 2024-09-16T00:34:52.843 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4197888 len 1024 2024-09-16T00:34:52.843 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4197888 len 2048 2024-09-16T00:34:52.843 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4197888 len 2048 2024-09-16T00:34:52.921 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4197888 len 2048 2024-09-16T00:34:52.922 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4197888 len 2048 2024-09-16T00:34:52.922 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4197888 len 4096 2024-09-16T00:34:52.922 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4197888 len 4096 2024-09-16T00:34:52.922 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4197888 len 4096 2024-09-16T00:34:52.922 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4197888 len 4096 2024-09-16T00:34:52.922 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4197888 len 8192 2024-09-16T00:34:52.922 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4197888 len 8192 2024-09-16T00:34:52.922 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4197888 len 8192 2024-09-16T00:34:52.922 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4197888 len 8192 2024-09-16T00:34:52.922 INFO:tasks.workunit.client.0.smithi038.stdout:write_direct buf_align 3584 offset 4197888 len 16384 2024-09-16T00:34:52.922 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4197888 len 16384 2024-09-16T00:34:52.922 INFO:tasks.workunit.client.0.smithi038.stdout:write_sync buf_align 3584 offset 4197888 len 16384 2024-09-16T00:34:52.923 INFO:tasks.workunit.client.0.smithi038.stdout:write_file buf_align 3584 offset 4197888 len 16384 2024-09-16T00:34:52.923 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-16T00:34:52.923 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-16T00:34:52.954 INFO:tasks.workunit:Stopping ['direct_io'] on client.0... 2024-09-16T00:34:52.954 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-16T00:34:53.358 DEBUG:teuthology.parallel:result is None 2024-09-16T00:34:53.358 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-16T00:34:53.389 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-16T00:34:53.389 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-09-16T00:34:53.399 INFO:tasks.quiescer:joining Quiescers 2024-09-16T00:34:53.399 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-09-16T00:34:53.399 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:34:53.685 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:34:53.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:53 smithi038 ceph-mon[30945]: pgmap v427: 129 pgs: 129 active+clean; 260 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 99 MiB/s rd, 1022 KiB/s wr, 336 op/s 2024-09-16T00:34:53.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:53 smithi122 ceph-mon[35412]: pgmap v427: 129 pgs: 129 active+clean; 260 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 99 MiB/s rd, 1022 KiB/s wr, 336 op/s 2024-09-16T00:34:53.793 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:53 smithi073 ceph-mon[35254]: pgmap v427: 129 pgs: 129 active+clean; 260 MiB data, 1022 MiB used, 1.0 TiB / 1.0 TiB avail; 99 MiB/s rd, 1022 KiB/s wr, 336 op/s 2024-09-16T00:34:54.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.195+0000 7fcb45a45640 1 -- 172.21.15.38:0/135284137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb40102790 msgr2=0x7fcb401067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:54.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.195+0000 7fcb45a45640 1 --2- 172.21.15.38:0/135284137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb40102790 0x7fcb401067e0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fcb34009900 tx=0x7fcb3402a410 comp rx=0 tx=0).stop 2024-09-16T00:34:54.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.195+0000 7fcb45a45640 1 -- 172.21.15.38:0/135284137 shutdown_connections 2024-09-16T00:34:54.213 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.195+0000 7fcb45a45640 1 --2- 172.21.15.38:0/135284137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb40106f10 0x7fcb4010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:54.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.195+0000 7fcb45a45640 1 --2- 172.21.15.38:0/135284137 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb40102790 0x7fcb401067e0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:54.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.195+0000 7fcb45a45640 1 --2- 172.21.15.38:0/135284137 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb40101de0 0x7fcb401021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:54.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.196+0000 7fcb45a45640 1 -- 172.21.15.38:0/135284137 >> 172.21.15.38:0/135284137 conn(0x7fcb400fd8c0 msgr2=0x7fcb400ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:54.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.196+0000 7fcb45a45640 1 -- 172.21.15.38:0/135284137 shutdown_connections 2024-09-16T00:34:54.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.196+0000 7fcb45a45640 1 -- 172.21.15.38:0/135284137 wait complete. 2024-09-16T00:34:54.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.196+0000 7fcb45a45640 1 Processor -- start 2024-09-16T00:34:54.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.196+0000 7fcb45a45640 1 -- start start 2024-09-16T00:34:54.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.196+0000 7fcb45a45640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb40101de0 0x7fcb40198410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:54.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.196+0000 7fcb45a45640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb40102790 0x7fcb40198950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:54.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.197+0000 7fcb45a45640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb40106f10 0x7fcb4019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:54.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.197+0000 7fcb45a45640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fcb40111f30 con 0x7fcb40102790 2024-09-16T00:34:54.214 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.197+0000 7fcb45a45640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fcb40111db0 con 0x7fcb40106f10 2024-09-16T00:34:54.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.197+0000 7fcb45a45640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fcb401120b0 con 0x7fcb40101de0 2024-09-16T00:34:54.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.197+0000 7fcb3e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb40102790 0x7fcb40198950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:54.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.197+0000 7fcb3e7fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb40102790 0x7fcb40198950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:41796/0 (socket says 172.21.15.38:41796) 2024-09-16T00:34:54.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.197+0000 7fcb3e7fc640 1 -- 172.21.15.38:0/2416593672 learned_addr learned my addr 172.21.15.38:0/2416593672 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:34:54.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.197+0000 7fcb3effd640 1 --2- 172.21.15.38:0/2416593672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb40101de0 0x7fcb40198410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:54.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.197+0000 7fcb3f7fe640 1 --2- 172.21.15.38:0/2416593672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb40106f10 0x7fcb4019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:54.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.197+0000 7fcb3e7fc640 1 -- 172.21.15.38:0/2416593672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb40101de0 msgr2=0x7fcb40198410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:54.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.197+0000 7fcb3e7fc640 1 --2- 172.21.15.38:0/2416593672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb40101de0 0x7fcb40198410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:54.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.197+0000 7fcb3e7fc640 1 -- 172.21.15.38:0/2416593672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb40106f10 msgr2=0x7fcb4019cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:54.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.197+0000 7fcb3e7fc640 1 --2- 172.21.15.38:0/2416593672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb40106f10 0x7fcb4019cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:54.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.197+0000 7fcb3e7fc640 1 -- 172.21.15.38:0/2416593672 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb4019d3c0 con 0x7fcb40102790 2024-09-16T00:34:54.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.197+0000 7fcb3e7fc640 1 --2- 172.21.15.38:0/2416593672 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb40102790 0x7fcb40198950 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fcb34006010 tx=0x7fcb340042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:54.215 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.197+0000 7fcb44a43640 1 -- 172.21.15.38:0/2416593672 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb3403f030 con 0x7fcb40102790 2024-09-16T00:34:54.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.197+0000 7fcb45a45640 1 -- 172.21.15.38:0/2416593672 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb4019d650 con 0x7fcb40102790 2024-09-16T00:34:54.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.198+0000 7fcb44a43640 1 -- 172.21.15.38:0/2416593672 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fcb34003ce0 con 0x7fcb40102790 2024-09-16T00:34:54.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.198+0000 7fcb44a43640 1 -- 172.21.15.38:0/2416593672 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb34035070 con 0x7fcb40102790 2024-09-16T00:34:54.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.198+0000 7fcb45a45640 1 -- 172.21.15.38:0/2416593672 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb401a4f90 con 0x7fcb40102790 2024-09-16T00:34:54.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.199+0000 7fcb45a45640 1 -- 172.21.15.38:0/2416593672 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb401021c0 con 0x7fcb40102790 2024-09-16T00:34:54.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.200+0000 7fcb44a43640 1 -- 172.21.15.38:0/2416593672 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fcb34042030 con 0x7fcb40102790 2024-09-16T00:34:54.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.200+0000 7fcb44a43640 1 --2- 172.21.15.38:0/2416593672 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fcb280a3240 0x7fcb280a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:54.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.201+0000 7fcb3effd640 1 --2- 172.21.15.38:0/2416593672 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fcb280a3240 0x7fcb280a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:54.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.201+0000 7fcb44a43640 1 -- 172.21.15.38:0/2416593672 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7fcb340f0000 con 0x7fcb40102790 2024-09-16T00:34:54.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.201+0000 7fcb3effd640 1 --2- 172.21.15.38:0/2416593672 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fcb280a3240 0x7fcb280a5700 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcb3000abe0 tx=0x7fcb3000a2d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:54.216 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.205+0000 7fcb44a43640 1 -- 172.21.15.38:0/2416593672 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fcb340bc9a0 con 0x7fcb40102790 2024-09-16T00:34:54.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.418+0000 7fcb45a45640 1 -- 172.21.15.38:0/2416593672 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcb4010a0f0 con 0x7fcb40102790 2024-09-16T00:34:54.419 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.419+0000 7fcb44a43640 1 -- 172.21.15.38:0/2416593672 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7fcb340bc2a0 con 0x7fcb40102790 2024-09-16T00:34:54.419 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:34:54.420 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"btime":"2024-09-16T00:32:41:853514+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/367974527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":367974527},{"type":"v1","addr":"172.21.15.38:6835","nonce":367974527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/335077136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":335077136},{"type":"v1","addr":"172.21.15.38:6837","nonce":335077136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/3501211160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3501211160},{"type":"v1","addr":"172.21.15.38:6839","nonce":3501211160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6841/2623130915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":2623130915},{"type":"v1","addr":"172.21.15.38:6841","nonce":2623130915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/412277519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":412277519},{"type":"v1","addr":"172.21.15.122:6833","nonce":412277519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3987234757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3987234757},{"type":"v1","addr":"172.21.15.122:6835","nonce":3987234757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:30:26.764565+0000","modified":"2024-09-16T00:32:41.355828+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24440},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1210328177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1210328177},{"type":"v1","addr":"172.21.15.73:6833","nonce":1210328177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6837/1858299680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1858299680},{"type":"v1","addr":"172.21.15.122:6837","nonce":1858299680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1983975311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1983975311},{"type":"v1","addr":"172.21.15.122:6839","nonce":1983975311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.73:6835/2866363113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2866363113},{"type":"v1","addr":"172.21.15.73:6835","nonce":2866363113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.73:6837/2681081265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2681081265},{"type":"v1","addr":"172.21.15.73:6837","nonce":2681081265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.73:6839/2396224525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2396224525},{"type":"v1","addr":"172.21.15.73:6839","nonce":2396224525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24464,24440]},"id":1}]} 2024-09-16T00:34:54.420 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 26 2024-09-16T00:34:54.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.421+0000 7fcb45a45640 1 -- 172.21.15.38:0/2416593672 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fcb280a3240 msgr2=0x7fcb280a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:54.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.421+0000 7fcb45a45640 1 --2- 172.21.15.38:0/2416593672 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fcb280a3240 0x7fcb280a5700 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fcb3000abe0 tx=0x7fcb3000a2d0 comp rx=0 tx=0).stop 2024-09-16T00:34:54.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.421+0000 7fcb45a45640 1 -- 172.21.15.38:0/2416593672 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb40102790 msgr2=0x7fcb40198950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:54.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.421+0000 7fcb45a45640 1 --2- 172.21.15.38:0/2416593672 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb40102790 0x7fcb40198950 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7fcb34006010 tx=0x7fcb340042f0 comp rx=0 tx=0).stop 2024-09-16T00:34:54.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.421+0000 7fcb45a45640 1 -- 172.21.15.38:0/2416593672 shutdown_connections 2024-09-16T00:34:54.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.421+0000 7fcb45a45640 1 --2- 172.21.15.38:0/2416593672 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fcb280a3240 0x7fcb280a5700 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:54.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.421+0000 7fcb45a45640 1 --2- 172.21.15.38:0/2416593672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb40106f10 0x7fcb4019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:54.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.421+0000 7fcb45a45640 1 --2- 172.21.15.38:0/2416593672 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fcb40102790 0x7fcb40198950 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:54.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.421+0000 7fcb45a45640 1 --2- 172.21.15.38:0/2416593672 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fcb40101de0 0x7fcb40198410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:54.422 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.421+0000 7fcb45a45640 1 -- 172.21.15.38:0/2416593672 >> 172.21.15.38:0/2416593672 conn(0x7fcb400fd8c0 msgr2=0x7fcb400fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:54.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.421+0000 7fcb45a45640 1 -- 172.21.15.38:0/2416593672 shutdown_connections 2024-09-16T00:34:54.423 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:54.421+0000 7fcb45a45640 1 -- 172.21.15.38:0/2416593672 wait complete. 2024-09-16T00:34:55.387 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24494 quiesce db --cancel --all' 2024-09-16T00:34:55.387 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- timeout 105 ceph tell mds.24494 quiesce db --cancel --all 2024-09-16T00:34:55.563 INFO:tasks.quiescer.fs.[cephfs]:Total duration (39.1) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-09-16T00:34:55.564 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.55 2024-09-16T00:34:55.564 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:34:55.652 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:34:55.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:55 smithi038 ceph-mon[30945]: pgmap v428: 129 pgs: 129 active+clean; 204 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 102 MiB/s rd, 1.0 MiB/s wr, 343 op/s 2024-09-16T00:34:55.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:55 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/2416593672' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:34:55.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:55 smithi122 ceph-mon[35412]: pgmap v428: 129 pgs: 129 active+clean; 204 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 102 MiB/s rd, 1.0 MiB/s wr, 343 op/s 2024-09-16T00:34:55.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:55 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/2416593672' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:34:55.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:55 smithi073 ceph-mon[35254]: pgmap v428: 129 pgs: 129 active+clean; 204 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 102 MiB/s rd, 1.0 MiB/s wr, 343 op/s 2024-09-16T00:34:55.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:55 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/2416593672' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:34:56.290 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.278+0000 7f36acfc7640 1 -- 172.21.15.38:0/4173367959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36a8102d40 msgr2=0x7f36a8103120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:56.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.278+0000 7f36acfc7640 1 --2- 172.21.15.38:0/4173367959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36a8102d40 0x7f36a8103120 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f3698009900 tx=0x7f369802a480 comp rx=0 tx=0).stop 2024-09-16T00:34:56.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.279+0000 7f36acfc7640 1 -- 172.21.15.38:0/4173367959 shutdown_connections 2024-09-16T00:34:56.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.279+0000 7f36acfc7640 1 --2- 172.21.15.38:0/4173367959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36a810c170 0x7f36a810e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:56.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.279+0000 7f36acfc7640 1 --2- 172.21.15.38:0/4173367959 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36a8103660 0x7f36a810bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:56.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.279+0000 7f36acfc7640 1 --2- 172.21.15.38:0/4173367959 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36a8102d40 0x7f36a8103120 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:56.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.279+0000 7f36acfc7640 1 -- 172.21.15.38:0/4173367959 >> 172.21.15.38:0/4173367959 conn(0x7f36a80fc7a0 msgr2=0x7f36a80febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:56.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.279+0000 7f36acfc7640 1 -- 172.21.15.38:0/4173367959 shutdown_connections 2024-09-16T00:34:56.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.279+0000 7f36acfc7640 1 -- 172.21.15.38:0/4173367959 wait complete. 2024-09-16T00:34:56.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.280+0000 7f36acfc7640 1 Processor -- start 2024-09-16T00:34:56.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.280+0000 7f36acfc7640 1 -- start start 2024-09-16T00:34:56.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.280+0000 7f36acfc7640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36a8102d40 0x7f36a81983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:56.291 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.280+0000 7f36acfc7640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36a8103660 0x7f36a8198930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:56.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.280+0000 7f36acfc7640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36a810c170 0x7f36a819ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:56.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.280+0000 7f36acfc7640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f36a8111fb0 con 0x7f36a8102d40 2024-09-16T00:34:56.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.280+0000 7f36acfc7640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f36a8111e30 con 0x7f36a8103660 2024-09-16T00:34:56.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.280+0000 7f36acfc7640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f36a8112130 con 0x7f36a810c170 2024-09-16T00:34:56.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.280+0000 7f36a659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36a8102d40 0x7f36a81983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:56.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.280+0000 7f36a659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36a8102d40 0x7f36a81983f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53832/0 (socket says 172.21.15.38:53832) 2024-09-16T00:34:56.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.280+0000 7f36a659c640 1 -- 172.21.15.38:0/68305568 learned_addr learned my addr 172.21.15.38:0/68305568 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:34:56.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.280+0000 7f36a659c640 1 -- 172.21.15.38:0/68305568 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36a810c170 msgr2=0x7f36a819ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:56.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.280+0000 7f36a659c640 1 --2- 172.21.15.38:0/68305568 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36a810c170 0x7f36a819ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:56.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.280+0000 7f36a5d9b640 1 --2- 172.21.15.38:0/68305568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36a8103660 0x7f36a8198930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:56.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.280+0000 7f36a659c640 1 -- 172.21.15.38:0/68305568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36a8103660 msgr2=0x7f36a8198930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:56.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.280+0000 7f36a659c640 1 --2- 172.21.15.38:0/68305568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36a8103660 0x7f36a8198930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:56.292 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.280+0000 7f36a659c640 1 -- 172.21.15.38:0/68305568 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36a819d3a0 con 0x7f36a8102d40 2024-09-16T00:34:56.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.280+0000 7f36a5d9b640 1 --2- 172.21.15.38:0/68305568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36a8103660 0x7f36a8198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:34:56.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.281+0000 7f36a659c640 1 --2- 172.21.15.38:0/68305568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36a8102d40 0x7f36a81983f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f3698005ab0 tx=0x7f36980042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:56.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.281+0000 7f36977fe640 1 -- 172.21.15.38:0/68305568 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369803f030 con 0x7f36a8102d40 2024-09-16T00:34:56.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.281+0000 7f36acfc7640 1 -- 172.21.15.38:0/68305568 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36a819d630 con 0x7f36a8102d40 2024-09-16T00:34:56.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.281+0000 7f36977fe640 1 -- 172.21.15.38:0/68305568 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f3698003ce0 con 0x7f36a8102d40 2024-09-16T00:34:56.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.281+0000 7f36977fe640 1 -- 172.21.15.38:0/68305568 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698035070 con 0x7f36a8102d40 2024-09-16T00:34:56.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.281+0000 7f36acfc7640 1 -- 172.21.15.38:0/68305568 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36a81a4f70 con 0x7f36a8102d40 2024-09-16T00:34:56.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.283+0000 7f36977fe640 1 -- 172.21.15.38:0/68305568 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f3698042030 con 0x7f36a8102d40 2024-09-16T00:34:56.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.283+0000 7f36af3a0740 1 Processor -- start 2024-09-16T00:34:56.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.284+0000 7f36af3a0740 1 -- start start 2024-09-16T00:34:56.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.284+0000 7f36af3a0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653cc090a00 0x5653cc090de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:56.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.284+0000 7f36af3a0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653cc091320 0x5653cc09b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:56.293 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.284+0000 7f36af3a0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653cc09bbe0 0x5653cc09dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:56.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.284+0000 7f36af3a0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5653cbff5200 con 0x5653cc091320 2024-09-16T00:34:56.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.284+0000 7f36af3a0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5653cbf9bdc0 con 0x5653cc090a00 2024-09-16T00:34:56.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.284+0000 7f36af3a0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5653cbff11c0 con 0x5653cc09bbe0 2024-09-16T00:34:56.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.284+0000 7f36a659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653cc091320 0x5653cc09b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:56.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.284+0000 7f36977fe640 1 --2- 172.21.15.38:0/68305568 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f36900a3240 0x7f36900a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:56.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.284+0000 7f36a659c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653cc091320 0x5653cc09b6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53840/0 (socket says 172.21.15.38:53840) 2024-09-16T00:34:56.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.284+0000 7f36a659c640 1 -- 172.21.15.38:0/1254250953 learned_addr learned my addr 172.21.15.38:0/1254250953 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:34:56.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.284+0000 7f36a6d9d640 1 --2- 172.21.15.38:0/68305568 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f36900a3240 0x7f36900a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:56.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.284+0000 7f36a5d9b640 1 --2- 172.21.15.38:0/1254250953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653cc09bbe0 0x5653cc09dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:56.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.284+0000 7f36a6d9d640 1 --2- 172.21.15.38:0/1254250953 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653cc090a00 0x5653cc090de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:56.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.284+0000 7f36977fe640 1 -- 172.21.15.38:0/68305568 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f36980f0310 con 0x7f36a8102d40 2024-09-16T00:34:56.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.284+0000 7f36a659c640 1 -- 172.21.15.38:0/1254250953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653cc09bbe0 msgr2=0x5653cc09dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:56.294 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.284+0000 7f36a659c640 1 --2- 172.21.15.38:0/1254250953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653cc09bbe0 0x5653cc09dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:56.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.284+0000 7f36a659c640 1 -- 172.21.15.38:0/1254250953 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653cc090a00 msgr2=0x5653cc090de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:56.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.284+0000 7f36a659c640 1 --2- 172.21.15.38:0/1254250953 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653cc090a00 0x5653cc090de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:56.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.284+0000 7f36a659c640 1 -- 172.21.15.38:0/1254250953 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653cbd66620 con 0x5653cc091320 2024-09-16T00:34:56.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.284+0000 7f36a6d9d640 1 --2- 172.21.15.38:0/1254250953 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653cc090a00 0x5653cc090de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:34:56.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.285+0000 7f36a659c640 1 --2- 172.21.15.38:0/1254250953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653cc091320 0x5653cc09b6a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f3698000c00 tx=0x7f369803b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=326a444173a5f672 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:56.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.285+0000 7f36a6d9d640 1 --2- 172.21.15.38:0/68305568 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f36900a3240 0x7f36900a5700 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f36a0008fd0 tx=0x7f36a000b250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:56.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.285+0000 7f3694ff9640 1 -- 172.21.15.38:0/1254250953 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698049070 con 0x5653cc091320 2024-09-16T00:34:56.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.285+0000 7f3694ff9640 1 -- 172.21.15.38:0/1254250953 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36980bcc50 con 0x5653cc091320 2024-09-16T00:34:56.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.285+0000 7f3694ff9640 1 -- 172.21.15.38:0/1254250953 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36980b8bc0 con 0x5653cc091320 2024-09-16T00:34:56.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.286+0000 7f36af3a0740 1 -- 172.21.15.38:0/1254250953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653cc091320 msgr2=0x5653cc09b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:56.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.286+0000 7f36af3a0740 1 --2- 172.21.15.38:0/1254250953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653cc091320 0x5653cc09b6a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f3698000c00 tx=0x7f369803b800 comp rx=0 tx=0).stop 2024-09-16T00:34:56.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.286+0000 7f36af3a0740 1 -- 172.21.15.38:0/1254250953 shutdown_connections 2024-09-16T00:34:56.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.286+0000 7f36af3a0740 1 --2- 172.21.15.38:0/1254250953 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653cc09bbe0 0x5653cc09dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:56.295 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.286+0000 7f36af3a0740 1 --2- 172.21.15.38:0/1254250953 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653cc091320 0x5653cc09b6a0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:56.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.286+0000 7f36af3a0740 1 --2- 172.21.15.38:0/1254250953 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653cc090a00 0x5653cc090de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:56.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.286+0000 7f36af3a0740 1 -- 172.21.15.38:0/1254250953 >> 172.21.15.38:0/1254250953 conn(0x5653cc08cf90 msgr2=0x5653cc08d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:56.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.286+0000 7f36af3a0740 1 -- 172.21.15.38:0/1254250953 shutdown_connections 2024-09-16T00:34:56.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.286+0000 7f36af3a0740 1 -- 172.21.15.38:0/1254250953 wait complete. 2024-09-16T00:34:56.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.286+0000 7f36af3a0740 1 -- start start 2024-09-16T00:34:56.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.286+0000 7f36af3a0740 1 Processor -- start 2024-09-16T00:34:56.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.286+0000 7f36af3a0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653cc090a00 0x5653cc09b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:56.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.286+0000 7f36af3a0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653cc091320 0x5653cc0987e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:56.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.286+0000 7f36af3a0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653cc09bbe0 0x5653cc098d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:56.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.286+0000 7f36af3a0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5653cc09e570 con 0x5653cc091320 2024-09-16T00:34:56.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.286+0000 7f36af3a0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x5653cbf8fc50 con 0x5653cc09bbe0 2024-09-16T00:34:56.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.286+0000 7f36af3a0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5653cc09e6f0 con 0x5653cc090a00 2024-09-16T00:34:56.296 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.286+0000 7f36af3a0740 10 client.0 _ll_register_callbacks cb 0x5653cbfa1ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:56.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.287+0000 7f36a5d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653cc091320 0x5653cc0987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:56.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.287+0000 7f36a5d9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653cc091320 0x5653cc0987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53850/0 (socket says 172.21.15.38:53850) 2024-09-16T00:34:56.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.287+0000 7f36a5d9b640 1 -- 172.21.15.38:0/4077941005 learned_addr learned my addr 172.21.15.38:0/4077941005 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:34:56.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.287+0000 7f36a659c640 1 --2- 172.21.15.38:0/4077941005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653cc090a00 0x5653cc09b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:56.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.287+0000 7f36a6d9d640 1 --2- 172.21.15.38:0/4077941005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653cc09bbe0 0x5653cc098d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:56.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.287+0000 7f36a5d9b640 1 -- 172.21.15.38:0/4077941005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653cc090a00 msgr2=0x5653cc09b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:56.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.287+0000 7f36a5d9b640 1 --2- 172.21.15.38:0/4077941005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653cc090a00 0x5653cc09b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:56.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.287+0000 7f36a5d9b640 1 -- 172.21.15.38:0/4077941005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653cc09bbe0 msgr2=0x5653cc098d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:56.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.287+0000 7f36a5d9b640 1 --2- 172.21.15.38:0/4077941005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653cc09bbe0 0x5653cc098d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:56.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.287+0000 7f36a5d9b640 1 -- 172.21.15.38:0/4077941005 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653cc099570 con 0x5653cc091320 2024-09-16T00:34:56.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.287+0000 7f36a5d9b640 1 --2- 172.21.15.38:0/4077941005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653cc091320 0x5653cc0987e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f369c00f6e0 tx=0x7f369c00fba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=452f91a15746c91a server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:56.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.287+0000 7f368dffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:34:56.297 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.287+0000 7f368dffb640 1 -- 172.21.15.38:0/4077941005 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369c00db20 con 0x5653cc091320 2024-09-16T00:34:56.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.287+0000 7f36af3a0740 1 -- 172.21.15.38:0/4077941005 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f36980bcc50 con 0x5653cc091320 2024-09-16T00:34:56.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.287+0000 7f368dffb640 1 -- 172.21.15.38:0/4077941005 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f369c00dcc0 con 0x5653cc091320 2024-09-16T00:34:56.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.287+0000 7f368dffb640 1 -- 172.21.15.38:0/4077941005 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f369c024050 con 0x5653cc091320 2024-09-16T00:34:56.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.287+0000 7f36af3a0740 10 client.15825 fetch_fsmap learned FSMap version 26 2024-09-16T00:34:56.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.287+0000 7f36af3a0740 1 -- 172.21.15.38:0/4077941005 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5653cc0aa140 con 0x5653cc091320 2024-09-16T00:34:56.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.288+0000 7f368dffb640 1 -- 172.21.15.38:0/4077941005 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f36980bcc50 con 0x5653cc091320 2024-09-16T00:34:56.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.288+0000 7f368dffb640 1 -- 172.21.15.38:0/4077941005 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f369c0105d0 con 0x5653cc091320 2024-09-16T00:34:56.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.288+0000 7f36af3a0740 10 client.15825 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:34:56.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.288+0000 7f36af3a0740 10 client.15825 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-16T00:34:56.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.288+0000 7f36af3a0740 20 client.15825 populate_metadata read hostname 'smithi038' 2024-09-16T00:34:56.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.288+0000 7f36af3a0740 1 --2- 172.21.15.38:0/4077941005 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5653cc0ab550 0x5653cc0ad990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:56.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.288+0000 7f36af3a0740 4 client.15825 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:34:56.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.288+0000 7f36af3a0740 1 -- 172.21.15.38:0/4077941005 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653cc002770 con 0x5653cc0ab550 2024-09-16T00:34:56.298 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.288+0000 7f36a659c640 1 --2- 172.21.15.38:0/4077941005 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5653cc0ab550 0x5653cc0ad990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:56.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.289+0000 7f36a659c640 1 --2- 172.21.15.38:0/4077941005 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5653cc0ab550 0x5653cc0ad990 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=392159249443784 server_cookie=a562b25fcfd124f0 in_seq=0 out_seq=0 2024-09-16T00:34:56.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.289+0000 7f368dffb640 10 client.15825 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:34:56.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.289+0000 7f368dffb640 1 -- 172.21.15.38:0/4077941005 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x7f36980a1e80 con 0x5653cc0ab550 2024-09-16T00:34:56.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.289+0000 7f368dffb640 10 client.15825 handle_command_reply: tid=0 2024-09-16T00:34:56.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.289+0000 7f36af3a0740 1 client.15825 shutdown 2024-09-16T00:34:56.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.290+0000 7f36af3a0740 1 -- 172.21.15.38:0/4077941005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653cc091320 msgr2=0x5653cc0987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:56.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.290+0000 7f36af3a0740 1 --2- 172.21.15.38:0/4077941005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653cc091320 0x5653cc0987e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f369c00f6e0 tx=0x7f369c00fba0 comp rx=0 tx=0).stop 2024-09-16T00:34:56.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.290+0000 7f36af3a0740 1 -- 172.21.15.38:0/4077941005 shutdown_connections 2024-09-16T00:34:56.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.290+0000 7f36af3a0740 1 --2- 172.21.15.38:0/4077941005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653cc09bbe0 0x5653cc098d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:56.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.290+0000 7f36af3a0740 1 --2- 172.21.15.38:0/4077941005 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5653cc0ab550 0x5653cc0ad990 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:56.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.290+0000 7f368dffb640 0 client.15825 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:34:56.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.290+0000 7f36af3a0740 1 --2- 172.21.15.38:0/4077941005 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653cc091320 0x5653cc0987e0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:56.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.290+0000 7f36af3a0740 1 --2- 172.21.15.38:0/4077941005 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653cc090a00 0x5653cc09b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:56.299 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.290+0000 7f36af3a0740 1 -- 172.21.15.38:0/4077941005 >> 172.21.15.38:0/4077941005 conn(0x5653cc08cf90 msgr2=0x5653cc08c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:56.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.290+0000 7f36af3a0740 1 -- 172.21.15.38:0/4077941005 shutdown_connections 2024-09-16T00:34:56.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.290+0000 7f36af3a0740 1 -- 172.21.15.38:0/4077941005 wait complete. 2024-09-16T00:34:56.300 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.290+0000 7f36af3a0740 20 client.15825 trim_cache size 0 max 16384 2024-09-16T00:34:56.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.309+0000 7f36af3a0740 1 Processor -- start 2024-09-16T00:34:56.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.309+0000 7f36af3a0740 1 -- start start 2024-09-16T00:34:56.309 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.309+0000 7f36af3a0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653cc088520 0x5653cc0a7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:56.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.309+0000 7f36af3a0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653cc08cf90 0x5653cc0b3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:56.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.309+0000 7f36af3a0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653cc090a00 0x5653cc0b37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:56.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.309+0000 7f36af3a0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f36a81af330 con 0x5653cc090a00 2024-09-16T00:34:56.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.309+0000 7f36af3a0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f36a81acc00 con 0x5653cc08cf90 2024-09-16T00:34:56.310 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.309+0000 7f36af3a0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f36a81aaa50 con 0x5653cc088520 2024-09-16T00:34:56.311 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.309+0000 7f36a659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653cc088520 0x5653cc0a7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:56.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.309+0000 7f36a659c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653cc088520 0x5653cc0a7740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.122:3300/0 says I am v2:172.21.15.38:59896/0 (socket says 172.21.15.38:59896) 2024-09-16T00:34:56.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.309+0000 7f36a659c640 1 -- 172.21.15.38:0/4145760203 learned_addr learned my addr 172.21.15.38:0/4145760203 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:34:56.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.310+0000 7f36a5d9b640 1 --2- 172.21.15.38:0/4145760203 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653cc08cf90 0x5653cc0b3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:56.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.310+0000 7f36a6d9d640 1 --2- 172.21.15.38:0/4145760203 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653cc090a00 0x5653cc0b37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:56.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.310+0000 7f36a659c640 1 -- 172.21.15.38:0/4145760203 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653cc08cf90 msgr2=0x5653cc0b3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:56.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.310+0000 7f36a659c640 1 --2- 172.21.15.38:0/4145760203 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653cc08cf90 0x5653cc0b3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:56.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.310+0000 7f36a659c640 1 -- 172.21.15.38:0/4145760203 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653cc090a00 msgr2=0x5653cc0b37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:56.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.310+0000 7f36a659c640 1 --2- 172.21.15.38:0/4145760203 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653cc090a00 0x5653cc0b37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:56.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.310+0000 7f36a659c640 1 -- 172.21.15.38:0/4145760203 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653cc090830 con 0x5653cc088520 2024-09-16T00:34:56.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.310+0000 7f36a6d9d640 1 --2- 172.21.15.38:0/4145760203 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653cc090a00 0x5653cc0b37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:34:56.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.310+0000 7f36a659c640 1 --2- 172.21.15.38:0/4145760203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653cc088520 0x5653cc0a7740 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3698031550 tx=0x7f369803b800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38b6c11de9a6ea25 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:56.312 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.310+0000 7f368ffff640 1 -- 172.21.15.38:0/4145760203 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36980da070 con 0x5653cc088520 2024-09-16T00:34:56.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.310+0000 7f368ffff640 1 -- 172.21.15.38:0/4145760203 <== mon.1 v2:172.21.15.122:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5653cc002770 con 0x5653cc088520 2024-09-16T00:34:56.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.310+0000 7f368ffff640 1 -- 172.21.15.38:0/4145760203 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3698049040 con 0x5653cc088520 2024-09-16T00:34:56.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.311+0000 7f36af3a0740 1 -- 172.21.15.38:0/4145760203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653cc088520 msgr2=0x5653cc0a7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:56.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.311+0000 7f36af3a0740 1 --2- 172.21.15.38:0/4145760203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653cc088520 0x5653cc0a7740 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f3698031550 tx=0x7f369803b800 comp rx=0 tx=0).stop 2024-09-16T00:34:56.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.311+0000 7f36af3a0740 1 -- 172.21.15.38:0/4145760203 shutdown_connections 2024-09-16T00:34:56.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.311+0000 7f36af3a0740 1 --2- 172.21.15.38:0/4145760203 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653cc090a00 0x5653cc0b37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:56.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.311+0000 7f36af3a0740 1 --2- 172.21.15.38:0/4145760203 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653cc08cf90 0x5653cc0b3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:56.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.311+0000 7f36af3a0740 1 --2- 172.21.15.38:0/4145760203 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653cc088520 0x5653cc0a7740 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:56.313 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.311+0000 7f36af3a0740 1 -- 172.21.15.38:0/4145760203 >> 172.21.15.38:0/4145760203 conn(0x5653cc08d910 msgr2=0x5653cc093e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:56.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.311+0000 7f36af3a0740 1 -- 172.21.15.38:0/4145760203 shutdown_connections 2024-09-16T00:34:56.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.311+0000 7f36af3a0740 1 -- 172.21.15.38:0/4145760203 wait complete. 2024-09-16T00:34:56.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.311+0000 7f36af3a0740 1 -- start start 2024-09-16T00:34:56.314 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.312+0000 7f36af3a0740 1 Processor -- start 2024-09-16T00:34:56.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.312+0000 7f36af3a0740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653cc088520 0x5653cc09f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:56.318 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.312+0000 7f36af3a0740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653cc08cf90 0x5653cc09fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:56.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.312+0000 7f36af3a0740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653cc090a00 0x5653cc0a0330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:56.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.312+0000 7f36af3a0740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f36a81ab0c0 con 0x5653cc090a00 2024-09-16T00:34:56.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.312+0000 7f36af3a0740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f36a81ac530 con 0x5653cc088520 2024-09-16T00:34:56.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.312+0000 7f36af3a0740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f36a81ab790 con 0x5653cc08cf90 2024-09-16T00:34:56.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.312+0000 7f36af3a0740 10 client.0 _ll_register_callbacks cb 0x7f36a806b7f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:34:56.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.312+0000 7f36a6d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653cc090a00 0x5653cc0a0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:56.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.312+0000 7f36a6d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653cc090a00 0x5653cc0a0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53878/0 (socket says 172.21.15.38:53878) 2024-09-16T00:34:56.319 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.312+0000 7f36a6d9d640 1 -- 172.21.15.38:0/3429706473 learned_addr learned my addr 172.21.15.38:0/3429706473 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:34:56.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.312+0000 7f36a5d9b640 1 --2- 172.21.15.38:0/3429706473 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653cc08cf90 0x5653cc09fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:56.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.312+0000 7f36a659c640 1 --2- 172.21.15.38:0/3429706473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653cc088520 0x5653cc09f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:56.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.313+0000 7f36a5d9b640 1 -- 172.21.15.38:0/3429706473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653cc088520 msgr2=0x5653cc09f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:56.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.313+0000 7f36a5d9b640 1 --2- 172.21.15.38:0/3429706473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653cc088520 0x5653cc09f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:56.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.313+0000 7f36a5d9b640 1 -- 172.21.15.38:0/3429706473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653cc090a00 msgr2=0x5653cc0a0330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:34:56.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.313+0000 7f36a5d9b640 1 --2- 172.21.15.38:0/3429706473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653cc090a00 0x5653cc0a0330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:56.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.313+0000 7f36a5d9b640 1 -- 172.21.15.38:0/3429706473 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653cc1154d0 con 0x5653cc08cf90 2024-09-16T00:34:56.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.313+0000 7f36a6d9d640 1 --2- 172.21.15.38:0/3429706473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653cc090a00 0x5653cc0a0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:34:56.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.313+0000 7f36a5d9b640 1 --2- 172.21.15.38:0/3429706473 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653cc08cf90 0x5653cc09fdf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f369c001eb0 tx=0x7f369c010ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c917fd0e6770f074 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:56.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.313+0000 7f368dffb640 10 client.0 ms_handle_connect on v2:172.21.15.122:3300/0 2024-09-16T00:34:56.320 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.313+0000 7f368dffb640 1 -- 172.21.15.38:0/3429706473 <== mon.1 v2:172.21.15.122:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f369c021040 con 0x5653cc08cf90 2024-09-16T00:34:56.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.313+0000 7f368dffb640 1 -- 172.21.15.38:0/3429706473 <== mon.1 v2:172.21.15.122:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f369c00d920 con 0x5653cc08cf90 2024-09-16T00:34:56.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.313+0000 7f36af3a0740 1 -- 172.21.15.38:0/3429706473 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5653cc002770 con 0x5653cc08cf90 2024-09-16T00:34:56.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.313+0000 7f368dffb640 1 -- 172.21.15.38:0/3429706473 <== mon.1 v2:172.21.15.122:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f369c03a050 con 0x5653cc08cf90 2024-09-16T00:34:56.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.313+0000 7f36af3a0740 10 client.25357 fetch_fsmap learned FSMap version 26 2024-09-16T00:34:56.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.313+0000 7f36af3a0740 1 -- 172.21.15.38:0/3429706473 --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5653cc116700 con 0x5653cc08cf90 2024-09-16T00:34:56.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.314+0000 7f368dffb640 1 -- 172.21.15.38:0/3429706473 <== mon.1 v2:172.21.15.122:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x5653cc002770 con 0x5653cc08cf90 2024-09-16T00:34:56.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.314+0000 7f368dffb640 1 -- 172.21.15.38:0/3429706473 <== mon.1 v2:172.21.15.122:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f369c0055b0 con 0x5653cc08cf90 2024-09-16T00:34:56.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.314+0000 7f36af3a0740 10 client.25357 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:34:56.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.314+0000 7f36af3a0740 10 client.25357 resolve_mds: validated gid 24494 aka daemon mds.b 2024-09-16T00:34:56.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.314+0000 7f36af3a0740 20 client.25357 populate_metadata read hostname 'smithi038' 2024-09-16T00:34:56.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.314+0000 7f36af3a0740 1 --2- 172.21.15.38:0/3429706473 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5653cc091320 0x5653cc118ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:56.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.314+0000 7f36af3a0740 4 client.25357 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-09-16T00:34:56.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.314+0000 7f36af3a0740 1 -- 172.21.15.38:0/3429706473 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x5653cbffcc80 con 0x5653cc091320 2024-09-16T00:34:56.321 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.314+0000 7f36a659c640 1 --2- 172.21.15.38:0/3429706473 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5653cc091320 0x5653cc118ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:56.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.314+0000 7f368dffb640 10 client.25357 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:34:56.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.314+0000 7f36a659c640 1 --2- 172.21.15.38:0/3429706473 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5653cc091320 0x5653cc118ea0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=266049f6ddaf2f6a server_cookie=195ffad4f319dc40 in_seq=0 out_seq=0 2024-09-16T00:34:56.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.315+0000 7f368dffb640 1 -- 172.21.15.38:0/3429706473 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+79 (crc 0 0 0) 0x5653cbffcc80 con 0x5653cc091320 2024-09-16T00:34:56.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.315+0000 7f368dffb640 10 client.25357 handle_command_reply: tid=0 2024-09-16T00:34:56.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.315+0000 7f36af3a0740 1 client.25357 shutdown 2024-09-16T00:34:56.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.315+0000 7f36af3a0740 1 -- 172.21.15.38:0/3429706473 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653cc08cf90 msgr2=0x5653cc09fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:56.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.315+0000 7f36af3a0740 1 --2- 172.21.15.38:0/3429706473 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653cc08cf90 0x5653cc09fdf0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f369c001eb0 tx=0x7f369c010ec0 comp rx=0 tx=0).stop 2024-09-16T00:34:56.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.315+0000 7f36af3a0740 1 -- 172.21.15.38:0/3429706473 shutdown_connections 2024-09-16T00:34:56.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.315+0000 7f36af3a0740 1 --2- 172.21.15.38:0/3429706473 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5653cc091320 0x5653cc118ea0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:56.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.315+0000 7f36af3a0740 1 --2- 172.21.15.38:0/3429706473 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x5653cc090a00 0x5653cc0a0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:56.322 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.315+0000 7f36af3a0740 1 --2- 172.21.15.38:0/3429706473 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5653cc08cf90 0x5653cc09fdf0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:56.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.315+0000 7f36af3a0740 1 --2- 172.21.15.38:0/3429706473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5653cc088520 0x5653cc09f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:56.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.315+0000 7f36af3a0740 1 -- 172.21.15.38:0/3429706473 >> 172.21.15.38:0/3429706473 conn(0x5653cc08d910 msgr2=0x5653cc09a040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:56.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.315+0000 7f368dffb640 0 client.25357 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:34:56.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.315+0000 7f36af3a0740 1 -- 172.21.15.38:0/3429706473 shutdown_connections 2024-09-16T00:34:56.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.315+0000 7f36af3a0740 1 -- 172.21.15.38:0/3429706473 wait complete. 2024-09-16T00:34:56.323 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.315+0000 7f36af3a0740 20 client.25357 trim_cache size 0 max 16384 2024-09-16T00:34:56.323 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-16T00:34:56.323 INFO:teuthology.orchestra.run.smithi038.stdout: "epoch": 26, 2024-09-16T00:34:56.323 INFO:teuthology.orchestra.run.smithi038.stdout: "leader": 24494, 2024-09-16T00:34:56.324 INFO:teuthology.orchestra.run.smithi038.stdout: "set_version": 4, 2024-09-16T00:34:56.324 INFO:teuthology.orchestra.run.smithi038.stdout: "sets": {} 2024-09-16T00:34:56.324 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-16T00:34:56.324 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.316+0000 7f36acfc7640 1 -- 172.21.15.38:0/68305568 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f36900a3240 msgr2=0x7f36900a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:56.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.316+0000 7f36acfc7640 1 --2- 172.21.15.38:0/68305568 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f36900a3240 0x7f36900a5700 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f36a0008fd0 tx=0x7f36a000b250 comp rx=0 tx=0).stop 2024-09-16T00:34:56.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.316+0000 7f36acfc7640 1 -- 172.21.15.38:0/68305568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36a8102d40 msgr2=0x7f36a81983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:56.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.316+0000 7f36acfc7640 1 --2- 172.21.15.38:0/68305568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36a8102d40 0x7f36a81983f0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f3698005ab0 tx=0x7f36980042f0 comp rx=0 tx=0).stop 2024-09-16T00:34:56.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.316+0000 7f36acfc7640 1 -- 172.21.15.38:0/68305568 shutdown_connections 2024-09-16T00:34:56.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.316+0000 7f36acfc7640 1 --2- 172.21.15.38:0/68305568 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f36900a3240 0x7f36900a5700 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:56.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.316+0000 7f36acfc7640 1 --2- 172.21.15.38:0/68305568 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f36a810c170 0x7f36a819ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:56.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.316+0000 7f36acfc7640 1 --2- 172.21.15.38:0/68305568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36a8103660 0x7f36a8198930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:56.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.316+0000 7f36acfc7640 1 --2- 172.21.15.38:0/68305568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f36a8102d40 0x7f36a81983f0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:56.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.316+0000 7f36acfc7640 1 -- 172.21.15.38:0/68305568 >> 172.21.15.38:0/68305568 conn(0x7f36a80fc7a0 msgr2=0x7f36a80fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:56.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.316+0000 7f36acfc7640 1 -- 172.21.15.38:0/68305568 shutdown_connections 2024-09-16T00:34:56.325 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:56.316+0000 7f36acfc7640 1 -- 172.21.15.38:0/68305568 wait complete. 2024-09-16T00:34:57.073 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:34:57.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:57 smithi038 ceph-mon[30945]: pgmap v429: 129 pgs: 129 active+clean; 124 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 86 MiB/s rd, 746 KiB/s wr, 278 op/s 2024-09-16T00:34:57.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.522+0000 7f4a50c72640 1 -- 172.21.15.38:0/4038162207 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a4c100a80 msgr2=0x7f4a4c100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:57.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.522+0000 7f4a50c72640 1 --2- 172.21.15.38:0/4038162207 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a4c100a80 0x7f4a4c100ee0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4a40009930 tx=0x7f4a4002a420 comp rx=0 tx=0).stop 2024-09-16T00:34:57.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.523+0000 7f4a50c72640 1 -- 172.21.15.38:0/4038162207 shutdown_connections 2024-09-16T00:34:57.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.523+0000 7f4a50c72640 1 --2- 172.21.15.38:0/4038162207 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a4c101420 0x7f4a4c10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:57.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.523+0000 7f4a50c72640 1 --2- 172.21.15.38:0/4038162207 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a4c100a80 0x7f4a4c100ee0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:57.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.523+0000 7f4a50c72640 1 --2- 172.21.15.38:0/4038162207 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a4c106a80 0x7f4a4c106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:57.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.523+0000 7f4a50c72640 1 -- 172.21.15.38:0/4038162207 >> 172.21.15.38:0/4038162207 conn(0x7f4a4c0fc7a0 msgr2=0x7f4a4c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:57.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.523+0000 7f4a50c72640 1 -- 172.21.15.38:0/4038162207 shutdown_connections 2024-09-16T00:34:57.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.523+0000 7f4a50c72640 1 -- 172.21.15.38:0/4038162207 wait complete. 2024-09-16T00:34:57.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.523+0000 7f4a50c72640 1 Processor -- start 2024-09-16T00:34:57.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.523+0000 7f4a50c72640 1 -- start start 2024-09-16T00:34:57.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.524+0000 7f4a50c72640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a4c100a80 0x7f4a4c197360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:57.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.524+0000 7f4a50c72640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a4c101420 0x7f4a4c1978a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:57.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.524+0000 7f4a50c72640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a4c106a80 0x7f4a4c19bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:57.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.524+0000 7f4a50c72640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c110e40 con 0x7f4a4c106a80 2024-09-16T00:34:57.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.524+0000 7f4a50c72640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c110cc0 con 0x7f4a4c100a80 2024-09-16T00:34:57.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.524+0000 7f4a50c72640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4c110fc0 con 0x7f4a4c101420 2024-09-16T00:34:57.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.524+0000 7f4a4ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a4c106a80 0x7f4a4c19bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:57.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.524+0000 7f4a4ad9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a4c106a80 0x7f4a4c19bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:53902/0 (socket says 172.21.15.38:53902) 2024-09-16T00:34:57.551 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.524+0000 7f4a4ad9d640 1 -- 172.21.15.38:0/3978023161 learned_addr learned my addr 172.21.15.38:0/3978023161 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:34:57.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.524+0000 7f4a4a59c640 1 --2- 172.21.15.38:0/3978023161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a4c100a80 0x7f4a4c197360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:57.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.524+0000 7f4a49d9b640 1 --2- 172.21.15.38:0/3978023161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a4c101420 0x7f4a4c1978a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:57.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.524+0000 7f4a4ad9d640 1 -- 172.21.15.38:0/3978023161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a4c101420 msgr2=0x7f4a4c1978a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:57.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.524+0000 7f4a4ad9d640 1 --2- 172.21.15.38:0/3978023161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a4c101420 0x7f4a4c1978a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:57.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.524+0000 7f4a4ad9d640 1 -- 172.21.15.38:0/3978023161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a4c100a80 msgr2=0x7f4a4c197360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:57.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.524+0000 7f4a4ad9d640 1 --2- 172.21.15.38:0/3978023161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a4c100a80 0x7f4a4c197360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:57.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.524+0000 7f4a4ad9d640 1 -- 172.21.15.38:0/3978023161 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a4c19c310 con 0x7f4a4c106a80 2024-09-16T00:34:57.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.524+0000 7f4a4ad9d640 1 --2- 172.21.15.38:0/3978023161 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a4c106a80 0x7f4a4c19bc30 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f4a4400eaa0 tx=0x7f4a4400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:57.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.524+0000 7f4a3b7fe640 1 -- 172.21.15.38:0/3978023161 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a4400cdb0 con 0x7f4a4c106a80 2024-09-16T00:34:57.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.525+0000 7f4a50c72640 1 -- 172.21.15.38:0/3978023161 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a4c19c600 con 0x7f4a4c106a80 2024-09-16T00:34:57.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.525+0000 7f4a3b7fe640 1 -- 172.21.15.38:0/3978023161 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f4a44005020 con 0x7f4a4c106a80 2024-09-16T00:34:57.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.525+0000 7f4a3b7fe640 1 -- 172.21.15.38:0/3978023161 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a440116b0 con 0x7f4a4c106a80 2024-09-16T00:34:57.552 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.525+0000 7f4a50c72640 1 -- 172.21.15.38:0/3978023161 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a4c1a3ee0 con 0x7f4a4c106a80 2024-09-16T00:34:57.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.526+0000 7f4a50c72640 1 -- 172.21.15.38:0/3978023161 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a4c1021c0 con 0x7f4a4c106a80 2024-09-16T00:34:57.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.527+0000 7f4a3b7fe640 1 -- 172.21.15.38:0/3978023161 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f4a44005560 con 0x7f4a4c106a80 2024-09-16T00:34:57.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.527+0000 7f4a3b7fe640 1 --2- 172.21.15.38:0/3978023161 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f4a340a3240 0x7f4a340a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:34:57.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.528+0000 7f4a4a59c640 1 --2- 172.21.15.38:0/3978023161 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f4a340a3240 0x7f4a340a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:34:57.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.528+0000 7f4a3b7fe640 1 -- 172.21.15.38:0/3978023161 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f4a440d0330 con 0x7f4a4c106a80 2024-09-16T00:34:57.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.528+0000 7f4a4a59c640 1 --2- 172.21.15.38:0/3978023161 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f4a340a3240 0x7f4a340a5700 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c00bb60 tx=0x7f4a3c00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:34:57.553 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.531+0000 7f4a3b7fe640 1 -- 172.21.15.38:0/3978023161 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f4a44016030 con 0x7f4a4c106a80 2024-09-16T00:34:57.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:57 smithi122 ceph-mon[35412]: pgmap v429: 129 pgs: 129 active+clean; 124 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 86 MiB/s rd, 746 KiB/s wr, 278 op/s 2024-09-16T00:34:57.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.748+0000 7f4a50c72640 1 -- 172.21.15.38:0/3978023161 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4a4c100ee0 con 0x7f4a4c106a80 2024-09-16T00:34:57.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.748+0000 7f4a3b7fe640 1 -- 172.21.15.38:0/3978023161 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f4a440bc030 con 0x7f4a4c106a80 2024-09-16T00:34:57.749 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:34:57.749 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"btime":"2024-09-16T00:32:41:853514+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/367974527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":367974527},{"type":"v1","addr":"172.21.15.38:6835","nonce":367974527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/335077136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":335077136},{"type":"v1","addr":"172.21.15.38:6837","nonce":335077136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/3501211160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3501211160},{"type":"v1","addr":"172.21.15.38:6839","nonce":3501211160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6841/2623130915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":2623130915},{"type":"v1","addr":"172.21.15.38:6841","nonce":2623130915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/412277519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":412277519},{"type":"v1","addr":"172.21.15.122:6833","nonce":412277519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3987234757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3987234757},{"type":"v1","addr":"172.21.15.122:6835","nonce":3987234757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:30:26.764565+0000","modified":"2024-09-16T00:32:41.355828+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24440},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1210328177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1210328177},{"type":"v1","addr":"172.21.15.73:6833","nonce":1210328177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6837/1858299680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1858299680},{"type":"v1","addr":"172.21.15.122:6837","nonce":1858299680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1983975311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1983975311},{"type":"v1","addr":"172.21.15.122:6839","nonce":1983975311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.73:6835/2866363113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2866363113},{"type":"v1","addr":"172.21.15.73:6835","nonce":2866363113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.73:6837/2681081265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2681081265},{"type":"v1","addr":"172.21.15.73:6837","nonce":2681081265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.73:6839/2396224525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2396224525},{"type":"v1","addr":"172.21.15.73:6839","nonce":2396224525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24464,24440]},"id":1}]} 2024-09-16T00:34:57.749 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 26 2024-09-16T00:34:57.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.751+0000 7f4a50c72640 1 -- 172.21.15.38:0/3978023161 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f4a340a3240 msgr2=0x7f4a340a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:57.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.751+0000 7f4a50c72640 1 --2- 172.21.15.38:0/3978023161 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f4a340a3240 0x7f4a340a5700 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c00bb60 tx=0x7f4a3c00b390 comp rx=0 tx=0).stop 2024-09-16T00:34:57.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.751+0000 7f4a50c72640 1 -- 172.21.15.38:0/3978023161 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a4c106a80 msgr2=0x7f4a4c19bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:34:57.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.751+0000 7f4a50c72640 1 --2- 172.21.15.38:0/3978023161 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a4c106a80 0x7f4a4c19bc30 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f4a4400eaa0 tx=0x7f4a4400ef60 comp rx=0 tx=0).stop 2024-09-16T00:34:57.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.751+0000 7f4a50c72640 1 -- 172.21.15.38:0/3978023161 shutdown_connections 2024-09-16T00:34:57.751 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.751+0000 7f4a50c72640 1 --2- 172.21.15.38:0/3978023161 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f4a340a3240 0x7f4a340a5700 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:57.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.751+0000 7f4a50c72640 1 --2- 172.21.15.38:0/3978023161 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f4a4c106a80 0x7f4a4c19bc30 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:57.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.751+0000 7f4a50c72640 1 --2- 172.21.15.38:0/3978023161 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f4a4c101420 0x7f4a4c1978a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:57.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.751+0000 7f4a50c72640 1 --2- 172.21.15.38:0/3978023161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a4c100a80 0x7f4a4c197360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:34:57.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.751+0000 7f4a50c72640 1 -- 172.21.15.38:0/3978023161 >> 172.21.15.38:0/3978023161 conn(0x7f4a4c0fc7a0 msgr2=0x7f4a4c0fcef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:34:57.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.751+0000 7f4a50c72640 1 -- 172.21.15.38:0/3978023161 shutdown_connections 2024-09-16T00:34:57.752 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:34:57.751+0000 7f4a50c72640 1 -- 172.21.15.38:0/3978023161 wait complete. 2024-09-16T00:34:57.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:57 smithi073 ceph-mon[35254]: pgmap v429: 129 pgs: 129 active+clean; 124 MiB data, 788 MiB used, 1.0 TiB / 1.0 TiB avail; 86 MiB/s rd, 746 KiB/s wr, 278 op/s 2024-09-16T00:34:58.510 INFO:tasks.quiescer:done joining Quiescers 2024-09-16T00:34:58.510 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-16T00:34:58.519 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-16T00:34:58.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:58 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/3978023161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:34:58.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:58 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/3978023161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:34:58.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:58 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/3978023161' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:34:59.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:34:59 smithi038 ceph-mon[30945]: pgmap v430: 129 pgs: 129 active+clean; 37 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 77 MiB/s rd, 808 KiB/s wr, 281 op/s 2024-09-16T00:34:59.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:34:59 smithi122 ceph-mon[35412]: pgmap v430: 129 pgs: 129 active+clean; 37 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 77 MiB/s rd, 808 KiB/s wr, 281 op/s 2024-09-16T00:34:59.793 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:34:59 smithi073 ceph-mon[35254]: pgmap v430: 129 pgs: 129 active+clean; 37 MiB data, 625 MiB used, 1.0 TiB / 1.0 TiB avail; 77 MiB/s rd, 808 KiB/s wr, 281 op/s 2024-09-16T00:35:01.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:01 smithi038 ceph-mon[30945]: pgmap v431: 129 pgs: 129 active+clean; 13 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 62 MiB/s rd, 757 KiB/s wr, 263 op/s 2024-09-16T00:35:01.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:01 smithi122 ceph-mon[35412]: pgmap v431: 129 pgs: 129 active+clean; 13 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 62 MiB/s rd, 757 KiB/s wr, 263 op/s 2024-09-16T00:35:01.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:01 smithi073 ceph-mon[35254]: pgmap v431: 129 pgs: 129 active+clean; 13 MiB data, 450 MiB used, 1.0 TiB / 1.0 TiB avail; 62 MiB/s rd, 757 KiB/s wr, 263 op/s 2024-09-16T00:35:03.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:03 smithi122 ceph-mon[35412]: pgmap v432: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s rd, 564 KiB/s wr, 201 op/s 2024-09-16T00:35:03.717 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:03 smithi038 ceph-mon[30945]: pgmap v432: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s rd, 564 KiB/s wr, 201 op/s 2024-09-16T00:35:03.793 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:03 smithi073 ceph-mon[35254]: pgmap v432: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 46 MiB/s rd, 564 KiB/s wr, 201 op/s 2024-09-16T00:35:05.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:05 smithi038 ceph-mon[30945]: pgmap v433: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 223 KiB/s wr, 111 op/s 2024-09-16T00:35:05.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:05 smithi122 ceph-mon[35412]: pgmap v433: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 223 KiB/s wr, 111 op/s 2024-09-16T00:35:05.793 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:05 smithi073 ceph-mon[35254]: pgmap v433: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 28 MiB/s rd, 223 KiB/s wr, 111 op/s 2024-09-16T00:35:07.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:07 smithi038 ceph-mon[30945]: pgmap v434: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 156 KiB/s wr, 68 op/s 2024-09-16T00:35:07.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:07 smithi122 ceph-mon[35412]: pgmap v434: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 156 KiB/s wr, 68 op/s 2024-09-16T00:35:07.793 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:07 smithi073 ceph-mon[35254]: pgmap v434: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 156 KiB/s wr, 68 op/s 2024-09-16T00:35:09.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:09 smithi038 ceph-mon[30945]: pgmap v435: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 127 KiB/s wr, 45 op/s 2024-09-16T00:35:09.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:09 smithi122 ceph-mon[35412]: pgmap v435: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 127 KiB/s wr, 45 op/s 2024-09-16T00:35:09.793 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:09 smithi073 ceph-mon[35254]: pgmap v435: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 127 KiB/s wr, 45 op/s 2024-09-16T00:35:11.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:11 smithi038 ceph-mon[30945]: pgmap v436: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.7 KiB/s wr, 15 op/s 2024-09-16T00:35:11.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:11 smithi122 ceph-mon[35412]: pgmap v436: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.7 KiB/s wr, 15 op/s 2024-09-16T00:35:11.793 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:11 smithi073 ceph-mon[35254]: pgmap v436: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 7.7 KiB/s wr, 15 op/s 2024-09-16T00:35:11.971 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 scrub status 2024-09-16T00:35:12.242 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:35:12.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.727+0000 7f00bd9e9640 1 -- 172.21.15.38:0/2263848084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00b8102790 msgr2=0x7f00b81067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:12.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.727+0000 7f00bd9e9640 1 --2- 172.21.15.38:0/2263848084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00b8102790 0x7f00b81067e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f00ac009900 tx=0x7f00ac02a410 comp rx=0 tx=0).stop 2024-09-16T00:35:12.761 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.727+0000 7f00bd9e9640 1 -- 172.21.15.38:0/2263848084 shutdown_connections 2024-09-16T00:35:12.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.727+0000 7f00bd9e9640 1 --2- 172.21.15.38:0/2263848084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00b8106f10 0x7f00b810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.727+0000 7f00bd9e9640 1 --2- 172.21.15.38:0/2263848084 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00b8102790 0x7f00b81067e0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.727+0000 7f00bd9e9640 1 --2- 172.21.15.38:0/2263848084 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00b8101de0 0x7f00b81021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.727+0000 7f00bd9e9640 1 -- 172.21.15.38:0/2263848084 >> 172.21.15.38:0/2263848084 conn(0x7f00b80fd8c0 msgr2=0x7f00b80ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:12.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.727+0000 7f00bd9e9640 1 -- 172.21.15.38:0/2263848084 shutdown_connections 2024-09-16T00:35:12.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.727+0000 7f00bd9e9640 1 -- 172.21.15.38:0/2263848084 wait complete. 2024-09-16T00:35:12.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.727+0000 7f00bd9e9640 1 Processor -- start 2024-09-16T00:35:12.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.728+0000 7f00bd9e9640 1 -- start start 2024-09-16T00:35:12.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.728+0000 7f00bd9e9640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00b8101de0 0x7f00b80744f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:12.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.728+0000 7f00bd9e9640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00b8102790 0x7f00b8074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:12.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.728+0000 7f00bd9e9640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00b8106f10 0x7f00b8074f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:12.762 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.728+0000 7f00bd9e9640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f00b8111f90 con 0x7f00b8106f10 2024-09-16T00:35:12.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.728+0000 7f00bd9e9640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f00b8111e10 con 0x7f00b8101de0 2024-09-16T00:35:12.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.728+0000 7f00bd9e9640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f00b8112110 con 0x7f00b8102790 2024-09-16T00:35:12.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.728+0000 7f00b77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00b8106f10 0x7f00b8074f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:12.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.728+0000 7f00b77fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00b8106f10 0x7f00b8074f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46478/0 (socket says 172.21.15.38:46478) 2024-09-16T00:35:12.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.728+0000 7f00b77fe640 1 -- 172.21.15.38:0/1287996499 learned_addr learned my addr 172.21.15.38:0/1287996499 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:35:12.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.728+0000 7f00b6ffd640 1 --2- 172.21.15.38:0/1287996499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00b8101de0 0x7f00b80744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:12.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.728+0000 7f00b67fc640 1 --2- 172.21.15.38:0/1287996499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00b8102790 0x7f00b8074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:12.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.728+0000 7f00b77fe640 1 -- 172.21.15.38:0/1287996499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00b8102790 msgr2=0x7f00b8074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:12.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.728+0000 7f00b77fe640 1 --2- 172.21.15.38:0/1287996499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00b8102790 0x7f00b8074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.728+0000 7f00b77fe640 1 -- 172.21.15.38:0/1287996499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00b8101de0 msgr2=0x7f00b80744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:12.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.728+0000 7f00b77fe640 1 --2- 172.21.15.38:0/1287996499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00b8101de0 0x7f00b80744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.763 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.728+0000 7f00b77fe640 1 -- 172.21.15.38:0/1287996499 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00b81a6980 con 0x7f00b8106f10 2024-09-16T00:35:12.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.728+0000 7f00b67fc640 1 --2- 172.21.15.38:0/1287996499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00b8102790 0x7f00b8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:35:12.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.729+0000 7f00b77fe640 1 --2- 172.21.15.38:0/1287996499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00b8106f10 0x7f00b8074f70 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f00b000c970 tx=0x7f00b000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:12.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.729+0000 7f00bc9e7640 1 -- 172.21.15.38:0/1287996499 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b0007bb0 con 0x7f00b8106f10 2024-09-16T00:35:12.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.729+0000 7f00bc9e7640 1 -- 172.21.15.38:0/1287996499 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00b0007d50 con 0x7f00b8106f10 2024-09-16T00:35:12.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.729+0000 7f00bd9e9640 1 -- 172.21.15.38:0/1287996499 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00b81a6c70 con 0x7f00b8106f10 2024-09-16T00:35:12.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.729+0000 7f00bc9e7640 1 -- 172.21.15.38:0/1287996499 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b000f450 con 0x7f00b8106f10 2024-09-16T00:35:12.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.729+0000 7f00bd9e9640 1 -- 172.21.15.38:0/1287996499 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00b81a7160 con 0x7f00b8106f10 2024-09-16T00:35:12.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.731+0000 7f00bc9e7640 1 -- 172.21.15.38:0/1287996499 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f00b0004f60 con 0x7f00b8106f10 2024-09-16T00:35:12.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.731+0000 7f00bfdc2740 1 Processor -- start 2024-09-16T00:35:12.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.731+0000 7f00bfdc2740 1 -- start start 2024-09-16T00:35:12.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.732+0000 7f00bfdc2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa9a96d4a0 0x55fa9a96d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:12.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.732+0000 7f00bfdc2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa9a96ddc0 0x55fa9a978140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:12.764 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.732+0000 7f00bfdc2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa9a978680 0x55fa9a97aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:12.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.732+0000 7f00bfdc2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55fa9a8d32e0 con 0x55fa9a96ddc0 2024-09-16T00:35:12.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.732+0000 7f00bfdc2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55fa9a87ace0 con 0x55fa9a96d4a0 2024-09-16T00:35:12.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.732+0000 7f00bfdc2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55fa9a8cf2a0 con 0x55fa9a978680 2024-09-16T00:35:12.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.732+0000 7f00b6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa9a96ddc0 0x55fa9a978140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:12.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.732+0000 7f00b6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa9a96ddc0 0x55fa9a978140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46484/0 (socket says 172.21.15.38:46484) 2024-09-16T00:35:12.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.732+0000 7f00b6ffd640 1 -- 172.21.15.38:0/2088472608 learned_addr learned my addr 172.21.15.38:0/2088472608 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:35:12.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.732+0000 7f00bc9e7640 1 --2- 172.21.15.38:0/1287996499 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f00a00a3240 0x7f00a00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:12.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.732+0000 7f00b67fc640 1 --2- 172.21.15.38:0/2088472608 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa9a96d4a0 0x55fa9a96d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:12.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.732+0000 7f00b77fe640 1 --2- 172.21.15.38:0/1287996499 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f00a00a3240 0x7f00a00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:12.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.732+0000 7f00b67fc640 1 --2- 172.21.15.38:0/2088472608 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa9a978680 0x55fa9a97aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:12.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.732+0000 7f00bc9e7640 1 -- 172.21.15.38:0/1287996499 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f00b00cf9c0 con 0x7f00b8106f10 2024-09-16T00:35:12.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.732+0000 7f00b6ffd640 1 -- 172.21.15.38:0/2088472608 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa9a978680 msgr2=0x55fa9a97aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:12.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.732+0000 7f00b6ffd640 1 --2- 172.21.15.38:0/2088472608 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa9a978680 0x55fa9a97aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.732+0000 7f00b6ffd640 1 -- 172.21.15.38:0/2088472608 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa9a96d4a0 msgr2=0x55fa9a96d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:12.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.732+0000 7f00b6ffd640 1 --2- 172.21.15.38:0/2088472608 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa9a96d4a0 0x55fa9a96d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.732+0000 7f00b6ffd640 1 -- 172.21.15.38:0/2088472608 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa9a85f440 con 0x55fa9a96ddc0 2024-09-16T00:35:12.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.732+0000 7f00b67fc640 1 --2- 172.21.15.38:0/2088472608 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa9a96d4a0 0x55fa9a96d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:35:12.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.732+0000 7f00b6ffd640 1 --2- 172.21.15.38:0/2088472608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa9a96ddc0 0x55fa9a978140 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f00a800a4b0 tx=0x7f00a80096d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d4d0b7f453ce7e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:12.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.733+0000 7f00b77fe640 1 --2- 172.21.15.38:0/1287996499 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f00a00a3240 0x7f00a00a5700 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f00b00cd4b0 tx=0x7f00b0091f90 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:12.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.733+0000 7f009dffb640 1 -- 172.21.15.38:0/2088472608 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a8018020 con 0x55fa9a96ddc0 2024-09-16T00:35:12.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.733+0000 7f009dffb640 1 -- 172.21.15.38:0/2088472608 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00a8014cf0 con 0x55fa9a96ddc0 2024-09-16T00:35:12.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.733+0000 7f009dffb640 1 -- 172.21.15.38:0/2088472608 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a8013de0 con 0x55fa9a96ddc0 2024-09-16T00:35:12.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.733+0000 7f00bfdc2740 1 -- 172.21.15.38:0/2088472608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa9a96ddc0 msgr2=0x55fa9a978140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:12.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.733+0000 7f00bfdc2740 1 --2- 172.21.15.38:0/2088472608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa9a96ddc0 0x55fa9a978140 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f00a800a4b0 tx=0x7f00a80096d0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.733+0000 7f00bfdc2740 1 -- 172.21.15.38:0/2088472608 shutdown_connections 2024-09-16T00:35:12.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.733+0000 7f00bfdc2740 1 --2- 172.21.15.38:0/2088472608 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa9a978680 0x55fa9a97aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.733+0000 7f00bfdc2740 1 --2- 172.21.15.38:0/2088472608 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa9a96ddc0 0x55fa9a978140 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.733+0000 7f00bfdc2740 1 --2- 172.21.15.38:0/2088472608 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa9a96d4a0 0x55fa9a96d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.733+0000 7f00bfdc2740 1 -- 172.21.15.38:0/2088472608 >> 172.21.15.38:0/2088472608 conn(0x55fa9a969a30 msgr2=0x55fa9a969e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:12.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.733+0000 7f00bfdc2740 1 -- 172.21.15.38:0/2088472608 shutdown_connections 2024-09-16T00:35:12.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.734+0000 7f00bfdc2740 1 -- 172.21.15.38:0/2088472608 wait complete. 2024-09-16T00:35:12.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.734+0000 7f00bfdc2740 1 -- start start 2024-09-16T00:35:12.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.734+0000 7f00bfdc2740 1 Processor -- start 2024-09-16T00:35:12.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.734+0000 7f00bfdc2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa9a96d4a0 0x55fa9a977fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:12.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.734+0000 7f00bfdc2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa9a96ddc0 0x55fa9a975250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:12.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.734+0000 7f00bfdc2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa9a978680 0x55fa9a975790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:12.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.734+0000 7f00bfdc2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x55fa9a97b250 con 0x55fa9a96ddc0 2024-09-16T00:35:12.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.734+0000 7f00bfdc2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x55fa9a88e050 con 0x55fa9a96d4a0 2024-09-16T00:35:12.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.734+0000 7f00bfdc2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x55fa9a97b3d0 con 0x55fa9a978680 2024-09-16T00:35:12.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.734+0000 7f00bfdc2740 10 client.0 _ll_register_callbacks cb 0x55fa9a8ce160 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:35:12.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.734+0000 7f00b6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa9a96ddc0 0x55fa9a975250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:12.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.734+0000 7f00b6ffd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa9a96ddc0 0x55fa9a975250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46488/0 (socket says 172.21.15.38:46488) 2024-09-16T00:35:12.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.734+0000 7f00b6ffd640 1 -- 172.21.15.38:0/3428512122 learned_addr learned my addr 172.21.15.38:0/3428512122 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:35:12.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.735+0000 7f00b67fc640 1 --2- 172.21.15.38:0/3428512122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa9a96d4a0 0x55fa9a977fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:12.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.735+0000 7f00b67fc640 1 --2- 172.21.15.38:0/3428512122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa9a978680 0x55fa9a975790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:12.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.735+0000 7f00b6ffd640 1 -- 172.21.15.38:0/3428512122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa9a978680 msgr2=0x55fa9a975790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:12.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.735+0000 7f00b6ffd640 1 --2- 172.21.15.38:0/3428512122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa9a978680 0x55fa9a975790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.735+0000 7f00b6ffd640 1 -- 172.21.15.38:0/3428512122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa9a96d4a0 msgr2=0x55fa9a977fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:12.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.735+0000 7f00b6ffd640 1 --2- 172.21.15.38:0/3428512122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa9a96d4a0 0x55fa9a977fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.735+0000 7f00b6ffd640 1 -- 172.21.15.38:0/3428512122 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa9a6443c0 con 0x55fa9a96ddc0 2024-09-16T00:35:12.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.735+0000 7f00b6ffd640 1 --2- 172.21.15.38:0/3428512122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa9a96ddc0 0x55fa9a975250 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f00a800afd0 tx=0x7f00a80136e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4fe969b38d32ef server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:12.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.735+0000 7f0096ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:35:12.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.735+0000 7f0096ffd640 1 -- 172.21.15.38:0/3428512122 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00a802c060 con 0x55fa9a96ddc0 2024-09-16T00:35:12.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.735+0000 7f0096ffd640 1 -- 172.21.15.38:0/3428512122 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f00a8014610 con 0x55fa9a96ddc0 2024-09-16T00:35:12.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.735+0000 7f00bfdc2740 1 -- 172.21.15.38:0/3428512122 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00a8014cf0 con 0x55fa9a96ddc0 2024-09-16T00:35:12.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.735+0000 7f0096ffd640 1 -- 172.21.15.38:0/3428512122 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f00a8014810 con 0x55fa9a96ddc0 2024-09-16T00:35:12.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.735+0000 7f00bfdc2740 10 client.15846 fetch_fsmap learned FSMap version 26 2024-09-16T00:35:12.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.735+0000 7f00bfdc2740 1 -- 172.21.15.38:0/3428512122 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fa9a986c00 con 0x55fa9a96ddc0 2024-09-16T00:35:12.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.736+0000 7f0096ffd640 1 -- 172.21.15.38:0/3428512122 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00a8014cf0 con 0x55fa9a96ddc0 2024-09-16T00:35:12.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.736+0000 7f0096ffd640 1 -- 172.21.15.38:0/3428512122 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f00a8027c60 con 0x55fa9a96ddc0 2024-09-16T00:35:12.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.736+0000 7f00bfdc2740 10 client.15846 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:35:12.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.736+0000 7f00bfdc2740 10 client.15846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:35:12.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.736+0000 7f00bfdc2740 20 client.15846 populate_metadata read hostname 'smithi038' 2024-09-16T00:35:12.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.736+0000 7f00bfdc2740 1 --2- 172.21.15.38:0/3428512122 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55fa9a9880c0 0x55fa9a98a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:12.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.736+0000 7f00bfdc2740 4 client.15846 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:35:12.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.736+0000 7f00bfdc2740 1 -- 172.21.15.38:0/3428512122 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa9a8e0850 con 0x55fa9a9880c0 2024-09-16T00:35:12.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.736+0000 7f00b67fc640 1 --2- 172.21.15.38:0/3428512122 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55fa9a9880c0 0x55fa9a98a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:12.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.736+0000 7f0096ffd640 10 client.15846 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:35:12.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.736+0000 7f00b67fc640 1 --2- 172.21.15.38:0/3428512122 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55fa9a9880c0 0x55fa9a98a4b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f10e0c6fde92cd1 server_cookie=f0adf68f17783b59 in_seq=0 out_seq=0 2024-09-16T00:35:12.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.737+0000 7f0096ffd640 1 -- 172.21.15.38:0/3428512122 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x55fa9a8e0850 con 0x55fa9a9880c0 2024-09-16T00:35:12.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.737+0000 7f0096ffd640 10 client.15846 handle_command_reply: tid=0 2024-09-16T00:35:12.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.737+0000 7f00bfdc2740 1 client.15846 shutdown 2024-09-16T00:35:12.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.737+0000 7f00bfdc2740 1 -- 172.21.15.38:0/3428512122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa9a96ddc0 msgr2=0x55fa9a975250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:12.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.737+0000 7f00bfdc2740 1 --2- 172.21.15.38:0/3428512122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa9a96ddc0 0x55fa9a975250 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f00a800afd0 tx=0x7f00a80136e0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.737+0000 7f00bfdc2740 1 -- 172.21.15.38:0/3428512122 shutdown_connections 2024-09-16T00:35:12.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.737+0000 7f00bfdc2740 1 --2- 172.21.15.38:0/3428512122 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa9a978680 0x55fa9a975790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.737+0000 7f00bfdc2740 1 --2- 172.21.15.38:0/3428512122 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55fa9a9880c0 0x55fa9a98a4b0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.737+0000 7f00bfdc2740 1 --2- 172.21.15.38:0/3428512122 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa9a96ddc0 0x55fa9a975250 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.737+0000 7f00bfdc2740 1 --2- 172.21.15.38:0/3428512122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa9a96d4a0 0x55fa9a977fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.737+0000 7f00bfdc2740 1 -- 172.21.15.38:0/3428512122 >> 172.21.15.38:0/3428512122 conn(0x55fa9a969a30 msgr2=0x55fa9a968b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:12.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.738+0000 7f0096ffd640 0 client.15846 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:35:12.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.738+0000 7f00bfdc2740 1 -- 172.21.15.38:0/3428512122 shutdown_connections 2024-09-16T00:35:12.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.738+0000 7f00bfdc2740 1 -- 172.21.15.38:0/3428512122 wait complete. 2024-09-16T00:35:12.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.738+0000 7f00bfdc2740 20 client.15846 trim_cache size 0 max 16384 2024-09-16T00:35:12.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.755+0000 7f00bfdc2740 1 Processor -- start 2024-09-16T00:35:12.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.755+0000 7f00bfdc2740 1 -- start start 2024-09-16T00:35:12.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.755+0000 7f00bfdc2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa9a96d4a0 0x55fa9a9844e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:12.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.755+0000 7f00bfdc2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa9a96ddc0 0x55fa9a98fc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:12.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.755+0000 7f00bfdc2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa9a978680 0x55fa9a9901d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:12.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.755+0000 7f00bfdc2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f00b81b3820 con 0x55fa9a978680 2024-09-16T00:35:12.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.755+0000 7f00bfdc2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f00b81b10f0 con 0x55fa9a96d4a0 2024-09-16T00:35:12.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.755+0000 7f00bfdc2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f00b81aef40 con 0x55fa9a96ddc0 2024-09-16T00:35:12.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.755+0000 7f00b67fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa9a978680 0x55fa9a9901d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:12.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.755+0000 7f00b67fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa9a978680 0x55fa9a9901d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46504/0 (socket says 172.21.15.38:46504) 2024-09-16T00:35:12.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.755+0000 7f00b67fc640 1 -- 172.21.15.38:0/2704341695 learned_addr learned my addr 172.21.15.38:0/2704341695 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:35:12.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.755+0000 7f00b67fc640 1 --2- 172.21.15.38:0/2704341695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa9a96d4a0 0x55fa9a9844e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:12.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.755+0000 7f00b6ffd640 1 --2- 172.21.15.38:0/2704341695 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa9a96ddc0 0x55fa9a98fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:12.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.755+0000 7f00b67fc640 1 -- 172.21.15.38:0/2704341695 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa9a96ddc0 msgr2=0x55fa9a98fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:12.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.755+0000 7f00b67fc640 1 --2- 172.21.15.38:0/2704341695 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa9a96ddc0 0x55fa9a98fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.755+0000 7f00b67fc640 1 -- 172.21.15.38:0/2704341695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa9a96d4a0 msgr2=0x55fa9a9844e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:12.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.755+0000 7f00b67fc640 1 --2- 172.21.15.38:0/2704341695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa9a96d4a0 0x55fa9a9844e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.755+0000 7f00b67fc640 1 -- 172.21.15.38:0/2704341695 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa9a9737a0 con 0x55fa9a978680 2024-09-16T00:35:12.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.756+0000 7f00b67fc640 1 --2- 172.21.15.38:0/2704341695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa9a978680 0x55fa9a9901d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f00ac031bf0 tx=0x7f00ac042040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6eae69b03e8156a6 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:12.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.756+0000 7f009d7fa640 1 -- 172.21.15.38:0/2704341695 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac051070 con 0x55fa9a978680 2024-09-16T00:35:12.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.756+0000 7f009d7fa640 1 -- 172.21.15.38:0/2704341695 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00ac003ad0 con 0x55fa9a978680 2024-09-16T00:35:12.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.756+0000 7f009d7fa640 1 -- 172.21.15.38:0/2704341695 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac049040 con 0x55fa9a978680 2024-09-16T00:35:12.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.756+0000 7f00bfdc2740 1 -- 172.21.15.38:0/2704341695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa9a978680 msgr2=0x55fa9a9901d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:12.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.756+0000 7f00bfdc2740 1 --2- 172.21.15.38:0/2704341695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa9a978680 0x55fa9a9901d0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f00ac031bf0 tx=0x7f00ac042040 comp rx=0 tx=0).stop 2024-09-16T00:35:12.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.756+0000 7f00bfdc2740 1 -- 172.21.15.38:0/2704341695 shutdown_connections 2024-09-16T00:35:12.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.756+0000 7f00bfdc2740 1 --2- 172.21.15.38:0/2704341695 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa9a978680 0x55fa9a9901d0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.756+0000 7f00bfdc2740 1 --2- 172.21.15.38:0/2704341695 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa9a96ddc0 0x55fa9a98fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.756+0000 7f00bfdc2740 1 --2- 172.21.15.38:0/2704341695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa9a96d4a0 0x55fa9a9844e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.757+0000 7f00bfdc2740 1 -- 172.21.15.38:0/2704341695 >> 172.21.15.38:0/2704341695 conn(0x55fa9a969a30 msgr2=0x55fa9a9707e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:12.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.757+0000 7f00bfdc2740 1 -- 172.21.15.38:0/2704341695 shutdown_connections 2024-09-16T00:35:12.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.757+0000 7f00bfdc2740 1 -- 172.21.15.38:0/2704341695 wait complete. 2024-09-16T00:35:12.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.757+0000 7f00bfdc2740 1 -- start start 2024-09-16T00:35:12.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.757+0000 7f00bfdc2740 1 Processor -- start 2024-09-16T00:35:12.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.757+0000 7f00bfdc2740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa9a96d4a0 0x55fa9a980b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:12.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.757+0000 7f00bfdc2740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa9a96ddc0 0x55fa9a97bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:12.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.757+0000 7f00bfdc2740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa9a978680 0x55fa9a97c400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:12.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.757+0000 7f00bfdc2740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f00b81af5b0 con 0x55fa9a96d4a0 2024-09-16T00:35:12.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.757+0000 7f00bfdc2740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f00b81b0a20 con 0x55fa9a978680 2024-09-16T00:35:12.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.757+0000 7f00bfdc2740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f00b81afc80 con 0x55fa9a96ddc0 2024-09-16T00:35:12.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.757+0000 7f00bfdc2740 10 client.0 _ll_register_callbacks cb 0x7f00a8009f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:35:12.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.757+0000 7f00b67fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa9a96d4a0 0x55fa9a980b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:12.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.757+0000 7f00b67fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa9a96d4a0 0x55fa9a980b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46506/0 (socket says 172.21.15.38:46506) 2024-09-16T00:35:12.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.757+0000 7f00b67fc640 1 -- 172.21.15.38:0/568496130 learned_addr learned my addr 172.21.15.38:0/568496130 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:35:12.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.758+0000 7f00b6ffd640 1 --2- 172.21.15.38:0/568496130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa9a96ddc0 0x55fa9a97bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:12.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.758+0000 7f00b67fc640 1 --2- 172.21.15.38:0/568496130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa9a978680 0x55fa9a97c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:12.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.758+0000 7f00b67fc640 1 -- 172.21.15.38:0/568496130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa9a96ddc0 msgr2=0x55fa9a97bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:12.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.758+0000 7f00b67fc640 1 --2- 172.21.15.38:0/568496130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa9a96ddc0 0x55fa9a97bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.758+0000 7f00b67fc640 1 -- 172.21.15.38:0/568496130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa9a978680 msgr2=0x55fa9a97c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:12.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.758+0000 7f00b67fc640 1 --2- 172.21.15.38:0/568496130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa9a978680 0x55fa9a97c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.758+0000 7f00b67fc640 1 -- 172.21.15.38:0/568496130 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa9a97c9f0 con 0x55fa9a96d4a0 2024-09-16T00:35:12.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.758+0000 7f00b6ffd640 1 --2- 172.21.15.38:0/568496130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa9a96ddc0 0x55fa9a97bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:35:12.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.758+0000 7f00b67fc640 1 --2- 172.21.15.38:0/568496130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa9a96d4a0 0x55fa9a980b90 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f00ac031670 tx=0x7f00ac03e680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16afa8ed7238b613 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:12.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.758+0000 7f0096ffd640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:35:12.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.758+0000 7f0096ffd640 1 -- 172.21.15.38:0/568496130 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00ac051040 con 0x55fa9a96d4a0 2024-09-16T00:35:12.775 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.758+0000 7f0096ffd640 1 -- 172.21.15.38:0/568496130 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f00ac003c80 con 0x55fa9a96d4a0 2024-09-16T00:35:12.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.758+0000 7f00bfdc2740 1 -- 172.21.15.38:0/568496130 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f00ac003ad0 con 0x55fa9a96d4a0 2024-09-16T00:35:12.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.758+0000 7f0096ffd640 1 -- 172.21.15.38:0/568496130 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f00ac05f050 con 0x55fa9a96d4a0 2024-09-16T00:35:12.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.758+0000 7f00bfdc2740 10 client.15852 fetch_fsmap learned FSMap version 26 2024-09-16T00:35:12.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.758+0000 7f00bfdc2740 1 -- 172.21.15.38:0/568496130 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55fa9a9f2a70 con 0x55fa9a96d4a0 2024-09-16T00:35:12.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.759+0000 7f0096ffd640 1 -- 172.21.15.38:0/568496130 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f00ac003ad0 con 0x55fa9a96d4a0 2024-09-16T00:35:12.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.759+0000 7f0096ffd640 1 -- 172.21.15.38:0/568496130 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f00ac057020 con 0x55fa9a96d4a0 2024-09-16T00:35:12.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.759+0000 7f00bfdc2740 10 client.15852 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:35:12.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.759+0000 7f00bfdc2740 10 client.15852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:35:12.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.759+0000 7f00bfdc2740 20 client.15852 populate_metadata read hostname 'smithi038' 2024-09-16T00:35:12.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.759+0000 7f00bfdc2740 1 --2- 172.21.15.38:0/568496130 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55fa9a9f3170 0x55fa9a9f5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:12.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.759+0000 7f00bfdc2740 4 client.15852 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-16T00:35:12.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.759+0000 7f00bfdc2740 1 -- 172.21.15.38:0/568496130 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55fa9a8dad60 con 0x55fa9a9f3170 2024-09-16T00:35:12.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.760+0000 7f00b6ffd640 1 --2- 172.21.15.38:0/568496130 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55fa9a9f3170 0x55fa9a9f5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:12.776 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.760+0000 7f00b6ffd640 1 --2- 172.21.15.38:0/568496130 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55fa9a9f3170 0x55fa9a9f5590 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b00c21ce98d132f6 server_cookie=f252088e061d3ca in_seq=0 out_seq=0 2024-09-16T00:35:12.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.760+0000 7f0096ffd640 10 client.15852 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:35:12.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.760+0000 7f0096ffd640 1 -- 172.21.15.38:0/568496130 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55fa9a8dad60 con 0x55fa9a9f3170 2024-09-16T00:35:12.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.760+0000 7f0096ffd640 10 client.15852 handle_command_reply: tid=0 2024-09-16T00:35:12.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.760+0000 7f00bfdc2740 1 client.15852 shutdown 2024-09-16T00:35:12.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.760+0000 7f00bfdc2740 1 -- 172.21.15.38:0/568496130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa9a96d4a0 msgr2=0x55fa9a980b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:12.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.760+0000 7f00bfdc2740 1 --2- 172.21.15.38:0/568496130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa9a96d4a0 0x55fa9a980b90 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f00ac031670 tx=0x7f00ac03e680 comp rx=0 tx=0).stop 2024-09-16T00:35:12.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.760+0000 7f00bfdc2740 1 -- 172.21.15.38:0/568496130 shutdown_connections 2024-09-16T00:35:12.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.760+0000 7f00bfdc2740 1 --2- 172.21.15.38:0/568496130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55fa9a978680 0x55fa9a97c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.760+0000 7f00bfdc2740 1 --2- 172.21.15.38:0/568496130 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x55fa9a96ddc0 0x55fa9a97bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.777 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.760+0000 7f00bfdc2740 1 --2- 172.21.15.38:0/568496130 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x55fa9a9f3170 0x55fa9a9f5590 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.760+0000 7f00bfdc2740 1 --2- 172.21.15.38:0/568496130 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x55fa9a96d4a0 0x55fa9a980b90 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.760+0000 7f0096ffd640 0 client.15852 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:35:12.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.760+0000 7f00bfdc2740 1 -- 172.21.15.38:0/568496130 >> 172.21.15.38:0/568496130 conn(0x55fa9a969a30 msgr2=0x55fa9a968350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:12.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.761+0000 7f00bfdc2740 1 -- 172.21.15.38:0/568496130 shutdown_connections 2024-09-16T00:35:12.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.761+0000 7f00bfdc2740 1 -- 172.21.15.38:0/568496130 wait complete. 2024-09-16T00:35:12.778 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.761+0000 7f00bfdc2740 20 client.15852 trim_cache size 0 max 16384 2024-09-16T00:35:12.778 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-16T00:35:12.778 INFO:teuthology.orchestra.run.smithi038.stdout: "status": "no active scrubs running", 2024-09-16T00:35:12.778 INFO:teuthology.orchestra.run.smithi038.stdout: "scrubs": {} 2024-09-16T00:35:12.779 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-16T00:35:12.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.761+0000 7f00bd9e9640 1 -- 172.21.15.38:0/1287996499 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f00a00a3240 msgr2=0x7f00a00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:12.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.761+0000 7f00bd9e9640 1 --2- 172.21.15.38:0/1287996499 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f00a00a3240 0x7f00a00a5700 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f00b00cd4b0 tx=0x7f00b0091f90 comp rx=0 tx=0).stop 2024-09-16T00:35:12.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.761+0000 7f00bd9e9640 1 -- 172.21.15.38:0/1287996499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00b8106f10 msgr2=0x7f00b8074f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:12.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.761+0000 7f00bd9e9640 1 --2- 172.21.15.38:0/1287996499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00b8106f10 0x7f00b8074f70 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f00b000c970 tx=0x7f00b000ce30 comp rx=0 tx=0).stop 2024-09-16T00:35:12.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.761+0000 7f00bd9e9640 1 -- 172.21.15.38:0/1287996499 shutdown_connections 2024-09-16T00:35:12.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.761+0000 7f00bd9e9640 1 --2- 172.21.15.38:0/1287996499 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f00a00a3240 0x7f00a00a5700 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.779 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.761+0000 7f00bd9e9640 1 --2- 172.21.15.38:0/1287996499 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f00b8106f10 0x7f00b8074f70 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.761+0000 7f00bd9e9640 1 --2- 172.21.15.38:0/1287996499 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f00b8102790 0x7f00b8074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.761+0000 7f00bd9e9640 1 --2- 172.21.15.38:0/1287996499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00b8101de0 0x7f00b80744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:12.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.761+0000 7f00bd9e9640 1 -- 172.21.15.38:0/1287996499 >> 172.21.15.38:0/1287996499 conn(0x7f00b80fd8c0 msgr2=0x7f00b80fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:12.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.761+0000 7f00bd9e9640 1 -- 172.21.15.38:0/1287996499 shutdown_connections 2024-09-16T00:35:12.780 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:12.762+0000 7f00bd9e9640 1 -- 172.21.15.38:0/1287996499 wait complete. 2024-09-16T00:35:13.533 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-16T00:35:13.533 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:35:13.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:13 smithi038 ceph-mon[30945]: pgmap v437: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.7 KiB/s wr, 10 op/s 2024-09-16T00:35:13.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:13 smithi122 ceph-mon[35412]: pgmap v437: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.7 KiB/s wr, 10 op/s 2024-09-16T00:35:13.793 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:13 smithi073 ceph-mon[35254]: pgmap v437: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 3.7 KiB/s wr, 10 op/s 2024-09-16T00:35:13.797 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:35:14.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.252+0000 7f05ec70a640 1 -- 172.21.15.38:0/1580735555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05e4102790 msgr2=0x7f05e41067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:14.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.252+0000 7f05ec70a640 1 --2- 172.21.15.38:0/1580735555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05e4102790 0x7f05e41067e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f05d4009900 tx=0x7f05d402a410 comp rx=0 tx=0).stop 2024-09-16T00:35:14.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.252+0000 7f05ec70a640 1 -- 172.21.15.38:0/1580735555 shutdown_connections 2024-09-16T00:35:14.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.252+0000 7f05ec70a640 1 --2- 172.21.15.38:0/1580735555 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05e4106f10 0x7f05e410ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:14.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.252+0000 7f05ec70a640 1 --2- 172.21.15.38:0/1580735555 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05e4102790 0x7f05e41067e0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:14.278 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.252+0000 7f05ec70a640 1 --2- 172.21.15.38:0/1580735555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05e4101de0 0x7f05e41021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:14.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.252+0000 7f05ec70a640 1 -- 172.21.15.38:0/1580735555 >> 172.21.15.38:0/1580735555 conn(0x7f05e40fd8c0 msgr2=0x7f05e40ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:14.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.252+0000 7f05ec70a640 1 -- 172.21.15.38:0/1580735555 shutdown_connections 2024-09-16T00:35:14.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.252+0000 7f05ec70a640 1 -- 172.21.15.38:0/1580735555 wait complete. 2024-09-16T00:35:14.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.252+0000 7f05ec70a640 1 Processor -- start 2024-09-16T00:35:14.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.252+0000 7f05ec70a640 1 -- start start 2024-09-16T00:35:14.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.253+0000 7f05ec70a640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05e4101de0 0x7f05e419a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:14.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.253+0000 7f05ec70a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05e4102790 0x7f05e419ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:14.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.253+0000 7f05ec70a640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05e4106f10 0x7f05e419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:14.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.253+0000 7f05ec70a640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f05e4111f30 con 0x7f05e4102790 2024-09-16T00:35:14.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.253+0000 7f05ec70a640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f05e4111db0 con 0x7f05e4101de0 2024-09-16T00:35:14.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.253+0000 7f05ec70a640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f05e41120b0 con 0x7f05e4106f10 2024-09-16T00:35:14.279 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.253+0000 7f05e9ca5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05e4102790 0x7f05e419ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:14.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.253+0000 7f05e9ca5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05e4102790 0x7f05e419ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46522/0 (socket says 172.21.15.38:46522) 2024-09-16T00:35:14.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.253+0000 7f05e9ca5640 1 -- 172.21.15.38:0/1685487864 learned_addr learned my addr 172.21.15.38:0/1685487864 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:35:14.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.253+0000 7f05ea4a6640 1 --2- 172.21.15.38:0/1685487864 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05e4101de0 0x7f05e419a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:14.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.253+0000 7f05eaca7640 1 --2- 172.21.15.38:0/1685487864 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05e4106f10 0x7f05e419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:14.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.253+0000 7f05e9ca5640 1 -- 172.21.15.38:0/1685487864 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05e4106f10 msgr2=0x7f05e419efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:14.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.253+0000 7f05e9ca5640 1 --2- 172.21.15.38:0/1685487864 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05e4106f10 0x7f05e419efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:14.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.253+0000 7f05e9ca5640 1 -- 172.21.15.38:0/1685487864 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05e4101de0 msgr2=0x7f05e419a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:14.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.253+0000 7f05e9ca5640 1 --2- 172.21.15.38:0/1685487864 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05e4101de0 0x7f05e419a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:14.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.253+0000 7f05e9ca5640 1 -- 172.21.15.38:0/1685487864 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05e419f6a0 con 0x7f05e4102790 2024-09-16T00:35:14.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.253+0000 7f05eaca7640 1 --2- 172.21.15.38:0/1685487864 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05e4106f10 0x7f05e419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:35:14.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.253+0000 7f05ea4a6640 1 --2- 172.21.15.38:0/1685487864 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05e4101de0 0x7f05e419a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:35:14.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.253+0000 7f05e9ca5640 1 --2- 172.21.15.38:0/1685487864 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05e4102790 0x7f05e419ac30 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f05d4005ab0 tx=0x7f05d40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:14.280 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.254+0000 7f05db7fe640 1 -- 172.21.15.38:0/1685487864 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d403f030 con 0x7f05e4102790 2024-09-16T00:35:14.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.254+0000 7f05db7fe640 1 -- 172.21.15.38:0/1685487864 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05d4003ce0 con 0x7f05e4102790 2024-09-16T00:35:14.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.254+0000 7f05ec70a640 1 -- 172.21.15.38:0/1685487864 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05e419f930 con 0x7f05e4102790 2024-09-16T00:35:14.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.254+0000 7f05db7fe640 1 -- 172.21.15.38:0/1685487864 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d4035070 con 0x7f05e4102790 2024-09-16T00:35:14.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.254+0000 7f05ec70a640 1 -- 172.21.15.38:0/1685487864 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05e41a7270 con 0x7f05e4102790 2024-09-16T00:35:14.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.255+0000 7f05ec70a640 1 -- 172.21.15.38:0/1685487864 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05e41032c0 con 0x7f05e4102790 2024-09-16T00:35:14.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.256+0000 7f05db7fe640 1 -- 172.21.15.38:0/1685487864 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f05d4042030 con 0x7f05e4102790 2024-09-16T00:35:14.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.256+0000 7f05db7fe640 1 --2- 172.21.15.38:0/1685487864 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f05d00a3240 0x7f05d00a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:14.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.257+0000 7f05db7fe640 1 -- 172.21.15.38:0/1685487864 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f05d40f08e0 con 0x7f05e4102790 2024-09-16T00:35:14.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.257+0000 7f05ea4a6640 1 --2- 172.21.15.38:0/1685487864 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f05d00a3240 0x7f05d00a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:14.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.257+0000 7f05ea4a6640 1 --2- 172.21.15.38:0/1685487864 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f05d00a3240 0x7f05d00a5700 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f05e00058c0 tx=0x7f05e0005ec0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:14.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.260+0000 7f05db7fe640 1 -- 172.21.15.38:0/1685487864 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f05d40bd280 con 0x7f05e4102790 2024-09-16T00:35:14.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.471+0000 7f05ec70a640 1 -- 172.21.15.38:0/1685487864 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f05e41021c0 con 0x7f05e4102790 2024-09-16T00:35:14.471 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.471+0000 7f05db7fe640 1 -- 172.21.15.38:0/1685487864 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f05e41021c0 con 0x7f05e4102790 2024-09-16T00:35:14.472 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:35:14.472 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"btime":"2024-09-16T00:32:41:853514+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/367974527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":367974527},{"type":"v1","addr":"172.21.15.38:6835","nonce":367974527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/335077136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":335077136},{"type":"v1","addr":"172.21.15.38:6837","nonce":335077136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/3501211160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3501211160},{"type":"v1","addr":"172.21.15.38:6839","nonce":3501211160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6841/2623130915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":2623130915},{"type":"v1","addr":"172.21.15.38:6841","nonce":2623130915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/412277519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":412277519},{"type":"v1","addr":"172.21.15.122:6833","nonce":412277519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3987234757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3987234757},{"type":"v1","addr":"172.21.15.122:6835","nonce":3987234757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:30:26.764565+0000","modified":"2024-09-16T00:32:41.355828+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24440},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1210328177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1210328177},{"type":"v1","addr":"172.21.15.73:6833","nonce":1210328177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6837/1858299680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1858299680},{"type":"v1","addr":"172.21.15.122:6837","nonce":1858299680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1983975311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1983975311},{"type":"v1","addr":"172.21.15.122:6839","nonce":1983975311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.73:6835/2866363113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2866363113},{"type":"v1","addr":"172.21.15.73:6835","nonce":2866363113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.73:6837/2681081265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2681081265},{"type":"v1","addr":"172.21.15.73:6837","nonce":2681081265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.73:6839/2396224525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2396224525},{"type":"v1","addr":"172.21.15.73:6839","nonce":2396224525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24464,24440]},"id":1}]} 2024-09-16T00:35:14.472 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 26 2024-09-16T00:35:14.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.473+0000 7f05ec70a640 1 -- 172.21.15.38:0/1685487864 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f05d00a3240 msgr2=0x7f05d00a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:14.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.473+0000 7f05ec70a640 1 --2- 172.21.15.38:0/1685487864 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f05d00a3240 0x7f05d00a5700 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f05e00058c0 tx=0x7f05e0005ec0 comp rx=0 tx=0).stop 2024-09-16T00:35:14.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.473+0000 7f05ec70a640 1 -- 172.21.15.38:0/1685487864 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05e4102790 msgr2=0x7f05e419ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:14.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.473+0000 7f05ec70a640 1 --2- 172.21.15.38:0/1685487864 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05e4102790 0x7f05e419ac30 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f05d4005ab0 tx=0x7f05d40042f0 comp rx=0 tx=0).stop 2024-09-16T00:35:14.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.473+0000 7f05ec70a640 1 -- 172.21.15.38:0/1685487864 shutdown_connections 2024-09-16T00:35:14.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.473+0000 7f05ec70a640 1 --2- 172.21.15.38:0/1685487864 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f05d00a3240 0x7f05d00a5700 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:14.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.473+0000 7f05ec70a640 1 --2- 172.21.15.38:0/1685487864 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05e4106f10 0x7f05e419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:14.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.473+0000 7f05ec70a640 1 --2- 172.21.15.38:0/1685487864 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05e4102790 0x7f05e419ac30 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:14.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.473+0000 7f05ec70a640 1 --2- 172.21.15.38:0/1685487864 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05e4101de0 0x7f05e419a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:14.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.473+0000 7f05ec70a640 1 -- 172.21.15.38:0/1685487864 >> 172.21.15.38:0/1685487864 conn(0x7f05e40fd8c0 msgr2=0x7f05e40fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:14.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.474+0000 7f05ec70a640 1 -- 172.21.15.38:0/1685487864 shutdown_connections 2024-09-16T00:35:14.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:14.474+0000 7f05ec70a640 1 -- 172.21.15.38:0/1685487864 wait complete. 2024-09-16T00:35:15.201 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:2 damage ls 2024-09-16T00:35:15.463 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:35:15.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:15 smithi038 ceph-mon[30945]: pgmap v438: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2024-09-16T00:35:15.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:15 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/1685487864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:35:15.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:15 smithi122 ceph-mon[35412]: pgmap v438: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2024-09-16T00:35:15.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:15 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/1685487864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:35:15.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:15 smithi073 ceph-mon[35254]: pgmap v438: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.3 KiB/s wr, 8 op/s 2024-09-16T00:35:15.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:15 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/1685487864' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:35:15.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.955+0000 7f05f23fb640 1 -- 172.21.15.38:0/2385433031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05ec102ef0 msgr2=0x7f05ec10da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:15.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.955+0000 7f05f23fb640 1 --2- 172.21.15.38:0/2385433031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05ec102ef0 0x7f05ec10da40 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f05e4009ca0 tx=0x7f05e402a950 comp rx=0 tx=0).stop 2024-09-16T00:35:15.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.955+0000 7f05f23fb640 1 -- 172.21.15.38:0/2385433031 shutdown_connections 2024-09-16T00:35:15.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.955+0000 7f05f23fb640 1 --2- 172.21.15.38:0/2385433031 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05ec102ef0 0x7f05ec10da40 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.955+0000 7f05f23fb640 1 --2- 172.21.15.38:0/2385433031 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05ec102570 0x7f05ec1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.955+0000 7f05f23fb640 1 --2- 172.21.15.38:0/2385433031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05ec100e20 0x7f05ec101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.955+0000 7f05f23fb640 1 -- 172.21.15.38:0/2385433031 >> 172.21.15.38:0/2385433031 conn(0x7f05ec0fc7d0 msgr2=0x7f05ec0febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:15.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.955+0000 7f05f23fb640 1 -- 172.21.15.38:0/2385433031 shutdown_connections 2024-09-16T00:35:15.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.955+0000 7f05f23fb640 1 -- 172.21.15.38:0/2385433031 wait complete. 2024-09-16T00:35:15.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.956+0000 7f05f23fb640 1 Processor -- start 2024-09-16T00:35:15.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.956+0000 7f05f23fb640 1 -- start start 2024-09-16T00:35:15.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.956+0000 7f05f23fb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05ec100e20 0x7f05ec1984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:15.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.956+0000 7f05f23fb640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05ec102570 0x7f05ec1989f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:15.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.956+0000 7f05f23fb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05ec102ef0 0x7f05ec19cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:15.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.956+0000 7f05f23fb640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec1120d0 con 0x7f05ec102ef0 2024-09-16T00:35:15.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.956+0000 7f05f23fb640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec111f50 con 0x7f05ec102570 2024-09-16T00:35:15.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.956+0000 7f05f23fb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec112250 con 0x7f05ec100e20 2024-09-16T00:35:15.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.956+0000 7f05f0998640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05ec102ef0 0x7f05ec19cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:15.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.956+0000 7f05f0998640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05ec102ef0 0x7f05ec19cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46542/0 (socket says 172.21.15.38:46542) 2024-09-16T00:35:15.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.956+0000 7f05f0998640 1 -- 172.21.15.38:0/3499002175 learned_addr learned my addr 172.21.15.38:0/3499002175 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:35:15.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.956+0000 7f05ebfff640 1 --2- 172.21.15.38:0/3499002175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05ec100e20 0x7f05ec1984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:15.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.956+0000 7f05e3fff640 1 --2- 172.21.15.38:0/3499002175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05ec102570 0x7f05ec1989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:15.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.956+0000 7f05f0998640 1 -- 172.21.15.38:0/3499002175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05ec100e20 msgr2=0x7f05ec1984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:15.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.956+0000 7f05f0998640 1 --2- 172.21.15.38:0/3499002175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05ec100e20 0x7f05ec1984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.956+0000 7f05f0998640 1 -- 172.21.15.38:0/3499002175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05ec102570 msgr2=0x7f05ec1989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:15.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.956+0000 7f05f0998640 1 --2- 172.21.15.38:0/3499002175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05ec102570 0x7f05ec1989f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.956+0000 7f05f0998640 1 -- 172.21.15.38:0/3499002175 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05ec19d460 con 0x7f05ec102ef0 2024-09-16T00:35:15.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.956+0000 7f05e3fff640 1 --2- 172.21.15.38:0/3499002175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05ec102570 0x7f05ec1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:35:15.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.957+0000 7f05f0998640 1 --2- 172.21.15.38:0/3499002175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05ec102ef0 0x7f05ec19cd80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f05e4009dc0 tx=0x7f05e4003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:15.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.957+0000 7f05e9ffb640 1 -- 172.21.15.38:0/3499002175 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e4034040 con 0x7f05ec102ef0 2024-09-16T00:35:15.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.957+0000 7f05e9ffb640 1 -- 172.21.15.38:0/3499002175 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05e402fe10 con 0x7f05ec102ef0 2024-09-16T00:35:15.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.957+0000 7f05f23fb640 1 -- 172.21.15.38:0/3499002175 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05ec19d6f0 con 0x7f05ec102ef0 2024-09-16T00:35:15.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.957+0000 7f05e9ffb640 1 -- 172.21.15.38:0/3499002175 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05e40047b0 con 0x7f05ec102ef0 2024-09-16T00:35:15.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.957+0000 7f05f23fb640 1 -- 172.21.15.38:0/3499002175 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05ec1a5030 con 0x7f05ec102ef0 2024-09-16T00:35:15.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.959+0000 7f05e9ffb640 1 -- 172.21.15.38:0/3499002175 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f05e4037020 con 0x7f05ec102ef0 2024-09-16T00:35:15.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.959+0000 7f05f47d4740 1 Processor -- start 2024-09-16T00:35:15.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.959+0000 7f05f47d4740 1 -- start start 2024-09-16T00:35:15.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.960+0000 7f05f47d4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564fc1b27b40 0x564fc1b26090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:15.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.960+0000 7f05f47d4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564fc1b265d0 0x564fc1b26a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:15.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.960+0000 7f05f47d4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564fc1b31370 0x564fc1b33760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:15.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.960+0000 7f05f47d4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564fc1a8c2f0 con 0x564fc1b27b40 2024-09-16T00:35:15.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.960+0000 7f05f47d4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564fc1a33cf0 con 0x564fc1b265d0 2024-09-16T00:35:15.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.960+0000 7f05f47d4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564fc1a882b0 con 0x564fc1b31370 2024-09-16T00:35:15.985 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.960+0000 7f05e3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564fc1b27b40 0x564fc1b26090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:15.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.960+0000 7f05e3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564fc1b27b40 0x564fc1b26090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46548/0 (socket says 172.21.15.38:46548) 2024-09-16T00:35:15.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.960+0000 7f05e3fff640 1 -- 172.21.15.38:0/494877323 learned_addr learned my addr 172.21.15.38:0/494877323 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:35:15.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.960+0000 7f05ebfff640 1 --2- 172.21.15.38:0/494877323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564fc1b265d0 0x564fc1b26a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:15.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.960+0000 7f05e3fff640 1 -- 172.21.15.38:0/494877323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564fc1b31370 msgr2=0x564fc1b33760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:15.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.960+0000 7f05e3fff640 1 --2- 172.21.15.38:0/494877323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564fc1b31370 0x564fc1b33760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.960+0000 7f05e3fff640 1 -- 172.21.15.38:0/494877323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564fc1b265d0 msgr2=0x564fc1b26a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:15.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.960+0000 7f05e3fff640 1 --2- 172.21.15.38:0/494877323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564fc1b265d0 0x564fc1b26a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.960+0000 7f05e3fff640 1 -- 172.21.15.38:0/494877323 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fc1a183a0 con 0x564fc1b27b40 2024-09-16T00:35:15.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.960+0000 7f05e9ffb640 1 --2- 172.21.15.38:0/3499002175 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f05d80a32c0 0x7f05d80a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:15.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.960+0000 7f05e3fff640 1 --2- 172.21.15.38:0/494877323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564fc1b27b40 0x564fc1b26090 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f05d400bbb0 tx=0x7f05d400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=813cddae010a6e68 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:15.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.960+0000 7f05ebfff640 1 --2- 172.21.15.38:0/3499002175 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f05d80a32c0 0x7f05d80a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:15.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.960+0000 7f05e27fc640 1 -- 172.21.15.38:0/494877323 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d4011670 con 0x564fc1b27b40 2024-09-16T00:35:15.988 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.960+0000 7f05e9ffb640 1 -- 172.21.15.38:0/3499002175 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f05e40f0b70 con 0x7f05ec102ef0 2024-09-16T00:35:15.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.961+0000 7f05e27fc640 1 -- 172.21.15.38:0/494877323 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05d4011cf0 con 0x564fc1b27b40 2024-09-16T00:35:15.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.961+0000 7f05e27fc640 1 -- 172.21.15.38:0/494877323 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d4010be0 con 0x564fc1b27b40 2024-09-16T00:35:15.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.961+0000 7f05f47d4740 1 -- 172.21.15.38:0/494877323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564fc1b27b40 msgr2=0x564fc1b26090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:15.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.961+0000 7f05ebfff640 1 --2- 172.21.15.38:0/3499002175 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f05d80a32c0 0x7f05d80a5780 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x564fc1b23ac0 tx=0x7f05dc00b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:15.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.961+0000 7f05f47d4740 1 --2- 172.21.15.38:0/494877323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564fc1b27b40 0x564fc1b26090 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f05d400bbb0 tx=0x7f05d400f730 comp rx=0 tx=0).stop 2024-09-16T00:35:15.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.961+0000 7f05f47d4740 1 -- 172.21.15.38:0/494877323 shutdown_connections 2024-09-16T00:35:15.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.961+0000 7f05f47d4740 1 --2- 172.21.15.38:0/494877323 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564fc1b31370 0x564fc1b33760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.961+0000 7f05f47d4740 1 --2- 172.21.15.38:0/494877323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564fc1b265d0 0x564fc1b26a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.961+0000 7f05f47d4740 1 --2- 172.21.15.38:0/494877323 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564fc1b27b40 0x564fc1b26090 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.961+0000 7f05f47d4740 1 -- 172.21.15.38:0/494877323 >> 172.21.15.38:0/494877323 conn(0x564fc1aef310 msgr2=0x564fc1b22c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:15.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.961+0000 7f05f47d4740 1 -- 172.21.15.38:0/494877323 shutdown_connections 2024-09-16T00:35:15.989 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.961+0000 7f05f47d4740 1 -- 172.21.15.38:0/494877323 wait complete. 2024-09-16T00:35:15.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.962+0000 7f05f47d4740 1 -- start start 2024-09-16T00:35:15.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.962+0000 7f05f47d4740 1 Processor -- start 2024-09-16T00:35:15.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.962+0000 7f05f47d4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564fc1b265d0 0x564fc1b30de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:15.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.962+0000 7f05f47d4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564fc1b27b40 0x564fc1b2e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:15.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.962+0000 7f05f47d4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564fc1b31370 0x564fc1b2e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:15.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.962+0000 7f05f47d4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x564fc1b34070 con 0x564fc1b265d0 2024-09-16T00:35:15.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.962+0000 7f05f47d4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x564fc1a47060 con 0x564fc1b31370 2024-09-16T00:35:15.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.962+0000 7f05f47d4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x564fc1b341f0 con 0x564fc1b27b40 2024-09-16T00:35:15.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.962+0000 7f05f47d4740 10 client.0 _ll_register_callbacks cb 0x564fc179a8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:35:15.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.962+0000 7f05ebfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564fc1b265d0 0x564fc1b30de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:15.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.962+0000 7f05ebfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564fc1b265d0 0x564fc1b30de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46550/0 (socket says 172.21.15.38:46550) 2024-09-16T00:35:15.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.962+0000 7f05ebfff640 1 -- 172.21.15.38:0/1151861498 learned_addr learned my addr 172.21.15.38:0/1151861498 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:35:15.990 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.963+0000 7f05e3fff640 1 --2- 172.21.15.38:0/1151861498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564fc1b27b40 0x564fc1b2e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:15.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.963+0000 7f05f0998640 1 --2- 172.21.15.38:0/1151861498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564fc1b31370 0x564fc1b2e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:15.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.963+0000 7f05ebfff640 1 -- 172.21.15.38:0/1151861498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564fc1b27b40 msgr2=0x564fc1b2e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:15.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.963+0000 7f05ebfff640 1 --2- 172.21.15.38:0/1151861498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564fc1b27b40 0x564fc1b2e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.963+0000 7f05ebfff640 1 -- 172.21.15.38:0/1151861498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564fc1b31370 msgr2=0x564fc1b2e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:15.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.963+0000 7f05ebfff640 1 --2- 172.21.15.38:0/1151861498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564fc1b31370 0x564fc1b2e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.963+0000 7f05ebfff640 1 -- 172.21.15.38:0/1151861498 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fc17fd3c0 con 0x564fc1b265d0 2024-09-16T00:35:15.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.963+0000 7f05ebfff640 1 --2- 172.21.15.38:0/1151861498 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564fc1b265d0 0x564fc1b30de0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f05dc003f10 tx=0x7f05dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cc6c40a87c7704c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:15.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.963+0000 7f05cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:35:15.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.963+0000 7f05cb7fe640 1 -- 172.21.15.38:0/1151861498 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc01b660 con 0x564fc1b265d0 2024-09-16T00:35:15.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.963+0000 7f05cb7fe640 1 -- 172.21.15.38:0/1151861498 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f05dc01b800 con 0x564fc1b265d0 2024-09-16T00:35:15.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.963+0000 7f05f47d4740 1 -- 172.21.15.38:0/1151861498 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f05d4011cf0 con 0x564fc1b265d0 2024-09-16T00:35:15.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.963+0000 7f05cb7fe640 1 -- 172.21.15.38:0/1151861498 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f05dc01ba00 con 0x564fc1b265d0 2024-09-16T00:35:15.991 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.963+0000 7f05f47d4740 10 client.15870 fetch_fsmap learned FSMap version 26 2024-09-16T00:35:15.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.963+0000 7f05f47d4740 1 -- 172.21.15.38:0/1151861498 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564fc1b3fc00 con 0x564fc1b265d0 2024-09-16T00:35:15.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.963+0000 7f05cb7fe640 1 -- 172.21.15.38:0/1151861498 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05d4011cf0 con 0x564fc1b265d0 2024-09-16T00:35:15.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.964+0000 7f05cb7fe640 1 -- 172.21.15.38:0/1151861498 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f05dc024070 con 0x564fc1b265d0 2024-09-16T00:35:15.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.964+0000 7f05f47d4740 10 client.15870 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:35:15.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.964+0000 7f05f47d4740 10 client.15870 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-16T00:35:15.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.964+0000 7f05f47d4740 20 client.15870 populate_metadata read hostname 'smithi038' 2024-09-16T00:35:15.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.964+0000 7f05f47d4740 1 --2- 172.21.15.38:0/1151861498 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x564fc1b410c0 0x564fc1b434b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:15.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.964+0000 7f05f47d4740 4 client.15870 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:35:15.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.964+0000 7f05f47d4740 1 -- 172.21.15.38:0/1151861498 --> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564fc1a99860 con 0x564fc1b410c0 2024-09-16T00:35:15.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.964+0000 7f05e3fff640 1 --2- 172.21.15.38:0/1151861498 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x564fc1b410c0 0x564fc1b434b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:15.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.965+0000 7f05e3fff640 1 --2- 172.21.15.38:0/1151861498 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x564fc1b410c0 0x564fc1b434b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=66b2d9893505ae30 server_cookie=6141f8a2df4eb27a in_seq=0 out_seq=0 2024-09-16T00:35:15.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.965+0000 7f05cb7fe640 10 client.15870 ms_handle_connect on v2:172.21.15.73:6834/2866363113 2024-09-16T00:35:15.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.965+0000 7f05cb7fe640 1 -- 172.21.15.38:0/1151861498 <== mds.2 v2:172.21.15.73:6834/2866363113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x564fc1a99860 con 0x564fc1b410c0 2024-09-16T00:35:15.992 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.965+0000 7f05cb7fe640 10 client.15870 handle_command_reply: tid=0 2024-09-16T00:35:15.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.965+0000 7f05f47d4740 1 client.15870 shutdown 2024-09-16T00:35:15.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.965+0000 7f05f47d4740 1 -- 172.21.15.38:0/1151861498 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564fc1b265d0 msgr2=0x564fc1b30de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:15.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.965+0000 7f05f47d4740 1 --2- 172.21.15.38:0/1151861498 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564fc1b265d0 0x564fc1b30de0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f05dc003f10 tx=0x7f05dc0046c0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.965+0000 7f05f47d4740 1 -- 172.21.15.38:0/1151861498 shutdown_connections 2024-09-16T00:35:15.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.965+0000 7f05f47d4740 1 --2- 172.21.15.38:0/1151861498 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x564fc1b410c0 0x564fc1b434b0 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.965+0000 7f05f47d4740 1 --2- 172.21.15.38:0/1151861498 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564fc1b31370 0x564fc1b2e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.965+0000 7f05f47d4740 1 --2- 172.21.15.38:0/1151861498 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564fc1b27b40 0x564fc1b2e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.966+0000 7f05cb7fe640 0 client.15870 ms_handle_reset on v2:172.21.15.73:6834/2866363113 2024-09-16T00:35:15.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.966+0000 7f05f47d4740 1 --2- 172.21.15.38:0/1151861498 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564fc1b265d0 0x564fc1b30de0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.966+0000 7f05f47d4740 1 -- 172.21.15.38:0/1151861498 >> 172.21.15.38:0/1151861498 conn(0x564fc1aef310 msgr2=0x564fc1b28930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:15.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.966+0000 7f05f47d4740 1 -- 172.21.15.38:0/1151861498 shutdown_connections 2024-09-16T00:35:15.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.966+0000 7f05f47d4740 1 -- 172.21.15.38:0/1151861498 wait complete. 2024-09-16T00:35:15.993 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.966+0000 7f05f47d4740 20 client.15870 trim_cache size 0 max 16384 2024-09-16T00:35:15.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.979+0000 7f05f47d4740 1 Processor -- start 2024-09-16T00:35:15.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.979+0000 7f05f47d4740 1 -- start start 2024-09-16T00:35:15.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.979+0000 7f05f47d4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564fc1b265d0 0x564fc1b3d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:15.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.979+0000 7f05f47d4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564fc1b27b40 0x564fc1b29120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:15.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.979+0000 7f05f47d4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564fc1b31370 0x564fc1b29660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:15.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.979+0000 7f05f47d4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec1b1460 con 0x564fc1b265d0 2024-09-16T00:35:15.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.979+0000 7f05f47d4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec1aed30 con 0x564fc1b27b40 2024-09-16T00:35:15.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.979+0000 7f05f47d4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec1acb80 con 0x564fc1b31370 2024-09-16T00:35:15.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.979+0000 7f05ebfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564fc1b265d0 0x564fc1b3d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:15.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.979+0000 7f05ebfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564fc1b265d0 0x564fc1b3d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46556/0 (socket says 172.21.15.38:46556) 2024-09-16T00:35:15.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.979+0000 7f05ebfff640 1 -- 172.21.15.38:0/1689450496 learned_addr learned my addr 172.21.15.38:0/1689450496 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:35:15.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.979+0000 7f05f0998640 1 --2- 172.21.15.38:0/1689450496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564fc1b31370 0x564fc1b29660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:15.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.979+0000 7f05e3fff640 1 --2- 172.21.15.38:0/1689450496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564fc1b27b40 0x564fc1b29120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:15.994 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.979+0000 7f05ebfff640 1 -- 172.21.15.38:0/1689450496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564fc1b31370 msgr2=0x564fc1b29660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:15.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.979+0000 7f05ebfff640 1 --2- 172.21.15.38:0/1689450496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564fc1b31370 0x564fc1b29660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.979+0000 7f05ebfff640 1 -- 172.21.15.38:0/1689450496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564fc1b27b40 msgr2=0x564fc1b29120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:15.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.979+0000 7f05ebfff640 1 --2- 172.21.15.38:0/1689450496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564fc1b27b40 0x564fc1b29120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.979+0000 7f05ebfff640 1 -- 172.21.15.38:0/1689450496 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564fc1b3d850 con 0x564fc1b265d0 2024-09-16T00:35:15.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.980+0000 7f05e3fff640 1 --2- 172.21.15.38:0/1689450496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564fc1b27b40 0x564fc1b29120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:35:15.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.980+0000 7f05ebfff640 1 --2- 172.21.15.38:0/1689450496 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564fc1b265d0 0x564fc1b3d310 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f05dc005940 tx=0x7f05dc004470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db450371b2b9c65c server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:15.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.980+0000 7f05e1ffb640 1 -- 172.21.15.38:0/1689450496 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc01b660 con 0x564fc1b265d0 2024-09-16T00:35:15.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.980+0000 7f05e1ffb640 1 -- 172.21.15.38:0/1689450496 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f05dc003ad0 con 0x564fc1b265d0 2024-09-16T00:35:15.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.980+0000 7f05e1ffb640 1 -- 172.21.15.38:0/1689450496 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05dc005450 con 0x564fc1b265d0 2024-09-16T00:35:15.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.981+0000 7f05f47d4740 1 -- 172.21.15.38:0/1689450496 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564fc1b265d0 msgr2=0x564fc1b3d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:15.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.981+0000 7f05f47d4740 1 --2- 172.21.15.38:0/1689450496 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564fc1b265d0 0x564fc1b3d310 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f05dc005940 tx=0x7f05dc004470 comp rx=0 tx=0).stop 2024-09-16T00:35:15.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.981+0000 7f05f47d4740 1 -- 172.21.15.38:0/1689450496 shutdown_connections 2024-09-16T00:35:15.995 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.981+0000 7f05f47d4740 1 --2- 172.21.15.38:0/1689450496 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564fc1b31370 0x564fc1b29660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.981+0000 7f05f47d4740 1 --2- 172.21.15.38:0/1689450496 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564fc1b27b40 0x564fc1b29120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.981+0000 7f05f47d4740 1 --2- 172.21.15.38:0/1689450496 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564fc1b265d0 0x564fc1b3d310 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:15.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.981+0000 7f05f47d4740 1 -- 172.21.15.38:0/1689450496 >> 172.21.15.38:0/1689450496 conn(0x564fc1aef310 msgr2=0x564fc1b2d180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:15.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.981+0000 7f05f47d4740 1 -- 172.21.15.38:0/1689450496 shutdown_connections 2024-09-16T00:35:15.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.981+0000 7f05f47d4740 1 -- 172.21.15.38:0/1689450496 wait complete. 2024-09-16T00:35:15.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.982+0000 7f05f47d4740 1 -- start start 2024-09-16T00:35:15.996 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.982+0000 7f05f47d4740 1 Processor -- start 2024-09-16T00:35:15.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.982+0000 7f05f47d4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564fc1b265d0 0x564fc1b39ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:15.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.982+0000 7f05f47d4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564fc1b27b40 0x564fc1b34e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:15.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.982+0000 7f05f47d4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564fc1b31370 0x564fc1b35380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:15.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.982+0000 7f05f47d4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec1ad1f0 con 0x564fc1b27b40 2024-09-16T00:35:15.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.982+0000 7f05f47d4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec1ae660 con 0x564fc1b265d0 2024-09-16T00:35:15.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.982+0000 7f05f47d4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f05ec1ad8c0 con 0x564fc1b31370 2024-09-16T00:35:15.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.982+0000 7f05f47d4740 10 client.0 _ll_register_callbacks cb 0x7f05dc004320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:35:15.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.982+0000 7f05e3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564fc1b27b40 0x564fc1b34e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:15.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.982+0000 7f05e3fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564fc1b27b40 0x564fc1b34e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:46558/0 (socket says 172.21.15.38:46558) 2024-09-16T00:35:15.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.982+0000 7f05e3fff640 1 -- 172.21.15.38:0/2256962693 learned_addr learned my addr 172.21.15.38:0/2256962693 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:35:15.999 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.983+0000 7f05f0998640 1 --2- 172.21.15.38:0/2256962693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564fc1b31370 0x564fc1b35380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:16.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.983+0000 7f05ebfff640 1 --2- 172.21.15.38:0/2256962693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564fc1b265d0 0x564fc1b39ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:16.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.983+0000 7f05e3fff640 1 -- 172.21.15.38:0/2256962693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564fc1b31370 msgr2=0x564fc1b35380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:16.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.983+0000 7f05e3fff640 1 --2- 172.21.15.38:0/2256962693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564fc1b31370 0x564fc1b35380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.983+0000 7f05e3fff640 1 -- 172.21.15.38:0/2256962693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564fc1b265d0 msgr2=0x564fc1b39ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:16.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.983+0000 7f05e3fff640 1 --2- 172.21.15.38:0/2256962693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564fc1b265d0 0x564fc1b39ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.983+0000 7f05e3fff640 1 -- 172.21.15.38:0/2256962693 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564fc1b35a70 con 0x564fc1b27b40 2024-09-16T00:35:16.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.983+0000 7f05e3fff640 1 --2- 172.21.15.38:0/2256962693 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564fc1b27b40 0x564fc1b34e40 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f05d400bbb0 tx=0x7f05d4004660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fabc5f765f510f15 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:16.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.983+0000 7f05cb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:35:16.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.983+0000 7f05cb7fe640 1 -- 172.21.15.38:0/2256962693 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05d4010860 con 0x564fc1b27b40 2024-09-16T00:35:16.000 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.983+0000 7f05cb7fe640 1 -- 172.21.15.38:0/2256962693 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f05d4010a00 con 0x564fc1b27b40 2024-09-16T00:35:16.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.983+0000 7f05f47d4740 1 -- 172.21.15.38:0/2256962693 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x564fc1a93d70 con 0x564fc1b27b40 2024-09-16T00:35:16.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.983+0000 7f05cb7fe640 1 -- 172.21.15.38:0/2256962693 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f05d4010c00 con 0x564fc1b27b40 2024-09-16T00:35:16.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.983+0000 7f05f47d4740 10 client.15876 fetch_fsmap learned FSMap version 26 2024-09-16T00:35:16.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.983+0000 7f05f47d4740 1 -- 172.21.15.38:0/2256962693 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x564fc1babc30 con 0x564fc1b27b40 2024-09-16T00:35:16.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.983+0000 7f05cb7fe640 1 -- 172.21.15.38:0/2256962693 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x564fc1a93d70 con 0x564fc1b27b40 2024-09-16T00:35:16.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.984+0000 7f05cb7fe640 1 -- 172.21.15.38:0/2256962693 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f05d4019490 con 0x564fc1b27b40 2024-09-16T00:35:16.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.984+0000 7f05f47d4740 10 client.15876 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:35:16.001 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.984+0000 7f05f47d4740 10 client.15876 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.e 2024-09-16T00:35:16.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.984+0000 7f05f47d4740 20 client.15876 populate_metadata read hostname 'smithi038' 2024-09-16T00:35:16.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.984+0000 7f05f47d4740 1 --2- 172.21.15.38:0/2256962693 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x564fc1bac250 0x564fc1bae670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:16.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.984+0000 7f05f47d4740 4 client.15876 mds_command: new command op to 24440 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:35:16.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.984+0000 7f05f47d4740 1 -- 172.21.15.38:0/2256962693 --> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564fc1a10170 con 0x564fc1bac250 2024-09-16T00:35:16.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.984+0000 7f05ebfff640 1 --2- 172.21.15.38:0/2256962693 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x564fc1bac250 0x564fc1bae670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:16.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.984+0000 7f05ebfff640 1 --2- 172.21.15.38:0/2256962693 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x564fc1bac250 0x564fc1bae670 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3b934b3b1813f21c server_cookie=eba4098b0e50a706 in_seq=0 out_seq=0 2024-09-16T00:35:16.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.984+0000 7f05cb7fe640 10 client.15876 ms_handle_connect on v2:172.21.15.73:6834/2866363113 2024-09-16T00:35:16.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.985+0000 7f05cb7fe640 1 -- 172.21.15.38:0/2256962693 <== mds.2 v2:172.21.15.73:6834/2866363113 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564fc1a10170 con 0x564fc1bac250 2024-09-16T00:35:16.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.985+0000 7f05cb7fe640 10 client.15876 handle_command_reply: tid=0 2024-09-16T00:35:16.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.985+0000 7f05f47d4740 1 client.15876 shutdown 2024-09-16T00:35:16.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.985+0000 7f05f47d4740 1 -- 172.21.15.38:0/2256962693 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564fc1b27b40 msgr2=0x564fc1b34e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:16.002 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.985+0000 7f05f47d4740 1 --2- 172.21.15.38:0/2256962693 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564fc1b27b40 0x564fc1b34e40 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f05d400bbb0 tx=0x7f05d4004660 comp rx=0 tx=0).stop 2024-09-16T00:35:16.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.985+0000 7f05f47d4740 1 -- 172.21.15.38:0/2256962693 shutdown_connections 2024-09-16T00:35:16.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.985+0000 7f05f47d4740 1 --2- 172.21.15.38:0/2256962693 >> [v2:172.21.15.73:6834/2866363113,v1:172.21.15.73:6835/2866363113] conn(0x564fc1bac250 0x564fc1bae670 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.985+0000 7f05f47d4740 1 --2- 172.21.15.38:0/2256962693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x564fc1b31370 0x564fc1b35380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.985+0000 7f05f47d4740 1 --2- 172.21.15.38:0/2256962693 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x564fc1b27b40 0x564fc1b34e40 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.985+0000 7f05f47d4740 1 --2- 172.21.15.38:0/2256962693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x564fc1b265d0 0x564fc1b39ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.985+0000 7f05cb7fe640 0 client.15876 ms_handle_reset on v2:172.21.15.73:6834/2866363113 2024-09-16T00:35:16.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.985+0000 7f05f47d4740 1 -- 172.21.15.38:0/2256962693 >> 172.21.15.38:0/2256962693 conn(0x564fc1aef310 msgr2=0x564fc1b214e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:16.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.985+0000 7f05f47d4740 1 -- 172.21.15.38:0/2256962693 shutdown_connections 2024-09-16T00:35:16.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.985+0000 7f05f47d4740 1 -- 172.21.15.38:0/2256962693 wait complete. 2024-09-16T00:35:16.003 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.985+0000 7f05f47d4740 20 client.15876 trim_cache size 0 max 16384 2024-09-16T00:35:16.003 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-09-16T00:35:16.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.986+0000 7f05f23fb640 1 -- 172.21.15.38:0/3499002175 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f05d80a32c0 msgr2=0x7f05d80a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:16.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.986+0000 7f05f23fb640 1 --2- 172.21.15.38:0/3499002175 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f05d80a32c0 0x7f05d80a5780 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x564fc1b23ac0 tx=0x7f05dc00b390 comp rx=0 tx=0).stop 2024-09-16T00:35:16.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.986+0000 7f05f23fb640 1 -- 172.21.15.38:0/3499002175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05ec102ef0 msgr2=0x7f05ec19cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:16.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.986+0000 7f05f23fb640 1 --2- 172.21.15.38:0/3499002175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05ec102ef0 0x7f05ec19cd80 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f05e4009dc0 tx=0x7f05e4003d00 comp rx=0 tx=0).stop 2024-09-16T00:35:16.004 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.986+0000 7f05f23fb640 1 -- 172.21.15.38:0/3499002175 shutdown_connections 2024-09-16T00:35:16.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.986+0000 7f05f23fb640 1 --2- 172.21.15.38:0/3499002175 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f05d80a32c0 0x7f05d80a5780 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.986+0000 7f05f23fb640 1 --2- 172.21.15.38:0/3499002175 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f05ec102ef0 0x7f05ec19cd80 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.986+0000 7f05f23fb640 1 --2- 172.21.15.38:0/3499002175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05ec102570 0x7f05ec1989f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.986+0000 7f05f23fb640 1 --2- 172.21.15.38:0/3499002175 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f05ec100e20 0x7f05ec1984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:16.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.986+0000 7f05f23fb640 1 -- 172.21.15.38:0/3499002175 >> 172.21.15.38:0/3499002175 conn(0x7f05ec0fc7d0 msgr2=0x7f05ec106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:16.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.986+0000 7f05f23fb640 1 -- 172.21.15.38:0/3499002175 shutdown_connections 2024-09-16T00:35:16.005 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:15.986+0000 7f05f23fb640 1 -- 172.21.15.38:0/3499002175 wait complete. 2024-09-16T00:35:16.684 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:1 damage ls 2024-09-16T00:35:16.943 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:35:17.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.432+0000 7f86c2fcb640 1 -- 172.21.15.38:0/1497748048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86bc101420 msgr2=0x7f86bc10d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:17.458 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.432+0000 7f86c2fcb640 1 --2- 172.21.15.38:0/1497748048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86bc101420 0x7f86bc10d950 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f86b4009cd0 tx=0x7f86b402a8e0 comp rx=0 tx=0).stop 2024-09-16T00:35:17.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.433+0000 7f86c2fcb640 1 -- 172.21.15.38:0/1497748048 shutdown_connections 2024-09-16T00:35:17.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.433+0000 7f86c2fcb640 1 --2- 172.21.15.38:0/1497748048 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86bc101420 0x7f86bc10d950 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:17.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.433+0000 7f86c2fcb640 1 --2- 172.21.15.38:0/1497748048 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86bc100a80 0x7f86bc100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:17.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.433+0000 7f86c2fcb640 1 --2- 172.21.15.38:0/1497748048 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86bc106a80 0x7f86bc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:17.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.433+0000 7f86c2fcb640 1 -- 172.21.15.38:0/1497748048 >> 172.21.15.38:0/1497748048 conn(0x7f86bc0fc7a0 msgr2=0x7f86bc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:17.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.433+0000 7f86c2fcb640 1 -- 172.21.15.38:0/1497748048 shutdown_connections 2024-09-16T00:35:17.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.433+0000 7f86c2fcb640 1 -- 172.21.15.38:0/1497748048 wait complete. 2024-09-16T00:35:17.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.433+0000 7f86c2fcb640 1 Processor -- start 2024-09-16T00:35:17.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.433+0000 7f86c2fcb640 1 -- start start 2024-09-16T00:35:17.459 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.433+0000 7f86c2fcb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86bc100a80 0x7f86bc19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:17.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.433+0000 7f86c2fcb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86bc101420 0x7f86bc19ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:17.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.433+0000 7f86c2fcb640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86bc106a80 0x7f86bc19eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:17.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.433+0000 7f86c2fcb640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc110e80 con 0x7f86bc101420 2024-09-16T00:35:17.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.433+0000 7f86c2fcb640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc110d00 con 0x7f86bc106a80 2024-09-16T00:35:17.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.433+0000 7f86c2fcb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc111000 con 0x7f86bc100a80 2024-09-16T00:35:17.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.434+0000 7f86bbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86bc101420 0x7f86bc19ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:17.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.434+0000 7f86bbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86bc101420 0x7f86bc19ab40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42722/0 (socket says 172.21.15.38:42722) 2024-09-16T00:35:17.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.434+0000 7f86bbfff640 1 -- 172.21.15.38:0/1760869753 learned_addr learned my addr 172.21.15.38:0/1760869753 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:35:17.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.434+0000 7f86c0d67640 1 --2- 172.21.15.38:0/1760869753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86bc100a80 0x7f86bc19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:17.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.434+0000 7f86c1568640 1 --2- 172.21.15.38:0/1760869753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86bc106a80 0x7f86bc19eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:17.460 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.434+0000 7f86bbfff640 1 -- 172.21.15.38:0/1760869753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86bc100a80 msgr2=0x7f86bc19a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:17.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.434+0000 7f86bbfff640 1 --2- 172.21.15.38:0/1760869753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86bc100a80 0x7f86bc19a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:17.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.434+0000 7f86bbfff640 1 -- 172.21.15.38:0/1760869753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86bc106a80 msgr2=0x7f86bc19eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:17.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.434+0000 7f86bbfff640 1 --2- 172.21.15.38:0/1760869753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86bc106a80 0x7f86bc19eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:17.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.434+0000 7f86bbfff640 1 -- 172.21.15.38:0/1760869753 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86bc19f5b0 con 0x7f86bc101420 2024-09-16T00:35:17.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.434+0000 7f86c1568640 1 --2- 172.21.15.38:0/1760869753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86bc106a80 0x7f86bc19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:35:17.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.434+0000 7f86c0d67640 1 --2- 172.21.15.38:0/1760869753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86bc100a80 0x7f86bc19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:35:17.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.434+0000 7f86bbfff640 1 --2- 172.21.15.38:0/1760869753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86bc101420 0x7f86bc19ab40 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f86a400d950 tx=0x7f86a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:17.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.434+0000 7f86b9ffb640 1 -- 172.21.15.38:0/1760869753 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a4017070 con 0x7f86bc101420 2024-09-16T00:35:17.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.434+0000 7f86c2fcb640 1 -- 172.21.15.38:0/1760869753 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86bc19f8a0 con 0x7f86bc101420 2024-09-16T00:35:17.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.434+0000 7f86b9ffb640 1 -- 172.21.15.38:0/1760869753 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86a4009c30 con 0x7f86bc101420 2024-09-16T00:35:17.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.434+0000 7f86b9ffb640 1 -- 172.21.15.38:0/1760869753 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a400f6b0 con 0x7f86bc101420 2024-09-16T00:35:17.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.435+0000 7f86c2fcb640 1 -- 172.21.15.38:0/1760869753 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86bc1a7180 con 0x7f86bc101420 2024-09-16T00:35:17.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.437+0000 7f86b9ffb640 1 -- 172.21.15.38:0/1760869753 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f86a40109d0 con 0x7f86bc101420 2024-09-16T00:35:17.461 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.437+0000 7f86c53a4740 1 Processor -- start 2024-09-16T00:35:17.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.437+0000 7f86c53a4740 1 -- start start 2024-09-16T00:35:17.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.437+0000 7f86b9ffb640 1 --2- 172.21.15.38:0/1760869753 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f86a80a3240 0x7f86a80a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:17.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.437+0000 7f86c53a4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56195d9a8b40 0x56195d9a7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:17.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.437+0000 7f86c53a4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56195d9a75d0 0x56195d9a7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:17.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.438+0000 7f86c53a4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56195d9b2370 0x56195d9b4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:17.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.438+0000 7f86c53a4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56195d90d2f0 con 0x56195d9a75d0 2024-09-16T00:35:17.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.438+0000 7f86c53a4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56195d8b4cf0 con 0x56195d9a8b40 2024-09-16T00:35:17.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.438+0000 7f86c53a4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56195d9092b0 con 0x56195d9b2370 2024-09-16T00:35:17.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.438+0000 7f86c1568640 1 --2- 172.21.15.38:0/1760869753 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f86a80a3240 0x7f86a80a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:17.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.438+0000 7f86bbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56195d9a75d0 0x56195d9a7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:17.462 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.438+0000 7f86b9ffb640 1 -- 172.21.15.38:0/1760869753 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f86a40cf9b0 con 0x7f86bc101420 2024-09-16T00:35:17.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.438+0000 7f86bbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56195d9a75d0 0x56195d9a7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42726/0 (socket says 172.21.15.38:42726) 2024-09-16T00:35:17.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.438+0000 7f86bbfff640 1 -- 172.21.15.38:0/3491250854 learned_addr learned my addr 172.21.15.38:0/3491250854 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:35:17.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.438+0000 7f86c0d67640 1 --2- 172.21.15.38:0/3491250854 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56195d9a8b40 0x56195d9a7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:17.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.438+0000 7f86bbfff640 1 -- 172.21.15.38:0/3491250854 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56195d9b2370 msgr2=0x56195d9b4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:17.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.438+0000 7f86bbfff640 1 --2- 172.21.15.38:0/3491250854 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56195d9b2370 0x56195d9b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:17.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.438+0000 7f86bbfff640 1 -- 172.21.15.38:0/3491250854 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56195d9a8b40 msgr2=0x56195d9a7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:17.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.438+0000 7f86bbfff640 1 --2- 172.21.15.38:0/3491250854 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56195d9a8b40 0x56195d9a7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:17.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.438+0000 7f86bbfff640 1 -- 172.21.15.38:0/3491250854 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56195d8993a0 con 0x56195d9a75d0 2024-09-16T00:35:17.463 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.438+0000 7f86c1568640 1 --2- 172.21.15.38:0/1760869753 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f86a80a3240 0x7f86a80a5700 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f86b4009430 tx=0x7f86b4031510 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:17.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.438+0000 7f86bbfff640 1 --2- 172.21.15.38:0/3491250854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56195d9a75d0 0x56195d9a7a10 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f86a4000c00 tx=0x7f86a4010d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d79a5e92511866b server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:17.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.438+0000 7f86b2ffd640 1 -- 172.21.15.38:0/3491250854 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a4063e40 con 0x56195d9a75d0 2024-09-16T00:35:17.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.438+0000 7f86b2ffd640 1 -- 172.21.15.38:0/3491250854 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86a4009de0 con 0x56195d9a75d0 2024-09-16T00:35:17.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.438+0000 7f86b2ffd640 1 -- 172.21.15.38:0/3491250854 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a406e740 con 0x56195d9a75d0 2024-09-16T00:35:17.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.439+0000 7f86c53a4740 1 -- 172.21.15.38:0/3491250854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56195d9a75d0 msgr2=0x56195d9a7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:17.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.439+0000 7f86c53a4740 1 --2- 172.21.15.38:0/3491250854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56195d9a75d0 0x56195d9a7a10 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f86a4000c00 tx=0x7f86a4010d70 comp rx=0 tx=0).stop 2024-09-16T00:35:17.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.439+0000 7f86c53a4740 1 -- 172.21.15.38:0/3491250854 shutdown_connections 2024-09-16T00:35:17.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.439+0000 7f86c53a4740 1 --2- 172.21.15.38:0/3491250854 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56195d9b2370 0x56195d9b4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:17.464 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.439+0000 7f86c53a4740 1 --2- 172.21.15.38:0/3491250854 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56195d9a75d0 0x56195d9a7a10 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:17.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.439+0000 7f86c53a4740 1 --2- 172.21.15.38:0/3491250854 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56195d9a8b40 0x56195d9a7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:17.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.439+0000 7f86c53a4740 1 -- 172.21.15.38:0/3491250854 >> 172.21.15.38:0/3491250854 conn(0x56195d970310 msgr2=0x56195d9a3c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:17.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.439+0000 7f86c53a4740 1 -- 172.21.15.38:0/3491250854 shutdown_connections 2024-09-16T00:35:17.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.439+0000 7f86c53a4740 1 -- 172.21.15.38:0/3491250854 wait complete. 2024-09-16T00:35:17.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.440+0000 7f86c53a4740 1 -- start start 2024-09-16T00:35:17.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.440+0000 7f86c53a4740 1 Processor -- start 2024-09-16T00:35:17.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.440+0000 7f86c53a4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56195d9a75d0 0x56195d9af0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:17.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.440+0000 7f86c53a4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56195d9a8b40 0x56195d9af630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:17.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.440+0000 7f86c53a4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56195d9b2370 0x56195d9afb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:17.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.440+0000 7f86c53a4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x56195d9b5070 con 0x56195d9a8b40 2024-09-16T00:35:17.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.440+0000 7f86c53a4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x56195d8c8060 con 0x56195d9b2370 2024-09-16T00:35:17.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.440+0000 7f86c53a4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x56195d9b51f0 con 0x56195d9a75d0 2024-09-16T00:35:17.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.440+0000 7f86c53a4740 10 client.0 _ll_register_callbacks cb 0x56195d61b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:35:17.465 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.440+0000 7f86bbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56195d9a8b40 0x56195d9af630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:17.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.440+0000 7f86bbfff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56195d9a8b40 0x56195d9af630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42728/0 (socket says 172.21.15.38:42728) 2024-09-16T00:35:17.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.440+0000 7f86bbfff640 1 -- 172.21.15.38:0/1764974722 learned_addr learned my addr 172.21.15.38:0/1764974722 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:35:17.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.440+0000 7f86c0d67640 1 --2- 172.21.15.38:0/1764974722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56195d9a75d0 0x56195d9af0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:17.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.440+0000 7f86c1568640 1 --2- 172.21.15.38:0/1764974722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56195d9b2370 0x56195d9afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:17.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.441+0000 7f86bbfff640 1 -- 172.21.15.38:0/1764974722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56195d9a75d0 msgr2=0x56195d9af0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:17.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.441+0000 7f86bbfff640 1 --2- 172.21.15.38:0/1764974722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56195d9a75d0 0x56195d9af0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:17.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.441+0000 7f86bbfff640 1 -- 172.21.15.38:0/1764974722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56195d9b2370 msgr2=0x56195d9afb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:17.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.441+0000 7f86bbfff640 1 --2- 172.21.15.38:0/1764974722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56195d9b2370 0x56195d9afb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:17.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.441+0000 7f86bbfff640 1 -- 172.21.15.38:0/1764974722 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56195d67e3c0 con 0x56195d9a8b40 2024-09-16T00:35:17.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.441+0000 7f86bbfff640 1 --2- 172.21.15.38:0/1764974722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56195d9a8b40 0x56195d9af630 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f86a4010fd0 tx=0x7f86a409bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86cc7fff1c9ed286 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:17.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.441+0000 7f869bfff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:35:17.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.441+0000 7f869bfff640 1 -- 172.21.15.38:0/1764974722 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86a4020030 con 0x56195d9a8b40 2024-09-16T00:35:17.466 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.441+0000 7f869bfff640 1 -- 172.21.15.38:0/1764974722 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f86a40a4430 con 0x56195d9a8b40 2024-09-16T00:35:17.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.441+0000 7f86c53a4740 1 -- 172.21.15.38:0/1764974722 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86a4009de0 con 0x56195d9a8b40 2024-09-16T00:35:17.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.441+0000 7f869bfff640 1 -- 172.21.15.38:0/1764974722 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f86a4063c20 con 0x56195d9a8b40 2024-09-16T00:35:17.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.441+0000 7f86c53a4740 10 client.15888 fetch_fsmap learned FSMap version 26 2024-09-16T00:35:17.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.441+0000 7f86c53a4740 1 -- 172.21.15.38:0/1764974722 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56195d9b5070 con 0x56195d9a8b40 2024-09-16T00:35:17.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.441+0000 7f869bfff640 1 -- 172.21.15.38:0/1764974722 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86a4009de0 con 0x56195d9a8b40 2024-09-16T00:35:17.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.442+0000 7f869bfff640 1 -- 172.21.15.38:0/1764974722 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f86a40640c0 con 0x56195d9a8b40 2024-09-16T00:35:17.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.442+0000 7f86c53a4740 10 client.15888 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:35:17.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.442+0000 7f86c53a4740 10 client.15888 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:35:17.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.442+0000 7f86c53a4740 20 client.15888 populate_metadata read hostname 'smithi038' 2024-09-16T00:35:17.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.442+0000 7f86c53a4740 1 --2- 172.21.15.38:0/1764974722 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x56195d9c1d80 0x56195d9c4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:17.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.442+0000 7f86c53a4740 4 client.15888 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:35:17.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.442+0000 7f86c53a4740 1 -- 172.21.15.38:0/1764974722 --> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56195d91a860 con 0x56195d9c1d80 2024-09-16T00:35:17.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.442+0000 7f86c0d67640 1 --2- 172.21.15.38:0/1764974722 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x56195d9c1d80 0x56195d9c4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:17.467 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.442+0000 7f86c0d67640 1 --2- 172.21.15.38:0/1764974722 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x56195d9c1d80 0x56195d9c4170 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=60de84d0de4ac160 server_cookie=b82cbfd9231e8b7b in_seq=0 out_seq=0 2024-09-16T00:35:17.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.442+0000 7f869bfff640 10 client.15888 ms_handle_connect on v2:172.21.15.73:6836/2681081265 2024-09-16T00:35:17.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.443+0000 7f869bfff640 1 -- 172.21.15.38:0/1764974722 <== mds.1 v2:172.21.15.73:6836/2681081265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x56195d91a860 con 0x56195d9c1d80 2024-09-16T00:35:17.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.443+0000 7f869bfff640 10 client.15888 handle_command_reply: tid=0 2024-09-16T00:35:17.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.443+0000 7f86c53a4740 1 client.15888 shutdown 2024-09-16T00:35:17.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.443+0000 7f86c53a4740 1 -- 172.21.15.38:0/1764974722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56195d9a8b40 msgr2=0x56195d9af630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:17.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.443+0000 7f86c53a4740 1 --2- 172.21.15.38:0/1764974722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56195d9a8b40 0x56195d9af630 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f86a4010fd0 tx=0x7f86a409bc50 comp rx=0 tx=0).stop 2024-09-16T00:35:17.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.443+0000 7f86c53a4740 1 -- 172.21.15.38:0/1764974722 shutdown_connections 2024-09-16T00:35:17.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.443+0000 7f86c53a4740 1 --2- 172.21.15.38:0/1764974722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56195d9b2370 0x56195d9afb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:17.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.443+0000 7f86c53a4740 1 --2- 172.21.15.38:0/1764974722 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x56195d9c1d80 0x56195d9c4170 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:17.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.443+0000 7f86c53a4740 1 --2- 172.21.15.38:0/1764974722 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56195d9a8b40 0x56195d9af630 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:17.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.443+0000 7f869bfff640 0 client.15888 ms_handle_reset on v2:172.21.15.73:6836/2681081265 2024-09-16T00:35:17.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.443+0000 7f86c53a4740 1 --2- 172.21.15.38:0/1764974722 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56195d9a75d0 0x56195d9af0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:17.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.443+0000 7f86c53a4740 1 -- 172.21.15.38:0/1764974722 >> 172.21.15.38:0/1764974722 conn(0x56195d970310 msgr2=0x56195d9a9990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:17.468 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.443+0000 7f86c53a4740 1 -- 172.21.15.38:0/1764974722 shutdown_connections 2024-09-16T00:35:17.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.443+0000 7f86c53a4740 1 -- 172.21.15.38:0/1764974722 wait complete. 2024-09-16T00:35:17.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.443+0000 7f86c53a4740 20 client.15888 trim_cache size 0 max 16384 2024-09-16T00:35:17.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.456+0000 7f86c53a4740 1 Processor -- start 2024-09-16T00:35:17.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.456+0000 7f86c53a4740 1 -- start start 2024-09-16T00:35:17.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.457+0000 7f86c53a4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56195d9a75d0 0x56195d9af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:17.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.457+0000 7f86c53a4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56195d9a8b40 0x56195d9c9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:17.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.457+0000 7f86c53a4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56195d9b2370 0x56195d9c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:17.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.457+0000 7f86c53a4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc1b3740 con 0x56195d9a75d0 2024-09-16T00:35:17.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.457+0000 7f86c53a4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc1b1010 con 0x56195d9a8b40 2024-09-16T00:35:17.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.457+0000 7f86c53a4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc1aee60 con 0x56195d9b2370 2024-09-16T00:35:17.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.457+0000 7f86c0d67640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56195d9a75d0 0x56195d9af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:17.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.457+0000 7f86c0d67640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56195d9a75d0 0x56195d9af9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42744/0 (socket says 172.21.15.38:42744) 2024-09-16T00:35:17.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.457+0000 7f86c0d67640 1 -- 172.21.15.38:0/415885873 learned_addr learned my addr 172.21.15.38:0/415885873 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:35:17.469 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.457+0000 7f86c1568640 1 --2- 172.21.15.38:0/415885873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56195d9b2370 0x56195d9c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:17.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.457+0000 7f86bbfff640 1 --2- 172.21.15.38:0/415885873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56195d9a8b40 0x56195d9c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:17.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.457+0000 7f86c0d67640 1 -- 172.21.15.38:0/415885873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56195d9b2370 msgr2=0x56195d9c9d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:17.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.457+0000 7f86c0d67640 1 --2- 172.21.15.38:0/415885873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56195d9b2370 0x56195d9c9d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:17.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.457+0000 7f86c0d67640 1 -- 172.21.15.38:0/415885873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56195d9a8b40 msgr2=0x56195d9c9830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:17.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.457+0000 7f86c0d67640 1 --2- 172.21.15.38:0/415885873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56195d9a8b40 0x56195d9c9830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:17.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.457+0000 7f86c0d67640 1 -- 172.21.15.38:0/415885873 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56195d9aa120 con 0x56195d9a75d0 2024-09-16T00:35:17.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.457+0000 7f86bbfff640 1 --2- 172.21.15.38:0/415885873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56195d9a8b40 0x56195d9c9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:35:17.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.457+0000 7f86c0d67640 1 --2- 172.21.15.38:0/415885873 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56195d9a75d0 0x56195d9af9c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f86ac004700 tx=0x7f86ac005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f11b4fdfabc83e8e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:17.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.458+0000 7f86b27fc640 1 -- 172.21.15.38:0/415885873 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac004020 con 0x56195d9a75d0 2024-09-16T00:35:17.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.458+0000 7f86b27fc640 1 -- 172.21.15.38:0/415885873 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86ac0041c0 con 0x56195d9a75d0 2024-09-16T00:35:17.470 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.458+0000 7f86b27fc640 1 -- 172.21.15.38:0/415885873 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86ac01b6c0 con 0x56195d9a75d0 2024-09-16T00:35:17.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.459+0000 7f86c53a4740 1 -- 172.21.15.38:0/415885873 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56195d9a75d0 msgr2=0x56195d9af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:17.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.459+0000 7f86c53a4740 1 --2- 172.21.15.38:0/415885873 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56195d9a75d0 0x56195d9af9c0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f86ac004700 tx=0x7f86ac005f90 comp rx=0 tx=0).stop 2024-09-16T00:35:17.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.459+0000 7f86c53a4740 1 -- 172.21.15.38:0/415885873 shutdown_connections 2024-09-16T00:35:17.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.459+0000 7f86c53a4740 1 --2- 172.21.15.38:0/415885873 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56195d9b2370 0x56195d9c9d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:17.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.459+0000 7f86c53a4740 1 --2- 172.21.15.38:0/415885873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56195d9a8b40 0x56195d9c9830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:17.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.459+0000 7f86c53a4740 1 --2- 172.21.15.38:0/415885873 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56195d9a75d0 0x56195d9af9c0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:17.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.459+0000 7f86c53a4740 1 -- 172.21.15.38:0/415885873 >> 172.21.15.38:0/415885873 conn(0x56195d970310 msgr2=0x56195d9a2d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:17.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.459+0000 7f86c53a4740 1 -- 172.21.15.38:0/415885873 shutdown_connections 2024-09-16T00:35:17.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.459+0000 7f86c53a4740 1 -- 172.21.15.38:0/415885873 wait complete. 2024-09-16T00:35:17.474 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.460+0000 7f86c53a4740 1 -- start start 2024-09-16T00:35:17.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.460+0000 7f86c53a4740 1 Processor -- start 2024-09-16T00:35:17.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.460+0000 7f86c53a4740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56195d9a75d0 0x56195da24af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:17.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.460+0000 7f86c53a4740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56195d9a8b40 0x56195da25030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:17.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.460+0000 7f86c53a4740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56195d9b2370 0x56195d9b5a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:17.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.460+0000 7f86c53a4740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc1af4d0 con 0x56195d9b2370 2024-09-16T00:35:17.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.460+0000 7f86c53a4740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc1b0940 con 0x56195d9a8b40 2024-09-16T00:35:17.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.460+0000 7f86c53a4740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f86bc1afba0 con 0x56195d9a75d0 2024-09-16T00:35:17.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.460+0000 7f86c53a4740 10 client.0 _ll_register_callbacks cb 0x7f86ac00c350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:35:17.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.460+0000 7f86c1568640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56195d9b2370 0x56195d9b5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:17.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.460+0000 7f86c1568640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56195d9b2370 0x56195d9b5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42758/0 (socket says 172.21.15.38:42758) 2024-09-16T00:35:17.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.460+0000 7f86c1568640 1 -- 172.21.15.38:0/3191274865 learned_addr learned my addr 172.21.15.38:0/3191274865 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:35:17.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.460+0000 7f86c0d67640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56195d9a75d0 0x56195da24af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:17.475 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.460+0000 7f86bbfff640 1 --2- 172.21.15.38:0/3191274865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56195d9a8b40 0x56195da25030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:17.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.461+0000 7f86c1568640 1 -- 172.21.15.38:0/3191274865 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56195d9a75d0 msgr2=0x56195da24af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:17.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.461+0000 7f86c1568640 1 --2- 172.21.15.38:0/3191274865 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56195d9a75d0 0x56195da24af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:17.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.461+0000 7f86c1568640 1 -- 172.21.15.38:0/3191274865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56195d9a8b40 msgr2=0x56195da25030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:17.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.461+0000 7f86c1568640 1 --2- 172.21.15.38:0/3191274865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56195d9a8b40 0x56195da25030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:17.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.461+0000 7f86c1568640 1 -- 172.21.15.38:0/3191274865 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56195d9b6150 con 0x56195d9b2370 2024-09-16T00:35:17.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.461+0000 7f86c0d67640 1 --2- 172.21.15.38:0/3191274865 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56195d9a75d0 0x56195da24af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-16T00:35:17.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.461+0000 7f86c1568640 1 --2- 172.21.15.38:0/3191274865 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56195d9b2370 0x56195d9b5a60 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f86b4009cd0 tx=0x7f86b402a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3721a133a536f0e2 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:17.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.461+0000 7f869bfff640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:35:17.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.464+0000 7f869bfff640 1 -- 172.21.15.38:0/3191274865 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86b404e070 con 0x56195d9b2370 2024-09-16T00:35:17.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.464+0000 7f869bfff640 1 -- 172.21.15.38:0/3191274865 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f86b40305a0 con 0x56195d9b2370 2024-09-16T00:35:17.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.464+0000 7f86c53a4740 1 -- 172.21.15.38:0/3191274865 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86ac0041c0 con 0x56195d9b2370 2024-09-16T00:35:17.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.464+0000 7f869bfff640 1 -- 172.21.15.38:0/3191274865 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f86b405b050 con 0x56195d9b2370 2024-09-16T00:35:17.476 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.464+0000 7f86c53a4740 10 client.15894 fetch_fsmap learned FSMap version 26 2024-09-16T00:35:17.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.464+0000 7f86c53a4740 1 -- 172.21.15.38:0/3191274865 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x56195da2c2c0 con 0x56195d9b2370 2024-09-16T00:35:17.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.465+0000 7f869bfff640 1 -- 172.21.15.38:0/3191274865 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f86ac0041c0 con 0x56195d9b2370 2024-09-16T00:35:17.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.465+0000 7f869bfff640 1 -- 172.21.15.38:0/3191274865 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7f86b4030070 con 0x56195d9b2370 2024-09-16T00:35:17.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.465+0000 7f86c53a4740 10 client.15894 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:35:17.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.465+0000 7f86c53a4740 10 client.15894 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2024-09-16T00:35:17.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.465+0000 7f86c53a4740 20 client.15894 populate_metadata read hostname 'smithi038' 2024-09-16T00:35:17.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.465+0000 7f86c53a4740 1 --2- 172.21.15.38:0/3191274865 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x56195da2ca40 0x56195da2ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:17.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.465+0000 7f86c53a4740 4 client.15894 mds_command: new command op to 24464 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:35:17.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.465+0000 7f86c53a4740 1 -- 172.21.15.38:0/3191274865 --> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56195d914d70 con 0x56195da2ca40 2024-09-16T00:35:17.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.465+0000 7f86c0d67640 1 --2- 172.21.15.38:0/3191274865 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x56195da2ca40 0x56195da2ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:17.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.466+0000 7f86c0d67640 1 --2- 172.21.15.38:0/3191274865 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x56195da2ca40 0x56195da2ee60 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7f884fc19e391f24 server_cookie=7157cd54317512c9 in_seq=0 out_seq=0 2024-09-16T00:35:17.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.466+0000 7f869bfff640 10 client.15894 ms_handle_connect on v2:172.21.15.73:6836/2681081265 2024-09-16T00:35:17.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.466+0000 7f869bfff640 1 -- 172.21.15.38:0/3191274865 <== mds.1 v2:172.21.15.73:6836/2681081265 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56195d914d70 con 0x56195da2ca40 2024-09-16T00:35:17.477 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.466+0000 7f869bfff640 10 client.15894 handle_command_reply: tid=0 2024-09-16T00:35:17.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.466+0000 7f86c53a4740 1 client.15894 shutdown 2024-09-16T00:35:17.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.466+0000 7f86c53a4740 1 -- 172.21.15.38:0/3191274865 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56195d9b2370 msgr2=0x56195d9b5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:17.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.466+0000 7f86c53a4740 1 --2- 172.21.15.38:0/3191274865 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56195d9b2370 0x56195d9b5a60 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f86b4009cd0 tx=0x7f86b402a8e0 comp rx=0 tx=0).stop 2024-09-16T00:35:17.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.466+0000 7f86c53a4740 1 -- 172.21.15.38:0/3191274865 shutdown_connections 2024-09-16T00:35:17.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.466+0000 7f86c53a4740 1 --2- 172.21.15.38:0/3191274865 >> [v2:172.21.15.73:6836/2681081265,v1:172.21.15.73:6837/2681081265] conn(0x56195da2ca40 0x56195da2ee60 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:17.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.466+0000 7f86c53a4740 1 --2- 172.21.15.38:0/3191274865 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x56195d9b2370 0x56195d9b5a60 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:17.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.466+0000 7f86c53a4740 1 --2- 172.21.15.38:0/3191274865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56195d9a8b40 0x56195da25030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:17.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.466+0000 7f86c53a4740 1 --2- 172.21.15.38:0/3191274865 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x56195d9a75d0 0x56195da24af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:17.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.466+0000 7f869bfff640 0 client.15894 ms_handle_reset on v2:172.21.15.73:6836/2681081265 2024-09-16T00:35:17.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.466+0000 7f86c53a4740 1 -- 172.21.15.38:0/3191274865 >> 172.21.15.38:0/3191274865 conn(0x56195d970310 msgr2=0x56195d9bd300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:17.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.466+0000 7f86c53a4740 1 -- 172.21.15.38:0/3191274865 shutdown_connections 2024-09-16T00:35:17.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.466+0000 7f86c53a4740 1 -- 172.21.15.38:0/3191274865 wait complete. 2024-09-16T00:35:17.478 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.466+0000 7f86c53a4740 20 client.15894 trim_cache size 0 max 16384 2024-09-16T00:35:17.479 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-09-16T00:35:17.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.467+0000 7f86c2fcb640 1 -- 172.21.15.38:0/1760869753 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f86a80a3240 msgr2=0x7f86a80a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:17.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.467+0000 7f86c2fcb640 1 --2- 172.21.15.38:0/1760869753 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f86a80a3240 0x7f86a80a5700 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f86b4009430 tx=0x7f86b4031510 comp rx=0 tx=0).stop 2024-09-16T00:35:17.479 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.467+0000 7f86c2fcb640 1 -- 172.21.15.38:0/1760869753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86bc101420 msgr2=0x7f86bc19ab40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:17.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.467+0000 7f86c2fcb640 1 --2- 172.21.15.38:0/1760869753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86bc101420 0x7f86bc19ab40 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f86a400d950 tx=0x7f86a400de10 comp rx=0 tx=0).stop 2024-09-16T00:35:17.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.467+0000 7f86c2fcb640 1 -- 172.21.15.38:0/1760869753 shutdown_connections 2024-09-16T00:35:17.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.467+0000 7f86c2fcb640 1 --2- 172.21.15.38:0/1760869753 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f86a80a3240 0x7f86a80a5700 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:17.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.467+0000 7f86c2fcb640 1 --2- 172.21.15.38:0/1760869753 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86bc106a80 0x7f86bc19eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:17.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.467+0000 7f86c2fcb640 1 --2- 172.21.15.38:0/1760869753 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f86bc101420 0x7f86bc19ab40 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:17.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.467+0000 7f86c2fcb640 1 --2- 172.21.15.38:0/1760869753 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f86bc100a80 0x7f86bc19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:17.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.467+0000 7f86c2fcb640 1 -- 172.21.15.38:0/1760869753 >> 172.21.15.38:0/1760869753 conn(0x7f86bc0fc7a0 msgr2=0x7f86bc06a790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:17.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.467+0000 7f86c2fcb640 1 -- 172.21.15.38:0/1760869753 shutdown_connections 2024-09-16T00:35:17.480 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:17.467+0000 7f86c2fcb640 1 -- 172.21.15.38:0/1760869753 wait complete. 2024-09-16T00:35:17.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:17 smithi038 ceph-mon[30945]: pgmap v439: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-16T00:35:17.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:17 smithi122 ceph-mon[35412]: pgmap v439: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-16T00:35:17.793 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:17 smithi073 ceph-mon[35254]: pgmap v439: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-16T00:35:18.204 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph tell mds.1:0 damage ls 2024-09-16T00:35:18.462 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:35:18.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.942+0000 7fb9a48ef640 1 -- 172.21.15.38:0/1582949361 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9a0101de0 msgr2=0x7fb9a01021c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:18.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.942+0000 7fb9a48ef640 1 --2- 172.21.15.38:0/1582949361 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9a0101de0 0x7fb9a01021c0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fb990009900 tx=0x7fb99002a410 comp rx=0 tx=0).stop 2024-09-16T00:35:18.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.942+0000 7fb9a48ef640 1 -- 172.21.15.38:0/1582949361 shutdown_connections 2024-09-16T00:35:18.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.942+0000 7fb9a48ef640 1 --2- 172.21.15.38:0/1582949361 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9a0106f10 0x7fb9a010ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:18.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.942+0000 7fb9a48ef640 1 --2- 172.21.15.38:0/1582949361 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9a0102790 0x7fb9a01067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:18.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.942+0000 7fb9a48ef640 1 --2- 172.21.15.38:0/1582949361 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9a0101de0 0x7fb9a01021c0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:18.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.942+0000 7fb9a48ef640 1 -- 172.21.15.38:0/1582949361 >> 172.21.15.38:0/1582949361 conn(0x7fb9a00fd8c0 msgr2=0x7fb9a00ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:18.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.942+0000 7fb9a48ef640 1 -- 172.21.15.38:0/1582949361 shutdown_connections 2024-09-16T00:35:18.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.942+0000 7fb9a48ef640 1 -- 172.21.15.38:0/1582949361 wait complete. 2024-09-16T00:35:18.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.943+0000 7fb9a48ef640 1 Processor -- start 2024-09-16T00:35:18.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.943+0000 7fb9a48ef640 1 -- start start 2024-09-16T00:35:18.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.943+0000 7fb9a48ef640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9a0101de0 0x7fb9a01985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:18.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.943+0000 7fb9a48ef640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9a0102790 0x7fb9a0198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:18.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.943+0000 7fb9a48ef640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9a0106f10 0x7fb9a019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:18.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.943+0000 7fb9a48ef640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a0112060 con 0x7fb9a0106f10 2024-09-16T00:35:18.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.943+0000 7fb9a48ef640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a0111ee0 con 0x7fb9a0101de0 2024-09-16T00:35:18.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.943+0000 7fb9a48ef640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a01121e0 con 0x7fb9a0102790 2024-09-16T00:35:18.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.943+0000 7fb99ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9a0106f10 0x7fb9a019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:18.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.943+0000 7fb99ed9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9a0106f10 0x7fb9a019ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42784/0 (socket says 172.21.15.38:42784) 2024-09-16T00:35:18.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.944+0000 7fb99ed9d640 1 -- 172.21.15.38:0/3061487745 learned_addr learned my addr 172.21.15.38:0/3061487745 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:35:18.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.944+0000 7fb99e59c640 1 --2- 172.21.15.38:0/3061487745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9a0101de0 0x7fb9a01985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:18.961 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.944+0000 7fb99dd9b640 1 --2- 172.21.15.38:0/3061487745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9a0102790 0x7fb9a0198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:18.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.944+0000 7fb99ed9d640 1 -- 172.21.15.38:0/3061487745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9a0102790 msgr2=0x7fb9a0198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:18.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.944+0000 7fb99ed9d640 1 --2- 172.21.15.38:0/3061487745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9a0102790 0x7fb9a0198b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:18.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.944+0000 7fb99ed9d640 1 -- 172.21.15.38:0/3061487745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9a0101de0 msgr2=0x7fb9a01985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:18.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.944+0000 7fb99ed9d640 1 --2- 172.21.15.38:0/3061487745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9a0101de0 0x7fb9a01985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:18.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.944+0000 7fb99ed9d640 1 -- 172.21.15.38:0/3061487745 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9a019d570 con 0x7fb9a0106f10 2024-09-16T00:35:18.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.944+0000 7fb99dd9b640 1 --2- 172.21.15.38:0/3061487745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9a0102790 0x7fb9a0198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:35:18.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.944+0000 7fb99ed9d640 1 --2- 172.21.15.38:0/3061487745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9a0106f10 0x7fb9a019ce90 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fb99800b970 tx=0x7fb99800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:18.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.944+0000 7fb9977fe640 1 -- 172.21.15.38:0/3061487745 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb998019070 con 0x7fb9a0106f10 2024-09-16T00:35:18.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.944+0000 7fb9a48ef640 1 -- 172.21.15.38:0/3061487745 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9a019d860 con 0x7fb9a0106f10 2024-09-16T00:35:18.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.944+0000 7fb9977fe640 1 -- 172.21.15.38:0/3061487745 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb99800cb40 con 0x7fb9a0106f10 2024-09-16T00:35:18.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.944+0000 7fb9977fe640 1 -- 172.21.15.38:0/3061487745 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb998007650 con 0x7fb9a0106f10 2024-09-16T00:35:18.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.944+0000 7fb9a48ef640 1 -- 172.21.15.38:0/3061487745 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9a01a5140 con 0x7fb9a0106f10 2024-09-16T00:35:18.962 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.947+0000 7fb9977fe640 1 -- 172.21.15.38:0/3061487745 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fb99801c070 con 0x7fb9a0106f10 2024-09-16T00:35:18.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.947+0000 7fb9a6cc8740 1 Processor -- start 2024-09-16T00:35:18.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.947+0000 7fb9a6cc8740 1 -- start start 2024-09-16T00:35:18.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.947+0000 7fb9a6cc8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557594777b40 0x557594776090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:18.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.947+0000 7fb9a6cc8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5575947765d0 0x557594776a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:18.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.947+0000 7fb9a6cc8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557594781370 0x557594783760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:18.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.947+0000 7fb9a6cc8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x5575946dc2f0 con 0x557594781370 2024-09-16T00:35:18.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.947+0000 7fb9a6cc8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557594683cf0 con 0x557594777b40 2024-09-16T00:35:18.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.947+0000 7fb9a6cc8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5575946d82b0 con 0x5575947765d0 2024-09-16T00:35:18.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.947+0000 7fb99dd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557594781370 0x557594783760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:18.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.948+0000 7fb99dd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557594781370 0x557594783760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42800/0 (socket says 172.21.15.38:42800) 2024-09-16T00:35:18.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.948+0000 7fb99dd9b640 1 -- 172.21.15.38:0/2757958191 learned_addr learned my addr 172.21.15.38:0/2757958191 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:35:18.963 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.948+0000 7fb99dd9b640 1 --2- 172.21.15.38:0/2757958191 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557594777b40 0x557594776090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:18.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.948+0000 7fb9977fe640 1 --2- 172.21.15.38:0/3061487745 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb98c0a3240 0x7fb98c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:18.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.948+0000 7fb99e59c640 1 --2- 172.21.15.38:0/2757958191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5575947765d0 0x557594776a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:18.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.948+0000 7fb99ed9d640 1 --2- 172.21.15.38:0/3061487745 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb98c0a3240 0x7fb98c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:18.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.948+0000 7fb9977fe640 1 -- 172.21.15.38:0/3061487745 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7fb9980cf350 con 0x7fb9a0106f10 2024-09-16T00:35:18.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.948+0000 7fb99dd9b640 1 -- 172.21.15.38:0/2757958191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5575947765d0 msgr2=0x557594776a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:18.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.948+0000 7fb99dd9b640 1 --2- 172.21.15.38:0/2757958191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5575947765d0 0x557594776a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:18.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.948+0000 7fb99dd9b640 1 -- 172.21.15.38:0/2757958191 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557594777b40 msgr2=0x557594776090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:18.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.948+0000 7fb99dd9b640 1 --2- 172.21.15.38:0/2757958191 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557594777b40 0x557594776090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:18.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.948+0000 7fb99dd9b640 1 -- 172.21.15.38:0/2757958191 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5575946683a0 con 0x557594781370 2024-09-16T00:35:18.964 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.948+0000 7fb99dd9b640 1 --2- 172.21.15.38:0/2757958191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557594781370 0x557594783760 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb988006fd0 tx=0x7fb98800c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5684230f20242ead server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:18.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.948+0000 7fb99ed9d640 1 --2- 172.21.15.38:0/3061487745 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb98c0a3240 0x7fb98c0a5700 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb99800b940 tx=0x7fb99800e040 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:18.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.948+0000 7fb994ff9640 1 -- 172.21.15.38:0/2757958191 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98801c070 con 0x557594781370 2024-09-16T00:35:18.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.948+0000 7fb994ff9640 1 -- 172.21.15.38:0/2757958191 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb988017af0 con 0x557594781370 2024-09-16T00:35:18.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.948+0000 7fb994ff9640 1 -- 172.21.15.38:0/2757958191 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98800f040 con 0x557594781370 2024-09-16T00:35:18.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.949+0000 7fb9a6cc8740 1 -- 172.21.15.38:0/2757958191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557594781370 msgr2=0x557594783760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:18.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.949+0000 7fb9a6cc8740 1 --2- 172.21.15.38:0/2757958191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557594781370 0x557594783760 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fb988006fd0 tx=0x7fb98800c230 comp rx=0 tx=0).stop 2024-09-16T00:35:18.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.949+0000 7fb9a6cc8740 1 -- 172.21.15.38:0/2757958191 shutdown_connections 2024-09-16T00:35:18.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.949+0000 7fb9a6cc8740 1 --2- 172.21.15.38:0/2757958191 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557594781370 0x557594783760 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:18.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.949+0000 7fb9a6cc8740 1 --2- 172.21.15.38:0/2757958191 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5575947765d0 0x557594776a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:18.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.949+0000 7fb9a6cc8740 1 --2- 172.21.15.38:0/2757958191 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557594777b40 0x557594776090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:18.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.949+0000 7fb9a6cc8740 1 -- 172.21.15.38:0/2757958191 >> 172.21.15.38:0/2757958191 conn(0x55759473f310 msgr2=0x557594772c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:18.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.949+0000 7fb9a6cc8740 1 -- 172.21.15.38:0/2757958191 shutdown_connections 2024-09-16T00:35:18.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.949+0000 7fb9a6cc8740 1 -- 172.21.15.38:0/2757958191 wait complete. 2024-09-16T00:35:18.965 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.950+0000 7fb9a6cc8740 1 -- start start 2024-09-16T00:35:18.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.950+0000 7fb9a6cc8740 1 Processor -- start 2024-09-16T00:35:18.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.950+0000 7fb9a6cc8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5575947765d0 0x557594780d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:18.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.950+0000 7fb9a6cc8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557594777b40 0x55759477e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:18.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.950+0000 7fb9a6cc8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557594781370 0x55759477e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:18.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.950+0000 7fb9a6cc8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x557594784070 con 0x557594781370 2024-09-16T00:35:18.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.950+0000 7fb9a6cc8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x557594697060 con 0x557594777b40 2024-09-16T00:35:18.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.950+0000 7fb9a6cc8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x5575947841f0 con 0x5575947765d0 2024-09-16T00:35:18.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.950+0000 7fb9a6cc8740 10 client.0 _ll_register_callbacks cb 0x5575943ea8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:35:18.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.950+0000 7fb99dd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557594781370 0x55759477e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:18.966 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.950+0000 7fb99dd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557594781370 0x55759477e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42806/0 (socket says 172.21.15.38:42806) 2024-09-16T00:35:18.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.950+0000 7fb99dd9b640 1 -- 172.21.15.38:0/2147001209 learned_addr learned my addr 172.21.15.38:0/2147001209 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:35:18.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.950+0000 7fb99dd9b640 1 --2- 172.21.15.38:0/2147001209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5575947765d0 0x557594780d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:18.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.950+0000 7fb99e59c640 1 --2- 172.21.15.38:0/2147001209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557594777b40 0x55759477e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:18.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.950+0000 7fb99dd9b640 1 -- 172.21.15.38:0/2147001209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5575947765d0 msgr2=0x557594780d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:18.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.950+0000 7fb99dd9b640 1 --2- 172.21.15.38:0/2147001209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5575947765d0 0x557594780d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:18.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.950+0000 7fb99dd9b640 1 -- 172.21.15.38:0/2147001209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557594777b40 msgr2=0x55759477e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:18.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.950+0000 7fb99dd9b640 1 --2- 172.21.15.38:0/2147001209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557594777b40 0x55759477e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:18.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.950+0000 7fb99dd9b640 1 -- 172.21.15.38:0/2147001209 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55759444d3c0 con 0x557594781370 2024-09-16T00:35:18.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.950+0000 7fb99e59c640 1 --2- 172.21.15.38:0/2147001209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557594777b40 0x55759477e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:35:18.967 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.951+0000 7fb99dd9b640 1 --2- 172.21.15.38:0/2147001209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557594781370 0x55759477e640 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb988016730 tx=0x7fb988016630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e6f6904cef24b51 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:18.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.951+0000 7fb985ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:35:18.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.951+0000 7fb985ffb640 1 -- 172.21.15.38:0/2147001209 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb988026020 con 0x557594781370 2024-09-16T00:35:18.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.951+0000 7fb985ffb640 1 -- 172.21.15.38:0/2147001209 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7fb988017610 con 0x557594781370 2024-09-16T00:35:18.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.951+0000 7fb9a6cc8740 1 -- 172.21.15.38:0/2147001209 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb988017af0 con 0x557594781370 2024-09-16T00:35:18.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.951+0000 7fb985ffb640 1 -- 172.21.15.38:0/2147001209 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb988031050 con 0x557594781370 2024-09-16T00:35:18.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.951+0000 7fb9a6cc8740 10 client.15906 fetch_fsmap learned FSMap version 26 2024-09-16T00:35:18.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.951+0000 7fb9a6cc8740 1 -- 172.21.15.38:0/2147001209 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x55759478fb40 con 0x557594781370 2024-09-16T00:35:18.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.951+0000 7fb985ffb640 1 -- 172.21.15.38:0/2147001209 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb988017af0 con 0x557594781370 2024-09-16T00:35:18.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.951+0000 7fb985ffb640 1 -- 172.21.15.38:0/2147001209 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb988015040 con 0x557594781370 2024-09-16T00:35:18.968 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.951+0000 7fb9a6cc8740 10 client.15906 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:35:18.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.951+0000 7fb9a6cc8740 10 client.15906 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:35:18.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.952+0000 7fb9a6cc8740 20 client.15906 populate_metadata read hostname 'smithi038' 2024-09-16T00:35:18.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.952+0000 7fb9a6cc8740 1 --2- 172.21.15.38:0/2147001209 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x557594790e90 0x557594793280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:18.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.952+0000 7fb9a6cc8740 4 client.15906 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-16T00:35:18.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.952+0000 7fb9a6cc8740 1 -- 172.21.15.38:0/2147001209 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5575946e9860 con 0x557594790e90 2024-09-16T00:35:18.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.952+0000 7fb99e59c640 1 --2- 172.21.15.38:0/2147001209 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x557594790e90 0x557594793280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:18.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.952+0000 7fb99e59c640 1 --2- 172.21.15.38:0/2147001209 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x557594790e90 0x557594793280 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30267c301cad2a90 server_cookie=c37feb827881343f in_seq=0 out_seq=0 2024-09-16T00:35:18.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.952+0000 7fb985ffb640 10 client.15906 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:35:18.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.953+0000 7fb985ffb640 1 -- 172.21.15.38:0/2147001209 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24171 (crc 0 0 0) 0x5575946e9860 con 0x557594790e90 2024-09-16T00:35:18.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.953+0000 7fb985ffb640 10 client.15906 handle_command_reply: tid=0 2024-09-16T00:35:18.969 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.953+0000 7fb9a6cc8740 1 client.15906 shutdown 2024-09-16T00:35:18.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.953+0000 7fb9a6cc8740 1 -- 172.21.15.38:0/2147001209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557594781370 msgr2=0x55759477e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:18.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.953+0000 7fb9a6cc8740 1 --2- 172.21.15.38:0/2147001209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557594781370 0x55759477e640 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fb988016730 tx=0x7fb988016630 comp rx=0 tx=0).stop 2024-09-16T00:35:18.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.953+0000 7fb9a6cc8740 1 -- 172.21.15.38:0/2147001209 shutdown_connections 2024-09-16T00:35:18.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.953+0000 7fb9a6cc8740 1 --2- 172.21.15.38:0/2147001209 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x557594790e90 0x557594793280 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:18.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.953+0000 7fb9a6cc8740 1 --2- 172.21.15.38:0/2147001209 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557594781370 0x55759477e640 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:18.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.953+0000 7fb9a6cc8740 1 --2- 172.21.15.38:0/2147001209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557594777b40 0x55759477e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:18.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.953+0000 7fb985ffb640 0 client.15906 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:35:18.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.953+0000 7fb9a6cc8740 1 --2- 172.21.15.38:0/2147001209 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x5575947765d0 0x557594780d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:18.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.953+0000 7fb9a6cc8740 1 -- 172.21.15.38:0/2147001209 >> 172.21.15.38:0/2147001209 conn(0x55759473f310 msgr2=0x557594778960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:18.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.953+0000 7fb9a6cc8740 1 -- 172.21.15.38:0/2147001209 shutdown_connections 2024-09-16T00:35:18.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.953+0000 7fb9a6cc8740 1 -- 172.21.15.38:0/2147001209 wait complete. 2024-09-16T00:35:18.970 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.953+0000 7fb9a6cc8740 20 client.15906 trim_cache size 0 max 16384 2024-09-16T00:35:18.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.966+0000 7fb9a6cc8740 1 Processor -- start 2024-09-16T00:35:18.975 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.966+0000 7fb9a6cc8740 1 -- start start 2024-09-16T00:35:18.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.966+0000 7fb9a6cc8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5575947765d0 0x55759478d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:18.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.966+0000 7fb9a6cc8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557594777b40 0x557594796690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:18.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.966+0000 7fb9a6cc8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557594781370 0x557594796bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:18.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.966+0000 7fb9a6cc8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a01b16a0 con 0x557594781370 2024-09-16T00:35:18.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.966+0000 7fb9a6cc8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a01aef70 con 0x5575947765d0 2024-09-16T00:35:18.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.966+0000 7fb9a6cc8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a01acdc0 con 0x557594777b40 2024-09-16T00:35:18.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.967+0000 7fb99dd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557594781370 0x557594796bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:18.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.967+0000 7fb99dd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557594781370 0x557594796bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42814/0 (socket says 172.21.15.38:42814) 2024-09-16T00:35:18.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.967+0000 7fb99dd9b640 1 -- 172.21.15.38:0/4274208898 learned_addr learned my addr 172.21.15.38:0/4274208898 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:35:18.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.967+0000 7fb99dd9b640 1 --2- 172.21.15.38:0/4274208898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5575947765d0 0x55759478d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:18.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.967+0000 7fb99e59c640 1 --2- 172.21.15.38:0/4274208898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557594777b40 0x557594796690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:18.976 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.967+0000 7fb99dd9b640 1 -- 172.21.15.38:0/4274208898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557594777b40 msgr2=0x557594796690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:18.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.967+0000 7fb99dd9b640 1 --2- 172.21.15.38:0/4274208898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557594777b40 0x557594796690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:18.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.967+0000 7fb99dd9b640 1 -- 172.21.15.38:0/4274208898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5575947765d0 msgr2=0x55759478d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:18.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.967+0000 7fb99dd9b640 1 --2- 172.21.15.38:0/4274208898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5575947765d0 0x55759478d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:18.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.967+0000 7fb99dd9b640 1 -- 172.21.15.38:0/4274208898 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557594777300 con 0x557594781370 2024-09-16T00:35:18.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.967+0000 7fb99e59c640 1 --2- 172.21.15.38:0/4274208898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557594777b40 0x557594796690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:35:18.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.967+0000 7fb99dd9b640 1 --2- 172.21.15.38:0/4274208898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557594781370 0x557594796bd0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb988015480 tx=0x7fb988016590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5826a4e3c18db6ed server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:18.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.967+0000 7fb987fff640 1 -- 172.21.15.38:0/4274208898 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb988026070 con 0x557594781370 2024-09-16T00:35:18.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.967+0000 7fb987fff640 1 -- 172.21.15.38:0/4274208898 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb98800c6f0 con 0x557594781370 2024-09-16T00:35:18.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.967+0000 7fb987fff640 1 -- 172.21.15.38:0/4274208898 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98801c040 con 0x557594781370 2024-09-16T00:35:18.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.968+0000 7fb9a6cc8740 1 -- 172.21.15.38:0/4274208898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557594781370 msgr2=0x557594796bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:18.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.968+0000 7fb9a6cc8740 1 --2- 172.21.15.38:0/4274208898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557594781370 0x557594796bd0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fb988015480 tx=0x7fb988016590 comp rx=0 tx=0).stop 2024-09-16T00:35:18.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.968+0000 7fb9a6cc8740 1 -- 172.21.15.38:0/4274208898 shutdown_connections 2024-09-16T00:35:18.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.968+0000 7fb9a6cc8740 1 --2- 172.21.15.38:0/4274208898 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557594781370 0x557594796bd0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:18.977 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.968+0000 7fb9a6cc8740 1 --2- 172.21.15.38:0/4274208898 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557594777b40 0x557594796690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:18.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.968+0000 7fb9a6cc8740 1 --2- 172.21.15.38:0/4274208898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5575947765d0 0x55759478d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:18.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.968+0000 7fb9a6cc8740 1 -- 172.21.15.38:0/4274208898 >> 172.21.15.38:0/4274208898 conn(0x55759473f310 msgr2=0x5575947800b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:18.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.968+0000 7fb9a6cc8740 1 -- 172.21.15.38:0/4274208898 shutdown_connections 2024-09-16T00:35:18.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.968+0000 7fb9a6cc8740 1 -- 172.21.15.38:0/4274208898 wait complete. 2024-09-16T00:35:18.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.968+0000 7fb9a6cc8740 1 -- start start 2024-09-16T00:35:18.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.968+0000 7fb9a6cc8740 1 Processor -- start 2024-09-16T00:35:18.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.969+0000 7fb9a6cc8740 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5575947765d0 0x5575947f6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:18.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.969+0000 7fb9a6cc8740 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557594777b40 0x5575947f70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:18.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.969+0000 7fb9a6cc8740 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557594781370 0x55759477abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:18.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.969+0000 7fb9a6cc8740 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a01ad430 con 0x557594777b40 2024-09-16T00:35:18.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.969+0000 7fb9a6cc8740 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a01ae8a0 con 0x5575947765d0 2024-09-16T00:35:18.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.969+0000 7fb9a6cc8740 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fb9a01adb00 con 0x557594781370 2024-09-16T00:35:18.978 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.969+0000 7fb9a6cc8740 10 client.0 _ll_register_callbacks cb 0x7fb98801a070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-16T00:35:18.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.969+0000 7fb99e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557594777b40 0x5575947f70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:18.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.969+0000 7fb99e59c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557594777b40 0x5575947f70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42822/0 (socket says 172.21.15.38:42822) 2024-09-16T00:35:18.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.969+0000 7fb99e59c640 1 -- 172.21.15.38:0/2652898109 learned_addr learned my addr 172.21.15.38:0/2652898109 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:35:18.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.969+0000 7fb99dd9b640 1 --2- 172.21.15.38:0/2652898109 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557594781370 0x55759477abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:18.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.969+0000 7fb99dd9b640 1 --2- 172.21.15.38:0/2652898109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5575947765d0 0x5575947f6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:18.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.969+0000 7fb99e59c640 1 -- 172.21.15.38:0/2652898109 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557594781370 msgr2=0x55759477abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:18.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.969+0000 7fb99e59c640 1 --2- 172.21.15.38:0/2652898109 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557594781370 0x55759477abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:18.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.969+0000 7fb99e59c640 1 -- 172.21.15.38:0/2652898109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5575947765d0 msgr2=0x5575947f6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-16T00:35:18.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.969+0000 7fb99e59c640 1 --2- 172.21.15.38:0/2652898109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5575947765d0 0x5575947f6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:18.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.969+0000 7fb99e59c640 1 -- 172.21.15.38:0/2652898109 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55759477b3e0 con 0x557594777b40 2024-09-16T00:35:18.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.969+0000 7fb99e59c640 1 --2- 172.21.15.38:0/2652898109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557594777b40 0x5575947f70f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fb99002a9d0 tx=0x7fb990005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f11f5a90be7d58e server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:18.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.969+0000 7fb985ffb640 10 client.0 ms_handle_connect on v2:172.21.15.38:3300/0 2024-09-16T00:35:18.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.969+0000 7fb985ffb640 1 -- 172.21.15.38:0/2652898109 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb990035040 con 0x557594777b40 2024-09-16T00:35:18.979 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.969+0000 7fb985ffb640 1 -- 172.21.15.38:0/2652898109 <== mon.0 v2:172.21.15.38:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7fb990005020 con 0x557594777b40 2024-09-16T00:35:18.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.969+0000 7fb9a6cc8740 1 -- 172.21.15.38:0/2652898109 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb98800c6f0 con 0x557594777b40 2024-09-16T00:35:18.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.970+0000 7fb985ffb640 1 -- 172.21.15.38:0/2652898109 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7fb990052050 con 0x557594777b40 2024-09-16T00:35:18.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.970+0000 7fb9a6cc8740 10 client.15912 fetch_fsmap learned FSMap version 26 2024-09-16T00:35:18.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.970+0000 7fb9a6cc8740 1 -- 172.21.15.38:0/2652898109 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) -- 0x5575947ff420 con 0x557594777b40 2024-09-16T00:35:18.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.970+0000 7fb985ffb640 1 -- 172.21.15.38:0/2652898109 <== mon.0 v2:172.21.15.38:3300/0 4 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fb98800c6f0 con 0x557594777b40 2024-09-16T00:35:18.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.970+0000 7fb985ffb640 1 -- 172.21.15.38:0/2652898109 <== mon.0 v2:172.21.15.38:3300/0 5 ==== fsmap(e 26) ==== 8354+0+0 (secure 0 0 0) 0x7fb9900047a0 con 0x557594777b40 2024-09-16T00:35:18.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.970+0000 7fb9a6cc8740 10 client.15912 fetch_fsmap finished waiting for FSMap version 26 2024-09-16T00:35:18.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.970+0000 7fb9a6cc8740 10 client.15912 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-16T00:35:18.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.970+0000 7fb9a6cc8740 20 client.15912 populate_metadata read hostname 'smithi038' 2024-09-16T00:35:18.980 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.970+0000 7fb9a6cc8740 1 --2- 172.21.15.38:0/2652898109 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5575947ffc20 0x557594802040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:18.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.970+0000 7fb9a6cc8740 4 client.15912 mds_command: new command op to 24494 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-16T00:35:18.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.970+0000 7fb9a6cc8740 1 -- 172.21.15.38:0/2652898109 --> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5575946e3d70 con 0x5575947ffc20 2024-09-16T00:35:18.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.970+0000 7fb99dd9b640 1 --2- 172.21.15.38:0/2652898109 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5575947ffc20 0x557594802040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:18.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.971+0000 7fb99dd9b640 1 --2- 172.21.15.38:0/2652898109 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5575947ffc20 0x557594802040 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d9887329b46337c4 server_cookie=b36adbaa0b4e425e in_seq=0 out_seq=0 2024-09-16T00:35:18.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.971+0000 7fb985ffb640 10 client.15912 ms_handle_connect on v2:172.21.15.73:6838/2396224525 2024-09-16T00:35:18.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.971+0000 7fb985ffb640 1 -- 172.21.15.38:0/2652898109 <== mds.0 v2:172.21.15.73:6838/2396224525 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5575946e3d70 con 0x5575947ffc20 2024-09-16T00:35:18.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.971+0000 7fb985ffb640 10 client.15912 handle_command_reply: tid=0 2024-09-16T00:35:18.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.971+0000 7fb9a6cc8740 1 client.15912 shutdown 2024-09-16T00:35:18.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.971+0000 7fb9a6cc8740 1 -- 172.21.15.38:0/2652898109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557594777b40 msgr2=0x5575947f70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:18.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.971+0000 7fb9a6cc8740 1 --2- 172.21.15.38:0/2652898109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557594777b40 0x5575947f70f0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fb99002a9d0 tx=0x7fb990005c00 comp rx=0 tx=0).stop 2024-09-16T00:35:18.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.971+0000 7fb9a6cc8740 1 -- 172.21.15.38:0/2652898109 shutdown_connections 2024-09-16T00:35:18.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.971+0000 7fb9a6cc8740 1 --2- 172.21.15.38:0/2652898109 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x557594781370 0x55759477abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:18.981 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.971+0000 7fb9a6cc8740 1 --2- 172.21.15.38:0/2652898109 >> [v2:172.21.15.73:6838/2396224525,v1:172.21.15.73:6839/2396224525] conn(0x5575947ffc20 0x557594802040 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:18.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.971+0000 7fb9a6cc8740 1 --2- 172.21.15.38:0/2652898109 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x557594777b40 0x5575947f70f0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:18.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.971+0000 7fb985ffb640 0 client.15912 ms_handle_reset on v2:172.21.15.73:6838/2396224525 2024-09-16T00:35:18.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.971+0000 7fb9a6cc8740 1 --2- 172.21.15.38:0/2652898109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5575947765d0 0x5575947f6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:18.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.971+0000 7fb9a6cc8740 1 -- 172.21.15.38:0/2652898109 >> 172.21.15.38:0/2652898109 conn(0x55759473f310 msgr2=0x55759477c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:18.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.971+0000 7fb9a6cc8740 1 -- 172.21.15.38:0/2652898109 shutdown_connections 2024-09-16T00:35:18.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.971+0000 7fb9a6cc8740 1 -- 172.21.15.38:0/2652898109 wait complete. 2024-09-16T00:35:18.982 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.971+0000 7fb9a6cc8740 20 client.15912 trim_cache size 0 max 16384 2024-09-16T00:35:18.982 INFO:teuthology.orchestra.run.smithi038.stdout:[] 2024-09-16T00:35:18.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.972+0000 7fb9a48ef640 1 -- 172.21.15.38:0/3061487745 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb98c0a3240 msgr2=0x7fb98c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:18.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.972+0000 7fb9a48ef640 1 --2- 172.21.15.38:0/3061487745 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb98c0a3240 0x7fb98c0a5700 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fb99800b940 tx=0x7fb99800e040 comp rx=0 tx=0).stop 2024-09-16T00:35:18.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.972+0000 7fb9a48ef640 1 -- 172.21.15.38:0/3061487745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9a0106f10 msgr2=0x7fb9a019ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:18.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.972+0000 7fb9a48ef640 1 --2- 172.21.15.38:0/3061487745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9a0106f10 0x7fb9a019ce90 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fb99800b970 tx=0x7fb99800be30 comp rx=0 tx=0).stop 2024-09-16T00:35:18.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.972+0000 7fb9a48ef640 1 -- 172.21.15.38:0/3061487745 shutdown_connections 2024-09-16T00:35:18.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.972+0000 7fb9a48ef640 1 --2- 172.21.15.38:0/3061487745 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fb98c0a3240 0x7fb98c0a5700 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:18.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.972+0000 7fb9a48ef640 1 --2- 172.21.15.38:0/3061487745 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fb9a0106f10 0x7fb9a019ce90 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:18.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.972+0000 7fb9a48ef640 1 --2- 172.21.15.38:0/3061487745 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fb9a0102790 0x7fb9a0198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:18.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.972+0000 7fb9a48ef640 1 --2- 172.21.15.38:0/3061487745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9a0101de0 0x7fb9a01985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:18.983 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.972+0000 7fb9a48ef640 1 -- 172.21.15.38:0/3061487745 >> 172.21.15.38:0/3061487745 conn(0x7fb9a00fd8c0 msgr2=0x7fb9a00fec50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:18.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.972+0000 7fb9a48ef640 1 -- 172.21.15.38:0/3061487745 shutdown_connections 2024-09-16T00:35:18.984 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:18.972+0000 7fb9a48ef640 1 -- 172.21.15.38:0/3061487745 wait complete. 2024-09-16T00:35:19.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:19 smithi038 ceph-mon[30945]: pgmap v440: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-16T00:35:19.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:19 smithi122 ceph-mon[35412]: pgmap v440: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-16T00:35:19.789 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph_ad35db06e8924912f01e32b35576297c0f1352b4/qa/tasks/thrasher.py", line 47, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-16T00:35:19.790 INFO:tasks.fwd_scrub:done joining 2024-09-16T00:35:19.790 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-16T00:35:19.801 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-16T00:35:19.802 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:19 smithi073 ceph-mon[35254]: pgmap v440: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-16T00:35:19.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:19.910+0000 7fa99745c640 1 -- 172.21.15.38:0/557227089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9901036e0 msgr2=0x7fa990109f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:19.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:19.910+0000 7fa99745c640 1 --2- 172.21.15.38:0/557227089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9901036e0 0x7fa990109f70 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fa988009ca0 tx=0x7fa98802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:35:19.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:19.911+0000 7fa99745c640 1 -- 172.21.15.38:0/557227089 shutdown_connections 2024-09-16T00:35:19.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:19.911+0000 7fa99745c640 1 --2- 172.21.15.38:0/557227089 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa9901036e0 0x7fa990109f70 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:19.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:19.911+0000 7fa99745c640 1 --2- 172.21.15.38:0/557227089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa990102d20 0x7fa9901031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:19.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:19.911+0000 7fa99745c640 1 --2- 172.21.15.38:0/557227089 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa990101b30 0x7fa990101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:19.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:19.911+0000 7fa99745c640 1 -- 172.21.15.38:0/557227089 >> 172.21.15.38:0/557227089 conn(0x7fa9900fd2b0 msgr2=0x7fa9900ff6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:19.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:19.911+0000 7fa99745c640 1 -- 172.21.15.38:0/557227089 shutdown_connections 2024-09-16T00:35:19.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:19.911+0000 7fa99745c640 1 -- 172.21.15.38:0/557227089 wait complete. 2024-09-16T00:35:19.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:19.911+0000 7fa99745c640 1 Processor -- start 2024-09-16T00:35:19.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:19.911+0000 7fa99745c640 1 -- start start 2024-09-16T00:35:19.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:19.911+0000 7fa99745c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa990101b30 0x7fa990075fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:19.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:19.911+0000 7fa99745c640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa990102d20 0x7fa9900745e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:19.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:19.912+0000 7fa99745c640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9901036e0 0x7fa990074b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:19.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:19.912+0000 7fa99745c640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa99010d800 con 0x7fa990101b30 2024-09-16T00:35:19.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:19.912+0000 7fa99745c640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa99010d680 con 0x7fa9901036e0 2024-09-16T00:35:19.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:19.912+0000 7fa99745c640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa99010d980 con 0x7fa990102d20 2024-09-16T00:35:19.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:19.912+0000 7fa9951f8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa990101b30 0x7fa990075fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:19.915 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:19.912+0000 7fa9951f8640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa990101b30 0x7fa990075fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42838/0 (socket says 172.21.15.38:42838) 2024-09-16T00:35:19.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:19.912+0000 7fa9951f8640 1 -- 172.21.15.38:0/1258454467 learned_addr learned my addr 172.21.15.38:0/1258454467 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:35:19.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:19.912+0000 7fa9959f9640 1 --2- 172.21.15.38:0/1258454467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9901036e0 0x7fa990074b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:19.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:19.912+0000 7fa9949f7640 1 --2- 172.21.15.38:0/1258454467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa990102d20 0x7fa9900745e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:19.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:19.912+0000 7fa9951f8640 1 -- 172.21.15.38:0/1258454467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa990102d20 msgr2=0x7fa9900745e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:19.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:19.912+0000 7fa9951f8640 1 --2- 172.21.15.38:0/1258454467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa990102d20 0x7fa9900745e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:19.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:19.912+0000 7fa9951f8640 1 -- 172.21.15.38:0/1258454467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9901036e0 msgr2=0x7fa990074b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:19.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:19.912+0000 7fa9951f8640 1 --2- 172.21.15.38:0/1258454467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9901036e0 0x7fa990074b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:19.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:19.912+0000 7fa9951f8640 1 -- 172.21.15.38:0/1258454467 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9900753b0 con 0x7fa990101b30 2024-09-16T00:35:19.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:19.912+0000 7fa9951f8640 1 --2- 172.21.15.38:0/1258454467 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa990101b30 0x7fa990075fa0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa98c00d950 tx=0x7fa98c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:19.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:19.912+0000 7fa9867fc640 1 -- 172.21.15.38:0/1258454467 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98c017070 con 0x7fa990101b30 2024-09-16T00:35:19.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:19.912+0000 7fa9867fc640 1 -- 172.21.15.38:0/1258454467 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa98c009c30 con 0x7fa990101b30 2024-09-16T00:35:19.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:19.912+0000 7fa99745c640 1 -- 172.21.15.38:0/1258454467 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9901a1f40 con 0x7fa990101b30 2024-09-16T00:35:19.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:19.912+0000 7fa9867fc640 1 -- 172.21.15.38:0/1258454467 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa98c00f600 con 0x7fa990101b30 2024-09-16T00:35:19.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:19.912+0000 7fa99745c640 1 -- 172.21.15.38:0/1258454467 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9901a2340 con 0x7fa990101b30 2024-09-16T00:35:19.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:19.913+0000 7fa99745c640 1 -- 172.21.15.38:0/1258454467 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa990108e50 con 0x7fa990101b30 2024-09-16T00:35:19.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:19.915+0000 7fa9867fc640 1 -- 172.21.15.38:0/1258454467 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fa98c01a070 con 0x7fa990101b30 2024-09-16T00:35:19.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:19.915+0000 7fa9867fc640 1 --2- 172.21.15.38:0/1258454467 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fa97c0a3250 0x7fa97c0a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:19.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:19.915+0000 7fa9949f7640 1 --2- 172.21.15.38:0/1258454467 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fa97c0a3250 0x7fa97c0a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:19.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:19.915+0000 7fa9867fc640 1 -- 172.21.15.38:0/1258454467 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7fa98c0cf750 con 0x7fa990101b30 2024-09-16T00:35:19.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:19.916+0000 7fa9949f7640 1 --2- 172.21.15.38:0/1258454467 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fa97c0a3250 0x7fa97c0a5710 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa980005fd0 tx=0x7fa980005850 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:19.921 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:19.920+0000 7fa9867fc640 1 -- 172.21.15.38:0/1258454467 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa98c009de0 con 0x7fa990101b30 2024-09-16T00:35:20.098 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.097+0000 7fa99745c640 1 -- 172.21.15.38:0/1258454467 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa990062ef0 con 0x7fa990101b30 2024-09-16T00:35:20.100 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.098+0000 7fa9867fc640 1 -- 172.21.15.38:0/1258454467 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24767 (secure 0 0 0) 0x7fa98c09c0f0 con 0x7fa990101b30 2024-09-16T00:35:20.100 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:35:20.100 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":92,"fsid":"983b2072-73c1-11ef-bceb-c7b262605968","created":"2024-09-16T00:22:34.897053+0000","modified":"2024-09-16T00:32:24.296695+0000","last_up_change":"2024-09-16T00:29:34.503134+0000","last_in_change":"2024-09-16T00:29:10.135656+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:25:41.931977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:30:22.741889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:30:25.042590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:30:31.554350+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3afc20e3-869f-43a9-a911-b0e8d07de86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6803","nonce":1622028921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6805","nonce":1622028921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6809","nonce":1622028921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6807","nonce":1622028921}]},"public_addr":"172.21.15.38:6803/1622028921","cluster_addr":"172.21.15.38:6805/1622028921","heartbeat_back_addr":"172.21.15.38:6809/1622028921","heartbeat_front_addr":"172.21.15.38:6807/1622028921","state":["exists","up"]},{"osd":1,"uuid":"6d9d7acb-45ed-4543-8afc-6433fc7f096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6801","nonce":3811690336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6803","nonce":3811690336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6807","nonce":3811690336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6805","nonce":3811690336}]},"public_addr":"172.21.15.73:6801/3811690336","cluster_addr":"172.21.15.73:6803/3811690336","heartbeat_back_addr":"172.21.15.73:6807/3811690336","heartbeat_front_addr":"172.21.15.73:6805/3811690336","state":["exists","up"]},{"osd":2,"uuid":"48690f35-99e9-4f45-9bee-487857caa8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6801","nonce":1754741655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6803","nonce":1754741655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6807","nonce":1754741655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6805","nonce":1754741655}]},"public_addr":"172.21.15.122:6801/1754741655","cluster_addr":"172.21.15.122:6803/1754741655","heartbeat_back_addr":"172.21.15.122:6807/1754741655","heartbeat_front_addr":"172.21.15.122:6805/1754741655","state":["exists","up"]},{"osd":3,"uuid":"28ebb4a0-92e7-4f57-82a5-b9b761eca269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6811","nonce":3523190079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6813","nonce":3523190079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6817","nonce":3523190079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6815","nonce":3523190079}]},"public_addr":"172.21.15.38:6811/3523190079","cluster_addr":"172.21.15.38:6813/3523190079","heartbeat_back_addr":"172.21.15.38:6817/3523190079","heartbeat_front_addr":"172.21.15.38:6815/3523190079","state":["exists","up"]},{"osd":4,"uuid":"5ea363f7-7ad8-45ed-b7e4-4b010fe7f8cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6809","nonce":1469927628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6811","nonce":1469927628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6815","nonce":1469927628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6813","nonce":1469927628}]},"public_addr":"172.21.15.73:6809/1469927628","cluster_addr":"172.21.15.73:6811/1469927628","heartbeat_back_addr":"172.21.15.73:6815/1469927628","heartbeat_front_addr":"172.21.15.73:6813/1469927628","state":["exists","up"]},{"osd":5,"uuid":"e3c66d43-dd28-4bb1-a886-62a869be82cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6809","nonce":2954413058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6811","nonce":2954413058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6815","nonce":2954413058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6813","nonce":2954413058}]},"public_addr":"172.21.15.122:6809/2954413058","cluster_addr":"172.21.15.122:6811/2954413058","heartbeat_back_addr":"172.21.15.122:6815/2954413058","heartbeat_front_addr":"172.21.15.122:6813/2954413058","state":["exists","up"]},{"osd":6,"uuid":"211497ac-7c48-4095-bfdb-1d4df0149f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6819","nonce":1415719550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6821","nonce":1415719550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6825","nonce":1415719550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6823","nonce":1415719550}]},"public_addr":"172.21.15.38:6819/1415719550","cluster_addr":"172.21.15.38:6821/1415719550","heartbeat_back_addr":"172.21.15.38:6825/1415719550","heartbeat_front_addr":"172.21.15.38:6823/1415719550","state":["exists","up"]},{"osd":7,"uuid":"06512315-3f5e-44f8-976d-d3f5f1ba00e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6817","nonce":2634361473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6819","nonce":2634361473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6823","nonce":2634361473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6821","nonce":2634361473}]},"public_addr":"172.21.15.73:6817/2634361473","cluster_addr":"172.21.15.73:6819/2634361473","heartbeat_back_addr":"172.21.15.73:6823/2634361473","heartbeat_front_addr":"172.21.15.73:6821/2634361473","state":["exists","up"]},{"osd":8,"uuid":"db703c0a-1f87-46f0-be5f-f826f7f9daac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6817","nonce":1800567633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6819","nonce":1800567633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6823","nonce":1800567633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6821","nonce":1800567633}]},"public_addr":"172.21.15.122:6817/1800567633","cluster_addr":"172.21.15.122:6819/1800567633","heartbeat_back_addr":"172.21.15.122:6823/1800567633","heartbeat_front_addr":"172.21.15.122:6821/1800567633","state":["exists","up"]},{"osd":9,"uuid":"f5384200-b3b4-4936-85e1-fc57a9fe2bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6827","nonce":445727833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6829","nonce":445727833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6833","nonce":445727833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6831","nonce":445727833}]},"public_addr":"172.21.15.38:6827/445727833","cluster_addr":"172.21.15.38:6829/445727833","heartbeat_back_addr":"172.21.15.38:6833/445727833","heartbeat_front_addr":"172.21.15.38:6831/445727833","state":["exists","up"]},{"osd":10,"uuid":"c22883d9-6672-4796-a961-963d21eb37fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6825","nonce":43297139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6827","nonce":43297139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6831","nonce":43297139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6829","nonce":43297139}]},"public_addr":"172.21.15.73:6825/43297139","cluster_addr":"172.21.15.73:6827/43297139","heartbeat_back_addr":"172.21.15.73:6831/43297139","heartbeat_front_addr":"172.21.15.73:6829/43297139","state":["exists","up"]},{"osd":11,"uuid":"9f98e8cb-1bff-4b8b-aa19-56b691387383","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6825","nonce":1886446061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6827","nonce":1886446061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6831","nonce":1886446061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6829","nonce":1886446061}]},"public_addr":"172.21.15.122:6825/1886446061","cluster_addr":"172.21.15.122:6827/1886446061","heartbeat_back_addr":"172.21.15.122:6831/1886446061","heartbeat_front_addr":"172.21.15.122:6829/1886446061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:47.384012+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:12.834488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:38.770055+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:04.800466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:30.808076+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:56.646967+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:27:22.721270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:27:48.510011+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:28:13.922944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:28:40.514947+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:29:06.031052+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:29:31.882062+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6801/1779324916":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/924150447":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/141237849":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/2302081750":"2024-09-17T00:22:56.844212+0000","172.21.15.38:0/2245468594":"2024-09-17T00:22:56.844212+0000","172.21.15.38:6801/583006373":"2024-09-17T00:22:56.844212+0000","172.21.15.38:0/2435492667":"2024-09-17T00:23:21.417483+0000","172.21.15.38:6800/583006373":"2024-09-17T00:22:56.844212+0000","172.21.15.38:6800/1779324916":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/1103516837":"2024-09-17T00:22:56.844212+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:35:20.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.100+0000 7fa99745c640 1 -- 172.21.15.38:0/1258454467 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fa97c0a3250 msgr2=0x7fa97c0a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:20.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.100+0000 7fa99745c640 1 --2- 172.21.15.38:0/1258454467 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fa97c0a3250 0x7fa97c0a5710 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fa980005fd0 tx=0x7fa980005850 comp rx=0 tx=0).stop 2024-09-16T00:35:20.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.100+0000 7fa99745c640 1 -- 172.21.15.38:0/1258454467 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa990101b30 msgr2=0x7fa990075fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:20.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.100+0000 7fa99745c640 1 --2- 172.21.15.38:0/1258454467 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa990101b30 0x7fa990075fa0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa98c00d950 tx=0x7fa98c00de10 comp rx=0 tx=0).stop 2024-09-16T00:35:20.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.100+0000 7fa99745c640 1 -- 172.21.15.38:0/1258454467 shutdown_connections 2024-09-16T00:35:20.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.100+0000 7fa99745c640 1 --2- 172.21.15.38:0/1258454467 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fa97c0a3250 0x7fa97c0a5710 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:20.102 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.100+0000 7fa99745c640 1 --2- 172.21.15.38:0/1258454467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9901036e0 0x7fa990074b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:20.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.100+0000 7fa99745c640 1 --2- 172.21.15.38:0/1258454467 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa990102d20 0x7fa9900745e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:20.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.100+0000 7fa99745c640 1 --2- 172.21.15.38:0/1258454467 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa990101b30 0x7fa990075fa0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:20.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.101+0000 7fa99745c640 1 -- 172.21.15.38:0/1258454467 >> 172.21.15.38:0/1258454467 conn(0x7fa9900fd2b0 msgr2=0x7fa990106020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:20.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.101+0000 7fa99745c640 1 -- 172.21.15.38:0/1258454467 shutdown_connections 2024-09-16T00:35:20.103 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.101+0000 7fa99745c640 1 -- 172.21.15.38:0/1258454467 wait complete. 2024-09-16T00:35:20.114 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-16T00:25:41.931977+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-16T00:30:22.741889+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-16T00:30:25.042590+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-09-16T00:30:31.554350+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-09-16T00:35:20.114 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-16T00:35:20.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.220+0000 7f30d96a4640 1 -- 172.21.15.38:0/2789936104 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30d4102640 msgr2=0x7f30d4104a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:20.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.220+0000 7f30d96a4640 1 --2- 172.21.15.38:0/2789936104 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30d4102640 0x7f30d4104a30 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f30c4009900 tx=0x7f30c402a410 comp rx=0 tx=0).stop 2024-09-16T00:35:20.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.220+0000 7f30d96a4640 1 -- 172.21.15.38:0/2789936104 shutdown_connections 2024-09-16T00:35:20.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.220+0000 7f30d96a4640 1 --2- 172.21.15.38:0/2789936104 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30d41078a0 0x7f30d4109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:20.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.220+0000 7f30d96a4640 1 --2- 172.21.15.38:0/2789936104 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30d4104f70 0x7f30d4107360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:20.222 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.220+0000 7f30d96a4640 1 --2- 172.21.15.38:0/2789936104 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30d4102640 0x7f30d4104a30 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:20.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.220+0000 7f30d96a4640 1 -- 172.21.15.38:0/2789936104 >> 172.21.15.38:0/2789936104 conn(0x7f30d40fc140 msgr2=0x7f30d40fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:20.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.220+0000 7f30d96a4640 1 -- 172.21.15.38:0/2789936104 shutdown_connections 2024-09-16T00:35:20.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.220+0000 7f30d96a4640 1 -- 172.21.15.38:0/2789936104 wait complete. 2024-09-16T00:35:20.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.221+0000 7f30d96a4640 1 Processor -- start 2024-09-16T00:35:20.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.221+0000 7f30d96a4640 1 -- start start 2024-09-16T00:35:20.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.221+0000 7f30d96a4640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30d4102640 0x7f30d4197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:20.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.221+0000 7f30d96a4640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30d4104f70 0x7f30d4198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:20.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.221+0000 7f30d96a4640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30d41078a0 0x7f30d419f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:20.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.221+0000 7f30d96a4640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f30d410d7b0 con 0x7f30d4104f70 2024-09-16T00:35:20.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.221+0000 7f30d96a4640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f30d410d630 con 0x7f30d4102640 2024-09-16T00:35:20.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.221+0000 7f30d96a4640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f30d410d930 con 0x7f30d41078a0 2024-09-16T00:35:20.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.222+0000 7f30d27fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30d4104f70 0x7f30d4198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:20.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.222+0000 7f30d27fc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30d4104f70 0x7f30d4198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42846/0 (socket says 172.21.15.38:42846) 2024-09-16T00:35:20.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.222+0000 7f30d27fc640 1 -- 172.21.15.38:0/548177607 learned_addr learned my addr 172.21.15.38:0/548177607 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:35:20.225 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.222+0000 7f30d37fe640 1 --2- 172.21.15.38:0/548177607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30d41078a0 0x7f30d419f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:20.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.222+0000 7f30d2ffd640 1 --2- 172.21.15.38:0/548177607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30d4102640 0x7f30d4197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:20.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.222+0000 7f30d27fc640 1 -- 172.21.15.38:0/548177607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30d41078a0 msgr2=0x7f30d419f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:20.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.222+0000 7f30d27fc640 1 --2- 172.21.15.38:0/548177607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30d41078a0 0x7f30d419f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:20.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.222+0000 7f30d27fc640 1 -- 172.21.15.38:0/548177607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30d4102640 msgr2=0x7f30d4197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:20.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.222+0000 7f30d27fc640 1 --2- 172.21.15.38:0/548177607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30d4102640 0x7f30d4197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:20.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.222+0000 7f30d27fc640 1 -- 172.21.15.38:0/548177607 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30d419fa50 con 0x7f30d4104f70 2024-09-16T00:35:20.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.222+0000 7f30d27fc640 1 --2- 172.21.15.38:0/548177607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30d4104f70 0x7f30d4198370 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f30c800c970 tx=0x7f30c800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:20.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.222+0000 7f30bbfff640 1 -- 172.21.15.38:0/548177607 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c8014070 con 0x7f30d4104f70 2024-09-16T00:35:20.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.222+0000 7f30bbfff640 1 -- 172.21.15.38:0/548177607 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f30c8004440 con 0x7f30d4104f70 2024-09-16T00:35:20.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.222+0000 7f30d96a4640 1 -- 172.21.15.38:0/548177607 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30d419fd40 con 0x7f30d4104f70 2024-09-16T00:35:20.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.222+0000 7f30bbfff640 1 -- 172.21.15.38:0/548177607 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c800f450 con 0x7f30d4104f70 2024-09-16T00:35:20.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.222+0000 7f30d96a4640 1 -- 172.21.15.38:0/548177607 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30d41a0230 con 0x7f30d4104f70 2024-09-16T00:35:20.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.223+0000 7f30d96a4640 1 -- 172.21.15.38:0/548177607 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30d40fdc30 con 0x7f30d4104f70 2024-09-16T00:35:20.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.224+0000 7f30bbfff640 1 -- 172.21.15.38:0/548177607 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f30c8017070 con 0x7f30d4104f70 2024-09-16T00:35:20.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.225+0000 7f30bbfff640 1 --2- 172.21.15.38:0/548177607 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f30bc0a3250 0x7f30bc0a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:20.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.225+0000 7f30d2ffd640 1 --2- 172.21.15.38:0/548177607 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f30bc0a3250 0x7f30bc0a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:20.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.225+0000 7f30bbfff640 1 -- 172.21.15.38:0/548177607 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f30c80cfb40 con 0x7f30d4104f70 2024-09-16T00:35:20.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.226+0000 7f30d2ffd640 1 --2- 172.21.15.38:0/548177607 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f30bc0a3250 0x7f30bc0a5710 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f30c4000c00 tx=0x7f30c40315d0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:20.230 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.229+0000 7f30bbfff640 1 -- 172.21.15.38:0/548177607 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f30c801e020 con 0x7f30d4104f70 2024-09-16T00:35:20.409 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.408+0000 7f30d96a4640 1 -- 172.21.15.38:0/548177607 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f30d4062e90 con 0x7f30d4104f70 2024-09-16T00:35:20.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.408+0000 7f30bbfff640 1 -- 172.21.15.38:0/548177607 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v92) ==== 93+0+10 (secure 0 0 0) 0x7f30c809c480 con 0x7f30d4104f70 2024-09-16T00:35:20.410 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 1 2024-09-16T00:35:20.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.410+0000 7f30d96a4640 1 -- 172.21.15.38:0/548177607 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f30bc0a3250 msgr2=0x7f30bc0a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:20.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.410+0000 7f30d96a4640 1 --2- 172.21.15.38:0/548177607 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f30bc0a3250 0x7f30bc0a5710 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f30c4000c00 tx=0x7f30c40315d0 comp rx=0 tx=0).stop 2024-09-16T00:35:20.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.410+0000 7f30d96a4640 1 -- 172.21.15.38:0/548177607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30d4104f70 msgr2=0x7f30d4198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:20.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.410+0000 7f30d96a4640 1 --2- 172.21.15.38:0/548177607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30d4104f70 0x7f30d4198370 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f30c800c970 tx=0x7f30c800ce30 comp rx=0 tx=0).stop 2024-09-16T00:35:20.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.410+0000 7f30d96a4640 1 -- 172.21.15.38:0/548177607 shutdown_connections 2024-09-16T00:35:20.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.410+0000 7f30d96a4640 1 --2- 172.21.15.38:0/548177607 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f30bc0a3250 0x7f30bc0a5710 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:20.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.410+0000 7f30d96a4640 1 --2- 172.21.15.38:0/548177607 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f30d41078a0 0x7f30d419f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:20.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.410+0000 7f30d96a4640 1 --2- 172.21.15.38:0/548177607 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f30d4104f70 0x7f30d4198370 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:20.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.410+0000 7f30d96a4640 1 --2- 172.21.15.38:0/548177607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30d4102640 0x7f30d4197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:20.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.410+0000 7f30d96a4640 1 -- 172.21.15.38:0/548177607 >> 172.21.15.38:0/548177607 conn(0x7f30d40fc140 msgr2=0x7f30d4103040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:20.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.411+0000 7f30d96a4640 1 -- 172.21.15.38:0/548177607 shutdown_connections 2024-09-16T00:35:20.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.411+0000 7f30d96a4640 1 -- 172.21.15.38:0/548177607 wait complete. 2024-09-16T00:35:20.424 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-16T00:35:20.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.532+0000 7f1a74c23640 1 -- 172.21.15.38:0/1423512698 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a70102610 msgr2=0x7f1a70104a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:20.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.532+0000 7f1a74c23640 1 --2- 172.21.15.38:0/1423512698 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a70102610 0x7f1a70104a00 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f1a60009900 tx=0x7f1a6002a410 comp rx=0 tx=0).stop 2024-09-16T00:35:20.533 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.532+0000 7f1a74c23640 1 -- 172.21.15.38:0/1423512698 shutdown_connections 2024-09-16T00:35:20.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.532+0000 7f1a74c23640 1 --2- 172.21.15.38:0/1423512698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a70107870 0x7f1a70109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:20.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.532+0000 7f1a74c23640 1 --2- 172.21.15.38:0/1423512698 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a70104f40 0x7f1a70107330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:20.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.532+0000 7f1a74c23640 1 --2- 172.21.15.38:0/1423512698 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a70102610 0x7f1a70104a00 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:20.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.532+0000 7f1a74c23640 1 -- 172.21.15.38:0/1423512698 >> 172.21.15.38:0/1423512698 conn(0x7f1a700fc1b0 msgr2=0x7f1a700fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:20.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.532+0000 7f1a74c23640 1 -- 172.21.15.38:0/1423512698 shutdown_connections 2024-09-16T00:35:20.534 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.532+0000 7f1a74c23640 1 -- 172.21.15.38:0/1423512698 wait complete. 2024-09-16T00:35:20.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.532+0000 7f1a74c23640 1 Processor -- start 2024-09-16T00:35:20.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.532+0000 7f1a74c23640 1 -- start start 2024-09-16T00:35:20.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.533+0000 7f1a74c23640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a70102610 0x7f1a70197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:20.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.533+0000 7f1a74c23640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a70104f40 0x7f1a701984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:20.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.533+0000 7f1a74c23640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a70107870 0x7f1a7019f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:20.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.533+0000 7f1a74c23640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7010d800 con 0x7f1a70104f40 2024-09-16T00:35:20.536 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.533+0000 7f1a74c23640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7010d680 con 0x7f1a70107870 2024-09-16T00:35:20.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.533+0000 7f1a74c23640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f1a7010d980 con 0x7f1a70102610 2024-09-16T00:35:20.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.533+0000 7f1a6dd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a70104f40 0x7f1a701984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:20.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.533+0000 7f1a6dd9b640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a70104f40 0x7f1a701984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42868/0 (socket says 172.21.15.38:42868) 2024-09-16T00:35:20.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.533+0000 7f1a6dd9b640 1 -- 172.21.15.38:0/3337608494 learned_addr learned my addr 172.21.15.38:0/3337608494 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:35:20.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.533+0000 7f1a6e59c640 1 --2- 172.21.15.38:0/3337608494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a70102610 0x7f1a70197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:20.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.533+0000 7f1a6ed9d640 1 --2- 172.21.15.38:0/3337608494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a70107870 0x7f1a7019f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:20.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.533+0000 7f1a6dd9b640 1 -- 172.21.15.38:0/3337608494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a70102610 msgr2=0x7f1a70197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:20.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.533+0000 7f1a6dd9b640 1 --2- 172.21.15.38:0/3337608494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a70102610 0x7f1a70197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:20.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.533+0000 7f1a6dd9b640 1 -- 172.21.15.38:0/3337608494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a70107870 msgr2=0x7f1a7019f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:20.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.533+0000 7f1a6dd9b640 1 --2- 172.21.15.38:0/3337608494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a70107870 0x7f1a7019f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:20.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.533+0000 7f1a6dd9b640 1 -- 172.21.15.38:0/3337608494 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a7019fba0 con 0x7f1a70104f40 2024-09-16T00:35:20.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.533+0000 7f1a6dd9b640 1 --2- 172.21.15.38:0/3337608494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a70104f40 0x7f1a701984c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f1a64005fd0 tx=0x7f1a6400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:20.537 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.533+0000 7f1a5f7fe640 1 -- 172.21.15.38:0/3337608494 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a64017070 con 0x7f1a70104f40 2024-09-16T00:35:20.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.533+0000 7f1a5f7fe640 1 -- 172.21.15.38:0/3337608494 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f1a64009c30 con 0x7f1a70104f40 2024-09-16T00:35:20.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.533+0000 7f1a74c23640 1 -- 172.21.15.38:0/3337608494 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a7019fe90 con 0x7f1a70104f40 2024-09-16T00:35:20.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.533+0000 7f1a5f7fe640 1 -- 172.21.15.38:0/3337608494 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a64010460 con 0x7f1a70104f40 2024-09-16T00:35:20.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.534+0000 7f1a74c23640 1 -- 172.21.15.38:0/3337608494 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a701a0350 con 0x7f1a70104f40 2024-09-16T00:35:20.538 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.535+0000 7f1a74c23640 1 -- 172.21.15.38:0/3337608494 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a700fdca0 con 0x7f1a70104f40 2024-09-16T00:35:20.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.536+0000 7f1a5f7fe640 1 -- 172.21.15.38:0/3337608494 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f1a64003e70 con 0x7f1a70104f40 2024-09-16T00:35:20.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.536+0000 7f1a5f7fe640 1 --2- 172.21.15.38:0/3337608494 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f1a580a3250 0x7f1a580a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:20.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.537+0000 7f1a6e59c640 1 --2- 172.21.15.38:0/3337608494 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f1a580a3250 0x7f1a580a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:20.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.537+0000 7f1a5f7fe640 1 -- 172.21.15.38:0/3337608494 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f1a640d0090 con 0x7f1a70104f40 2024-09-16T00:35:20.539 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.537+0000 7f1a6e59c640 1 --2- 172.21.15.38:0/3337608494 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f1a580a3250 0x7f1a580a5710 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1a60000c00 tx=0x7f1a600315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:20.541 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.540+0000 7f1a5f7fe640 1 -- 172.21.15.38:0/3337608494 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f1a64020030 con 0x7f1a70104f40 2024-09-16T00:35:20.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:20 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/1258454467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:35:20.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:20 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/548177607' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T00:35:20.717 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.716+0000 7f1a74c23640 1 -- 172.21.15.38:0/3337608494 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1a7002da10 con 0x7f1a70104f40 2024-09-16T00:35:20.718 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.717+0000 7f1a5f7fe640 1 -- 172.21.15.38:0/3337608494 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v92) ==== 104+0+11 (secure 0 0 0) 0x7f1a6409c9d0 con 0x7f1a70104f40 2024-09-16T00:35:20.718 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-09-16T00:35:20.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.719+0000 7f1a74c23640 1 -- 172.21.15.38:0/3337608494 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f1a580a3250 msgr2=0x7f1a580a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:20.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.719+0000 7f1a74c23640 1 --2- 172.21.15.38:0/3337608494 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f1a580a3250 0x7f1a580a5710 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f1a60000c00 tx=0x7f1a600315b0 comp rx=0 tx=0).stop 2024-09-16T00:35:20.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.719+0000 7f1a74c23640 1 -- 172.21.15.38:0/3337608494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a70104f40 msgr2=0x7f1a701984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:20.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.719+0000 7f1a74c23640 1 --2- 172.21.15.38:0/3337608494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a70104f40 0x7f1a701984c0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f1a64005fd0 tx=0x7f1a6400dde0 comp rx=0 tx=0).stop 2024-09-16T00:35:20.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.719+0000 7f1a74c23640 1 -- 172.21.15.38:0/3337608494 shutdown_connections 2024-09-16T00:35:20.721 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.719+0000 7f1a74c23640 1 --2- 172.21.15.38:0/3337608494 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f1a580a3250 0x7f1a580a5710 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:20.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.719+0000 7f1a74c23640 1 --2- 172.21.15.38:0/3337608494 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a70107870 0x7f1a7019f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:20.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.719+0000 7f1a74c23640 1 --2- 172.21.15.38:0/3337608494 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f1a70104f40 0x7f1a701984c0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:20.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.719+0000 7f1a74c23640 1 --2- 172.21.15.38:0/3337608494 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f1a70102610 0x7f1a70197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:20.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.719+0000 7f1a74c23640 1 -- 172.21.15.38:0/3337608494 >> 172.21.15.38:0/3337608494 conn(0x7f1a700fc1b0 msgr2=0x7f1a70103070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:20.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.719+0000 7f1a74c23640 1 -- 172.21.15.38:0/3337608494 shutdown_connections 2024-09-16T00:35:20.722 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.719+0000 7f1a74c23640 1 -- 172.21.15.38:0/3337608494 wait complete. 2024-09-16T00:35:20.731 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-16T00:35:20.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:20 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/1258454467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:35:20.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:20 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/548177607' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T00:35:20.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.836+0000 7f8dcfc27640 1 -- 172.21.15.38:0/3989559782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8dc8101c20 msgr2=0x7f8dc81020a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:20.839 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.836+0000 7f8dcfc27640 1 --2- 172.21.15.38:0/3989559782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8dc8101c20 0x7f8dc81020a0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f8db8009900 tx=0x7f8db802a410 comp rx=0 tx=0).stop 2024-09-16T00:35:20.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.836+0000 7f8dcfc27640 1 -- 172.21.15.38:0/3989559782 shutdown_connections 2024-09-16T00:35:20.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.836+0000 7f8dcfc27640 1 --2- 172.21.15.38:0/3989559782 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8dc81025e0 0x7f8dc8108e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:20.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.836+0000 7f8dcfc27640 1 --2- 172.21.15.38:0/3989559782 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8dc8101c20 0x7f8dc81020a0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:20.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.836+0000 7f8dcfc27640 1 --2- 172.21.15.38:0/3989559782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8dc8100a30 0x7f8dc8100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:20.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.836+0000 7f8dcfc27640 1 -- 172.21.15.38:0/3989559782 >> 172.21.15.38:0/3989559782 conn(0x7f8dc80fc1b0 msgr2=0x7f8dc80fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:20.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.836+0000 7f8dcfc27640 1 -- 172.21.15.38:0/3989559782 shutdown_connections 2024-09-16T00:35:20.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.836+0000 7f8dcfc27640 1 -- 172.21.15.38:0/3989559782 wait complete. 2024-09-16T00:35:20.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.837+0000 7f8dcfc27640 1 Processor -- start 2024-09-16T00:35:20.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.837+0000 7f8dcfc27640 1 -- start start 2024-09-16T00:35:20.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.837+0000 7f8dcfc27640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8dc8100a30 0x7f8dc81049c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:20.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.837+0000 7f8dcfc27640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8dc8101c20 0x7f8dc8103000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:20.840 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.837+0000 7f8dcfc27640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8dc81025e0 0x7f8dc8103540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:20.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.837+0000 7f8dcfc27640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc810c6b0 con 0x7f8dc8100a30 2024-09-16T00:35:20.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.837+0000 7f8dcfc27640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc810c530 con 0x7f8dc81025e0 2024-09-16T00:35:20.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.837+0000 7f8dcfc27640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc810c830 con 0x7f8dc8101c20 2024-09-16T00:35:20.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.837+0000 7f8dcd9c3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8dc8100a30 0x7f8dc81049c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:20.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.837+0000 7f8dcd9c3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8dc8100a30 0x7f8dc81049c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42886/0 (socket says 172.21.15.38:42886) 2024-09-16T00:35:20.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.837+0000 7f8dcd9c3640 1 -- 172.21.15.38:0/4184445693 learned_addr learned my addr 172.21.15.38:0/4184445693 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:35:20.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.837+0000 7f8dcd1c2640 1 --2- 172.21.15.38:0/4184445693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8dc8101c20 0x7f8dc8103000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:20.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.837+0000 7f8dcd9c3640 1 -- 172.21.15.38:0/4184445693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8dc8101c20 msgr2=0x7f8dc8103000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:20.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.837+0000 7f8dcd9c3640 1 --2- 172.21.15.38:0/4184445693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8dc8101c20 0x7f8dc8103000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:20.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.837+0000 7f8dcd9c3640 1 -- 172.21.15.38:0/4184445693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8dc81025e0 msgr2=0x7f8dc8103540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:20.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.837+0000 7f8dcd9c3640 1 --2- 172.21.15.38:0/4184445693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8dc81025e0 0x7f8dc8103540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:20.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.837+0000 7f8dcd9c3640 1 -- 172.21.15.38:0/4184445693 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dc8103dd0 con 0x7f8dc8100a30 2024-09-16T00:35:20.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.838+0000 7f8dcd9c3640 1 --2- 172.21.15.38:0/4184445693 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8dc8100a30 0x7f8dc81049c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f8dc400d770 tx=0x7f8dc400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:20.841 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.838+0000 7f8dbeffd640 1 -- 172.21.15.38:0/4184445693 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4017070 con 0x7f8dc8100a30 2024-09-16T00:35:20.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.838+0000 7f8dcfc27640 1 -- 172.21.15.38:0/4184445693 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dc81a1fe0 con 0x7f8dc8100a30 2024-09-16T00:35:20.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.838+0000 7f8dbeffd640 1 -- 172.21.15.38:0/4184445693 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f8dc4009c30 con 0x7f8dc8100a30 2024-09-16T00:35:20.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.838+0000 7f8dbeffd640 1 -- 172.21.15.38:0/4184445693 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4003dd0 con 0x7f8dc8100a30 2024-09-16T00:35:20.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.838+0000 7f8dcfc27640 1 -- 172.21.15.38:0/4184445693 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dc81a24a0 con 0x7f8dc8100a30 2024-09-16T00:35:20.842 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.839+0000 7f8dcfc27640 1 -- 172.21.15.38:0/4184445693 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8dc81a2790 con 0x7f8dc8100a30 2024-09-16T00:35:20.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.840+0000 7f8dbeffd640 1 -- 172.21.15.38:0/4184445693 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f8dc4004000 con 0x7f8dc8100a30 2024-09-16T00:35:20.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.840+0000 7f8dbeffd640 1 --2- 172.21.15.38:0/4184445693 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f8db40a32d0 0x7f8db40a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:20.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.841+0000 7f8dbeffd640 1 -- 172.21.15.38:0/4184445693 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f8dc40cf580 con 0x7f8dc8100a30 2024-09-16T00:35:20.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.841+0000 7f8dcd1c2640 1 --2- 172.21.15.38:0/4184445693 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f8db40a32d0 0x7f8db40a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:20.843 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.841+0000 7f8dcd1c2640 1 --2- 172.21.15.38:0/4184445693 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f8db40a32d0 0x7f8db40a5790 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8104730 tx=0x7f8db80315b0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:20.846 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:20.845+0000 7f8dbeffd640 1 -- 172.21.15.38:0/4184445693 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f8dc409bf20 con 0x7f8dc8100a30 2024-09-16T00:35:20.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:20 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/1258454467' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:35:20.967 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:20 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/548177607' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-16T00:35:21.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.020+0000 7f8dcfc27640 1 -- 172.21.15.38:0/4184445693 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f8dc81a2a00 con 0x7f8dc8100a30 2024-09-16T00:35:21.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.020+0000 7f8dbeffd640 1 -- 172.21.15.38:0/4184445693 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v92) ==== 100+0+11 (secure 0 0 0) 0x7f8dc4009de0 con 0x7f8dc8100a30 2024-09-16T00:35:21.021 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 64 2024-09-16T00:35:21.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.022+0000 7f8dcfc27640 1 -- 172.21.15.38:0/4184445693 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f8db40a32d0 msgr2=0x7f8db40a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:21.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.022+0000 7f8dcfc27640 1 --2- 172.21.15.38:0/4184445693 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f8db40a32d0 0x7f8db40a5790 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8104730 tx=0x7f8db80315b0 comp rx=0 tx=0).stop 2024-09-16T00:35:21.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.022+0000 7f8dcfc27640 1 -- 172.21.15.38:0/4184445693 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8dc8100a30 msgr2=0x7f8dc81049c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:21.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.022+0000 7f8dcfc27640 1 --2- 172.21.15.38:0/4184445693 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8dc8100a30 0x7f8dc81049c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f8dc400d770 tx=0x7f8dc400dc30 comp rx=0 tx=0).stop 2024-09-16T00:35:21.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.022+0000 7f8dcfc27640 1 -- 172.21.15.38:0/4184445693 shutdown_connections 2024-09-16T00:35:21.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.022+0000 7f8dcfc27640 1 --2- 172.21.15.38:0/4184445693 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f8db40a32d0 0x7f8db40a5790 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:21.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.022+0000 7f8dcfc27640 1 --2- 172.21.15.38:0/4184445693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8dc81025e0 0x7f8dc8103540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:21.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.022+0000 7f8dcfc27640 1 --2- 172.21.15.38:0/4184445693 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f8dc8101c20 0x7f8dc8103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:21.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.022+0000 7f8dcfc27640 1 --2- 172.21.15.38:0/4184445693 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f8dc8100a30 0x7f8dc81049c0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:21.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.022+0000 7f8dcfc27640 1 -- 172.21.15.38:0/4184445693 >> 172.21.15.38:0/4184445693 conn(0x7f8dc80fc1b0 msgr2=0x7f8dc806a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:21.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.022+0000 7f8dcfc27640 1 -- 172.21.15.38:0/4184445693 shutdown_connections 2024-09-16T00:35:21.024 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.022+0000 7f8dcfc27640 1 -- 172.21.15.38:0/4184445693 wait complete. 2024-09-16T00:35:21.034 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-09-16T00:35:21.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.140+0000 7f15fb1cd640 1 -- 172.21.15.38:0/1396055589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15f4102580 msgr2=0x7f15f4104970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:21.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.140+0000 7f15fb1cd640 1 --2- 172.21.15.38:0/1396055589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15f4102580 0x7f15f4104970 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f15ec009900 tx=0x7f15ec02a410 comp rx=0 tx=0).stop 2024-09-16T00:35:21.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.140+0000 7f15fb1cd640 1 -- 172.21.15.38:0/1396055589 shutdown_connections 2024-09-16T00:35:21.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.140+0000 7f15fb1cd640 1 --2- 172.21.15.38:0/1396055589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15f41077e0 0x7f15f4109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:21.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.140+0000 7f15fb1cd640 1 --2- 172.21.15.38:0/1396055589 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15f4104eb0 0x7f15f41072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:21.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.140+0000 7f15fb1cd640 1 --2- 172.21.15.38:0/1396055589 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15f4102580 0x7f15f4104970 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:21.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.140+0000 7f15fb1cd640 1 -- 172.21.15.38:0/1396055589 >> 172.21.15.38:0/1396055589 conn(0x7f15f40fc140 msgr2=0x7f15f40fe5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:21.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.140+0000 7f15fb1cd640 1 -- 172.21.15.38:0/1396055589 shutdown_connections 2024-09-16T00:35:21.143 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.140+0000 7f15fb1cd640 1 -- 172.21.15.38:0/1396055589 wait complete. 2024-09-16T00:35:21.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.140+0000 7f15fb1cd640 1 Processor -- start 2024-09-16T00:35:21.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.140+0000 7f15fb1cd640 1 -- start start 2024-09-16T00:35:21.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.140+0000 7f15fb1cd640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15f4102580 0x7f15f4197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:21.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.140+0000 7f15fb1cd640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15f4104eb0 0x7f15f4198430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:21.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.140+0000 7f15fb1cd640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15f41077e0 0x7f15f419f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:21.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.140+0000 7f15fb1cd640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f15f410d7e0 con 0x7f15f41077e0 2024-09-16T00:35:21.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.140+0000 7f15fb1cd640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f15f410d660 con 0x7f15f4102580 2024-09-16T00:35:21.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.140+0000 7f15fb1cd640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f15f410d960 con 0x7f15f4104eb0 2024-09-16T00:35:21.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.141+0000 7f15f976a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15f41077e0 0x7f15f419f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:21.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.141+0000 7f15f976a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15f41077e0 0x7f15f419f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42900/0 (socket says 172.21.15.38:42900) 2024-09-16T00:35:21.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.141+0000 7f15f976a640 1 -- 172.21.15.38:0/3949833680 learned_addr learned my addr 172.21.15.38:0/3949833680 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:35:21.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.141+0000 7f15f8f69640 1 --2- 172.21.15.38:0/3949833680 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15f4102580 0x7f15f4197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:21.144 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.141+0000 7f15f3fff640 1 --2- 172.21.15.38:0/3949833680 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15f4104eb0 0x7f15f4198430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:21.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.141+0000 7f15f976a640 1 -- 172.21.15.38:0/3949833680 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15f4104eb0 msgr2=0x7f15f4198430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:21.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.141+0000 7f15f976a640 1 --2- 172.21.15.38:0/3949833680 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15f4104eb0 0x7f15f4198430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:21.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.141+0000 7f15f976a640 1 -- 172.21.15.38:0/3949833680 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15f4102580 msgr2=0x7f15f4197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:21.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.141+0000 7f15f976a640 1 --2- 172.21.15.38:0/3949833680 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15f4102580 0x7f15f4197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:21.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.141+0000 7f15f976a640 1 -- 172.21.15.38:0/3949833680 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f419fb10 con 0x7f15f41077e0 2024-09-16T00:35:21.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.141+0000 7f15f976a640 1 --2- 172.21.15.38:0/3949833680 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15f41077e0 0x7f15f419f4a0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f15dc00d950 tx=0x7f15dc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:21.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.141+0000 7f15f1ffb640 1 -- 172.21.15.38:0/3949833680 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc017070 con 0x7f15f41077e0 2024-09-16T00:35:21.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.141+0000 7f15fb1cd640 1 -- 172.21.15.38:0/3949833680 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15f419fe00 con 0x7f15f41077e0 2024-09-16T00:35:21.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.141+0000 7f15f1ffb640 1 -- 172.21.15.38:0/3949833680 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f15dc009c30 con 0x7f15f41077e0 2024-09-16T00:35:21.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.141+0000 7f15f1ffb640 1 -- 172.21.15.38:0/3949833680 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc003d60 con 0x7f15f41077e0 2024-09-16T00:35:21.145 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.142+0000 7f15fb1cd640 1 -- 172.21.15.38:0/3949833680 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15f41a0270 con 0x7f15f41077e0 2024-09-16T00:35:21.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.143+0000 7f15fb1cd640 1 -- 172.21.15.38:0/3949833680 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15f40fdc30 con 0x7f15f41077e0 2024-09-16T00:35:21.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.144+0000 7f15f1ffb640 1 -- 172.21.15.38:0/3949833680 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f15dc003f00 con 0x7f15f41077e0 2024-09-16T00:35:21.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.144+0000 7f15f1ffb640 1 --2- 172.21.15.38:0/3949833680 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f15e00a3250 0x7f15e00a5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:21.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.144+0000 7f15f8f69640 1 --2- 172.21.15.38:0/3949833680 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f15e00a3250 0x7f15e00a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:21.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.144+0000 7f15f1ffb640 1 -- 172.21.15.38:0/3949833680 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f15dc0cf4f0 con 0x7f15f41077e0 2024-09-16T00:35:21.146 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.145+0000 7f15f8f69640 1 --2- 172.21.15.38:0/3949833680 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f15e00a3250 0x7f15e00a5710 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f15ec0314e0 tx=0x7f15ec000bc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:21.150 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.149+0000 7f15f1ffb640 1 -- 172.21.15.38:0/3949833680 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f15dc09b840 con 0x7f15f41077e0 2024-09-16T00:35:21.329 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.328+0000 7f15fb1cd640 1 -- 172.21.15.38:0/3949833680 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f15f4062e90 con 0x7f15f41077e0 2024-09-16T00:35:21.330 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.328+0000 7f15f1ffb640 1 -- 172.21.15.38:0/3949833680 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v92) ==== 103+0+11 (secure 0 0 0) 0x7f15dc09fb40 con 0x7f15f41077e0 2024-09-16T00:35:21.330 INFO:teuthology.orchestra.run.smithi038.stdout:pg_num: 32 2024-09-16T00:35:21.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.330+0000 7f15fb1cd640 1 -- 172.21.15.38:0/3949833680 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f15e00a3250 msgr2=0x7f15e00a5710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:21.331 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.330+0000 7f15fb1cd640 1 --2- 172.21.15.38:0/3949833680 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f15e00a3250 0x7f15e00a5710 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f15ec0314e0 tx=0x7f15ec000bc0 comp rx=0 tx=0).stop 2024-09-16T00:35:21.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.330+0000 7f15fb1cd640 1 -- 172.21.15.38:0/3949833680 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15f41077e0 msgr2=0x7f15f419f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:21.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.330+0000 7f15fb1cd640 1 --2- 172.21.15.38:0/3949833680 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15f41077e0 0x7f15f419f4a0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f15dc00d950 tx=0x7f15dc00de10 comp rx=0 tx=0).stop 2024-09-16T00:35:21.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.330+0000 7f15fb1cd640 1 -- 172.21.15.38:0/3949833680 shutdown_connections 2024-09-16T00:35:21.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.330+0000 7f15fb1cd640 1 --2- 172.21.15.38:0/3949833680 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f15e00a3250 0x7f15e00a5710 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:21.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.330+0000 7f15fb1cd640 1 --2- 172.21.15.38:0/3949833680 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f15f41077e0 0x7f15f419f4a0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:21.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.330+0000 7f15fb1cd640 1 --2- 172.21.15.38:0/3949833680 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f15f4104eb0 0x7f15f4198430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:21.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.330+0000 7f15fb1cd640 1 --2- 172.21.15.38:0/3949833680 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15f4102580 0x7f15f4197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:21.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.330+0000 7f15fb1cd640 1 -- 172.21.15.38:0/3949833680 >> 172.21.15.38:0/3949833680 conn(0x7f15f40fc140 msgr2=0x7f15f4102ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:21.332 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.330+0000 7f15fb1cd640 1 -- 172.21.15.38:0/3949833680 shutdown_connections 2024-09-16T00:35:21.333 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.330+0000 7f15fb1cd640 1 -- 172.21.15.38:0/3949833680 wait complete. 2024-09-16T00:35:21.345 DEBUG:teuthology.orchestra.run.smithi038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-16T00:35:21.450 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.446+0000 7f5693cf1640 1 -- 172.21.15.38:0/46906679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f568c1077f0 msgr2=0x7f568c109c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:21.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.446+0000 7f5693cf1640 1 --2- 172.21.15.38:0/46906679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f568c1077f0 0x7f568c109c60 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f567c009ca0 tx=0x7f567c02a8c0 comp rx=0 tx=0).stop 2024-09-16T00:35:21.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.447+0000 7f5693cf1640 1 -- 172.21.15.38:0/46906679 shutdown_connections 2024-09-16T00:35:21.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.447+0000 7f5693cf1640 1 --2- 172.21.15.38:0/46906679 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f568c1077f0 0x7f568c109c60 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:21.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.447+0000 7f5693cf1640 1 --2- 172.21.15.38:0/46906679 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f568c104ec0 0x7f568c1072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:21.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.447+0000 7f5693cf1640 1 --2- 172.21.15.38:0/46906679 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f568c102590 0x7f568c104980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:21.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.447+0000 7f5693cf1640 1 -- 172.21.15.38:0/46906679 >> 172.21.15.38:0/46906679 conn(0x7f568c0fc170 msgr2=0x7f568c0fe5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:21.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.447+0000 7f5693cf1640 1 -- 172.21.15.38:0/46906679 shutdown_connections 2024-09-16T00:35:21.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.447+0000 7f5693cf1640 1 -- 172.21.15.38:0/46906679 wait complete. 2024-09-16T00:35:21.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.447+0000 7f5693cf1640 1 Processor -- start 2024-09-16T00:35:21.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.447+0000 7f5693cf1640 1 -- start start 2024-09-16T00:35:21.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.447+0000 7f5693cf1640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f568c102590 0x7f568c197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:21.451 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.447+0000 7f5693cf1640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f568c104ec0 0x7f568c1983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:21.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.447+0000 7f5693cf1640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f568c1077f0 0x7f568c19f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:21.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.447+0000 7f5693cf1640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f568c10d820 con 0x7f568c104ec0 2024-09-16T00:35:21.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.447+0000 7f5693cf1640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f568c10d6a0 con 0x7f568c102590 2024-09-16T00:35:21.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.447+0000 7f5693cf1640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f568c10d9a0 con 0x7f568c1077f0 2024-09-16T00:35:21.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.448+0000 7f569128c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f568c104ec0 0x7f568c1983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:21.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.448+0000 7f569128c640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f568c104ec0 0x7f568c1983b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:42926/0 (socket says 172.21.15.38:42926) 2024-09-16T00:35:21.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.448+0000 7f569128c640 1 -- 172.21.15.38:0/4278011359 learned_addr learned my addr 172.21.15.38:0/4278011359 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:35:21.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.448+0000 7f5691a8d640 1 --2- 172.21.15.38:0/4278011359 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f568c102590 0x7f568c197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:21.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.448+0000 7f569228e640 1 --2- 172.21.15.38:0/4278011359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f568c1077f0 0x7f568c19f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:21.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.448+0000 7f569128c640 1 -- 172.21.15.38:0/4278011359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f568c1077f0 msgr2=0x7f568c19f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:21.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.448+0000 7f569128c640 1 --2- 172.21.15.38:0/4278011359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f568c1077f0 0x7f568c19f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:21.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.448+0000 7f569128c640 1 -- 172.21.15.38:0/4278011359 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f568c102590 msgr2=0x7f568c197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:21.452 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.448+0000 7f569128c640 1 --2- 172.21.15.38:0/4278011359 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f568c102590 0x7f568c197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:21.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.448+0000 7f569128c640 1 -- 172.21.15.38:0/4278011359 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f568c19fa90 con 0x7f568c104ec0 2024-09-16T00:35:21.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.448+0000 7f569128c640 1 --2- 172.21.15.38:0/4278011359 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f568c104ec0 0x7f568c1983b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f568000d740 tx=0x7f568000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:21.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.448+0000 7f568affd640 1 -- 172.21.15.38:0/4278011359 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5680017070 con 0x7f568c104ec0 2024-09-16T00:35:21.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.448+0000 7f568affd640 1 -- 172.21.15.38:0/4278011359 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f568000de50 con 0x7f568c104ec0 2024-09-16T00:35:21.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.448+0000 7f5693cf1640 1 -- 172.21.15.38:0/4278011359 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f568c19fd80 con 0x7f568c104ec0 2024-09-16T00:35:21.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.448+0000 7f568affd640 1 -- 172.21.15.38:0/4278011359 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5680004e00 con 0x7f568c104ec0 2024-09-16T00:35:21.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.448+0000 7f5693cf1640 1 -- 172.21.15.38:0/4278011359 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f568c1a0270 con 0x7f568c104ec0 2024-09-16T00:35:21.453 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.449+0000 7f5693cf1640 1 -- 172.21.15.38:0/4278011359 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f568c0fdc60 con 0x7f568c104ec0 2024-09-16T00:35:21.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.450+0000 7f568affd640 1 -- 172.21.15.38:0/4278011359 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f5680004fa0 con 0x7f568c104ec0 2024-09-16T00:35:21.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.451+0000 7f568affd640 1 --2- 172.21.15.38:0/4278011359 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f56740a32d0 0x7f56740a5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:21.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.451+0000 7f5691a8d640 1 --2- 172.21.15.38:0/4278011359 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f56740a32d0 0x7f56740a5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:21.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.451+0000 7f568affd640 1 -- 172.21.15.38:0/4278011359 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f56800cf500 con 0x7f568c104ec0 2024-09-16T00:35:21.454 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.452+0000 7f5691a8d640 1 --2- 172.21.15.38:0/4278011359 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f56740a32d0 0x7f56740a5790 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f568400bbe0 tx=0x7f568400b390 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:21.456 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.455+0000 7f568affd640 1 -- 172.21.15.38:0/4278011359 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f568009b850 con 0x7f568c104ec0 2024-09-16T00:35:21.684 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.682+0000 7f5693cf1640 1 -- 172.21.15.38:0/4278011359 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f568c062e90 con 0x7f568c104ec0 2024-09-16T00:35:21.692 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.690+0000 7f568affd640 1 -- 172.21.15.38:0/4278011359 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v20) ==== 81+0+742341 (secure 0 0 0) 0x7f568009fb50 con 0x7f568c104ec0 2024-09-16T00:35:21.704 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:35:21.705 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-16T00:35:21.705 INFO:teuthology.orchestra.run.smithi038.stdout: "epoch": 20, 2024-09-16T00:35:21.705 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.705 INFO:teuthology.orchestra.run.smithi038.stdout: "active_gid": 14150, 2024-09-16T00:35:21.705 INFO:teuthology.orchestra.run.smithi038.stdout: "active_name": "x", 2024-09-16T00:35:21.705 INFO:teuthology.orchestra.run.smithi038.stdout: "active_addrs": { 2024-09-16T00:35:21.705 INFO:teuthology.orchestra.run.smithi038.stdout: "addrvec": [ 2024-09-16T00:35:21.705 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:21.705 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v2", 2024-09-16T00:35:21.705 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "172.21.15.38:6800", 2024-09-16T00:35:21.705 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 1461245815 2024-09-16T00:35:21.705 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.705 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:21.706 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v1", 2024-09-16T00:35:21.706 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "172.21.15.38:6801", 2024-09-16T00:35:21.706 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 1461245815 2024-09-16T00:35:21.706 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:21.706 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-09-16T00:35:21.706 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.706 INFO:teuthology.orchestra.run.smithi038.stdout: "active_addr": "172.21.15.38:6801/1461245815", 2024-09-16T00:35:21.706 INFO:teuthology.orchestra.run.smithi038.stdout: "active_change": "2024-09-16T00:23:21.417599+0000", 2024-09-16T00:35:21.706 INFO:teuthology.orchestra.run.smithi038.stdout: "active_mgr_features": 4540701547738038271, 2024-09-16T00:35:21.706 INFO:teuthology.orchestra.run.smithi038.stdout: "available": true, 2024-09-16T00:35:21.706 INFO:teuthology.orchestra.run.smithi038.stdout: "standbys": [ 2024-09-16T00:35:21.706 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:21.707 INFO:teuthology.orchestra.run.smithi038.stdout: "gid": 24106, 2024-09-16T00:35:21.707 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "z", 2024-09-16T00:35:21.707 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr_features": 4540701547738038271, 2024-09-16T00:35:21.707 INFO:teuthology.orchestra.run.smithi038.stdout: "available_modules": [ 2024-09-16T00:35:21.707 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:21.707 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "alerts", 2024-09-16T00:35:21.707 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:21.707 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:21.707 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:21.707 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-09-16T00:35:21.707 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-09-16T00:35:21.707 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:21.708 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.708 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.708 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-09-16T00:35:21.708 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.708 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.708 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.708 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How frequently to reexamine health status", 2024-09-16T00:35:21.708 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.708 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.708 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.708 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.708 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:21.709 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:21.709 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.709 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.709 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.709 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.709 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.709 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.709 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:21.709 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:21.709 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:21.709 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:21.709 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:21.709 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:21.709 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:21.710 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:21.710 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.710 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.710 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.710 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.710 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.710 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:21.710 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:21.710 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.710 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.710 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.710 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.710 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.710 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.711 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.711 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.711 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.711 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.711 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.711 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.711 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:21.711 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:21.711 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.711 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.711 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.711 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:21.711 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.711 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.712 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:21.712 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:21.712 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:21.712 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:21.712 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:21.712 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:21.712 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:21.712 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:21.712 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.712 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.712 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.713 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.713 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.713 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:21.713 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:21.713 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.713 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.713 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.713 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.713 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.713 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.713 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.714 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.714 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.714 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.714 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.714 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.714 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_destination": { 2024-09-16T00:35:21.714 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_destination", 2024-09-16T00:35:21.714 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.714 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.714 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.714 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.714 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.714 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.714 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.715 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Email address to send alerts to", 2024-09-16T00:35:21.715 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.715 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.715 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.715 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.715 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_from_name": { 2024-09-16T00:35:21.715 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_from_name", 2024-09-16T00:35:21.715 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.715 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.715 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.715 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "Ceph", 2024-09-16T00:35:21.715 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.715 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.715 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.716 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Email From: name", 2024-09-16T00:35:21.716 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.716 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.716 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.716 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.716 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_host": { 2024-09-16T00:35:21.716 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_host", 2024-09-16T00:35:21.716 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.716 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.716 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.716 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.716 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.716 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.716 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.717 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SMTP server", 2024-09-16T00:35:21.717 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.717 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.717 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.717 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.717 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_password": { 2024-09-16T00:35:21.717 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_password", 2024-09-16T00:35:21.717 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.717 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.717 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.717 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.717 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.717 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.717 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.718 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Password to authenticate with", 2024-09-16T00:35:21.718 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.718 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.718 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.718 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.718 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_port": { 2024-09-16T00:35:21.718 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_port", 2024-09-16T00:35:21.718 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:21.718 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.718 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.718 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "465", 2024-09-16T00:35:21.718 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.718 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.718 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.719 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SMTP port", 2024-09-16T00:35:21.719 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.719 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.719 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.719 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.719 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_sender": { 2024-09-16T00:35:21.719 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_sender", 2024-09-16T00:35:21.719 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.719 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.719 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.719 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.719 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.720 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.720 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.720 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SMTP envelope sender", 2024-09-16T00:35:21.720 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.720 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.720 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.720 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.720 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_ssl": { 2024-09-16T00:35:21.720 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_ssl", 2024-09-16T00:35:21.720 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.720 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.721 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.721 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:21.721 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.721 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.721 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.721 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-16T00:35:21.721 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.721 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.721 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.721 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.721 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_user": { 2024-09-16T00:35:21.721 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_user", 2024-09-16T00:35:21.721 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.721 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.722 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.722 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.722 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.722 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.722 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.722 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "User to authenticate as", 2024-09-16T00:35:21.722 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.722 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.722 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.722 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:21.722 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:21.722 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.722 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:21.722 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "balancer", 2024-09-16T00:35:21.723 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:21.723 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:21.723 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:21.723 INFO:teuthology.orchestra.run.smithi038.stdout: "active": { 2024-09-16T00:35:21.723 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "active", 2024-09-16T00:35:21.723 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.723 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.723 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.723 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:21.723 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.723 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.723 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.723 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "automatically balance PGs across cluster", 2024-09-16T00:35:21.723 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.724 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.724 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.724 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.724 INFO:teuthology.orchestra.run.smithi038.stdout: "begin_time": { 2024-09-16T00:35:21.724 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "begin_time", 2024-09-16T00:35:21.724 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.724 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.724 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.724 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0000", 2024-09-16T00:35:21.724 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.724 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.724 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.724 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "beginning time of day to automatically balance", 2024-09-16T00:35:21.725 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T00:35:21.725 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.725 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.725 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.725 INFO:teuthology.orchestra.run.smithi038.stdout: "begin_weekday": { 2024-09-16T00:35:21.725 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "begin_weekday", 2024-09-16T00:35:21.725 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-09-16T00:35:21.725 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.725 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.725 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-09-16T00:35:21.725 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "0", 2024-09-16T00:35:21.725 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "6", 2024-09-16T00:35:21.725 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.726 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-16T00:35:21.726 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T00:35:21.726 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.726 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.726 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.726 INFO:teuthology.orchestra.run.smithi038.stdout: "crush_compat_max_iterations": { 2024-09-16T00:35:21.726 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crush_compat_max_iterations", 2024-09-16T00:35:21.726 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-09-16T00:35:21.726 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.726 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.726 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "25", 2024-09-16T00:35:21.727 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-09-16T00:35:21.727 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "250", 2024-09-16T00:35:21.727 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.727 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-16T00:35:21.727 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.727 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.727 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.727 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.727 INFO:teuthology.orchestra.run.smithi038.stdout: "crush_compat_metrics": { 2024-09-16T00:35:21.727 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crush_compat_metrics", 2024-09-16T00:35:21.727 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.727 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.728 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.728 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "pgs,objects,bytes", 2024-09-16T00:35:21.728 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.728 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.728 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.728 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-16T00:35:21.728 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-16T00:35:21.728 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.728 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.728 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.728 INFO:teuthology.orchestra.run.smithi038.stdout: "crush_compat_step": { 2024-09-16T00:35:21.728 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crush_compat_step", 2024-09-16T00:35:21.729 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-16T00:35:21.729 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.729 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.729 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0.5", 2024-09-16T00:35:21.729 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "0.001", 2024-09-16T00:35:21.729 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "0.999", 2024-09-16T00:35:21.729 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.729 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "aggressiveness of optimization", 2024-09-16T00:35:21.729 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-16T00:35:21.729 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.730 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.730 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.730 INFO:teuthology.orchestra.run.smithi038.stdout: "end_time": { 2024-09-16T00:35:21.730 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "end_time", 2024-09-16T00:35:21.730 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.730 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.730 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.730 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "2359", 2024-09-16T00:35:21.730 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.730 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.730 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.730 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "ending time of day to automatically balance", 2024-09-16T00:35:21.730 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T00:35:21.731 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.731 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.731 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.731 INFO:teuthology.orchestra.run.smithi038.stdout: "end_weekday": { 2024-09-16T00:35:21.731 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "end_weekday", 2024-09-16T00:35:21.731 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-09-16T00:35:21.731 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.731 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.731 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-09-16T00:35:21.731 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "0", 2024-09-16T00:35:21.732 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "6", 2024-09-16T00:35:21.732 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.732 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-16T00:35:21.732 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T00:35:21.732 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.732 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.732 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.732 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:21.732 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:21.732 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.733 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.733 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.733 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.733 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.733 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.733 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:21.733 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:21.733 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:21.733 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:21.734 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:21.734 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:21.734 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:21.734 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:21.734 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.734 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.734 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.734 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.734 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.734 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:21.734 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:21.735 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.735 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.735 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.735 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.735 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.735 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.735 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.735 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.735 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.735 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.735 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.736 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.736 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:21.736 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:21.736 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.736 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.736 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.736 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:21.736 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.737 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.737 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:21.737 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:21.737 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:21.737 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:21.737 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:21.737 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:21.737 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:21.737 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:21.737 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.738 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.738 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.738 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.738 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.738 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:21.738 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:21.738 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.738 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.738 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.739 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.739 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.739 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.739 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.739 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.739 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.739 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.739 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.739 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.739 INFO:teuthology.orchestra.run.smithi038.stdout: "min_score": { 2024-09-16T00:35:21.739 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "min_score", 2024-09-16T00:35:21.739 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-16T00:35:21.740 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.740 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.740 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-09-16T00:35:21.740 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.740 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.740 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.740 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-16T00:35:21.740 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.740 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.740 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.740 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.740 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": { 2024-09-16T00:35:21.740 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mode", 2024-09-16T00:35:21.741 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.741 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.741 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.741 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "upmap", 2024-09-16T00:35:21.741 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.741 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.741 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:21.741 INFO:teuthology.orchestra.run.smithi038.stdout: "crush-compat", 2024-09-16T00:35:21.741 INFO:teuthology.orchestra.run.smithi038.stdout: "none", 2024-09-16T00:35:21.741 INFO:teuthology.orchestra.run.smithi038.stdout: "read", 2024-09-16T00:35:21.741 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap", 2024-09-16T00:35:21.741 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap-read" 2024-09-16T00:35:21.741 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:21.742 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Balancer mode", 2024-09-16T00:35:21.742 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.742 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.742 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.742 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.742 INFO:teuthology.orchestra.run.smithi038.stdout: "pool_ids": { 2024-09-16T00:35:21.742 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pool_ids", 2024-09-16T00:35:21.744 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.744 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.744 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.744 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.744 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.744 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.744 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.744 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-16T00:35:21.744 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.744 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.744 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.744 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.745 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-09-16T00:35:21.745 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-09-16T00:35:21.745 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:21.745 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.745 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.745 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-09-16T00:35:21.745 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.745 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.745 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.745 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-16T00:35:21.745 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.745 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.745 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.745 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.746 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap_max_deviation": { 2024-09-16T00:35:21.746 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "upmap_max_deviation", 2024-09-16T00:35:21.746 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:21.746 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.746 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.746 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-09-16T00:35:21.746 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-09-16T00:35:21.746 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.746 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.746 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-16T00:35:21.746 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-16T00:35:21.746 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.746 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.746 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.747 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap_max_optimizations": { 2024-09-16T00:35:21.747 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "upmap_max_optimizations", 2024-09-16T00:35:21.747 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-09-16T00:35:21.747 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.747 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.747 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-16T00:35:21.747 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.747 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.747 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.747 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-16T00:35:21.747 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.747 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.747 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.747 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:21.748 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:21.748 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.748 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:21.748 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cephadm", 2024-09-16T00:35:21.748 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:21.748 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:21.748 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:21.748 INFO:teuthology.orchestra.run.smithi038.stdout: "agent_down_multiplier": { 2024-09-16T00:35:21.748 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "agent_down_multiplier", 2024-09-16T00:35:21.748 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-16T00:35:21.748 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.748 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.748 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "3.0", 2024-09-16T00:35:21.748 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.749 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.749 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.749 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-16T00:35:21.749 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.749 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.749 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.749 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.749 INFO:teuthology.orchestra.run.smithi038.stdout: "agent_refresh_rate": { 2024-09-16T00:35:21.749 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "agent_refresh_rate", 2024-09-16T00:35:21.749 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:21.749 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.749 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.749 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "20", 2024-09-16T00:35:21.749 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.750 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.750 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.750 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-16T00:35:21.750 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.750 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.750 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.750 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.750 INFO:teuthology.orchestra.run.smithi038.stdout: "agent_starting_port": { 2024-09-16T00:35:21.750 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "agent_starting_port", 2024-09-16T00:35:21.750 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:21.750 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.750 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.750 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "4721", 2024-09-16T00:35:21.750 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.750 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.751 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.751 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-16T00:35:21.751 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.751 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.751 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.751 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.751 INFO:teuthology.orchestra.run.smithi038.stdout: "allow_ptrace": { 2024-09-16T00:35:21.751 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "allow_ptrace", 2024-09-16T00:35:21.751 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.751 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.751 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.751 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.752 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.752 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.752 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.752 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-16T00:35:21.752 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-16T00:35:21.752 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.752 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.752 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.752 INFO:teuthology.orchestra.run.smithi038.stdout: "autotune_interval": { 2024-09-16T00:35:21.752 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "autotune_interval", 2024-09-16T00:35:21.752 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:21.752 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.752 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.752 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-09-16T00:35:21.753 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.753 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.753 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.753 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-16T00:35:21.753 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.753 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.753 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.753 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.753 INFO:teuthology.orchestra.run.smithi038.stdout: "autotune_memory_target_ratio": { 2024-09-16T00:35:21.753 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "autotune_memory_target_ratio", 2024-09-16T00:35:21.753 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-16T00:35:21.753 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.753 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.754 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0.7", 2024-09-16T00:35:21.754 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.754 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.754 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.754 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-16T00:35:21.754 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.754 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.754 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.754 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.754 INFO:teuthology.orchestra.run.smithi038.stdout: "cephadm_log_destination": { 2024-09-16T00:35:21.754 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cephadm_log_destination", 2024-09-16T00:35:21.754 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.755 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.755 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.755 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.755 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.755 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.755 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:21.755 INFO:teuthology.orchestra.run.smithi038.stdout: "file", 2024-09-16T00:35:21.755 INFO:teuthology.orchestra.run.smithi038.stdout: "file,syslog", 2024-09-16T00:35:21.755 INFO:teuthology.orchestra.run.smithi038.stdout: "syslog" 2024-09-16T00:35:21.755 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:21.755 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-16T00:35:21.755 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.755 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.756 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.756 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.756 INFO:teuthology.orchestra.run.smithi038.stdout: "cgroups_split": { 2024-09-16T00:35:21.756 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cgroups_split", 2024-09-16T00:35:21.756 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.756 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.756 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.756 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:21.756 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.756 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.756 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.756 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-16T00:35:21.756 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.757 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.757 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.757 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.757 INFO:teuthology.orchestra.run.smithi038.stdout: "config_checks_enabled": { 2024-09-16T00:35:21.757 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "config_checks_enabled", 2024-09-16T00:35:21.757 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.757 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.757 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.757 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.757 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.757 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.757 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.758 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-16T00:35:21.758 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.758 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.758 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.758 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.758 INFO:teuthology.orchestra.run.smithi038.stdout: "config_dashboard": { 2024-09-16T00:35:21.758 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "config_dashboard", 2024-09-16T00:35:21.758 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.758 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.758 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.758 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:21.758 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.758 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.758 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.758 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-16T00:35:21.759 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.759 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.759 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.759 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.759 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_alertmanager": { 2024-09-16T00:35:21.759 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_alertmanager", 2024-09-16T00:35:21.759 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.759 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.759 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.759 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-16T00:35:21.759 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.759 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.759 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.759 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-09-16T00:35:21.760 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.760 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.760 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.760 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.760 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_base": { 2024-09-16T00:35:21.760 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_base", 2024-09-16T00:35:21.760 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.760 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.760 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.760 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-16T00:35:21.760 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.760 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.760 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.760 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Container image name, without the tag", 2024-09-16T00:35:21.761 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.761 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.761 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.761 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.761 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_elasticsearch": { 2024-09-16T00:35:21.761 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_elasticsearch", 2024-09-16T00:35:21.761 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.761 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.761 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.761 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-16T00:35:21.761 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.761 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.761 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.761 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "elasticsearch container image", 2024-09-16T00:35:21.762 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.762 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.762 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.762 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.762 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_grafana": { 2024-09-16T00:35:21.762 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_grafana", 2024-09-16T00:35:21.762 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.762 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.762 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.762 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-16T00:35:21.762 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.762 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.762 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.762 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-09-16T00:35:21.763 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.763 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.763 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.763 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.763 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_haproxy": { 2024-09-16T00:35:21.763 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_haproxy", 2024-09-16T00:35:21.763 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.763 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.763 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.763 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-16T00:35:21.763 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.763 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.763 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.764 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "HAproxy container image", 2024-09-16T00:35:21.764 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.764 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.764 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.764 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.764 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_jaeger_agent": { 2024-09-16T00:35:21.764 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_jaeger_agent", 2024-09-16T00:35:21.764 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.764 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.765 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.765 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-16T00:35:21.765 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.765 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.765 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.765 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Jaeger agent container image", 2024-09-16T00:35:21.765 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.765 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.765 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.765 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.765 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_jaeger_collector": { 2024-09-16T00:35:21.765 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_jaeger_collector", 2024-09-16T00:35:21.765 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.766 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.766 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.766 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-16T00:35:21.766 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.766 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.766 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.766 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Jaeger collector container image", 2024-09-16T00:35:21.766 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.766 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.766 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.766 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.767 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_jaeger_query": { 2024-09-16T00:35:21.767 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_jaeger_query", 2024-09-16T00:35:21.767 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.767 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.767 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.767 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-16T00:35:21.767 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.767 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.767 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.767 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Jaeger query container image", 2024-09-16T00:35:21.767 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.767 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.768 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.768 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.768 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_keepalived": { 2024-09-16T00:35:21.768 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_keepalived", 2024-09-16T00:35:21.768 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.768 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.768 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.768 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-16T00:35:21.768 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.768 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.768 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.769 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Keepalived container image", 2024-09-16T00:35:21.769 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.769 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.769 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.769 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.769 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_loki": { 2024-09-16T00:35:21.769 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_loki", 2024-09-16T00:35:21.769 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.769 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.769 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.769 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-16T00:35:21.769 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.770 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.770 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.770 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Loki container image", 2024-09-16T00:35:21.770 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.770 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.770 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.770 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.770 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_node_exporter": { 2024-09-16T00:35:21.770 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_node_exporter", 2024-09-16T00:35:21.770 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.770 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.771 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.771 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-16T00:35:21.771 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.771 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.771 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.771 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-09-16T00:35:21.771 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.771 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.771 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.771 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.771 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_nvmeof": { 2024-09-16T00:35:21.772 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_nvmeof", 2024-09-16T00:35:21.772 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.772 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.772 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.772 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-16T00:35:21.772 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.772 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.772 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.772 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Nvme-of container image", 2024-09-16T00:35:21.773 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.773 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.773 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.773 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.773 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_prometheus": { 2024-09-16T00:35:21.773 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_prometheus", 2024-09-16T00:35:21.773 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.773 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.773 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.773 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-16T00:35:21.773 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.773 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.773 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.773 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-09-16T00:35:21.774 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.774 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.774 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.774 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.774 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_promtail": { 2024-09-16T00:35:21.774 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_promtail", 2024-09-16T00:35:21.774 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.774 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.774 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.774 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-16T00:35:21.774 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.774 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.774 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.774 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Promtail container image", 2024-09-16T00:35:21.775 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.775 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.775 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.775 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.775 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_samba": { 2024-09-16T00:35:21.775 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_samba", 2024-09-16T00:35:21.775 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.775 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.775 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.775 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-16T00:35:21.775 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.775 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.775 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.776 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Samba/SMB container image", 2024-09-16T00:35:21.776 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.776 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.776 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.776 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.776 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_snmp_gateway": { 2024-09-16T00:35:21.776 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_snmp_gateway", 2024-09-16T00:35:21.776 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.776 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.776 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.776 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-16T00:35:21.776 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.776 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.776 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.777 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SNMP Gateway container image", 2024-09-16T00:35:21.777 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.777 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.777 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.777 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.777 INFO:teuthology.orchestra.run.smithi038.stdout: "container_init": { 2024-09-16T00:35:21.777 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_init", 2024-09-16T00:35:21.777 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.777 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.777 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.777 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:21.777 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.777 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.778 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.778 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Run podman/docker with `--init`", 2024-09-16T00:35:21.778 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.778 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.778 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.778 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.778 INFO:teuthology.orchestra.run.smithi038.stdout: "daemon_cache_timeout": { 2024-09-16T00:35:21.778 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "daemon_cache_timeout", 2024-09-16T00:35:21.778 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:21.779 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.779 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.779 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-09-16T00:35:21.779 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.779 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.779 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.779 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-16T00:35:21.779 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.779 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.780 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.780 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.780 INFO:teuthology.orchestra.run.smithi038.stdout: "default_cephadm_command_timeout": { 2024-09-16T00:35:21.780 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "default_cephadm_command_timeout", 2024-09-16T00:35:21.780 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:21.780 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.780 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.780 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "900", 2024-09-16T00:35:21.780 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.781 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.781 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.781 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-16T00:35:21.781 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.781 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.781 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.781 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.781 INFO:teuthology.orchestra.run.smithi038.stdout: "default_registry": { 2024-09-16T00:35:21.781 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "default_registry", 2024-09-16T00:35:21.781 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.781 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.782 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.782 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io", 2024-09-16T00:35:21.782 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.782 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.782 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.782 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-16T00:35:21.782 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.782 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.782 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.782 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.782 INFO:teuthology.orchestra.run.smithi038.stdout: "device_cache_timeout": { 2024-09-16T00:35:21.782 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "device_cache_timeout", 2024-09-16T00:35:21.782 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:21.782 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.783 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.783 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "1800", 2024-09-16T00:35:21.783 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.783 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.783 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.783 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "seconds to cache device inventory", 2024-09-16T00:35:21.783 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.783 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.783 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.783 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.783 INFO:teuthology.orchestra.run.smithi038.stdout: "device_enhanced_scan": { 2024-09-16T00:35:21.783 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "device_enhanced_scan", 2024-09-16T00:35:21.783 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.783 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.784 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.784 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.784 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.784 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.784 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.784 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-16T00:35:21.784 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.784 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.784 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.784 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.784 INFO:teuthology.orchestra.run.smithi038.stdout: "facts_cache_timeout": { 2024-09-16T00:35:21.784 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "facts_cache_timeout", 2024-09-16T00:35:21.784 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:21.784 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.785 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.785 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-09-16T00:35:21.785 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.785 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.785 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.785 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "seconds to cache host facts data", 2024-09-16T00:35:21.785 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.785 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.785 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.785 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.785 INFO:teuthology.orchestra.run.smithi038.stdout: "grafana_dashboards_path": { 2024-09-16T00:35:21.785 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "grafana_dashboards_path", 2024-09-16T00:35:21.785 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.785 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.786 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.786 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-16T00:35:21.786 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.786 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.786 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.786 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-16T00:35:21.786 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.786 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.786 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.786 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.786 INFO:teuthology.orchestra.run.smithi038.stdout: "host_check_interval": { 2024-09-16T00:35:21.786 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "host_check_interval", 2024-09-16T00:35:21.786 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:21.786 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.787 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.787 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-09-16T00:35:21.787 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.787 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.787 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.787 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to perform a host check", 2024-09-16T00:35:21.787 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.787 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.787 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.787 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.787 INFO:teuthology.orchestra.run.smithi038.stdout: "hw_monitoring": { 2024-09-16T00:35:21.787 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "hw_monitoring", 2024-09-16T00:35:21.787 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.787 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.788 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.788 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.788 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.788 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.788 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.788 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-16T00:35:21.788 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.788 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.788 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.788 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.788 INFO:teuthology.orchestra.run.smithi038.stdout: "inventory_list_all": { 2024-09-16T00:35:21.788 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "inventory_list_all", 2024-09-16T00:35:21.788 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.788 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.789 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.789 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.789 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.789 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.789 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.789 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-16T00:35:21.789 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.789 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.789 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.789 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.789 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:21.789 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:21.789 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.789 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.790 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.790 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.790 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.790 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.790 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:21.790 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:21.790 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:21.790 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:21.790 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:21.790 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:21.790 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:21.790 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:21.790 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.791 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.791 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.791 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.791 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.791 INFO:teuthology.orchestra.run.smithi038.stdout: "log_refresh_metadata": { 2024-09-16T00:35:21.791 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_refresh_metadata", 2024-09-16T00:35:21.791 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.791 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.791 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.791 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.791 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.791 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.792 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.792 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-16T00:35:21.792 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.792 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.792 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.792 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.792 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:21.792 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:21.792 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.792 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.792 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.792 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:21.792 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.792 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.793 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.793 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-16T00:35:21.793 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.793 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.793 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.793 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.793 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:21.793 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:21.793 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.793 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.794 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.794 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:21.794 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.794 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.794 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:21.794 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:21.794 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:21.794 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:21.794 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:21.794 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:21.794 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:21.794 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:21.794 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.794 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.795 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.795 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.795 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.795 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:21.795 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:21.795 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.795 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.795 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.795 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.795 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.795 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.795 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.795 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.795 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.796 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.796 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.796 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.796 INFO:teuthology.orchestra.run.smithi038.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-16T00:35:21.796 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-16T00:35:21.796 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.796 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.796 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.796 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.796 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.796 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.796 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.796 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-16T00:35:21.797 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.797 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.797 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.797 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.797 INFO:teuthology.orchestra.run.smithi038.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-16T00:35:21.797 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-16T00:35:21.797 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.797 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.797 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.797 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "*", 2024-09-16T00:35:21.797 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.797 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.797 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.798 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-16T00:35:21.798 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.798 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.798 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.798 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.798 INFO:teuthology.orchestra.run.smithi038.stdout: "max_count_per_host": { 2024-09-16T00:35:21.798 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_count_per_host", 2024-09-16T00:35:21.798 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:21.798 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.798 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.798 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-16T00:35:21.798 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.798 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.798 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.799 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "max number of daemons per service per host", 2024-09-16T00:35:21.799 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.799 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.799 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.799 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.799 INFO:teuthology.orchestra.run.smithi038.stdout: "max_osd_draining_count": { 2024-09-16T00:35:21.799 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_osd_draining_count", 2024-09-16T00:35:21.799 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:21.799 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.799 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.799 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-16T00:35:21.799 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.799 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.800 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.800 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-16T00:35:21.800 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.800 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.800 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.800 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.800 INFO:teuthology.orchestra.run.smithi038.stdout: "migration_current": { 2024-09-16T00:35:21.800 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "migration_current", 2024-09-16T00:35:21.800 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:21.800 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.800 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.800 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.800 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.800 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.801 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.801 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "internal - do not modify", 2024-09-16T00:35:21.801 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.801 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.801 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.801 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.801 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": { 2024-09-16T00:35:21.801 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mode", 2024-09-16T00:35:21.801 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.801 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.801 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.801 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "root", 2024-09-16T00:35:21.801 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.802 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.802 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:21.802 INFO:teuthology.orchestra.run.smithi038.stdout: "cephadm-package", 2024-09-16T00:35:21.802 INFO:teuthology.orchestra.run.smithi038.stdout: "root" 2024-09-16T00:35:21.802 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:21.802 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "mode for remote execution of cephadm", 2024-09-16T00:35:21.802 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.802 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.802 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.802 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.802 INFO:teuthology.orchestra.run.smithi038.stdout: "oob_default_addr": { 2024-09-16T00:35:21.802 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "oob_default_addr", 2024-09-16T00:35:21.802 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.802 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.803 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.803 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "169.254.1.1", 2024-09-16T00:35:21.803 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.803 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.803 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.803 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-16T00:35:21.803 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.803 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.803 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.803 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.803 INFO:teuthology.orchestra.run.smithi038.stdout: "prometheus_alerts_path": { 2024-09-16T00:35:21.803 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "prometheus_alerts_path", 2024-09-16T00:35:21.804 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.804 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.804 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.804 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-16T00:35:21.804 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.804 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.804 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.804 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-16T00:35:21.804 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.804 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.805 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.805 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.805 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_insecure": { 2024-09-16T00:35:21.805 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_insecure", 2024-09-16T00:35:21.805 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.805 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.805 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.805 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.805 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.805 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.805 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.805 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-16T00:35:21.806 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.806 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.806 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.806 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.806 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_password": { 2024-09-16T00:35:21.806 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_password", 2024-09-16T00:35:21.806 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.806 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.806 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.807 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.807 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.807 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.807 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.807 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-16T00:35:21.807 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.807 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.807 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.807 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.807 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_url": { 2024-09-16T00:35:21.807 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_url", 2024-09-16T00:35:21.807 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.807 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.808 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.808 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.808 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.808 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.808 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.808 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-16T00:35:21.808 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.808 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.808 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.808 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.808 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_username": { 2024-09-16T00:35:21.809 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_username", 2024-09-16T00:35:21.809 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.809 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.809 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.809 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.809 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.809 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.809 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.809 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-16T00:35:21.809 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.810 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.810 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.810 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.810 INFO:teuthology.orchestra.run.smithi038.stdout: "secure_monitoring_stack": { 2024-09-16T00:35:21.810 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "secure_monitoring_stack", 2024-09-16T00:35:21.810 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.810 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.810 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.810 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.810 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.811 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.811 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.811 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-16T00:35:21.811 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.811 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.811 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.811 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.811 INFO:teuthology.orchestra.run.smithi038.stdout: "service_discovery_port": { 2024-09-16T00:35:21.811 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "service_discovery_port", 2024-09-16T00:35:21.811 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:21.811 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.811 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.812 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8765", 2024-09-16T00:35:21.812 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.812 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.812 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.812 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "cephadm service discovery port", 2024-09-16T00:35:21.812 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.812 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.812 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.812 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.812 INFO:teuthology.orchestra.run.smithi038.stdout: "ssh_config_file": { 2024-09-16T00:35:21.812 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssh_config_file", 2024-09-16T00:35:21.813 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.813 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.813 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.813 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.813 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.813 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.813 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.813 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-16T00:35:21.813 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.813 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.814 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.814 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.814 INFO:teuthology.orchestra.run.smithi038.stdout: "use_agent": { 2024-09-16T00:35:21.814 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "use_agent", 2024-09-16T00:35:21.814 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.814 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.814 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.814 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.814 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.814 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.814 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.815 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-16T00:35:21.815 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.815 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.815 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.815 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.815 INFO:teuthology.orchestra.run.smithi038.stdout: "use_repo_digest": { 2024-09-16T00:35:21.815 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "use_repo_digest", 2024-09-16T00:35:21.815 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.816 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.816 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.816 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:21.816 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.816 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.816 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.816 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-16T00:35:21.816 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.816 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.816 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.817 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.817 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_on_failed_host_check": { 2024-09-16T00:35:21.817 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_on_failed_host_check", 2024-09-16T00:35:21.817 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.817 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.817 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.817 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:21.817 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.817 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.817 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.817 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise a health warning if the host check fails", 2024-09-16T00:35:21.817 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.817 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.817 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.818 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.818 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_on_stray_daemons": { 2024-09-16T00:35:21.818 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_on_stray_daemons", 2024-09-16T00:35:21.818 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.818 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.818 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.818 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:21.818 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.818 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.818 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.818 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-16T00:35:21.818 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.818 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.818 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.819 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.819 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_on_stray_hosts": { 2024-09-16T00:35:21.819 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_on_stray_hosts", 2024-09-16T00:35:21.819 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.819 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.819 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.819 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:21.819 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.819 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.819 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.819 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-16T00:35:21.819 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.819 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.819 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.820 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:21.820 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:21.820 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.820 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:21.820 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crash", 2024-09-16T00:35:21.820 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:21.820 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:21.820 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:21.820 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:21.820 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:21.820 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.820 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.820 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.821 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.821 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.821 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.821 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:21.821 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:21.821 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:21.821 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:21.821 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:21.821 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:21.821 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:21.821 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:21.821 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.821 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.821 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.822 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.822 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.822 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:21.822 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:21.822 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.822 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.822 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.822 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.822 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.822 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.822 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.822 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.822 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.822 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.823 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.823 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.823 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:21.823 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:21.823 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.823 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.823 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.823 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:21.823 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.823 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.823 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:21.823 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:21.823 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:21.824 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:21.824 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:21.824 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:21.824 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:21.824 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:21.824 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.824 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.824 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.824 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.824 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.824 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:21.824 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:21.824 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.824 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.825 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.825 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.825 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.825 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.825 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.825 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.825 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.825 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.825 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.825 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.825 INFO:teuthology.orchestra.run.smithi038.stdout: "retain_interval": { 2024-09-16T00:35:21.825 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "retain_interval", 2024-09-16T00:35:21.825 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:21.825 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.826 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.826 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "31536000", 2024-09-16T00:35:21.826 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.826 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.826 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.826 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-16T00:35:21.826 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.826 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.826 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.826 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.826 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_recent_interval": { 2024-09-16T00:35:21.826 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_recent_interval", 2024-09-16T00:35:21.826 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:21.826 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.826 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.827 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "1209600", 2024-09-16T00:35:21.827 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.827 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.827 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.827 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-16T00:35:21.827 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.827 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.827 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.827 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:21.827 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:21.827 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.827 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:21.827 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "dashboard", 2024-09-16T00:35:21.827 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:21.828 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:21.828 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:21.828 INFO:teuthology.orchestra.run.smithi038.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-16T00:35:21.828 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-16T00:35:21.828 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:21.828 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.828 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.828 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-16T00:35:21.828 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.828 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.828 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.828 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.828 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.828 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.829 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.829 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.829 INFO:teuthology.orchestra.run.smithi038.stdout: "ALERTMANAGER_API_HOST": { 2024-09-16T00:35:21.829 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-16T00:35:21.829 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.829 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.829 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.829 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.829 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.829 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.829 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.829 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.829 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.829 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.829 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.830 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.830 INFO:teuthology.orchestra.run.smithi038.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-16T00:35:21.830 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-16T00:35:21.830 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.830 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.830 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.830 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:21.830 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.830 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.830 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.830 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.830 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.830 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.830 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.831 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.831 INFO:teuthology.orchestra.run.smithi038.stdout: "AUDIT_API_ENABLED": { 2024-09-16T00:35:21.831 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "AUDIT_API_ENABLED", 2024-09-16T00:35:21.831 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.831 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.831 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.831 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.831 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.831 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.831 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.831 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.831 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.831 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.831 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.831 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.832 INFO:teuthology.orchestra.run.smithi038.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-16T00:35:21.832 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-16T00:35:21.832 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.832 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.832 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.832 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:21.832 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.832 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.832 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.832 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.832 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.832 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.832 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.832 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.833 INFO:teuthology.orchestra.run.smithi038.stdout: "ENABLE_BROWSABLE_API": { 2024-09-16T00:35:21.833 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-16T00:35:21.833 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.833 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.833 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.833 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:21.833 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.833 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.833 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.833 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.834 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.834 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.834 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.834 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.834 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-16T00:35:21.834 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-16T00:35:21.834 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.834 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.834 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.834 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:21.834 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.834 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.834 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.835 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.835 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.835 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.835 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.835 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.835 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-16T00:35:21.835 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-16T00:35:21.835 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.835 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.835 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.835 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:21.835 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.835 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.835 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.836 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.836 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.836 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.836 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.836 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.836 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-16T00:35:21.836 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-16T00:35:21.836 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.836 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.836 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.836 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:21.836 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.836 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.836 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.837 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.837 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.837 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.837 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.837 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.837 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-16T00:35:21.837 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-16T00:35:21.837 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.837 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.837 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.837 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:21.837 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.837 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.837 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.838 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.838 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.838 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.838 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.838 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.838 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-16T00:35:21.838 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-16T00:35:21.838 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.838 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.838 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.838 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:21.838 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.838 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.838 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.839 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.839 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.839 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.839 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.839 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.839 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-16T00:35:21.839 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-16T00:35:21.839 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.839 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.839 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.839 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:21.839 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.840 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.840 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.840 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.840 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.840 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.840 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.840 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.840 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-16T00:35:21.840 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-16T00:35:21.840 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.841 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.841 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.841 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:21.841 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.841 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.841 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.841 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.841 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.841 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.841 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.841 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.841 INFO:teuthology.orchestra.run.smithi038.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-16T00:35:21.841 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-16T00:35:21.842 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.842 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.842 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.842 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.842 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.842 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.842 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.842 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.842 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.842 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.842 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.843 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.843 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_PASSWORD": { 2024-09-16T00:35:21.843 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-16T00:35:21.843 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.843 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.843 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.843 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "admin", 2024-09-16T00:35:21.843 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.843 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.843 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.843 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.843 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.844 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.844 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.844 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.844 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-16T00:35:21.844 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-16T00:35:21.844 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.844 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.844 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.844 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:21.844 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.844 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.845 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.845 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.845 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.845 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.845 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.845 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.845 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_URL": { 2024-09-16T00:35:21.845 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_URL", 2024-09-16T00:35:21.845 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.845 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.845 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.845 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.846 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.846 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.846 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.846 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.846 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.846 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.846 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.846 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.846 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_USERNAME": { 2024-09-16T00:35:21.846 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-16T00:35:21.846 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.846 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.847 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.847 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "admin", 2024-09-16T00:35:21.847 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.847 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.847 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.847 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.847 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.847 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.847 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.847 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.847 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-16T00:35:21.848 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-16T00:35:21.848 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.848 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.848 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.848 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.848 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.848 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.848 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.849 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.849 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.849 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.849 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.849 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.849 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-16T00:35:21.849 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-16T00:35:21.849 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.849 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.850 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.850 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.850 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.850 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.850 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.850 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.850 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.850 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.850 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.850 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.850 INFO:teuthology.orchestra.run.smithi038.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-16T00:35:21.850 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-16T00:35:21.850 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.851 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.851 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.851 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:21.851 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.851 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.851 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.851 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.851 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.851 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.851 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.851 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.851 INFO:teuthology.orchestra.run.smithi038.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-16T00:35:21.851 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-16T00:35:21.851 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.852 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.852 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.852 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.852 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.852 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.852 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.852 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.852 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.852 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.852 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.852 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.852 INFO:teuthology.orchestra.run.smithi038.stdout: "PROMETHEUS_API_HOST": { 2024-09-16T00:35:21.852 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-16T00:35:21.853 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.853 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.853 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.853 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.853 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.853 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.853 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.853 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.853 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.853 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.854 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.854 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.854 INFO:teuthology.orchestra.run.smithi038.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-16T00:35:21.854 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-16T00:35:21.854 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.854 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.854 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.854 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:21.855 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.855 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.855 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.855 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.855 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.855 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.855 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.855 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.855 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-16T00:35:21.855 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-16T00:35:21.855 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.855 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.855 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.856 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.856 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.856 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.856 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.856 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.856 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.856 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.856 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.856 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.856 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-16T00:35:21.856 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-16T00:35:21.856 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.856 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.856 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.857 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.857 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.857 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.857 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.857 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.857 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.857 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.857 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.857 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.857 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-16T00:35:21.857 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-16T00:35:21.857 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.857 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.858 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.858 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:21.858 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.858 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.858 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.858 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.858 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.858 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.858 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.858 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.858 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-16T00:35:21.858 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-16T00:35:21.858 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.858 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.858 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.859 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:21.859 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.859 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.859 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.859 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.859 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.859 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.859 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.859 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.859 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-16T00:35:21.859 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-16T00:35:21.859 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.859 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.860 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.860 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.860 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.860 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.860 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.860 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.860 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.860 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.860 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.860 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.860 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-16T00:35:21.860 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-16T00:35:21.860 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.860 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.861 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.861 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.861 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.861 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.861 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.861 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.861 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.861 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.861 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.861 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.861 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-16T00:35:21.861 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-16T00:35:21.861 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.861 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.861 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.862 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.862 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.862 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.862 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.862 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.862 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.862 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.862 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.862 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.862 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_ENABLED": { 2024-09-16T00:35:21.862 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-16T00:35:21.862 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.862 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.862 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.863 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:21.863 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.863 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.863 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.863 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.863 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.863 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.863 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.863 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.863 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-16T00:35:21.863 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-16T00:35:21.863 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.863 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.864 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.864 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-16T00:35:21.864 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.864 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.864 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.864 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.864 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.864 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.864 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.864 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.864 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-16T00:35:21.864 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-16T00:35:21.864 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:21.864 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.865 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.865 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-16T00:35:21.865 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.865 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.865 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.865 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.865 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.865 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.865 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.865 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.865 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-16T00:35:21.865 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-16T00:35:21.865 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:21.865 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.866 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.866 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8", 2024-09-16T00:35:21.866 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.866 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.866 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.866 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.866 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.866 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.866 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.866 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.866 INFO:teuthology.orchestra.run.smithi038.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-16T00:35:21.866 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-16T00:35:21.866 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:21.866 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.867 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.867 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "45", 2024-09-16T00:35:21.867 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.867 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.867 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.867 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.867 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.867 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.867 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.867 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.867 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_ACCESS_KEY": { 2024-09-16T00:35:21.867 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-16T00:35:21.867 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.867 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.868 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.868 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.868 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.868 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.868 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.868 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.868 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.868 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.868 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.868 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.868 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-16T00:35:21.868 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-16T00:35:21.868 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.868 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.869 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.869 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "admin", 2024-09-16T00:35:21.869 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.869 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.869 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.869 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.869 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.869 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.869 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.869 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.869 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_SECRET_KEY": { 2024-09-16T00:35:21.869 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-16T00:35:21.869 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.869 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.870 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.870 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.870 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.870 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.870 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.870 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.870 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.870 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.870 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.870 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.870 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_SSL_VERIFY": { 2024-09-16T00:35:21.870 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-16T00:35:21.870 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.870 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.871 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.871 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:21.871 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.871 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.871 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.871 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.871 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.871 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.871 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.871 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.871 INFO:teuthology.orchestra.run.smithi038.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-16T00:35:21.871 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-16T00:35:21.871 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:21.871 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.871 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.872 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-09-16T00:35:21.872 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.872 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.872 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.872 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.872 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.872 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.872 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.872 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.872 INFO:teuthology.orchestra.run.smithi038.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-16T00:35:21.872 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-16T00:35:21.872 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:21.872 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.872 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.873 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-16T00:35:21.873 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.873 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.873 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.873 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.873 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.873 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.873 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.873 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.873 INFO:teuthology.orchestra.run.smithi038.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-16T00:35:21.873 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-16T00:35:21.873 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:21.873 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.873 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.873 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-09-16T00:35:21.874 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.874 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.874 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.874 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.874 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.874 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.874 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.874 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.874 INFO:teuthology.orchestra.run.smithi038.stdout: "cross_origin_url": { 2024-09-16T00:35:21.874 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cross_origin_url", 2024-09-16T00:35:21.874 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.874 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.874 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.874 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.875 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.875 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.875 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.875 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.875 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.875 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.875 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.875 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.875 INFO:teuthology.orchestra.run.smithi038.stdout: "crt_file": { 2024-09-16T00:35:21.875 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crt_file", 2024-09-16T00:35:21.875 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.875 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.875 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.876 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.876 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.876 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.876 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.876 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.876 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.876 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.876 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.876 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.876 INFO:teuthology.orchestra.run.smithi038.stdout: "debug": { 2024-09-16T00:35:21.877 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "debug", 2024-09-16T00:35:21.877 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.877 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.877 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.877 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.877 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.877 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.877 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.877 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Enable/disable debug options", 2024-09-16T00:35:21.877 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.877 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.877 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.878 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.878 INFO:teuthology.orchestra.run.smithi038.stdout: "jwt_token_ttl": { 2024-09-16T00:35:21.878 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "jwt_token_ttl", 2024-09-16T00:35:21.878 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:21.878 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.878 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.878 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "28800", 2024-09-16T00:35:21.878 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.878 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.878 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.878 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.879 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.879 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.879 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.879 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.879 INFO:teuthology.orchestra.run.smithi038.stdout: "key_file": { 2024-09-16T00:35:21.879 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "key_file", 2024-09-16T00:35:21.879 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.879 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.879 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.879 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.879 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.880 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.880 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.880 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.880 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.880 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.880 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.880 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.880 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:21.880 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:21.880 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.880 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.881 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.881 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.881 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.881 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.881 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:21.881 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:21.881 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:21.881 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:21.881 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:21.881 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:21.881 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:21.881 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:21.881 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.882 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.882 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.882 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.882 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.882 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:21.882 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:21.882 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.882 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.882 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.882 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.883 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.883 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.883 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.883 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.883 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.883 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.883 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.883 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.883 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:21.883 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:21.883 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.883 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.883 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.884 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:21.884 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.884 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.884 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:21.884 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:21.884 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:21.884 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:21.884 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:21.884 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:21.884 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:21.884 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:21.885 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.885 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.885 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.885 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.885 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.885 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:21.885 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:21.885 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.885 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.885 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.885 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.886 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.886 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.886 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.886 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.886 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.886 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.886 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.886 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.886 INFO:teuthology.orchestra.run.smithi038.stdout: "motd": { 2024-09-16T00:35:21.886 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "motd", 2024-09-16T00:35:21.886 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.887 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.887 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.887 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.887 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.887 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.887 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.887 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "The message of the day", 2024-09-16T00:35:21.887 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.887 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.887 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.888 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.888 INFO:teuthology.orchestra.run.smithi038.stdout: "redirect_resolve_ip_addr": { 2024-09-16T00:35:21.888 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "redirect_resolve_ip_addr", 2024-09-16T00:35:21.888 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.888 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.888 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.888 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.888 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.888 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.888 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.888 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.889 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.889 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.889 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.889 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.889 INFO:teuthology.orchestra.run.smithi038.stdout: "server_addr": { 2024-09-16T00:35:21.889 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_addr", 2024-09-16T00:35:21.889 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.889 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.889 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.889 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "::", 2024-09-16T00:35:21.889 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.889 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.889 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.889 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.889 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.890 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.890 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.890 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.890 INFO:teuthology.orchestra.run.smithi038.stdout: "server_port": { 2024-09-16T00:35:21.890 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_port", 2024-09-16T00:35:21.890 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:21.890 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.890 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.890 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8080", 2024-09-16T00:35:21.890 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.890 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.890 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.890 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.890 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.891 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.891 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.891 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.891 INFO:teuthology.orchestra.run.smithi038.stdout: "ssl": { 2024-09-16T00:35:21.891 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssl", 2024-09-16T00:35:21.891 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.891 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.891 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.891 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:21.891 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.891 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.891 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.891 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.891 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.891 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.892 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.892 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.892 INFO:teuthology.orchestra.run.smithi038.stdout: "ssl_server_port": { 2024-09-16T00:35:21.892 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssl_server_port", 2024-09-16T00:35:21.892 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:21.892 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.892 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.892 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8443", 2024-09-16T00:35:21.892 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.892 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.892 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.892 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.892 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.893 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.893 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.893 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.893 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_behaviour": { 2024-09-16T00:35:21.893 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_behaviour", 2024-09-16T00:35:21.893 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.893 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.893 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.893 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "redirect", 2024-09-16T00:35:21.893 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.893 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.893 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:21.893 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:21.894 INFO:teuthology.orchestra.run.smithi038.stdout: "redirect" 2024-09-16T00:35:21.894 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:21.894 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.894 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.894 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.894 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.894 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.894 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_error_status_code": { 2024-09-16T00:35:21.894 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_error_status_code", 2024-09-16T00:35:21.894 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:21.894 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.894 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.894 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "500", 2024-09-16T00:35:21.894 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "400", 2024-09-16T00:35:21.895 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "599", 2024-09-16T00:35:21.895 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.895 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.895 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.895 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.895 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.895 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.895 INFO:teuthology.orchestra.run.smithi038.stdout: "url_prefix": { 2024-09-16T00:35:21.895 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "url_prefix", 2024-09-16T00:35:21.895 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.895 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.895 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.895 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.895 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.896 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.896 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.896 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.896 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.896 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.896 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.896 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:21.896 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:21.896 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.896 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:21.896 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "devicehealth", 2024-09-16T00:35:21.896 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:21.896 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:21.896 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:21.897 INFO:teuthology.orchestra.run.smithi038.stdout: "enable_monitoring": { 2024-09-16T00:35:21.897 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enable_monitoring", 2024-09-16T00:35:21.897 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.897 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.897 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.897 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:21.897 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.897 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.897 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.897 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "monitor device health metrics", 2024-09-16T00:35:21.897 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.897 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.897 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.897 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.898 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:21.898 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:21.898 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.898 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.898 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.898 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.898 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.898 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.898 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:21.898 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:21.898 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:21.898 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:21.898 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:21.898 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:21.898 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:21.899 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:21.899 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.899 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.899 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.899 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.899 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.899 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:21.899 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:21.899 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.899 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.899 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.899 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.899 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.899 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.900 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.900 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.900 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.900 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.900 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.900 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.900 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:21.900 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:21.900 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.900 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.900 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.900 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:21.900 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.900 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.901 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:21.901 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:21.901 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:21.901 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:21.901 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:21.901 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:21.901 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:21.901 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:21.901 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.901 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.901 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.901 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.901 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.901 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:21.902 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:21.902 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.902 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.902 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.902 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.902 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.902 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.902 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.902 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.902 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.902 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.902 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.902 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.902 INFO:teuthology.orchestra.run.smithi038.stdout: "mark_out_threshold": { 2024-09-16T00:35:21.903 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mark_out_threshold", 2024-09-16T00:35:21.903 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:21.903 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.903 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.903 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "2419200", 2024-09-16T00:35:21.903 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.903 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.903 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.903 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-16T00:35:21.903 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.903 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.903 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.903 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.903 INFO:teuthology.orchestra.run.smithi038.stdout: "pool_name": { 2024-09-16T00:35:21.903 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pool_name", 2024-09-16T00:35:21.904 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.904 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.904 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.904 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "device_health_metrics", 2024-09-16T00:35:21.904 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.904 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.904 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.904 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-16T00:35:21.904 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.904 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.904 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.904 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.904 INFO:teuthology.orchestra.run.smithi038.stdout: "retention_period": { 2024-09-16T00:35:21.904 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "retention_period", 2024-09-16T00:35:21.905 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:21.905 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.905 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.905 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "15552000", 2024-09-16T00:35:21.905 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.905 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.905 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.905 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how long to retain device health metrics", 2024-09-16T00:35:21.905 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.905 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.905 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.905 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.905 INFO:teuthology.orchestra.run.smithi038.stdout: "scrape_frequency": { 2024-09-16T00:35:21.905 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "scrape_frequency", 2024-09-16T00:35:21.905 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:21.906 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.906 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.906 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "86400", 2024-09-16T00:35:21.906 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.906 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.906 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.906 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-16T00:35:21.906 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.906 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.906 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.906 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.906 INFO:teuthology.orchestra.run.smithi038.stdout: "self_heal": { 2024-09-16T00:35:21.906 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "self_heal", 2024-09-16T00:35:21.907 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.907 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.907 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.907 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:21.907 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.907 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.907 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.907 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-16T00:35:21.907 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.907 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.907 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.907 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.907 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-09-16T00:35:21.907 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-09-16T00:35:21.908 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:21.908 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.908 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.908 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-09-16T00:35:21.908 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.908 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.908 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.908 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to wake up and check device health", 2024-09-16T00:35:21.908 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.908 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.908 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.908 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.908 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_threshold": { 2024-09-16T00:35:21.908 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_threshold", 2024-09-16T00:35:21.909 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:21.909 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.909 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.909 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "7257600", 2024-09-16T00:35:21.909 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.909 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.909 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.909 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-16T00:35:21.909 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.909 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.909 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.909 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:21.909 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:21.909 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.910 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:21.910 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "diskprediction_local", 2024-09-16T00:35:21.910 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:21.910 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:21.910 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:21.910 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:21.910 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:21.910 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.910 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.911 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.911 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.911 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.911 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.911 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:21.911 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:21.911 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:21.911 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:21.911 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:21.911 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:21.911 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:21.911 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:21.911 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.912 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.912 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.912 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.912 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.912 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:21.912 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:21.912 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.912 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.912 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.912 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.912 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.913 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.913 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.913 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.913 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.913 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.913 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.913 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.913 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:21.913 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:21.913 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.913 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.913 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.913 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:21.914 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.914 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.914 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:21.914 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:21.914 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:21.914 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:21.914 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:21.914 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:21.914 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:21.914 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:21.914 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.915 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.915 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.915 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.915 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.915 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:21.915 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:21.915 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.915 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.915 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.915 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.915 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.915 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.915 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.916 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.916 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.916 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.916 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.916 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.916 INFO:teuthology.orchestra.run.smithi038.stdout: "predict_interval": { 2024-09-16T00:35:21.916 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "predict_interval", 2024-09-16T00:35:21.916 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.917 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.917 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.917 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "86400", 2024-09-16T00:35:21.917 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.917 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.917 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.917 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.917 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.917 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.917 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.917 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.917 INFO:teuthology.orchestra.run.smithi038.stdout: "predictor_model": { 2024-09-16T00:35:21.918 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "predictor_model", 2024-09-16T00:35:21.918 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.918 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.918 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.918 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "prophetstor", 2024-09-16T00:35:21.918 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.918 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.918 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.918 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.919 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.919 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.919 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.919 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.919 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-09-16T00:35:21.919 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-09-16T00:35:21.919 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.919 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.919 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.919 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-09-16T00:35:21.919 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.920 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.920 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.920 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.920 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.920 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.920 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.920 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:21.920 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:21.920 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.920 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:21.921 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "influx", 2024-09-16T00:35:21.921 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": false, 2024-09-16T00:35:21.921 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "influxdb python module not found", 2024-09-16T00:35:21.921 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:21.921 INFO:teuthology.orchestra.run.smithi038.stdout: "batch_size": { 2024-09-16T00:35:21.921 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "batch_size", 2024-09-16T00:35:21.921 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:21.921 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.921 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.922 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5000", 2024-09-16T00:35:21.922 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.922 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.922 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.922 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-16T00:35:21.922 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.922 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.922 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.922 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.922 INFO:teuthology.orchestra.run.smithi038.stdout: "database": { 2024-09-16T00:35:21.922 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "database", 2024-09-16T00:35:21.922 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.922 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.923 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.923 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "ceph", 2024-09-16T00:35:21.923 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.923 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.923 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.923 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-16T00:35:21.923 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.923 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.923 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.923 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.923 INFO:teuthology.orchestra.run.smithi038.stdout: "hostname": { 2024-09-16T00:35:21.923 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "hostname", 2024-09-16T00:35:21.923 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.923 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.924 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.924 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.924 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.924 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.924 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.924 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "InfluxDB server hostname", 2024-09-16T00:35:21.924 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.924 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.925 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.925 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.925 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-09-16T00:35:21.925 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-09-16T00:35:21.925 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:21.925 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.925 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.925 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "30", 2024-09-16T00:35:21.925 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "5", 2024-09-16T00:35:21.925 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.925 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.925 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-16T00:35:21.925 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.926 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.926 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.926 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.926 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:21.926 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:21.926 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.926 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.926 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.926 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.926 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.926 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.926 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:21.926 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:21.926 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:21.926 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:21.927 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:21.927 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:21.927 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:21.927 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:21.927 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.927 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.927 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.927 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.927 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.927 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:21.927 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:21.927 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.927 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.927 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.928 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.928 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.928 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.928 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.928 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.928 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.928 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.928 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.928 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.928 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:21.928 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:21.928 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.928 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.929 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.929 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:21.929 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.929 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.929 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:21.929 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:21.929 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:21.929 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:21.929 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:21.929 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:21.929 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:21.929 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:21.929 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.929 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.929 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.930 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.930 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.930 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:21.930 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:21.930 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.930 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.930 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.930 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.930 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.930 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.930 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.930 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.930 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.930 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.931 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.931 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.931 INFO:teuthology.orchestra.run.smithi038.stdout: "password": { 2024-09-16T00:35:21.931 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "password", 2024-09-16T00:35:21.931 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.931 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.931 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.931 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.931 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.931 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.931 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.931 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "password of InfluxDB server user", 2024-09-16T00:35:21.931 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.931 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.931 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.932 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.932 INFO:teuthology.orchestra.run.smithi038.stdout: "port": { 2024-09-16T00:35:21.932 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "port", 2024-09-16T00:35:21.932 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:21.932 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.932 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.932 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8086", 2024-09-16T00:35:21.932 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.932 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.932 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.932 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "InfluxDB server port", 2024-09-16T00:35:21.932 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.932 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.933 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.933 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.933 INFO:teuthology.orchestra.run.smithi038.stdout: "ssl": { 2024-09-16T00:35:21.933 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssl", 2024-09-16T00:35:21.933 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.933 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.933 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.933 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "false", 2024-09-16T00:35:21.933 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.933 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.933 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.934 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T00:35:21.934 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.934 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.934 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.934 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.934 INFO:teuthology.orchestra.run.smithi038.stdout: "threads": { 2024-09-16T00:35:21.934 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "threads", 2024-09-16T00:35:21.934 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:21.934 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.934 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.934 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-09-16T00:35:21.934 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-09-16T00:35:21.934 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "32", 2024-09-16T00:35:21.934 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.935 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-16T00:35:21.935 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.935 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.935 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.935 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.935 INFO:teuthology.orchestra.run.smithi038.stdout: "username": { 2024-09-16T00:35:21.935 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "username", 2024-09-16T00:35:21.935 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.935 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.935 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.935 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.935 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.935 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.936 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.936 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "username of InfluxDB server user", 2024-09-16T00:35:21.936 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.936 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.936 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.936 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.936 INFO:teuthology.orchestra.run.smithi038.stdout: "verify_ssl": { 2024-09-16T00:35:21.936 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "verify_ssl", 2024-09-16T00:35:21.936 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.936 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.936 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.936 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "true", 2024-09-16T00:35:21.936 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.937 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.937 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.937 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T00:35:21.937 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.937 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.937 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.937 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:21.937 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:21.937 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.937 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:21.937 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "insights", 2024-09-16T00:35:21.937 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:21.937 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:21.937 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:21.938 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:21.938 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:21.938 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.938 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.938 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.938 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.938 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.938 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.938 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:21.938 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:21.938 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:21.938 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:21.938 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:21.938 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:21.939 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:21.939 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:21.939 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.939 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.939 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.939 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.939 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.939 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:21.939 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:21.939 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.939 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.939 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.939 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.939 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.940 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.940 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.940 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.940 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.940 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.940 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.940 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.940 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:21.940 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:21.940 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.940 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.940 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.940 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:21.940 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.941 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.941 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:21.941 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:21.941 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:21.941 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:21.941 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:21.941 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:21.941 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:21.941 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:21.941 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.941 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.941 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.941 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.941 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.942 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:21.942 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:21.942 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.942 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.942 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.942 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.942 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.942 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.942 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.942 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.942 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.942 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.943 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.943 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:21.943 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:21.943 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.943 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:21.943 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "iostat", 2024-09-16T00:35:21.943 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:21.943 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:21.943 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:21.943 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:21.943 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:21.943 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.943 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.943 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.943 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.944 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.944 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.944 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:21.944 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:21.944 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:21.944 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:21.944 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:21.944 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:21.944 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:21.944 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:21.944 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.944 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.944 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.944 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.945 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.945 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:21.945 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:21.945 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.945 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.945 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.945 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.945 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.945 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.945 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.945 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.945 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.945 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.948 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.948 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.948 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:21.948 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:21.948 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.948 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.948 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.948 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:21.948 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.948 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.948 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:21.948 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:21.948 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:21.949 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:21.949 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:21.949 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:21.949 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:21.949 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:21.949 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.949 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.949 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.949 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.949 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.949 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:21.949 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:21.950 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.950 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.950 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.950 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.950 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.950 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.950 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.950 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.950 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.950 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.950 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.951 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:21.951 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:21.951 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.951 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:21.951 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "k8sevents", 2024-09-16T00:35:21.951 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:21.951 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:21.951 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:21.951 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_event_retention_days": { 2024-09-16T00:35:21.951 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ceph_event_retention_days", 2024-09-16T00:35:21.951 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:21.951 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.951 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.952 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "7", 2024-09-16T00:35:21.952 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.952 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.952 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.952 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-16T00:35:21.952 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.952 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.952 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.952 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.952 INFO:teuthology.orchestra.run.smithi038.stdout: "config_check_secs": { 2024-09-16T00:35:21.952 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "config_check_secs", 2024-09-16T00:35:21.953 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:21.953 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.953 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.953 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-16T00:35:21.953 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "10", 2024-09-16T00:35:21.953 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.953 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.953 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-16T00:35:21.953 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.953 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.954 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.954 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.954 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:21.954 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:21.954 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.954 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.954 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.954 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.954 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.954 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.954 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:21.955 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:21.955 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:21.955 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:21.955 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:21.955 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:21.955 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:21.955 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:21.955 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.955 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.955 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.956 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.956 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.956 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:21.956 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:21.956 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.956 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.956 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.956 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.956 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.956 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.956 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.956 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.957 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.957 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.957 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.957 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.957 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:21.957 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:21.957 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.957 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.958 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.958 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:21.958 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.958 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.958 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:21.958 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:21.958 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:21.958 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:21.958 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:21.958 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:21.959 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:21.959 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:21.959 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.959 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.959 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.959 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.959 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.960 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:21.960 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:21.960 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.960 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.960 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.960 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.960 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.960 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.960 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.960 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.960 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.960 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.960 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.961 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:21.961 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:21.961 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.961 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:21.961 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "localpool", 2024-09-16T00:35:21.961 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:21.961 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:21.961 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:21.961 INFO:teuthology.orchestra.run.smithi038.stdout: "failure_domain": { 2024-09-16T00:35:21.961 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "failure_domain", 2024-09-16T00:35:21.961 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.961 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.961 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.961 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "host", 2024-09-16T00:35:21.962 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.962 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.962 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.962 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "failure domain for any created local pool", 2024-09-16T00:35:21.962 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-16T00:35:21.962 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.962 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.962 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.962 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:21.962 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:21.962 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.962 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.962 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.962 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.963 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.963 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.963 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:21.963 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:21.963 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:21.963 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:21.963 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:21.963 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:21.963 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:21.963 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:21.963 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.963 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.964 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.964 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.964 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.964 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:21.964 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:21.964 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.964 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.964 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.964 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.964 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.964 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.964 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.964 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.964 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.965 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.965 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.965 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.965 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:21.965 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:21.965 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.965 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.965 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.965 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:21.965 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.965 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.965 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:21.965 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:21.965 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:21.965 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:21.966 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:21.966 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:21.966 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:21.966 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:21.966 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.966 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.966 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.966 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.966 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.966 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:21.966 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:21.966 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.966 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.966 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.967 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.967 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.967 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.967 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.967 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.967 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.967 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.967 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.967 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.967 INFO:teuthology.orchestra.run.smithi038.stdout: "min_size": { 2024-09-16T00:35:21.967 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "min_size", 2024-09-16T00:35:21.967 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:21.967 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.967 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.968 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.968 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.968 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.968 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.968 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "default min_size for any created local pool", 2024-09-16T00:35:21.968 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-16T00:35:21.968 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.968 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.968 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.968 INFO:teuthology.orchestra.run.smithi038.stdout: "num_rep": { 2024-09-16T00:35:21.968 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "num_rep", 2024-09-16T00:35:21.968 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:21.968 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.969 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.969 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "3", 2024-09-16T00:35:21.969 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.969 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.969 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.969 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "default replica count for any created local pool", 2024-09-16T00:35:21.969 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.969 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.969 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.969 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.969 INFO:teuthology.orchestra.run.smithi038.stdout: "pg_num": { 2024-09-16T00:35:21.969 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pg_num", 2024-09-16T00:35:21.970 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:21.970 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.970 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.970 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "128", 2024-09-16T00:35:21.970 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.970 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.970 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.970 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "default pg_num for any created local pool", 2024-09-16T00:35:21.970 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.970 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.970 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.970 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.970 INFO:teuthology.orchestra.run.smithi038.stdout: "prefix": { 2024-09-16T00:35:21.970 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "prefix", 2024-09-16T00:35:21.971 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.971 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.971 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.971 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.971 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.971 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.971 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.971 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "name prefix for any created local pool", 2024-09-16T00:35:21.971 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.971 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.971 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.971 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.971 INFO:teuthology.orchestra.run.smithi038.stdout: "subtree": { 2024-09-16T00:35:21.971 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "subtree", 2024-09-16T00:35:21.972 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.972 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.972 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.972 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "rack", 2024-09-16T00:35:21.972 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.972 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.972 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.972 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-16T00:35:21.972 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-16T00:35:21.972 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.972 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.972 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:21.972 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:21.973 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.973 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:21.973 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mds_autoscaler", 2024-09-16T00:35:21.973 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:21.973 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:21.973 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:21.973 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:21.973 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:21.973 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.973 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.973 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.973 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.973 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.973 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.974 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:21.974 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:21.974 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:21.974 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:21.974 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:21.974 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:21.974 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:21.974 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:21.974 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.974 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.974 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.974 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.974 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.974 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:21.974 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:21.975 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.975 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.975 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.975 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.975 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.975 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.975 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.975 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.975 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.975 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.975 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.975 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.975 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:21.975 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:21.976 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.976 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.976 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.976 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:21.976 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.976 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.976 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:21.976 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:21.976 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:21.976 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:21.976 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:21.976 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:21.976 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:21.976 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:21.977 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.977 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.977 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.977 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.977 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.977 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:21.977 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:21.977 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.977 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.977 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.977 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.977 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.977 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.977 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.978 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.978 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.978 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.978 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.978 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:21.978 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:21.978 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.978 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:21.978 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mirroring", 2024-09-16T00:35:21.978 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:21.978 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:21.978 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:21.978 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:21.978 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:21.979 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.979 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.979 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.979 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.979 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.979 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.979 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:21.979 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:21.979 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:21.979 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:21.979 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:21.979 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:21.979 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:21.979 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:21.980 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.980 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.980 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.980 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.980 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.980 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:21.980 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:21.980 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.980 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.980 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.980 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.980 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.980 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.981 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.981 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.981 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.981 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.981 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.981 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.981 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:21.981 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:21.981 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.982 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.982 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.982 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:21.982 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.982 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.982 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:21.982 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:21.982 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:21.982 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:21.982 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:21.982 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:21.982 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:21.982 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:21.982 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.983 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.983 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.983 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.983 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.983 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:21.983 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:21.983 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.983 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.983 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.984 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.984 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.984 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.984 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.984 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.984 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.984 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.984 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.984 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:21.984 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:21.984 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.984 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:21.984 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "nfs", 2024-09-16T00:35:21.984 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:21.985 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:21.985 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:21.985 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:21.985 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:21.985 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.985 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.985 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.985 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.985 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.985 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.986 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:21.986 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:21.986 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:21.986 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:21.986 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:21.986 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:21.986 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:21.986 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:21.986 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.986 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.986 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.986 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.986 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.987 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:21.987 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:21.987 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.987 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.987 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.987 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.987 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.987 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.987 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.987 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.988 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.988 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.988 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.988 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.988 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:21.988 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:21.988 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.988 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.988 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.988 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:21.988 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.988 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.989 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:21.989 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:21.989 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:21.989 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:21.989 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:21.989 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:21.989 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:21.989 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:21.989 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.989 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.990 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.990 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.990 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.990 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:21.990 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:21.990 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.990 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.990 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.990 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.990 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.990 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.991 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.991 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.991 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.991 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.991 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.991 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:21.991 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:21.991 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.991 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:21.992 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "orchestrator", 2024-09-16T00:35:21.992 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:21.992 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:21.992 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:21.992 INFO:teuthology.orchestra.run.smithi038.stdout: "fail_fs": { 2024-09-16T00:35:21.992 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "fail_fs", 2024-09-16T00:35:21.992 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.993 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.993 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:21.993 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.993 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.993 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.993 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.993 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-16T00:35:21.993 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.993 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.993 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.993 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.993 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:21.994 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:21.994 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.994 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.994 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.994 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.994 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.994 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.994 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:21.994 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:21.994 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:21.994 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:21.994 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:21.994 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:21.994 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:21.995 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:21.995 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.995 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.995 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.995 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.995 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.995 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:21.995 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:21.995 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.995 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.995 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.995 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.995 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.995 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.996 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.996 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.996 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.996 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.996 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.996 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.996 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:21.996 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:21.996 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.996 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.996 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.996 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:21.996 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.996 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.997 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:21.997 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:21.997 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:21.997 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:21.997 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:21.997 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:21.997 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:21.997 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:21.997 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.997 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.997 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.997 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.997 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.997 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:21.998 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:21.998 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:21.998 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.998 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.998 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:21.998 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.998 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.998 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:21.998 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:21.998 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.998 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:21.998 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:21.998 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:21.998 INFO:teuthology.orchestra.run.smithi038.stdout: "orchestrator": { 2024-09-16T00:35:21.999 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "orchestrator", 2024-09-16T00:35:21.999 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:21.999 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:21.999 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:21.999 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:21.999 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:21.999 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:21.999 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:21.999 INFO:teuthology.orchestra.run.smithi038.stdout: "cephadm", 2024-09-16T00:35:21.999 INFO:teuthology.orchestra.run.smithi038.stdout: "rook", 2024-09-16T00:35:21.999 INFO:teuthology.orchestra.run.smithi038.stdout: "test_orchestrator" 2024-09-16T00:35:21.999 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:21.999 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Orchestrator backend", 2024-09-16T00:35:21.999 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:21.999 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.000 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.000 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.000 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.000 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.000 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.000 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "osd_perf_query", 2024-09-16T00:35:22.000 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.000 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.000 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.000 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.000 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.000 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.001 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.001 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.001 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.001 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.001 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.001 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.001 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.001 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.001 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.001 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.001 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.001 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.001 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.001 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.002 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.002 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.002 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.002 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.002 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.002 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.002 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.002 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.002 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.002 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.002 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.002 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.002 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.003 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.003 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.003 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.003 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.003 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.003 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.003 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.003 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.003 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.003 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.003 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.003 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.003 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.003 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.004 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.004 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.004 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.004 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.004 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.004 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.004 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.004 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.004 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.004 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.004 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.004 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.004 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.004 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.005 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.005 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.005 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.005 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.005 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.005 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.005 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.005 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.005 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.005 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.005 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.005 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.005 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.006 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.006 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.006 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "osd_support", 2024-09-16T00:35:22.006 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.006 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.006 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.006 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.006 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.006 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.006 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.006 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.006 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.006 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.006 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.007 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.007 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.007 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.007 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.007 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.007 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.007 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.007 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.007 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.007 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.007 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.007 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.007 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.007 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.007 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.008 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.008 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.008 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.008 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.008 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.008 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.008 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.008 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.008 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.008 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.008 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.008 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.008 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.008 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.009 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.009 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.009 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.009 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.009 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.009 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.009 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.009 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.009 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.009 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.009 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.009 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.009 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.009 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.010 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.010 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.010 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.010 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.010 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.010 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.010 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.010 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.010 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.010 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.010 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.010 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.010 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.010 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.011 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.011 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.011 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.011 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.011 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.011 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.011 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.011 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.011 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pg_autoscaler", 2024-09-16T00:35:22.011 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.011 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.011 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.011 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.011 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.012 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.012 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.012 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.012 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.012 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.012 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.012 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.012 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.012 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.012 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.012 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.012 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.012 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.012 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.012 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.013 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.013 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.013 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.013 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.013 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.013 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.013 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.013 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.013 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.013 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.013 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.013 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.013 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.014 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.014 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.014 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.014 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.014 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.014 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.014 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.014 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.014 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.014 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.014 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.014 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.015 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.015 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.015 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.015 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.015 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.015 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.015 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.015 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.015 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.015 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.016 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.016 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.016 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.016 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.016 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.016 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.016 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.016 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.016 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.016 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.016 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.016 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.016 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.017 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.017 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.017 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.017 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.017 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.017 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-09-16T00:35:22.017 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-09-16T00:35:22.017 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:22.017 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.017 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.017 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-09-16T00:35:22.017 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.017 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.017 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.018 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.018 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.018 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.018 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.018 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.018 INFO:teuthology.orchestra.run.smithi038.stdout: "threshold": { 2024-09-16T00:35:22.018 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "threshold", 2024-09-16T00:35:22.018 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-16T00:35:22.018 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.018 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.018 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "3.0", 2024-09-16T00:35:22.018 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1.0", 2024-09-16T00:35:22.018 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.018 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.018 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "scaling threshold", 2024-09-16T00:35:22.019 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-16T00:35:22.019 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.019 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.019 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.019 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.019 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.019 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.019 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "progress", 2024-09-16T00:35:22.019 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.019 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.019 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.019 INFO:teuthology.orchestra.run.smithi038.stdout: "allow_pg_recovery_event": { 2024-09-16T00:35:22.019 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "allow_pg_recovery_event", 2024-09-16T00:35:22.019 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.020 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.020 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.020 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.020 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.020 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.020 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.020 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-16T00:35:22.020 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.020 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.021 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.021 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.021 INFO:teuthology.orchestra.run.smithi038.stdout: "enabled": { 2024-09-16T00:35:22.021 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enabled", 2024-09-16T00:35:22.021 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.021 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.021 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.021 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.021 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.021 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.021 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.021 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.021 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.021 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.022 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.022 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.022 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.022 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.022 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.022 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.022 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.022 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.022 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.023 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.023 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.023 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.023 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.023 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.023 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.023 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.023 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.023 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.023 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.023 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.023 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.023 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.023 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.024 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.024 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.024 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.024 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.024 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.024 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.024 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.024 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.024 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.025 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.025 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.025 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.025 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.025 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.025 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.025 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.025 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.025 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.025 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.025 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.025 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.026 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.026 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.026 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.026 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.026 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.026 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.026 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.026 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.026 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.026 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.027 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.027 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.027 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.027 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.027 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.027 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.027 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.027 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.027 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.027 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.027 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.028 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.028 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.028 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.028 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.028 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.028 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.028 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.028 INFO:teuthology.orchestra.run.smithi038.stdout: "max_completed_events": { 2024-09-16T00:35:22.028 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_completed_events", 2024-09-16T00:35:22.029 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.029 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.029 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.029 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "50", 2024-09-16T00:35:22.029 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.029 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.029 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.029 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "number of past completed events to remember", 2024-09-16T00:35:22.029 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.030 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.030 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.030 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.030 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-09-16T00:35:22.030 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-09-16T00:35:22.030 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:22.030 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.030 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.030 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-09-16T00:35:22.030 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.030 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.030 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.031 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how long the module is going to sleep", 2024-09-16T00:35:22.031 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.031 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.031 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.031 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.031 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.031 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.031 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.031 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "prometheus", 2024-09-16T00:35:22.031 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.031 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.031 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.031 INFO:teuthology.orchestra.run.smithi038.stdout: "cache": { 2024-09-16T00:35:22.031 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cache", 2024-09-16T00:35:22.032 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.032 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.032 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.032 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.032 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.032 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.032 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.032 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.032 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.032 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.032 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.032 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.032 INFO:teuthology.orchestra.run.smithi038.stdout: "exclude_perf_counters": { 2024-09-16T00:35:22.032 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "exclude_perf_counters", 2024-09-16T00:35:22.033 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.033 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.033 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.033 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.033 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.033 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.033 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.033 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-16T00:35:22.033 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-16T00:35:22.033 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.033 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.034 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.034 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.034 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.034 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.034 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.034 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.034 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.034 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.034 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.034 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.034 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.034 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.035 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.035 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.035 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.035 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.035 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.035 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.035 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.035 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.035 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.035 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.035 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.035 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.035 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.036 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.036 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.036 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.036 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.036 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.036 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.036 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.036 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.036 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.036 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.036 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.036 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.036 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.036 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.037 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.037 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.037 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.037 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.037 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.037 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.037 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.037 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.037 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.037 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.037 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.037 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.037 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.037 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.038 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.038 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.038 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.038 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.038 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.038 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.038 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.038 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.038 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.038 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.038 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.038 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.038 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.038 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.038 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.039 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.039 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.039 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.039 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_stats_pools": { 2024-09-16T00:35:22.039 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rbd_stats_pools", 2024-09-16T00:35:22.039 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.039 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.039 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.039 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.039 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.039 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.039 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.039 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.039 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.040 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.040 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.040 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.040 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-16T00:35:22.040 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-16T00:35:22.040 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.040 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.040 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.040 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "300", 2024-09-16T00:35:22.040 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.040 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.040 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.040 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.040 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.041 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.041 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.041 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.041 INFO:teuthology.orchestra.run.smithi038.stdout: "scrape_interval": { 2024-09-16T00:35:22.041 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "scrape_interval", 2024-09-16T00:35:22.041 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-16T00:35:22.041 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.041 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.041 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "15.0", 2024-09-16T00:35:22.041 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.041 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.041 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.041 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.041 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.042 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.042 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.042 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.042 INFO:teuthology.orchestra.run.smithi038.stdout: "server_addr": { 2024-09-16T00:35:22.042 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_addr", 2024-09-16T00:35:22.042 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.042 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.042 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.042 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "::", 2024-09-16T00:35:22.042 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.042 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.042 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.042 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-16T00:35:22.042 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.043 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.043 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.043 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.043 INFO:teuthology.orchestra.run.smithi038.stdout: "server_port": { 2024-09-16T00:35:22.043 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_port", 2024-09-16T00:35:22.043 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.043 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.043 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.043 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "9283", 2024-09-16T00:35:22.043 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.043 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.043 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.043 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-16T00:35:22.043 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.043 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.044 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.044 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.044 INFO:teuthology.orchestra.run.smithi038.stdout: "stale_cache_strategy": { 2024-09-16T00:35:22.044 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "stale_cache_strategy", 2024-09-16T00:35:22.044 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.044 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.044 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.044 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "log", 2024-09-16T00:35:22.044 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.044 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.044 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.044 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.045 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.045 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.045 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.045 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.045 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_behaviour": { 2024-09-16T00:35:22.045 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_behaviour", 2024-09-16T00:35:22.045 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.045 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.045 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.045 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "default", 2024-09-16T00:35:22.045 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.045 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.045 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.045 INFO:teuthology.orchestra.run.smithi038.stdout: "default", 2024-09-16T00:35:22.046 INFO:teuthology.orchestra.run.smithi038.stdout: "error" 2024-09-16T00:35:22.046 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.046 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.046 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.046 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.046 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.046 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.046 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_error_status_code": { 2024-09-16T00:35:22.046 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_error_status_code", 2024-09-16T00:35:22.046 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.046 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.046 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.046 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "500", 2024-09-16T00:35:22.046 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "400", 2024-09-16T00:35:22.046 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "599", 2024-09-16T00:35:22.047 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.047 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.047 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.047 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.047 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.047 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.047 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.047 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.047 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.047 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rbd_support", 2024-09-16T00:35:22.047 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.047 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.047 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.047 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.048 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.048 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.048 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.048 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.048 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.048 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.048 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.048 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.048 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.048 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.048 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.048 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.048 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.048 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.049 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.049 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.049 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.049 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.049 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.049 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.049 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.049 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.049 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.049 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.049 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.049 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.049 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.049 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.050 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.050 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.050 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.050 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.050 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.050 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.050 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.050 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.050 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.050 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.050 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.050 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.050 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.051 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.051 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.051 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.051 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.051 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.051 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.051 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.051 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.051 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.051 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.051 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.051 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.051 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.052 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.052 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.052 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.052 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.052 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.052 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.052 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.052 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.052 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.052 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.053 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.053 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.053 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.053 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.053 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.053 INFO:teuthology.orchestra.run.smithi038.stdout: "max_concurrent_snap_create": { 2024-09-16T00:35:22.053 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_concurrent_snap_create", 2024-09-16T00:35:22.053 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.053 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.053 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.053 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-16T00:35:22.053 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.053 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.053 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.053 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.054 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.054 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.054 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.054 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.054 INFO:teuthology.orchestra.run.smithi038.stdout: "mirror_snapshot_schedule": { 2024-09-16T00:35:22.054 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mirror_snapshot_schedule", 2024-09-16T00:35:22.054 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.054 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.054 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.054 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.054 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.055 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.055 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.055 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.055 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.055 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.055 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.055 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.055 INFO:teuthology.orchestra.run.smithi038.stdout: "trash_purge_schedule": { 2024-09-16T00:35:22.055 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "trash_purge_schedule", 2024-09-16T00:35:22.055 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.055 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.055 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.055 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.056 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.056 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.056 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.056 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.056 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.056 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.056 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.056 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.056 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.056 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.056 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.056 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "restful", 2024-09-16T00:35:22.057 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.057 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.057 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.057 INFO:teuthology.orchestra.run.smithi038.stdout: "enable_auth": { 2024-09-16T00:35:22.057 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enable_auth", 2024-09-16T00:35:22.057 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.057 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.057 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.057 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.057 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.057 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.057 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.057 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.058 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.058 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.058 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.058 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.058 INFO:teuthology.orchestra.run.smithi038.stdout: "key_file": { 2024-09-16T00:35:22.058 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "key_file", 2024-09-16T00:35:22.058 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.059 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.059 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.059 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.059 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.059 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.059 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.059 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.059 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.059 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.059 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.060 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.060 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.060 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.060 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.060 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.060 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.060 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.060 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.060 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.061 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.061 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.061 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.061 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.061 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.061 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.061 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.061 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.061 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.061 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.061 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.062 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.062 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.062 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.062 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.062 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.062 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.062 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.062 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.063 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.063 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.063 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.063 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.063 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.063 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.063 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.063 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.063 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.063 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.064 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.064 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.064 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.064 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.064 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.064 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.064 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.064 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.064 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.064 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.065 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.065 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.065 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.065 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.065 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.065 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.065 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.065 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.065 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.065 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.065 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.065 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.065 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.066 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.066 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.066 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.066 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.066 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.066 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.066 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.066 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.066 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.066 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.066 INFO:teuthology.orchestra.run.smithi038.stdout: "server_addr": { 2024-09-16T00:35:22.066 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_addr", 2024-09-16T00:35:22.066 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.066 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.067 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.067 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.067 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.067 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.067 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.067 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.067 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.067 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.067 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.067 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.067 INFO:teuthology.orchestra.run.smithi038.stdout: "server_port": { 2024-09-16T00:35:22.067 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_port", 2024-09-16T00:35:22.067 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.067 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.068 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.068 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.068 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.068 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.068 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.068 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.068 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.068 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.068 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.068 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.068 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.068 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.068 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.068 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rgw", 2024-09-16T00:35:22.069 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.069 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.069 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.069 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.069 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.069 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.069 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.069 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.069 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.069 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.069 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.069 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.069 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.069 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.070 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.070 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.070 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.070 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.070 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.070 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.070 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.070 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.070 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.070 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.070 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.070 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.070 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.070 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.070 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.071 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.071 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.071 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.071 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.071 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.071 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.071 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.071 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.071 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.071 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.071 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.071 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.071 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.071 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.072 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.072 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.072 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.072 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.072 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.072 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.072 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.072 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.072 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.072 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.072 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.072 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.072 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.073 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.073 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.073 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.073 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.073 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.073 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.073 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.073 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.073 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.073 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.073 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.073 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.073 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.073 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.074 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.074 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.074 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.074 INFO:teuthology.orchestra.run.smithi038.stdout: "secondary_zone_period_retry_limit": { 2024-09-16T00:35:22.074 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-16T00:35:22.074 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.074 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.074 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.074 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-09-16T00:35:22.074 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.074 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.074 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.074 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-16T00:35:22.074 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.074 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.075 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.075 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.075 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.075 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.075 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.075 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rook", 2024-09-16T00:35:22.075 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.075 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.075 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.075 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.075 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.075 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.076 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.076 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.076 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.076 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.076 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.076 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.076 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.076 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.076 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.076 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.076 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.076 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.076 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.077 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.077 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.077 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.077 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.077 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.077 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.077 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.077 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.077 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.077 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.077 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.077 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.077 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.077 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.078 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.078 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.078 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.078 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.078 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.078 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.078 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.078 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.078 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.078 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.078 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.078 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.078 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.078 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.079 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.079 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.079 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.079 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.079 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.079 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.079 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.079 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.079 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.079 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.079 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.079 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.079 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.079 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.080 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.080 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.080 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.080 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.080 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.080 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.080 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.080 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.080 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.080 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.080 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.080 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.080 INFO:teuthology.orchestra.run.smithi038.stdout: "storage_class": { 2024-09-16T00:35:22.081 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "storage_class", 2024-09-16T00:35:22.081 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.081 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.081 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.081 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "local", 2024-09-16T00:35:22.081 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.081 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.081 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.081 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-16T00:35:22.081 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.081 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.081 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.081 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.081 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.082 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.082 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.082 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "selftest", 2024-09-16T00:35:22.082 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.082 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.082 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.082 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.082 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.082 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.082 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.082 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.082 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.082 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.082 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.083 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.083 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.083 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.083 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.083 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.083 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.083 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.083 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.083 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.083 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.083 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.083 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.083 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.083 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.084 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.084 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.084 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.084 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.084 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.084 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.084 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.084 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.084 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.084 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.084 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.084 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.084 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.084 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.085 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.085 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.085 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.085 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.085 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.085 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.085 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.085 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.085 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.085 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.085 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.085 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.085 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.085 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.085 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.086 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.086 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.086 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.086 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.086 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.086 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.086 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.086 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.086 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.086 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.086 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.086 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.086 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.086 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.087 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.087 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.087 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.087 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.087 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.087 INFO:teuthology.orchestra.run.smithi038.stdout: "roption1": { 2024-09-16T00:35:22.087 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "roption1", 2024-09-16T00:35:22.087 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.087 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.087 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.087 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.087 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.087 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.087 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.088 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.088 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.088 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.088 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.088 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.088 INFO:teuthology.orchestra.run.smithi038.stdout: "roption2": { 2024-09-16T00:35:22.088 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "roption2", 2024-09-16T00:35:22.088 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.088 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.088 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.088 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "xyz", 2024-09-16T00:35:22.089 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.089 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.089 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.089 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.089 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.089 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.089 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.089 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.089 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption1": { 2024-09-16T00:35:22.089 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption1", 2024-09-16T00:35:22.089 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.090 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.090 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.090 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.090 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.090 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.090 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.090 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.090 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.090 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.090 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.090 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.090 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption2": { 2024-09-16T00:35:22.090 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption2", 2024-09-16T00:35:22.091 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.091 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.091 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.091 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.091 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.091 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.091 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.091 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.091 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.092 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.092 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.092 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.092 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption3": { 2024-09-16T00:35:22.092 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption3", 2024-09-16T00:35:22.092 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-16T00:35:22.092 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.092 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.092 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.092 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.092 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.092 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.092 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.093 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.093 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.093 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.093 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.093 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption4": { 2024-09-16T00:35:22.093 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption4", 2024-09-16T00:35:22.093 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.093 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.093 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.093 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.093 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.093 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.093 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.093 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.094 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.094 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.094 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.094 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.094 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption5": { 2024-09-16T00:35:22.094 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption5", 2024-09-16T00:35:22.094 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.094 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.094 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.094 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.094 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.094 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.094 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.094 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.095 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.095 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.095 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.095 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.095 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption6": { 2024-09-16T00:35:22.095 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption6", 2024-09-16T00:35:22.095 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.095 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.095 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.095 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.095 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.095 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.095 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.095 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.096 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.096 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.096 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.096 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.096 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption7": { 2024-09-16T00:35:22.096 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption7", 2024-09-16T00:35:22.096 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.096 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.096 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.097 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.097 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-09-16T00:35:22.097 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "42", 2024-09-16T00:35:22.097 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.097 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.097 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.097 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.097 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.097 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.097 INFO:teuthology.orchestra.run.smithi038.stdout: "testkey": { 2024-09-16T00:35:22.097 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "testkey", 2024-09-16T00:35:22.097 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.097 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.097 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.098 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.098 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.098 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.098 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.098 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.098 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.098 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.098 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.098 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.098 INFO:teuthology.orchestra.run.smithi038.stdout: "testlkey": { 2024-09-16T00:35:22.099 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "testlkey", 2024-09-16T00:35:22.099 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.099 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.099 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.099 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.099 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.099 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.099 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.099 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.099 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.099 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.099 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.100 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.100 INFO:teuthology.orchestra.run.smithi038.stdout: "testnewline": { 2024-09-16T00:35:22.100 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "testnewline", 2024-09-16T00:35:22.100 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.100 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.100 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.100 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.100 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.100 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.100 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.100 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.100 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.101 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.101 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.101 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.101 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.101 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.101 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.101 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "snap_schedule", 2024-09-16T00:35:22.101 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.101 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.101 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.101 INFO:teuthology.orchestra.run.smithi038.stdout: "allow_m_granularity": { 2024-09-16T00:35:22.102 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "allow_m_granularity", 2024-09-16T00:35:22.102 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.102 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.102 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.102 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.102 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.102 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.102 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.102 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "allow minute scheduled snapshots", 2024-09-16T00:35:22.102 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.103 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.103 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.103 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.103 INFO:teuthology.orchestra.run.smithi038.stdout: "dump_on_update": { 2024-09-16T00:35:22.103 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "dump_on_update", 2024-09-16T00:35:22.103 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.103 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.103 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.103 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.103 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.104 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.104 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.104 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "dump database to debug log on update", 2024-09-16T00:35:22.104 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.104 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.104 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.104 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.104 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.104 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.104 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.104 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.105 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.105 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.105 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.105 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.105 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.105 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.105 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.105 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.105 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.105 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.105 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.105 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.105 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.105 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.106 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.106 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.106 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.106 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.106 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.106 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.106 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.106 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.106 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.106 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.106 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.106 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.107 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.107 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.107 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.107 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.107 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.107 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.107 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.107 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.107 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.107 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.107 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.107 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.107 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.107 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.108 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.108 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.108 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.108 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.108 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.108 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.108 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.108 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.108 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.108 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.108 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.108 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.108 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.109 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.109 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.109 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.109 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.109 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.109 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.109 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.109 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.109 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.109 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.109 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.109 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.109 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.109 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.110 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.110 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.110 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "stats", 2024-09-16T00:35:22.110 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.110 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.110 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.110 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.110 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.110 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.110 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.110 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.110 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.110 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.110 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.111 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.111 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.111 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.111 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.111 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.111 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.111 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.111 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.111 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.111 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.111 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.111 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.111 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.111 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.112 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.112 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.112 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.112 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.112 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.112 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.112 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.112 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.112 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.112 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.112 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.112 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.112 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.112 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.113 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.113 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.113 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.113 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.113 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.113 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.113 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.113 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.113 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.113 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.113 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.113 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.113 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.113 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.113 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.114 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.114 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.114 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.114 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.114 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.114 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.114 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.114 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.114 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.114 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.114 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.114 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.114 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.114 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.115 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.115 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.115 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.115 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.115 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.115 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.115 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.115 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.115 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "status", 2024-09-16T00:35:22.115 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.115 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.115 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.115 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.115 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.115 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.116 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.116 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.116 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.116 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.116 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.116 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.116 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.116 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.116 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.116 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.116 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.116 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.116 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.116 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.117 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.117 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.117 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.117 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.117 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.117 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.117 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.117 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.117 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.117 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.117 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.117 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.117 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.117 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.118 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.118 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.118 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.118 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.118 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.118 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.118 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.118 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.118 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.118 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.118 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.118 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.118 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.118 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.119 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.119 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.119 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.119 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.119 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.119 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.119 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.119 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.119 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.119 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.119 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.119 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.119 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.119 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.119 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.120 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.120 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.120 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.120 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.120 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.120 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.120 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.120 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.120 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.120 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.120 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.120 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.121 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.121 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "telegraf", 2024-09-16T00:35:22.121 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.121 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.121 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.121 INFO:teuthology.orchestra.run.smithi038.stdout: "address": { 2024-09-16T00:35:22.121 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "address", 2024-09-16T00:35:22.121 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.121 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.121 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.121 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-16T00:35:22.121 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.121 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.121 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.122 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.122 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.122 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.122 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.122 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.122 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-09-16T00:35:22.122 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-09-16T00:35:22.122 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:22.122 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.122 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.122 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "15", 2024-09-16T00:35:22.122 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.122 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.122 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.123 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.123 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.123 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.123 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.123 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.123 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.123 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.123 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.123 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.123 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.123 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.123 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.123 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.123 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.124 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.124 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.124 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.124 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.124 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.124 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.124 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.124 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.124 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.124 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.124 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.124 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.124 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.124 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.125 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.125 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.125 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.125 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.125 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.125 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.125 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.125 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.125 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.125 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.125 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.125 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.125 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.126 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.126 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.126 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.126 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.126 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.126 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.126 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.126 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.126 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.126 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.127 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.127 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.127 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.127 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.127 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.127 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.127 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.127 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.127 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.127 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.127 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.127 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.127 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.127 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.128 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.128 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.128 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.128 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.128 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.128 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.128 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.128 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.128 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.129 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.129 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.129 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.129 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.129 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "telemetry", 2024-09-16T00:35:22.129 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.129 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.129 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.129 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_basic": { 2024-09-16T00:35:22.129 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_basic", 2024-09-16T00:35:22.129 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.129 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.129 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.130 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.130 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.130 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.130 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.130 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-16T00:35:22.130 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.130 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.130 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.130 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.130 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_crash": { 2024-09-16T00:35:22.131 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_crash", 2024-09-16T00:35:22.131 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.131 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.131 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.131 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.131 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.131 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.131 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.131 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-16T00:35:22.131 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.131 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.131 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.131 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.131 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_device": { 2024-09-16T00:35:22.132 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_device", 2024-09-16T00:35:22.132 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.132 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.132 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.132 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.132 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.132 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.132 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.132 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-16T00:35:22.132 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.132 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.133 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.133 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.133 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_ident": { 2024-09-16T00:35:22.133 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_ident", 2024-09-16T00:35:22.133 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.133 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.133 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.133 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.133 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.133 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.133 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.133 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-16T00:35:22.134 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.134 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.134 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.134 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.134 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_perf": { 2024-09-16T00:35:22.134 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_perf", 2024-09-16T00:35:22.134 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.134 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.134 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.134 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.134 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.135 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.135 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.135 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-16T00:35:22.135 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.135 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.135 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.135 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.135 INFO:teuthology.orchestra.run.smithi038.stdout: "contact": { 2024-09-16T00:35:22.135 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "contact", 2024-09-16T00:35:22.135 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.136 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.136 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.136 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.136 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.136 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.136 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.136 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.136 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.136 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.136 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.137 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.137 INFO:teuthology.orchestra.run.smithi038.stdout: "description": { 2024-09-16T00:35:22.137 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "description", 2024-09-16T00:35:22.137 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.137 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.137 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.137 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.137 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.137 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.137 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.138 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.138 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.138 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.138 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.138 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.138 INFO:teuthology.orchestra.run.smithi038.stdout: "device_url": { 2024-09-16T00:35:22.138 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "device_url", 2024-09-16T00:35:22.138 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.138 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.139 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.139 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-16T00:35:22.139 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.139 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.139 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.139 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.139 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.139 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.139 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.139 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.139 INFO:teuthology.orchestra.run.smithi038.stdout: "enabled": { 2024-09-16T00:35:22.139 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enabled", 2024-09-16T00:35:22.140 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.140 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.140 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.140 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.140 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.140 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.140 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.140 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.140 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.140 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.140 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.140 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.140 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-09-16T00:35:22.140 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-09-16T00:35:22.141 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.141 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.141 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.141 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "24", 2024-09-16T00:35:22.141 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "8", 2024-09-16T00:35:22.141 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.141 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.141 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.141 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.141 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.141 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.141 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.141 INFO:teuthology.orchestra.run.smithi038.stdout: "last_opt_revision": { 2024-09-16T00:35:22.141 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "last_opt_revision", 2024-09-16T00:35:22.142 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.142 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.142 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.142 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "1", 2024-09-16T00:35:22.142 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.142 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.142 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.142 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.142 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.142 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.142 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.142 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.142 INFO:teuthology.orchestra.run.smithi038.stdout: "leaderboard": { 2024-09-16T00:35:22.142 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "leaderboard", 2024-09-16T00:35:22.143 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.143 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.143 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.143 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.143 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.143 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.143 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.143 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.143 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.143 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.143 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.143 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.143 INFO:teuthology.orchestra.run.smithi038.stdout: "leaderboard_description": { 2024-09-16T00:35:22.143 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "leaderboard_description", 2024-09-16T00:35:22.144 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.144 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.144 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.144 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.144 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.144 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.144 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.144 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.144 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.144 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.144 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.144 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.144 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.144 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.145 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.145 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.145 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.145 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.145 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.145 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.145 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.145 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.145 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.145 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.145 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.145 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.145 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.145 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.145 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.146 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.146 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.146 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.146 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.146 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.146 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.146 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.146 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.146 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.146 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.146 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.146 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.146 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.146 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.147 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.147 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.147 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.147 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.147 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.147 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.147 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.147 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.147 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.147 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.147 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.147 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.147 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.147 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.147 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.148 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.148 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.148 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.148 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.148 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.148 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.148 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.148 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.148 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.148 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.148 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.148 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.148 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.148 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.149 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.149 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.149 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.149 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.149 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.149 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.149 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.149 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.149 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.149 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.149 INFO:teuthology.orchestra.run.smithi038.stdout: "organization": { 2024-09-16T00:35:22.149 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "organization", 2024-09-16T00:35:22.149 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.149 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.150 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.150 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.150 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.150 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.150 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.150 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.150 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.152 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.152 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.152 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.152 INFO:teuthology.orchestra.run.smithi038.stdout: "proxy": { 2024-09-16T00:35:22.152 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "proxy", 2024-09-16T00:35:22.152 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.152 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.152 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.152 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.152 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.152 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.152 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.153 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.153 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.153 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.153 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.153 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.153 INFO:teuthology.orchestra.run.smithi038.stdout: "url": { 2024-09-16T00:35:22.153 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "url", 2024-09-16T00:35:22.153 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.153 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.153 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.153 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-16T00:35:22.153 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.153 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.153 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.154 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.154 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.154 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.154 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.154 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.154 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.154 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.154 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.154 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "test_orchestrator", 2024-09-16T00:35:22.154 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.154 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.154 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.154 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.154 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.155 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.155 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.155 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.155 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.155 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.155 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.155 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.155 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.155 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.155 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.155 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.155 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.155 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.155 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.156 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.156 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.156 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.156 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.156 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.156 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.156 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.156 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.156 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.156 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.156 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.156 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.156 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.157 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.157 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.157 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.157 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.157 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.157 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.157 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.157 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.157 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.157 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.157 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.157 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.157 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.157 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.158 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.158 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.158 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.158 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.158 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.158 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.158 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.158 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.158 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.158 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.158 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.158 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.158 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.158 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.158 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.159 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.159 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.159 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.159 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.159 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.159 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.159 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.159 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.159 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.159 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.160 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.160 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.160 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.160 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.160 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.160 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "volumes", 2024-09-16T00:35:22.160 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.160 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.160 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.160 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.161 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.161 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.161 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.161 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.161 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.161 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.161 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.161 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.161 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.161 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.161 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.161 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.161 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.161 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.162 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.162 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.162 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.162 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.162 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.162 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.162 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.162 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.162 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.162 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.163 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.163 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.163 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.163 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.163 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.163 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.163 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.163 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.163 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.163 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.163 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.163 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.163 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.163 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.164 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.164 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.164 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.164 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.164 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.164 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.164 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.164 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.164 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.164 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.164 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.164 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.164 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.165 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.165 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.165 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.165 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.165 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.165 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.165 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.165 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.165 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.165 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.166 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.166 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.166 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.166 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.166 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.166 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.166 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.166 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.166 INFO:teuthology.orchestra.run.smithi038.stdout: "max_concurrent_clones": { 2024-09-16T00:35:22.166 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_concurrent_clones", 2024-09-16T00:35:22.166 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.166 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.167 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.167 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "4", 2024-09-16T00:35:22.167 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.167 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.167 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.167 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-16T00:35:22.167 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.167 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.167 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.167 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.168 INFO:teuthology.orchestra.run.smithi038.stdout: "periodic_async_work": { 2024-09-16T00:35:22.168 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "periodic_async_work", 2024-09-16T00:35:22.168 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.168 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.168 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.168 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.168 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.168 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.168 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.168 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Periodically check for async work", 2024-09-16T00:35:22.168 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.169 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.169 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.169 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.169 INFO:teuthology.orchestra.run.smithi038.stdout: "snapshot_clone_delay": { 2024-09-16T00:35:22.169 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "snapshot_clone_delay", 2024-09-16T00:35:22.169 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.169 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.169 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.169 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-09-16T00:35:22.169 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.169 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.169 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.170 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-16T00:35:22.170 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.170 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.170 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.170 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.170 INFO:teuthology.orchestra.run.smithi038.stdout: "snapshot_clone_no_wait": { 2024-09-16T00:35:22.170 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "snapshot_clone_no_wait", 2024-09-16T00:35:22.170 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.170 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.170 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.170 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.170 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.170 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.171 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.171 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-16T00:35:22.171 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.171 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.171 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.171 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.171 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.171 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.171 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.171 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix", 2024-09-16T00:35:22.171 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.171 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.171 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.172 INFO:teuthology.orchestra.run.smithi038.stdout: "discovery_interval": { 2024-09-16T00:35:22.172 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "discovery_interval", 2024-09-16T00:35:22.172 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-09-16T00:35:22.172 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.172 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.172 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "100", 2024-09-16T00:35:22.172 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.172 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.172 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.172 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.172 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.172 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.172 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.172 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.173 INFO:teuthology.orchestra.run.smithi038.stdout: "identifier": { 2024-09-16T00:35:22.173 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "identifier", 2024-09-16T00:35:22.173 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.173 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.173 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.173 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.173 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.173 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.173 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.174 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.174 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.174 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.174 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.174 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.174 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-09-16T00:35:22.174 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-09-16T00:35:22.174 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:22.174 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.175 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.175 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-09-16T00:35:22.175 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.175 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.175 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.175 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.175 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.175 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.175 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.175 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.175 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.176 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.176 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.176 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.176 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.176 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.176 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.176 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.176 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.176 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.176 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.177 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.177 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.177 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.177 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.177 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.177 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.177 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.177 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.177 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.177 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.178 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.178 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.178 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.178 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.178 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.178 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.178 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.178 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.178 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.178 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.178 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.178 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.178 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.178 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.179 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.179 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.179 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.179 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.179 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.179 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.179 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.179 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.179 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.179 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.179 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.179 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.179 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.179 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.180 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.180 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.180 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.180 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.180 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.180 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.180 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.180 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.180 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.180 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.180 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.180 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.180 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.180 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.181 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.181 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.181 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.181 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.181 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.181 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.181 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.181 INFO:teuthology.orchestra.run.smithi038.stdout: "zabbix_host": { 2024-09-16T00:35:22.181 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix_host", 2024-09-16T00:35:22.181 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.181 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.181 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.181 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.181 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.182 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.182 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.182 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.182 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.182 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.182 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.182 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.182 INFO:teuthology.orchestra.run.smithi038.stdout: "zabbix_port": { 2024-09-16T00:35:22.182 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix_port", 2024-09-16T00:35:22.182 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.182 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.182 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.182 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10051", 2024-09-16T00:35:22.182 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.182 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.183 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.183 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.183 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.183 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.183 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.183 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.183 INFO:teuthology.orchestra.run.smithi038.stdout: "zabbix_sender": { 2024-09-16T00:35:22.183 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix_sender", 2024-09-16T00:35:22.183 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.183 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.183 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.183 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-16T00:35:22.183 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.184 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.184 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.184 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.184 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.184 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.184 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.184 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.184 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.184 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.184 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-09-16T00:35:22.184 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.184 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.184 INFO:teuthology.orchestra.run.smithi038.stdout: "gid": 24110, 2024-09-16T00:35:22.184 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "y", 2024-09-16T00:35:22.185 INFO:teuthology.orchestra.run.smithi038.stdout: "mgr_features": 4540701547738038271, 2024-09-16T00:35:22.185 INFO:teuthology.orchestra.run.smithi038.stdout: "available_modules": [ 2024-09-16T00:35:22.185 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.185 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "alerts", 2024-09-16T00:35:22.185 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.185 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.185 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.185 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-09-16T00:35:22.185 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-09-16T00:35:22.185 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:22.185 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.185 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.185 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-09-16T00:35:22.185 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.186 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.186 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.186 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How frequently to reexamine health status", 2024-09-16T00:35:22.186 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.186 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.186 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.186 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.186 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.186 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.186 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.186 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.186 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.186 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.186 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.187 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.187 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.187 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.187 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.187 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.187 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.187 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.187 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.187 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.187 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.187 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.187 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.187 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.187 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.188 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.188 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.188 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.188 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.188 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.188 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.188 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.188 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.188 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.188 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.188 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.188 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.188 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.189 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.189 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.189 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.189 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.189 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.189 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.189 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.189 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.189 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.189 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.189 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.189 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.189 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.189 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.190 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.190 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.190 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.190 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.190 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.190 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.190 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.190 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.190 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.190 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.190 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.190 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.190 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.190 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.191 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.191 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.191 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.191 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.191 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.191 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.191 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.191 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.191 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_destination": { 2024-09-16T00:35:22.191 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_destination", 2024-09-16T00:35:22.191 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.191 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.191 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.191 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.191 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.192 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.192 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.192 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Email address to send alerts to", 2024-09-16T00:35:22.192 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.192 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.192 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.192 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.192 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_from_name": { 2024-09-16T00:35:22.192 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_from_name", 2024-09-16T00:35:22.192 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.192 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.192 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.192 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "Ceph", 2024-09-16T00:35:22.192 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.193 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.193 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.193 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Email From: name", 2024-09-16T00:35:22.193 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.193 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.193 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.193 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.193 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_host": { 2024-09-16T00:35:22.193 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_host", 2024-09-16T00:35:22.193 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.193 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.193 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.193 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.193 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.194 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.194 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.194 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SMTP server", 2024-09-16T00:35:22.194 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.194 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.194 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.194 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.194 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_password": { 2024-09-16T00:35:22.194 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_password", 2024-09-16T00:35:22.194 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.194 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.194 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.194 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.194 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.194 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.195 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.195 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Password to authenticate with", 2024-09-16T00:35:22.195 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.195 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.195 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.195 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.195 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_port": { 2024-09-16T00:35:22.195 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_port", 2024-09-16T00:35:22.195 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.195 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.195 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.195 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "465", 2024-09-16T00:35:22.195 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.195 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.196 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.196 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SMTP port", 2024-09-16T00:35:22.196 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.196 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.196 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.196 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.196 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_sender": { 2024-09-16T00:35:22.196 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_sender", 2024-09-16T00:35:22.196 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.196 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.196 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.196 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.196 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.196 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.196 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.197 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SMTP envelope sender", 2024-09-16T00:35:22.197 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.197 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.197 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.197 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.197 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_ssl": { 2024-09-16T00:35:22.197 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_ssl", 2024-09-16T00:35:22.197 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.197 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.197 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.197 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.197 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.197 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.198 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.198 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-16T00:35:22.198 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.198 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.198 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.198 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.198 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_user": { 2024-09-16T00:35:22.198 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_user", 2024-09-16T00:35:22.198 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.198 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.198 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.198 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.198 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.198 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.199 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.199 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "User to authenticate as", 2024-09-16T00:35:22.199 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.199 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.199 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.199 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.199 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.199 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.199 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.199 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "balancer", 2024-09-16T00:35:22.199 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.199 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.199 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.200 INFO:teuthology.orchestra.run.smithi038.stdout: "active": { 2024-09-16T00:35:22.200 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "active", 2024-09-16T00:35:22.200 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.200 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.200 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.200 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.200 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.200 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.201 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.201 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "automatically balance PGs across cluster", 2024-09-16T00:35:22.201 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.201 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.201 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.201 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.201 INFO:teuthology.orchestra.run.smithi038.stdout: "begin_time": { 2024-09-16T00:35:22.201 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "begin_time", 2024-09-16T00:35:22.201 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.201 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.201 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.201 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0000", 2024-09-16T00:35:22.201 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.202 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.202 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.202 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "beginning time of day to automatically balance", 2024-09-16T00:35:22.202 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T00:35:22.202 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.202 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.202 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.202 INFO:teuthology.orchestra.run.smithi038.stdout: "begin_weekday": { 2024-09-16T00:35:22.202 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "begin_weekday", 2024-09-16T00:35:22.202 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-09-16T00:35:22.203 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.203 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.203 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-09-16T00:35:22.203 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "0", 2024-09-16T00:35:22.203 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "6", 2024-09-16T00:35:22.203 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.203 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-16T00:35:22.203 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T00:35:22.203 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.203 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.203 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.203 INFO:teuthology.orchestra.run.smithi038.stdout: "crush_compat_max_iterations": { 2024-09-16T00:35:22.203 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crush_compat_max_iterations", 2024-09-16T00:35:22.204 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-09-16T00:35:22.204 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.204 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.204 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "25", 2024-09-16T00:35:22.204 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-09-16T00:35:22.204 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "250", 2024-09-16T00:35:22.204 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.204 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-16T00:35:22.204 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.204 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.205 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.205 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.205 INFO:teuthology.orchestra.run.smithi038.stdout: "crush_compat_metrics": { 2024-09-16T00:35:22.205 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crush_compat_metrics", 2024-09-16T00:35:22.205 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.205 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.205 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.205 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "pgs,objects,bytes", 2024-09-16T00:35:22.205 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.205 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.205 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.205 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-16T00:35:22.205 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-16T00:35:22.206 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.206 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.206 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.206 INFO:teuthology.orchestra.run.smithi038.stdout: "crush_compat_step": { 2024-09-16T00:35:22.206 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crush_compat_step", 2024-09-16T00:35:22.206 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-16T00:35:22.206 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.206 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.206 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0.5", 2024-09-16T00:35:22.206 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "0.001", 2024-09-16T00:35:22.207 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "0.999", 2024-09-16T00:35:22.207 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.207 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "aggressiveness of optimization", 2024-09-16T00:35:22.207 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-16T00:35:22.207 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.207 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.207 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.207 INFO:teuthology.orchestra.run.smithi038.stdout: "end_time": { 2024-09-16T00:35:22.207 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "end_time", 2024-09-16T00:35:22.207 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.207 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.207 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.207 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "2359", 2024-09-16T00:35:22.208 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.208 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.208 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.208 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "ending time of day to automatically balance", 2024-09-16T00:35:22.208 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T00:35:22.208 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.208 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.208 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.208 INFO:teuthology.orchestra.run.smithi038.stdout: "end_weekday": { 2024-09-16T00:35:22.208 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "end_weekday", 2024-09-16T00:35:22.209 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-09-16T00:35:22.209 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.209 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.209 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-09-16T00:35:22.209 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "0", 2024-09-16T00:35:22.209 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "6", 2024-09-16T00:35:22.209 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.209 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-16T00:35:22.209 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T00:35:22.209 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.209 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.210 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.210 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.210 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.210 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.210 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.210 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.210 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.210 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.210 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.210 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.210 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.211 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.211 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.211 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.211 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.211 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.211 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.211 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.211 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.211 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.211 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.212 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.212 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.212 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.212 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.212 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.212 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.212 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.212 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.212 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.212 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.213 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.213 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.213 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.213 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.213 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.213 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.213 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.213 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.213 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.213 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.213 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.213 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.213 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.213 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.214 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.214 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.214 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.214 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.214 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.214 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.214 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.214 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.214 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.214 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.214 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.214 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.214 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.214 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.215 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.215 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.215 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.215 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.215 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.215 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.215 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.215 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.215 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.215 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.215 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.215 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.215 INFO:teuthology.orchestra.run.smithi038.stdout: "min_score": { 2024-09-16T00:35:22.215 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "min_score", 2024-09-16T00:35:22.216 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-16T00:35:22.216 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.216 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.216 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-09-16T00:35:22.216 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.216 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.216 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.216 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-16T00:35:22.216 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.216 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.216 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.216 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.216 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": { 2024-09-16T00:35:22.217 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mode", 2024-09-16T00:35:22.217 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.217 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.217 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.217 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "upmap", 2024-09-16T00:35:22.217 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.217 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.217 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.217 INFO:teuthology.orchestra.run.smithi038.stdout: "crush-compat", 2024-09-16T00:35:22.217 INFO:teuthology.orchestra.run.smithi038.stdout: "none", 2024-09-16T00:35:22.217 INFO:teuthology.orchestra.run.smithi038.stdout: "read", 2024-09-16T00:35:22.217 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap", 2024-09-16T00:35:22.217 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap-read" 2024-09-16T00:35:22.217 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.217 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Balancer mode", 2024-09-16T00:35:22.218 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.218 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.218 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.218 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.218 INFO:teuthology.orchestra.run.smithi038.stdout: "pool_ids": { 2024-09-16T00:35:22.218 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pool_ids", 2024-09-16T00:35:22.218 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.218 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.218 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.218 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.218 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.218 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.218 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.218 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-16T00:35:22.219 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.219 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.219 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.219 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.219 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-09-16T00:35:22.219 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-09-16T00:35:22.219 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:22.219 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.219 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.219 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-09-16T00:35:22.219 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.219 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.219 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.219 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-16T00:35:22.219 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.220 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.220 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.220 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.220 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap_max_deviation": { 2024-09-16T00:35:22.220 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "upmap_max_deviation", 2024-09-16T00:35:22.220 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.220 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.220 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.220 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-09-16T00:35:22.220 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-09-16T00:35:22.220 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.220 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.220 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-16T00:35:22.220 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-16T00:35:22.221 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.221 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.221 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.221 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap_max_optimizations": { 2024-09-16T00:35:22.221 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "upmap_max_optimizations", 2024-09-16T00:35:22.221 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-09-16T00:35:22.221 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.221 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.221 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-16T00:35:22.221 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.221 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.221 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.221 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-16T00:35:22.221 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.222 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.222 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.222 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.222 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.222 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.222 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.222 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cephadm", 2024-09-16T00:35:22.222 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.222 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.222 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.222 INFO:teuthology.orchestra.run.smithi038.stdout: "agent_down_multiplier": { 2024-09-16T00:35:22.222 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "agent_down_multiplier", 2024-09-16T00:35:22.222 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-16T00:35:22.223 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.223 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.223 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "3.0", 2024-09-16T00:35:22.223 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.223 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.223 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.223 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-16T00:35:22.223 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.223 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.223 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.223 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.223 INFO:teuthology.orchestra.run.smithi038.stdout: "agent_refresh_rate": { 2024-09-16T00:35:22.223 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "agent_refresh_rate", 2024-09-16T00:35:22.223 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:22.224 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.224 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.224 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "20", 2024-09-16T00:35:22.224 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.224 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.224 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.224 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-16T00:35:22.224 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.224 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.224 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.224 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.224 INFO:teuthology.orchestra.run.smithi038.stdout: "agent_starting_port": { 2024-09-16T00:35:22.224 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "agent_starting_port", 2024-09-16T00:35:22.224 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.225 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.225 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.225 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "4721", 2024-09-16T00:35:22.225 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.225 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.225 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.225 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-16T00:35:22.225 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.225 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.225 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.225 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.225 INFO:teuthology.orchestra.run.smithi038.stdout: "allow_ptrace": { 2024-09-16T00:35:22.226 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "allow_ptrace", 2024-09-16T00:35:22.226 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.226 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.226 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.226 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.226 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.226 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.226 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.226 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-16T00:35:22.226 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-16T00:35:22.226 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.226 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.226 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.226 INFO:teuthology.orchestra.run.smithi038.stdout: "autotune_interval": { 2024-09-16T00:35:22.227 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "autotune_interval", 2024-09-16T00:35:22.227 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:22.227 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.227 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.227 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-09-16T00:35:22.227 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.227 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.227 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.227 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-16T00:35:22.227 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.227 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.227 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.227 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.227 INFO:teuthology.orchestra.run.smithi038.stdout: "autotune_memory_target_ratio": { 2024-09-16T00:35:22.228 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "autotune_memory_target_ratio", 2024-09-16T00:35:22.228 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-16T00:35:22.228 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.228 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.228 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0.7", 2024-09-16T00:35:22.228 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.228 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.228 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.228 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-16T00:35:22.228 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.228 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.228 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.228 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.228 INFO:teuthology.orchestra.run.smithi038.stdout: "cephadm_log_destination": { 2024-09-16T00:35:22.229 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cephadm_log_destination", 2024-09-16T00:35:22.229 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.229 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.229 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.229 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.229 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.229 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.229 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.229 INFO:teuthology.orchestra.run.smithi038.stdout: "file", 2024-09-16T00:35:22.229 INFO:teuthology.orchestra.run.smithi038.stdout: "file,syslog", 2024-09-16T00:35:22.229 INFO:teuthology.orchestra.run.smithi038.stdout: "syslog" 2024-09-16T00:35:22.229 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.229 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-16T00:35:22.229 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.230 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.230 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.230 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.230 INFO:teuthology.orchestra.run.smithi038.stdout: "cgroups_split": { 2024-09-16T00:35:22.230 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cgroups_split", 2024-09-16T00:35:22.230 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.230 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.230 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.230 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.230 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.230 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.230 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.230 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-16T00:35:22.230 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.230 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.231 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.231 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.231 INFO:teuthology.orchestra.run.smithi038.stdout: "config_checks_enabled": { 2024-09-16T00:35:22.231 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "config_checks_enabled", 2024-09-16T00:35:22.231 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.231 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.231 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.231 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.231 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.231 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.231 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.231 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-16T00:35:22.231 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.231 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.232 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.232 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.232 INFO:teuthology.orchestra.run.smithi038.stdout: "config_dashboard": { 2024-09-16T00:35:22.232 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "config_dashboard", 2024-09-16T00:35:22.232 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.232 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.232 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.232 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.232 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.232 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.232 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.233 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-16T00:35:22.233 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.233 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.233 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.233 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.233 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_alertmanager": { 2024-09-16T00:35:22.233 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_alertmanager", 2024-09-16T00:35:22.233 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.233 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.233 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.233 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-16T00:35:22.233 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.234 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.234 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.234 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-09-16T00:35:22.234 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.234 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.234 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.234 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.234 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_base": { 2024-09-16T00:35:22.234 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_base", 2024-09-16T00:35:22.234 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.234 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.234 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.235 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-16T00:35:22.235 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.235 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.235 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.235 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Container image name, without the tag", 2024-09-16T00:35:22.235 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.235 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.235 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.235 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.235 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_elasticsearch": { 2024-09-16T00:35:22.235 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_elasticsearch", 2024-09-16T00:35:22.236 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.236 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.236 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.236 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-16T00:35:22.236 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.236 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.236 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.236 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "elasticsearch container image", 2024-09-16T00:35:22.236 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.236 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.236 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.236 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.236 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_grafana": { 2024-09-16T00:35:22.236 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_grafana", 2024-09-16T00:35:22.237 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.237 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.237 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.237 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-16T00:35:22.237 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.237 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.237 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.237 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-09-16T00:35:22.237 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.237 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.237 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.237 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.238 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_haproxy": { 2024-09-16T00:35:22.238 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_haproxy", 2024-09-16T00:35:22.238 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.238 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.238 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.238 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-16T00:35:22.238 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.238 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.238 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.238 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "HAproxy container image", 2024-09-16T00:35:22.239 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.239 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.239 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.239 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.239 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_jaeger_agent": { 2024-09-16T00:35:22.239 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_jaeger_agent", 2024-09-16T00:35:22.239 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.239 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.239 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.239 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-16T00:35:22.239 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.239 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.239 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.240 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Jaeger agent container image", 2024-09-16T00:35:22.240 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.240 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.240 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.240 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.240 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_jaeger_collector": { 2024-09-16T00:35:22.240 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_jaeger_collector", 2024-09-16T00:35:22.240 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.240 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.240 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.241 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-16T00:35:22.241 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.241 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.241 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.241 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Jaeger collector container image", 2024-09-16T00:35:22.241 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.241 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.241 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.241 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.241 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_jaeger_query": { 2024-09-16T00:35:22.241 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_jaeger_query", 2024-09-16T00:35:22.241 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.241 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.241 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.242 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-16T00:35:22.242 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.242 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.242 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.242 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Jaeger query container image", 2024-09-16T00:35:22.242 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.242 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.242 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.242 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.243 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_keepalived": { 2024-09-16T00:35:22.243 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_keepalived", 2024-09-16T00:35:22.243 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.243 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.243 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.243 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-16T00:35:22.243 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.243 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.243 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.243 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Keepalived container image", 2024-09-16T00:35:22.243 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.243 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.243 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.244 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.244 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_loki": { 2024-09-16T00:35:22.244 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_loki", 2024-09-16T00:35:22.244 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.244 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.244 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.244 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-16T00:35:22.244 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.244 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.245 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.245 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Loki container image", 2024-09-16T00:35:22.245 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.245 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.245 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.245 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.245 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_node_exporter": { 2024-09-16T00:35:22.245 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_node_exporter", 2024-09-16T00:35:22.245 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.245 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.245 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.245 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-16T00:35:22.245 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.245 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.245 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.246 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-09-16T00:35:22.246 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.246 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.246 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.246 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.246 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_nvmeof": { 2024-09-16T00:35:22.246 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_nvmeof", 2024-09-16T00:35:22.246 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.246 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.246 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.246 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-16T00:35:22.246 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.246 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.246 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.247 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Nvme-of container image", 2024-09-16T00:35:22.247 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.247 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.247 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.247 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.247 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_prometheus": { 2024-09-16T00:35:22.247 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_prometheus", 2024-09-16T00:35:22.247 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.247 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.247 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.247 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-16T00:35:22.247 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.247 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.247 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.247 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-09-16T00:35:22.248 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.248 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.248 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.248 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.248 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_promtail": { 2024-09-16T00:35:22.248 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_promtail", 2024-09-16T00:35:22.248 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.248 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.248 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.248 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-16T00:35:22.248 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.248 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.248 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.248 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Promtail container image", 2024-09-16T00:35:22.249 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.249 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.249 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.249 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.249 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_samba": { 2024-09-16T00:35:22.249 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_samba", 2024-09-16T00:35:22.249 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.249 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.249 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.249 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-16T00:35:22.249 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.249 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.249 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.250 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Samba/SMB container image", 2024-09-16T00:35:22.250 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.250 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.250 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.250 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.250 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_snmp_gateway": { 2024-09-16T00:35:22.250 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_snmp_gateway", 2024-09-16T00:35:22.250 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.250 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.250 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.250 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-16T00:35:22.251 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.251 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.251 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.251 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SNMP Gateway container image", 2024-09-16T00:35:22.251 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.251 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.251 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.251 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.251 INFO:teuthology.orchestra.run.smithi038.stdout: "container_init": { 2024-09-16T00:35:22.252 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_init", 2024-09-16T00:35:22.252 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.252 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.252 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.252 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.252 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.252 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.252 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.252 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Run podman/docker with `--init`", 2024-09-16T00:35:22.253 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.253 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.253 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.253 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.253 INFO:teuthology.orchestra.run.smithi038.stdout: "daemon_cache_timeout": { 2024-09-16T00:35:22.253 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "daemon_cache_timeout", 2024-09-16T00:35:22.253 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:22.253 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.253 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.254 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-09-16T00:35:22.254 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.254 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.254 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.254 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-16T00:35:22.254 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.254 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.254 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.254 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.254 INFO:teuthology.orchestra.run.smithi038.stdout: "default_cephadm_command_timeout": { 2024-09-16T00:35:22.254 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "default_cephadm_command_timeout", 2024-09-16T00:35:22.254 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.254 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.254 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.255 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "900", 2024-09-16T00:35:22.255 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.255 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.255 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.255 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-16T00:35:22.255 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.255 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.255 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.255 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.255 INFO:teuthology.orchestra.run.smithi038.stdout: "default_registry": { 2024-09-16T00:35:22.255 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "default_registry", 2024-09-16T00:35:22.255 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.255 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.255 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.256 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io", 2024-09-16T00:35:22.256 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.256 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.256 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.256 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-16T00:35:22.256 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.256 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.256 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.256 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.256 INFO:teuthology.orchestra.run.smithi038.stdout: "device_cache_timeout": { 2024-09-16T00:35:22.256 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "device_cache_timeout", 2024-09-16T00:35:22.256 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:22.256 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.256 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.257 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "1800", 2024-09-16T00:35:22.257 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.257 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.257 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.257 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "seconds to cache device inventory", 2024-09-16T00:35:22.257 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.257 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.257 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.257 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.257 INFO:teuthology.orchestra.run.smithi038.stdout: "device_enhanced_scan": { 2024-09-16T00:35:22.257 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "device_enhanced_scan", 2024-09-16T00:35:22.257 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.257 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.257 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.258 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.258 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.258 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.258 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.258 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-16T00:35:22.258 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.258 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.258 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.258 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.258 INFO:teuthology.orchestra.run.smithi038.stdout: "facts_cache_timeout": { 2024-09-16T00:35:22.258 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "facts_cache_timeout", 2024-09-16T00:35:22.258 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:22.258 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.258 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.259 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-09-16T00:35:22.259 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.259 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.259 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.259 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "seconds to cache host facts data", 2024-09-16T00:35:22.259 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.259 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.259 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.259 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.259 INFO:teuthology.orchestra.run.smithi038.stdout: "grafana_dashboards_path": { 2024-09-16T00:35:22.259 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "grafana_dashboards_path", 2024-09-16T00:35:22.259 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.259 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.259 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.259 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-16T00:35:22.260 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.260 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.260 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.260 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-16T00:35:22.260 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.260 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.260 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.260 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.260 INFO:teuthology.orchestra.run.smithi038.stdout: "host_check_interval": { 2024-09-16T00:35:22.260 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "host_check_interval", 2024-09-16T00:35:22.260 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:22.260 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.260 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.260 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-09-16T00:35:22.261 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.261 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.261 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.261 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to perform a host check", 2024-09-16T00:35:22.261 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.261 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.261 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.261 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.261 INFO:teuthology.orchestra.run.smithi038.stdout: "hw_monitoring": { 2024-09-16T00:35:22.261 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "hw_monitoring", 2024-09-16T00:35:22.261 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.261 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.261 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.262 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.262 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.262 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.262 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.262 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-16T00:35:22.262 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.262 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.262 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.262 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.262 INFO:teuthology.orchestra.run.smithi038.stdout: "inventory_list_all": { 2024-09-16T00:35:22.262 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "inventory_list_all", 2024-09-16T00:35:22.262 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.262 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.262 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.263 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.263 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.263 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.263 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.263 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-16T00:35:22.263 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.263 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.263 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.263 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.263 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.263 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.263 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.263 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.263 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.264 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.264 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.264 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.264 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.264 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.264 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.264 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.264 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.264 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.264 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.264 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.264 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.264 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.264 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.265 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.265 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.265 INFO:teuthology.orchestra.run.smithi038.stdout: "log_refresh_metadata": { 2024-09-16T00:35:22.265 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_refresh_metadata", 2024-09-16T00:35:22.265 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.265 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.265 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.265 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.265 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.265 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.265 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.265 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-16T00:35:22.265 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.266 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.266 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.266 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.266 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.266 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.266 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.266 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.266 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.266 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.266 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.266 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.266 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.266 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-16T00:35:22.266 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.267 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.267 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.267 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.267 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.267 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.267 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.267 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.267 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.267 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.267 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.267 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.267 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.267 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.267 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.268 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.268 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.268 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.268 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.268 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.268 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.268 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.268 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.268 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.268 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.268 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.268 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.268 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.268 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.269 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.269 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.269 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.269 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.269 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.269 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.269 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.269 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.269 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.269 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.269 INFO:teuthology.orchestra.run.smithi038.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-16T00:35:22.269 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-16T00:35:22.269 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.269 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.270 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.270 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.270 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.270 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.270 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.270 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-16T00:35:22.270 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.270 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.270 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.270 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.270 INFO:teuthology.orchestra.run.smithi038.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-16T00:35:22.271 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-16T00:35:22.271 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.271 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.271 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.271 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "*", 2024-09-16T00:35:22.271 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.271 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.271 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.271 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-16T00:35:22.271 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.271 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.271 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.271 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.272 INFO:teuthology.orchestra.run.smithi038.stdout: "max_count_per_host": { 2024-09-16T00:35:22.272 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_count_per_host", 2024-09-16T00:35:22.272 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.272 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.272 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.272 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-16T00:35:22.272 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.272 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.272 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.272 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "max number of daemons per service per host", 2024-09-16T00:35:22.272 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.272 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.272 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.272 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.273 INFO:teuthology.orchestra.run.smithi038.stdout: "max_osd_draining_count": { 2024-09-16T00:35:22.273 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_osd_draining_count", 2024-09-16T00:35:22.273 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.273 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.273 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.273 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-16T00:35:22.273 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.273 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.273 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.273 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-16T00:35:22.273 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.273 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.273 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.273 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.274 INFO:teuthology.orchestra.run.smithi038.stdout: "migration_current": { 2024-09-16T00:35:22.274 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "migration_current", 2024-09-16T00:35:22.274 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.274 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.274 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.274 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.274 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.274 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.274 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.274 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "internal - do not modify", 2024-09-16T00:35:22.274 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.274 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.274 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.274 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.275 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": { 2024-09-16T00:35:22.275 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mode", 2024-09-16T00:35:22.275 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.275 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.275 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.275 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "root", 2024-09-16T00:35:22.275 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.275 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.275 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.275 INFO:teuthology.orchestra.run.smithi038.stdout: "cephadm-package", 2024-09-16T00:35:22.275 INFO:teuthology.orchestra.run.smithi038.stdout: "root" 2024-09-16T00:35:22.275 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.275 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "mode for remote execution of cephadm", 2024-09-16T00:35:22.276 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.276 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.276 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.276 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.276 INFO:teuthology.orchestra.run.smithi038.stdout: "oob_default_addr": { 2024-09-16T00:35:22.276 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "oob_default_addr", 2024-09-16T00:35:22.276 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.276 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.276 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.276 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "169.254.1.1", 2024-09-16T00:35:22.277 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.277 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.277 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.277 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-16T00:35:22.277 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.277 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.277 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.277 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.277 INFO:teuthology.orchestra.run.smithi038.stdout: "prometheus_alerts_path": { 2024-09-16T00:35:22.277 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "prometheus_alerts_path", 2024-09-16T00:35:22.277 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.277 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.277 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.277 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-16T00:35:22.278 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.278 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.278 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.278 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-16T00:35:22.278 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.278 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.279 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.279 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.279 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_insecure": { 2024-09-16T00:35:22.279 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_insecure", 2024-09-16T00:35:22.279 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.279 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.279 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.279 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.279 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.279 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.279 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.279 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-16T00:35:22.279 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.280 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.280 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.280 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.280 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_password": { 2024-09-16T00:35:22.280 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_password", 2024-09-16T00:35:22.280 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.280 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.280 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.280 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.280 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.280 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.280 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.280 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-16T00:35:22.281 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.281 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.281 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.281 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.281 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_url": { 2024-09-16T00:35:22.281 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_url", 2024-09-16T00:35:22.281 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.281 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.281 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.281 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.282 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.282 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.282 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.282 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-16T00:35:22.282 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.282 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.282 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.282 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.282 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_username": { 2024-09-16T00:35:22.282 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_username", 2024-09-16T00:35:22.282 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.282 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.282 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.283 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.283 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.283 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.283 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.283 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-16T00:35:22.283 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.283 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.283 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.283 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.283 INFO:teuthology.orchestra.run.smithi038.stdout: "secure_monitoring_stack": { 2024-09-16T00:35:22.284 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "secure_monitoring_stack", 2024-09-16T00:35:22.284 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.284 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.284 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.284 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.284 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.284 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.284 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.284 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-16T00:35:22.284 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.285 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.285 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.285 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.285 INFO:teuthology.orchestra.run.smithi038.stdout: "service_discovery_port": { 2024-09-16T00:35:22.285 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "service_discovery_port", 2024-09-16T00:35:22.285 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.285 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.285 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.285 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8765", 2024-09-16T00:35:22.285 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.285 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.285 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.286 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "cephadm service discovery port", 2024-09-16T00:35:22.286 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.286 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.286 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.286 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.286 INFO:teuthology.orchestra.run.smithi038.stdout: "ssh_config_file": { 2024-09-16T00:35:22.286 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssh_config_file", 2024-09-16T00:35:22.286 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.286 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.286 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.287 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.287 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.287 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.287 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.287 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-16T00:35:22.287 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.287 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.287 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.287 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.288 INFO:teuthology.orchestra.run.smithi038.stdout: "use_agent": { 2024-09-16T00:35:22.288 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "use_agent", 2024-09-16T00:35:22.288 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.288 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.288 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.288 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.288 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.288 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.288 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.288 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-16T00:35:22.288 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.289 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.289 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.289 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.289 INFO:teuthology.orchestra.run.smithi038.stdout: "use_repo_digest": { 2024-09-16T00:35:22.289 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "use_repo_digest", 2024-09-16T00:35:22.289 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.289 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.289 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.289 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.289 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.289 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.290 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.290 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-16T00:35:22.290 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.290 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.290 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.290 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.290 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_on_failed_host_check": { 2024-09-16T00:35:22.290 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_on_failed_host_check", 2024-09-16T00:35:22.290 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.290 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.290 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.290 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.290 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.290 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.291 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.291 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise a health warning if the host check fails", 2024-09-16T00:35:22.291 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.291 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.291 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.291 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.291 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_on_stray_daemons": { 2024-09-16T00:35:22.291 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_on_stray_daemons", 2024-09-16T00:35:22.291 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.291 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.291 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.291 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.291 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.291 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.292 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.292 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-16T00:35:22.292 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.292 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.292 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.292 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.292 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_on_stray_hosts": { 2024-09-16T00:35:22.292 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_on_stray_hosts", 2024-09-16T00:35:22.292 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.292 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.292 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.292 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.292 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.292 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.293 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.293 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-16T00:35:22.293 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.293 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.293 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.293 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.293 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.293 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.293 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.293 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crash", 2024-09-16T00:35:22.293 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.293 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.293 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.294 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.294 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.294 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.294 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.294 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.294 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.294 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.294 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.294 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.294 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.294 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.294 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.294 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.294 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.295 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.295 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.295 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.295 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.295 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.295 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.295 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.295 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.295 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.295 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.295 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.295 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.295 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.295 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.296 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.296 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.296 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.296 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.296 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.296 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.296 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.296 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.296 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.296 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.296 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.296 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.297 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.297 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.297 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.297 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.297 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.297 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.297 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.297 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.297 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.297 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.297 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.297 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.297 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.297 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.298 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.298 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.298 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.298 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.298 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.298 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.298 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.298 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.298 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.298 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.298 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.298 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.298 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.298 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.299 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.299 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.299 INFO:teuthology.orchestra.run.smithi038.stdout: "retain_interval": { 2024-09-16T00:35:22.299 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "retain_interval", 2024-09-16T00:35:22.299 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:22.299 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.299 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.299 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "31536000", 2024-09-16T00:35:22.299 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.299 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.299 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.299 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-16T00:35:22.299 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.299 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.300 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.300 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.300 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_recent_interval": { 2024-09-16T00:35:22.300 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_recent_interval", 2024-09-16T00:35:22.300 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:22.300 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.300 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.300 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "1209600", 2024-09-16T00:35:22.300 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.300 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.300 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.300 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-16T00:35:22.300 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.300 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.301 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.301 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.301 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.301 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.301 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.301 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "dashboard", 2024-09-16T00:35:22.301 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.301 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.301 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.301 INFO:teuthology.orchestra.run.smithi038.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-16T00:35:22.301 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-16T00:35:22.301 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.301 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.301 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.302 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-16T00:35:22.302 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.302 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.302 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.302 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.302 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.302 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.302 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.302 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.302 INFO:teuthology.orchestra.run.smithi038.stdout: "ALERTMANAGER_API_HOST": { 2024-09-16T00:35:22.302 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-16T00:35:22.302 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.302 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.302 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.302 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.303 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.303 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.303 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.303 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.303 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.303 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.303 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.303 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.303 INFO:teuthology.orchestra.run.smithi038.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-16T00:35:22.303 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-16T00:35:22.303 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.303 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.303 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.303 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.304 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.304 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.304 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.304 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.304 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.304 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.304 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.304 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.304 INFO:teuthology.orchestra.run.smithi038.stdout: "AUDIT_API_ENABLED": { 2024-09-16T00:35:22.304 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "AUDIT_API_ENABLED", 2024-09-16T00:35:22.304 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.304 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.304 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.304 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.304 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.305 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.305 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.305 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.305 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.305 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.305 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.305 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.305 INFO:teuthology.orchestra.run.smithi038.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-16T00:35:22.305 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-16T00:35:22.305 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.305 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.305 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.306 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.306 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.306 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.306 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.306 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.306 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.306 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.306 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.306 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.306 INFO:teuthology.orchestra.run.smithi038.stdout: "ENABLE_BROWSABLE_API": { 2024-09-16T00:35:22.306 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-16T00:35:22.306 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.306 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.306 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.307 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.307 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.307 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.307 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.307 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.307 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.307 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.307 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.307 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.307 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-16T00:35:22.307 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-16T00:35:22.307 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.307 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.308 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.308 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.308 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.308 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.308 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.308 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.308 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.308 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.308 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.308 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.308 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-16T00:35:22.308 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-16T00:35:22.308 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.308 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.309 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.309 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.309 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.309 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.309 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.309 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.309 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.309 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.309 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.309 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.309 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-16T00:35:22.309 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-16T00:35:22.309 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.309 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.310 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.310 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.310 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.310 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.310 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.310 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.310 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.310 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.310 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.310 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.310 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-16T00:35:22.310 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-16T00:35:22.310 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.310 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.311 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.311 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.311 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.311 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.311 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.311 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.311 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.311 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.311 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.311 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.312 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-16T00:35:22.312 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-16T00:35:22.312 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.312 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.312 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.312 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.312 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.312 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.312 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.312 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.312 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.312 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.313 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.313 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.313 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-16T00:35:22.313 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-16T00:35:22.313 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.313 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.313 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.313 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.313 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.313 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.313 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.313 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.313 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.313 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.314 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.314 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.314 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-16T00:35:22.314 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-16T00:35:22.314 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.314 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.314 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.314 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.314 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.315 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.315 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.315 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.315 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.315 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.315 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.315 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.315 INFO:teuthology.orchestra.run.smithi038.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-16T00:35:22.315 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-16T00:35:22.315 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.315 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.315 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.315 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.315 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.316 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.316 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.316 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.316 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.316 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.316 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.316 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.316 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_PASSWORD": { 2024-09-16T00:35:22.316 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-16T00:35:22.316 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.317 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.317 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.317 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "admin", 2024-09-16T00:35:22.317 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.317 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.317 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.317 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.317 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.317 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.317 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.317 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.318 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-16T00:35:22.318 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-16T00:35:22.318 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.318 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.318 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.318 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.318 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.318 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.318 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.318 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.319 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.319 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.319 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.319 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.319 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_URL": { 2024-09-16T00:35:22.319 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_URL", 2024-09-16T00:35:22.319 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.319 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.319 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.319 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.319 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.319 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.319 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.320 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.320 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.320 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.320 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.320 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.320 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_USERNAME": { 2024-09-16T00:35:22.320 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-16T00:35:22.320 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.321 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.321 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.321 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "admin", 2024-09-16T00:35:22.321 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.321 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.321 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.321 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.321 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.321 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.321 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.321 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.321 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-16T00:35:22.322 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-16T00:35:22.322 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.322 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.322 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.322 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.322 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.322 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.322 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.323 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.323 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.323 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.323 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.323 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.323 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-16T00:35:22.323 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-16T00:35:22.323 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.323 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.323 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.324 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.324 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.324 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.324 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.324 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.324 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.324 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.324 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.324 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.324 INFO:teuthology.orchestra.run.smithi038.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-16T00:35:22.324 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-16T00:35:22.324 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.325 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.325 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.325 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.325 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.325 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.325 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.325 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.325 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.325 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.325 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.325 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.325 INFO:teuthology.orchestra.run.smithi038.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-16T00:35:22.325 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-16T00:35:22.325 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.326 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.326 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.326 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.326 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.326 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.326 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.326 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.326 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.326 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.326 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.326 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.326 INFO:teuthology.orchestra.run.smithi038.stdout: "PROMETHEUS_API_HOST": { 2024-09-16T00:35:22.326 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-16T00:35:22.326 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.327 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.327 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.327 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.327 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.327 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.327 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.327 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.327 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.327 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.327 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.327 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.327 INFO:teuthology.orchestra.run.smithi038.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-16T00:35:22.327 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-16T00:35:22.328 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.328 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.328 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.328 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.328 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.328 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.328 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.328 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.328 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.328 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.328 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.328 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.328 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-16T00:35:22.329 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-16T00:35:22.329 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.329 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.329 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.329 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.329 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.329 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.329 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.329 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.329 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.329 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.330 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.330 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.330 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-16T00:35:22.330 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-16T00:35:22.330 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.330 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.330 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.330 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.330 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.331 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.331 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.331 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.331 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.331 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.331 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.331 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.331 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-16T00:35:22.331 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-16T00:35:22.331 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.331 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.331 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.331 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.332 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.332 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.332 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.332 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.332 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.332 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.332 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.332 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.332 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-16T00:35:22.332 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-16T00:35:22.332 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.332 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.333 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.333 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.333 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.333 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.333 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.333 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.333 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.333 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.333 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.333 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.333 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-16T00:35:22.333 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-16T00:35:22.333 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.334 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.334 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.334 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.334 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.334 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.334 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.334 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.334 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.334 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.334 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.334 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.334 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-16T00:35:22.334 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-16T00:35:22.334 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.334 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.335 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.335 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.335 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.335 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.335 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.335 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.335 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.335 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.335 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.335 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.335 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-16T00:35:22.335 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-16T00:35:22.335 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.336 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.336 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.336 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.336 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.336 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.336 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.336 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.336 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.336 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.336 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.336 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.336 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_ENABLED": { 2024-09-16T00:35:22.337 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-16T00:35:22.337 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.337 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.337 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.337 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.337 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.337 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.337 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.337 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.337 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.337 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.337 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.338 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.338 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-16T00:35:22.338 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-16T00:35:22.338 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.338 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.338 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.338 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-16T00:35:22.338 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.338 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.338 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.338 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.338 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.338 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.339 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.339 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.339 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-16T00:35:22.339 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-16T00:35:22.339 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.339 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.339 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.339 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-16T00:35:22.339 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.339 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.339 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.339 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.339 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.340 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.340 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.340 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.340 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-16T00:35:22.340 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-16T00:35:22.340 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.340 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.340 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.340 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8", 2024-09-16T00:35:22.340 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.340 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.340 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.340 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.341 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.341 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.341 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.341 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.341 INFO:teuthology.orchestra.run.smithi038.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-16T00:35:22.341 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-16T00:35:22.341 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.341 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.341 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.341 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "45", 2024-09-16T00:35:22.342 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.342 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.342 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.342 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.342 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.342 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.342 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.342 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.342 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_ACCESS_KEY": { 2024-09-16T00:35:22.342 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-16T00:35:22.342 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.342 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.342 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.343 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.343 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.343 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.343 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.343 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.343 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.343 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.343 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.343 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.343 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-16T00:35:22.343 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-16T00:35:22.343 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.343 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.344 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.344 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "admin", 2024-09-16T00:35:22.344 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.344 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.344 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.344 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.344 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.344 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.344 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.344 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.344 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_SECRET_KEY": { 2024-09-16T00:35:22.344 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-16T00:35:22.344 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.345 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.345 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.345 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.345 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.345 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.345 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.345 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.345 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.345 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.345 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.345 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.345 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_SSL_VERIFY": { 2024-09-16T00:35:22.345 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-16T00:35:22.346 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.346 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.346 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.346 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.346 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.346 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.346 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.346 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.346 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.346 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.346 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.346 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.347 INFO:teuthology.orchestra.run.smithi038.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-16T00:35:22.347 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-16T00:35:22.347 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.347 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.347 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.347 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-09-16T00:35:22.347 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.347 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.347 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.347 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.347 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.347 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.348 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.348 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.348 INFO:teuthology.orchestra.run.smithi038.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-16T00:35:22.348 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-16T00:35:22.348 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.348 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.348 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.348 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-16T00:35:22.348 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.348 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.348 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.348 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.348 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.348 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.349 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.349 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.349 INFO:teuthology.orchestra.run.smithi038.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-16T00:35:22.349 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-16T00:35:22.349 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.349 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.349 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.349 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-09-16T00:35:22.350 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.350 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.350 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.350 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.350 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.350 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.350 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.350 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.350 INFO:teuthology.orchestra.run.smithi038.stdout: "cross_origin_url": { 2024-09-16T00:35:22.350 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cross_origin_url", 2024-09-16T00:35:22.350 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.350 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.351 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.351 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.351 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.351 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.351 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.351 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.351 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.351 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.351 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.351 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.351 INFO:teuthology.orchestra.run.smithi038.stdout: "crt_file": { 2024-09-16T00:35:22.351 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crt_file", 2024-09-16T00:35:22.351 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.351 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.351 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.352 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.352 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.352 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.352 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.352 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.352 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.352 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.352 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.352 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.352 INFO:teuthology.orchestra.run.smithi038.stdout: "debug": { 2024-09-16T00:35:22.352 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "debug", 2024-09-16T00:35:22.353 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.353 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.353 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.353 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.353 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.353 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.353 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.353 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Enable/disable debug options", 2024-09-16T00:35:22.353 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.353 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.353 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.354 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.355 INFO:teuthology.orchestra.run.smithi038.stdout: "jwt_token_ttl": { 2024-09-16T00:35:22.356 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "jwt_token_ttl", 2024-09-16T00:35:22.356 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.356 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.356 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.356 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "28800", 2024-09-16T00:35:22.356 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.356 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.356 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.356 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.356 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.356 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.356 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.356 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.357 INFO:teuthology.orchestra.run.smithi038.stdout: "key_file": { 2024-09-16T00:35:22.357 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "key_file", 2024-09-16T00:35:22.357 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.357 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.357 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.357 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.357 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.357 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.357 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.357 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.357 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.357 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.357 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.357 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.358 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.358 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.358 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.358 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.358 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.358 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.358 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.358 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.358 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.358 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.358 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.358 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.358 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.359 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.359 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.359 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.359 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.359 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.359 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.359 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.360 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.360 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.360 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.360 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.360 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.360 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.360 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.360 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.360 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.361 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.361 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.361 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.361 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.361 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.361 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.361 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.361 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.361 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.361 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.361 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.361 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.361 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.362 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.362 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.362 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.362 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.362 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.362 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.362 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.362 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.362 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.362 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.362 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.362 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.362 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.363 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.363 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.363 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.363 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.363 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.363 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.363 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.363 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.363 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.363 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.363 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.363 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.364 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.364 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.364 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.364 INFO:teuthology.orchestra.run.smithi038.stdout: "motd": { 2024-09-16T00:35:22.364 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "motd", 2024-09-16T00:35:22.364 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.364 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.364 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.364 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.364 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.364 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.364 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.364 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "The message of the day", 2024-09-16T00:35:22.365 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.365 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.365 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.365 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.365 INFO:teuthology.orchestra.run.smithi038.stdout: "redirect_resolve_ip_addr": { 2024-09-16T00:35:22.365 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "redirect_resolve_ip_addr", 2024-09-16T00:35:22.365 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.365 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.365 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.366 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.366 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.366 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.366 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.366 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.366 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.366 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.366 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.366 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.366 INFO:teuthology.orchestra.run.smithi038.stdout: "server_addr": { 2024-09-16T00:35:22.366 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_addr", 2024-09-16T00:35:22.367 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.367 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.367 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.367 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "::", 2024-09-16T00:35:22.367 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.367 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.367 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.367 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.367 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.367 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.367 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.367 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.367 INFO:teuthology.orchestra.run.smithi038.stdout: "server_port": { 2024-09-16T00:35:22.368 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_port", 2024-09-16T00:35:22.368 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.368 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.368 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.368 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8080", 2024-09-16T00:35:22.368 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.368 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.368 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.368 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.368 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.368 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.368 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.369 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.369 INFO:teuthology.orchestra.run.smithi038.stdout: "ssl": { 2024-09-16T00:35:22.369 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssl", 2024-09-16T00:35:22.369 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.369 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.369 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.369 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.369 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.369 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.369 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.369 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.369 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.369 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.370 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.370 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.370 INFO:teuthology.orchestra.run.smithi038.stdout: "ssl_server_port": { 2024-09-16T00:35:22.370 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssl_server_port", 2024-09-16T00:35:22.370 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.370 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.370 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.370 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8443", 2024-09-16T00:35:22.370 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.370 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.371 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.371 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.371 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.371 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.371 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.371 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.371 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_behaviour": { 2024-09-16T00:35:22.371 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_behaviour", 2024-09-16T00:35:22.371 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.371 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.371 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.371 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "redirect", 2024-09-16T00:35:22.371 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.372 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.372 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.372 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.372 INFO:teuthology.orchestra.run.smithi038.stdout: "redirect" 2024-09-16T00:35:22.372 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.372 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.372 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.372 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.372 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.372 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.372 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_error_status_code": { 2024-09-16T00:35:22.372 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_error_status_code", 2024-09-16T00:35:22.373 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.373 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.373 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.373 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "500", 2024-09-16T00:35:22.373 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "400", 2024-09-16T00:35:22.373 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "599", 2024-09-16T00:35:22.373 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.373 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.373 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.373 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.373 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.373 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.373 INFO:teuthology.orchestra.run.smithi038.stdout: "url_prefix": { 2024-09-16T00:35:22.373 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "url_prefix", 2024-09-16T00:35:22.374 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.374 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.374 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.374 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.374 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.374 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.374 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.374 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.374 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.374 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.374 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.374 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.375 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.375 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.375 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.375 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "devicehealth", 2024-09-16T00:35:22.375 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.375 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.375 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.375 INFO:teuthology.orchestra.run.smithi038.stdout: "enable_monitoring": { 2024-09-16T00:35:22.376 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enable_monitoring", 2024-09-16T00:35:22.376 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.376 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.376 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.376 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.376 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.376 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.376 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.376 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "monitor device health metrics", 2024-09-16T00:35:22.376 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.376 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.376 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.376 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.377 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.377 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.377 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.377 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.377 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.377 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.377 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.377 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.377 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.378 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.378 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.378 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.378 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.378 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.378 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.378 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.378 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.378 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.378 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.378 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.378 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.379 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.379 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.379 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.379 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.379 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.379 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.379 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.379 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.379 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.379 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.379 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.380 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.380 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.380 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.380 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.380 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.380 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.380 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.380 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.380 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.380 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.380 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.380 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.380 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.381 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.381 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.381 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.381 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.381 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.381 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.381 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.381 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.381 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.381 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.381 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.381 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.381 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.381 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.382 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.382 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.382 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.382 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.382 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.382 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.382 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.382 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.382 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.382 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.382 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.382 INFO:teuthology.orchestra.run.smithi038.stdout: "mark_out_threshold": { 2024-09-16T00:35:22.382 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mark_out_threshold", 2024-09-16T00:35:22.382 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:22.383 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.383 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.383 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "2419200", 2024-09-16T00:35:22.383 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.383 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.383 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.383 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-16T00:35:22.383 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.383 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.383 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.383 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.383 INFO:teuthology.orchestra.run.smithi038.stdout: "pool_name": { 2024-09-16T00:35:22.383 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pool_name", 2024-09-16T00:35:22.384 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.384 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.384 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.384 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "device_health_metrics", 2024-09-16T00:35:22.384 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.384 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.384 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.384 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-16T00:35:22.384 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.384 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.384 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.385 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.385 INFO:teuthology.orchestra.run.smithi038.stdout: "retention_period": { 2024-09-16T00:35:22.385 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "retention_period", 2024-09-16T00:35:22.385 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:22.385 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.385 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.385 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "15552000", 2024-09-16T00:35:22.385 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.385 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.385 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.385 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how long to retain device health metrics", 2024-09-16T00:35:22.386 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.386 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.386 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.386 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.386 INFO:teuthology.orchestra.run.smithi038.stdout: "scrape_frequency": { 2024-09-16T00:35:22.386 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "scrape_frequency", 2024-09-16T00:35:22.386 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:22.386 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.386 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.386 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "86400", 2024-09-16T00:35:22.386 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.387 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.387 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.387 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-16T00:35:22.387 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.387 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.387 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.387 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.387 INFO:teuthology.orchestra.run.smithi038.stdout: "self_heal": { 2024-09-16T00:35:22.387 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "self_heal", 2024-09-16T00:35:22.387 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.388 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.388 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.388 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.388 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.388 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.388 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.388 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-16T00:35:22.388 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.388 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.388 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.388 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.389 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-09-16T00:35:22.389 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-09-16T00:35:22.389 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:22.389 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.389 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.389 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-09-16T00:35:22.389 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.389 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.389 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.389 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to wake up and check device health", 2024-09-16T00:35:22.389 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.389 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.389 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.390 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.390 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_threshold": { 2024-09-16T00:35:22.390 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_threshold", 2024-09-16T00:35:22.390 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:22.390 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.390 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.390 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "7257600", 2024-09-16T00:35:22.390 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.390 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.390 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.390 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-16T00:35:22.390 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.390 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.391 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.391 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.391 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.391 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.391 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.391 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "diskprediction_local", 2024-09-16T00:35:22.391 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.391 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.391 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.391 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.391 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.391 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.391 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.391 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.392 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.392 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.392 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.392 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.392 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.392 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.392 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.392 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.392 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.392 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.392 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.392 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.392 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.393 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.393 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.393 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.393 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.393 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.393 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.393 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.393 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.393 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.393 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.393 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.393 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.393 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.394 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.394 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.394 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.394 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.394 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.394 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.394 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.394 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.394 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.394 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.394 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.394 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.394 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.394 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.395 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.395 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.395 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.395 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.395 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.395 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.395 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.395 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.395 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.395 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.395 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.395 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.395 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.395 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.395 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.396 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.396 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.396 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.396 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.396 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.396 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.396 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.396 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.396 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.396 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.396 INFO:teuthology.orchestra.run.smithi038.stdout: "predict_interval": { 2024-09-16T00:35:22.396 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "predict_interval", 2024-09-16T00:35:22.396 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.397 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.397 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.397 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "86400", 2024-09-16T00:35:22.397 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.397 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.397 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.397 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.397 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.397 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.397 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.397 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.398 INFO:teuthology.orchestra.run.smithi038.stdout: "predictor_model": { 2024-09-16T00:35:22.398 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "predictor_model", 2024-09-16T00:35:22.398 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.398 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.398 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.398 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "prophetstor", 2024-09-16T00:35:22.398 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.398 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.398 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.398 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.398 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.398 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.399 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.399 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.399 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-09-16T00:35:22.399 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-09-16T00:35:22.399 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.399 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.399 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.399 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-09-16T00:35:22.399 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.399 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.399 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.399 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.399 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.399 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.400 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.400 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.400 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.400 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.400 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.400 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "influx", 2024-09-16T00:35:22.400 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": false, 2024-09-16T00:35:22.400 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "influxdb python module not found", 2024-09-16T00:35:22.400 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.400 INFO:teuthology.orchestra.run.smithi038.stdout: "batch_size": { 2024-09-16T00:35:22.400 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "batch_size", 2024-09-16T00:35:22.400 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.400 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.400 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.401 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5000", 2024-09-16T00:35:22.401 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.401 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.401 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.401 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-16T00:35:22.401 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.401 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.402 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.402 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.402 INFO:teuthology.orchestra.run.smithi038.stdout: "database": { 2024-09-16T00:35:22.402 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "database", 2024-09-16T00:35:22.402 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.402 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.402 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.402 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "ceph", 2024-09-16T00:35:22.402 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.402 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.402 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.402 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-16T00:35:22.403 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.403 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.403 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.403 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.403 INFO:teuthology.orchestra.run.smithi038.stdout: "hostname": { 2024-09-16T00:35:22.403 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "hostname", 2024-09-16T00:35:22.403 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.403 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.403 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.403 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.403 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.403 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.403 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.403 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "InfluxDB server hostname", 2024-09-16T00:35:22.404 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.404 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.404 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.404 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.404 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-09-16T00:35:22.404 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-09-16T00:35:22.404 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:22.404 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.404 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.404 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "30", 2024-09-16T00:35:22.405 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "5", 2024-09-16T00:35:22.405 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.405 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.405 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-16T00:35:22.405 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.405 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.405 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.405 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.405 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.405 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.405 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.405 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.405 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.406 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.406 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.406 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.406 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.406 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.406 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.406 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.406 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.406 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.406 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.406 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.406 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.406 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.406 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.407 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.407 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.407 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.407 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.407 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.407 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.407 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.407 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.407 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.408 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.408 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.408 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.408 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.408 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.408 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.408 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.408 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.408 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.408 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.408 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.408 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.408 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.409 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.409 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.409 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.409 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.409 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.409 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.409 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.409 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.409 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.409 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.410 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.410 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.410 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.410 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.410 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.410 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.410 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.410 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.410 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.410 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.410 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.410 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.411 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.411 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.411 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.411 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.411 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.411 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.411 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.411 INFO:teuthology.orchestra.run.smithi038.stdout: "password": { 2024-09-16T00:35:22.411 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "password", 2024-09-16T00:35:22.412 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.412 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.412 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.412 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.412 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.412 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.412 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.413 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "password of InfluxDB server user", 2024-09-16T00:35:22.413 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.413 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.413 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.413 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.413 INFO:teuthology.orchestra.run.smithi038.stdout: "port": { 2024-09-16T00:35:22.413 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "port", 2024-09-16T00:35:22.413 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.413 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.413 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.414 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8086", 2024-09-16T00:35:22.414 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.414 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.414 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.414 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "InfluxDB server port", 2024-09-16T00:35:22.414 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.414 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.414 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.414 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.414 INFO:teuthology.orchestra.run.smithi038.stdout: "ssl": { 2024-09-16T00:35:22.414 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssl", 2024-09-16T00:35:22.414 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.414 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.414 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.415 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "false", 2024-09-16T00:35:22.415 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.415 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.415 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.415 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T00:35:22.415 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.415 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.415 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.415 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.415 INFO:teuthology.orchestra.run.smithi038.stdout: "threads": { 2024-09-16T00:35:22.415 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "threads", 2024-09-16T00:35:22.415 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.415 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.416 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.416 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-09-16T00:35:22.416 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-09-16T00:35:22.416 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "32", 2024-09-16T00:35:22.416 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.416 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-16T00:35:22.416 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.416 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.416 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.416 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.417 INFO:teuthology.orchestra.run.smithi038.stdout: "username": { 2024-09-16T00:35:22.417 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "username", 2024-09-16T00:35:22.417 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.417 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.417 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.417 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.417 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.417 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.417 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.417 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "username of InfluxDB server user", 2024-09-16T00:35:22.417 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.418 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.418 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.418 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.418 INFO:teuthology.orchestra.run.smithi038.stdout: "verify_ssl": { 2024-09-16T00:35:22.418 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "verify_ssl", 2024-09-16T00:35:22.418 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.418 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.418 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.418 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "true", 2024-09-16T00:35:22.418 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.418 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.419 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.419 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T00:35:22.419 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.419 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.419 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.419 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.419 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.419 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.419 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.419 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "insights", 2024-09-16T00:35:22.419 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.419 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.419 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.419 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.420 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.420 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.420 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.420 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.420 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.420 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.420 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.420 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.420 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.420 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.420 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.420 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.420 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.420 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.421 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.421 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.421 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.421 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.421 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.421 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.421 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.421 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.421 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.421 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.421 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.421 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.421 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.421 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.422 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.422 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.422 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.422 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.422 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.422 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.422 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.422 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.422 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.422 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.422 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.422 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.422 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.423 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.423 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.423 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.423 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.423 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.423 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.423 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.423 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.423 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.423 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.423 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.423 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.423 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.423 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.424 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.424 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.424 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.424 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.424 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.424 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.424 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.424 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.424 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.424 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.424 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.424 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.424 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.424 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.425 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.425 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.425 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.425 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "iostat", 2024-09-16T00:35:22.425 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.425 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.425 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.425 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.425 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.425 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.425 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.425 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.425 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.425 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.426 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.426 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.426 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.426 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.426 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.426 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.426 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.426 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.426 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.426 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.426 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.426 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.426 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.427 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.427 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.427 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.427 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.427 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.427 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.427 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.427 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.427 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.427 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.427 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.427 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.427 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.427 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.428 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.428 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.428 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.428 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.428 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.428 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.428 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.428 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.428 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.428 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.428 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.428 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.428 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.429 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.429 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.429 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.429 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.429 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.429 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.429 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.429 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.429 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.429 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.429 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.429 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.429 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.429 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.430 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.430 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.430 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.430 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.430 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.430 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.430 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.430 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.430 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.430 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.430 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.431 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.431 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "k8sevents", 2024-09-16T00:35:22.431 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.431 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.431 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.431 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_event_retention_days": { 2024-09-16T00:35:22.431 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ceph_event_retention_days", 2024-09-16T00:35:22.431 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.431 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.431 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.431 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "7", 2024-09-16T00:35:22.431 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.432 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.432 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.432 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-16T00:35:22.432 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.432 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.432 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.433 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.433 INFO:teuthology.orchestra.run.smithi038.stdout: "config_check_secs": { 2024-09-16T00:35:22.433 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "config_check_secs", 2024-09-16T00:35:22.433 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.433 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.433 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.433 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-16T00:35:22.433 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "10", 2024-09-16T00:35:22.433 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.433 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.434 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-16T00:35:22.434 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.434 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.434 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.434 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.434 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.434 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.434 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.434 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.434 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.434 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.435 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.435 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.435 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.435 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.435 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.435 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.435 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.435 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.435 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.435 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.435 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.435 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.435 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.436 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.436 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.436 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.436 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.436 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.436 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.436 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.436 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.436 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.436 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.436 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.436 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.436 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.436 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.437 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.437 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.437 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.437 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.437 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.437 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.437 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.437 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.437 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.437 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.437 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.437 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.437 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.437 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.438 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.438 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.438 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.438 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.438 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.438 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.438 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.438 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.438 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.438 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.438 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.438 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.439 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.439 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.439 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.439 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.439 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.439 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.439 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.439 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.439 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.439 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.439 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.439 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.439 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.440 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.440 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "localpool", 2024-09-16T00:35:22.440 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.440 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.440 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.440 INFO:teuthology.orchestra.run.smithi038.stdout: "failure_domain": { 2024-09-16T00:35:22.440 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "failure_domain", 2024-09-16T00:35:22.440 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.440 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.440 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.440 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "host", 2024-09-16T00:35:22.441 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.441 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.441 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.441 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "failure domain for any created local pool", 2024-09-16T00:35:22.441 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-16T00:35:22.441 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.441 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.441 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.441 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.441 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.442 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.442 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.442 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.442 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.442 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.442 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.442 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.442 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.442 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.442 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.442 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.442 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.442 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.442 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.443 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.443 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.443 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.443 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.443 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.443 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.443 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.444 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.444 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.444 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.444 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.444 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.444 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.444 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.444 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.444 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.444 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.444 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.444 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.445 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.445 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.445 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.445 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.445 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.445 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.445 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.445 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.445 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.445 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.446 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.446 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.446 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.446 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.446 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.446 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.446 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.446 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.446 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.446 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.446 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.446 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.446 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.447 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.447 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.447 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.447 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.447 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.447 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.447 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.447 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.447 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.447 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.447 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.448 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.448 INFO:teuthology.orchestra.run.smithi038.stdout: "min_size": { 2024-09-16T00:35:22.448 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "min_size", 2024-09-16T00:35:22.448 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.448 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.448 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.448 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.448 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.448 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.448 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.448 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "default min_size for any created local pool", 2024-09-16T00:35:22.449 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-16T00:35:22.449 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.449 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.449 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.449 INFO:teuthology.orchestra.run.smithi038.stdout: "num_rep": { 2024-09-16T00:35:22.449 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "num_rep", 2024-09-16T00:35:22.449 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.449 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.449 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.450 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "3", 2024-09-16T00:35:22.450 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.450 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.450 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.450 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "default replica count for any created local pool", 2024-09-16T00:35:22.450 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.450 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.450 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.450 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.450 INFO:teuthology.orchestra.run.smithi038.stdout: "pg_num": { 2024-09-16T00:35:22.450 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pg_num", 2024-09-16T00:35:22.451 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.451 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.451 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.451 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "128", 2024-09-16T00:35:22.451 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.451 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.451 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.451 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "default pg_num for any created local pool", 2024-09-16T00:35:22.451 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.451 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.452 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.452 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.452 INFO:teuthology.orchestra.run.smithi038.stdout: "prefix": { 2024-09-16T00:35:22.452 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "prefix", 2024-09-16T00:35:22.452 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.452 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.452 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.452 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.452 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.453 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.453 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.453 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "name prefix for any created local pool", 2024-09-16T00:35:22.453 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.453 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.453 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.453 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.453 INFO:teuthology.orchestra.run.smithi038.stdout: "subtree": { 2024-09-16T00:35:22.453 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "subtree", 2024-09-16T00:35:22.453 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.453 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.453 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.454 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "rack", 2024-09-16T00:35:22.454 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.454 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.454 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.454 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-16T00:35:22.454 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-16T00:35:22.454 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.454 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.454 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.454 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.454 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.454 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.454 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mds_autoscaler", 2024-09-16T00:35:22.455 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.455 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.455 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.455 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.455 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.455 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.455 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.455 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.455 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.455 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.455 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.455 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.455 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.455 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.456 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.456 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.456 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.456 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.456 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.456 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.456 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.456 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.456 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.456 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.456 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.456 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.456 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.456 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.457 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.457 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.457 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.457 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.457 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.457 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.457 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.457 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.457 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.457 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.457 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.457 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.457 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.457 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.458 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.458 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.458 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.458 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.458 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.458 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.458 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.458 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.458 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.458 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.458 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.458 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.458 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.458 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.459 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.459 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.459 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.459 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.459 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.459 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.459 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.459 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.459 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.459 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.459 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.459 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.459 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.460 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.460 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.460 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.460 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.460 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.460 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.460 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.460 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mirroring", 2024-09-16T00:35:22.460 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.460 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.460 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.460 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.460 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.461 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.461 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.461 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.461 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.461 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.461 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.461 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.461 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.461 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.461 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.461 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.461 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.461 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.461 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.462 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.462 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.462 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.462 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.462 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.462 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.462 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.462 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.462 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.462 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.463 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.463 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.463 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.463 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.463 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.463 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.463 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.463 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.463 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.463 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.463 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.463 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.463 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.463 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.464 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.464 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.464 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.464 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.464 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.464 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.464 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.464 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.464 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.464 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.464 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.464 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.464 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.465 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.465 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.465 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.465 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.465 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.465 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.465 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.465 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.465 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.465 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.465 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.465 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.466 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.466 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.466 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.466 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.466 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.466 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.466 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.466 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.466 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "nfs", 2024-09-16T00:35:22.466 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.466 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.466 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.466 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.466 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.467 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.467 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.467 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.467 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.467 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.467 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.467 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.467 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.467 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.467 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.467 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.467 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.467 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.467 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.468 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.468 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.468 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.468 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.468 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.468 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.468 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.468 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.468 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.468 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.468 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.468 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.468 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.469 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.469 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.469 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.469 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.469 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.469 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.469 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.469 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.469 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.469 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.469 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.469 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.469 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.469 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.470 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.470 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.470 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.470 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.470 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.470 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.470 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.470 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.470 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.470 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.470 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.470 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.470 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.470 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.471 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.471 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.471 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.471 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.471 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.471 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.471 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.471 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.471 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.471 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.471 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.471 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.471 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.471 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.472 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.472 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.472 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "orchestrator", 2024-09-16T00:35:22.472 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.472 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.472 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.472 INFO:teuthology.orchestra.run.smithi038.stdout: "fail_fs": { 2024-09-16T00:35:22.472 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "fail_fs", 2024-09-16T00:35:22.472 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.472 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.472 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.472 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.472 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.473 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.473 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.473 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-16T00:35:22.473 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.473 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.473 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.473 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.473 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.473 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.473 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.473 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.474 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.474 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.474 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.474 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.474 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.474 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.474 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.474 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.474 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.474 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.474 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.474 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.474 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.474 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.475 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.475 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.475 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.475 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.475 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.475 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.475 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.475 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.475 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.475 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.476 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.476 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.476 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.476 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.476 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.476 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.476 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.476 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.476 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.476 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.476 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.476 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.476 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.477 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.477 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.477 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.477 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.477 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.477 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.477 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.477 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.477 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.477 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.477 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.477 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.478 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.478 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.478 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.478 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.478 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.478 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.478 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.478 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.478 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.478 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.478 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.478 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.479 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.479 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.479 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.479 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.479 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.479 INFO:teuthology.orchestra.run.smithi038.stdout: "orchestrator": { 2024-09-16T00:35:22.479 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "orchestrator", 2024-09-16T00:35:22.479 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.479 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.479 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.479 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.479 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.479 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.479 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.479 INFO:teuthology.orchestra.run.smithi038.stdout: "cephadm", 2024-09-16T00:35:22.480 INFO:teuthology.orchestra.run.smithi038.stdout: "rook", 2024-09-16T00:35:22.480 INFO:teuthology.orchestra.run.smithi038.stdout: "test_orchestrator" 2024-09-16T00:35:22.480 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.480 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Orchestrator backend", 2024-09-16T00:35:22.480 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.480 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.480 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.480 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.480 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.480 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.480 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.480 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "osd_perf_query", 2024-09-16T00:35:22.481 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.481 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.481 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.481 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.481 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.481 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.481 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.481 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.481 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.481 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.481 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.481 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.481 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.482 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.482 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.482 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.482 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.482 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.482 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.482 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.482 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.482 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.482 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.482 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.482 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.482 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.483 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.483 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.483 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.483 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.483 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.483 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.483 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.483 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.483 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.483 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.483 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.483 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.483 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.483 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.484 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.484 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.484 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.484 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.484 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.484 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.484 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.484 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.484 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.484 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.484 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.484 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.485 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.485 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.485 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.485 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.485 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.485 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.485 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.485 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.485 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.485 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.485 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.485 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.485 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.485 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.486 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.486 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.486 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.486 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.486 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.486 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.486 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.486 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.486 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.486 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.486 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "osd_support", 2024-09-16T00:35:22.486 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.486 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.486 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.487 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.487 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.487 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.487 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.487 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.487 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.487 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.487 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.487 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.487 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.487 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.487 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.487 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.487 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.487 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.488 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.488 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.488 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.488 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.488 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.488 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.488 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.488 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.488 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.488 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.488 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.489 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.489 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.489 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.489 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.489 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.489 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.489 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.489 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.489 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.489 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.489 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.489 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.489 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.489 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.490 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.490 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.490 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.490 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.490 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.490 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.490 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.490 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.490 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.490 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.490 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.490 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.490 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.490 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.490 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.491 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.491 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.491 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.491 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.491 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.491 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.491 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.491 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.491 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.491 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.491 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.491 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.491 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.491 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.492 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.492 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.492 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.492 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.492 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pg_autoscaler", 2024-09-16T00:35:22.492 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.492 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.492 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.492 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.492 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.492 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.492 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.492 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.492 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.493 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.493 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.493 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.493 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.493 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.493 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.493 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.493 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.493 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.493 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.493 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.493 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.493 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.493 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.494 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.494 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.494 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.494 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.494 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.494 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.494 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.494 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.494 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.494 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.494 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.494 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.494 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.494 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.495 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.495 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.495 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.495 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.495 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.495 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.495 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.495 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.495 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.496 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.496 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.496 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.496 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.496 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.496 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.496 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.496 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.496 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.496 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.496 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.496 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.496 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.497 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.497 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.497 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.497 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.497 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.497 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.497 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.497 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.497 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.497 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.497 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.497 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.497 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.498 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.498 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-09-16T00:35:22.498 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-09-16T00:35:22.498 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:22.499 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.499 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.499 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-09-16T00:35:22.499 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.499 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.499 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.499 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.499 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.499 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.499 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.499 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.499 INFO:teuthology.orchestra.run.smithi038.stdout: "threshold": { 2024-09-16T00:35:22.499 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "threshold", 2024-09-16T00:35:22.499 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-16T00:35:22.500 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.500 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.500 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "3.0", 2024-09-16T00:35:22.500 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1.0", 2024-09-16T00:35:22.500 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.500 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.500 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "scaling threshold", 2024-09-16T00:35:22.500 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-16T00:35:22.500 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.500 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.500 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.500 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.501 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.501 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.501 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "progress", 2024-09-16T00:35:22.501 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.501 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.501 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.501 INFO:teuthology.orchestra.run.smithi038.stdout: "allow_pg_recovery_event": { 2024-09-16T00:35:22.501 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "allow_pg_recovery_event", 2024-09-16T00:35:22.501 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.502 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.502 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.502 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.502 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.502 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.502 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.502 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-16T00:35:22.502 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.502 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.502 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.502 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.502 INFO:teuthology.orchestra.run.smithi038.stdout: "enabled": { 2024-09-16T00:35:22.503 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enabled", 2024-09-16T00:35:22.503 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.503 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.503 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.503 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.503 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.503 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.503 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.503 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.503 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.503 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.503 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.503 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.503 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.504 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.504 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.504 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.504 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.504 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.504 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.504 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.504 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.504 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.504 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.504 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.504 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.504 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.504 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.505 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.505 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.505 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.505 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.505 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.505 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.505 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.505 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.505 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.505 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.505 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.505 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.505 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.506 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.506 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.506 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.506 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.506 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.506 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.506 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.506 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.506 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.506 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.506 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.506 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.507 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.507 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.507 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.507 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.507 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.507 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.507 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.507 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.507 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.507 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.507 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.507 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.507 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.507 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.507 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.508 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.508 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.508 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.508 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.508 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.508 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.508 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.509 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.509 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.509 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.509 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.509 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.509 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.509 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.509 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.509 INFO:teuthology.orchestra.run.smithi038.stdout: "max_completed_events": { 2024-09-16T00:35:22.509 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_completed_events", 2024-09-16T00:35:22.509 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.509 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.510 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.510 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "50", 2024-09-16T00:35:22.510 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.510 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.510 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.510 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "number of past completed events to remember", 2024-09-16T00:35:22.510 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.510 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.510 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.510 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.510 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-09-16T00:35:22.510 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-09-16T00:35:22.510 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:22.510 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.511 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.511 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-09-16T00:35:22.511 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.511 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.511 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.511 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how long the module is going to sleep", 2024-09-16T00:35:22.511 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.511 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.511 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.511 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.511 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.511 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.511 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.511 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "prometheus", 2024-09-16T00:35:22.512 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.512 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.512 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.512 INFO:teuthology.orchestra.run.smithi038.stdout: "cache": { 2024-09-16T00:35:22.512 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cache", 2024-09-16T00:35:22.512 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.512 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.512 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.512 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.512 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.512 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.512 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.512 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.512 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.513 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.513 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.513 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.513 INFO:teuthology.orchestra.run.smithi038.stdout: "exclude_perf_counters": { 2024-09-16T00:35:22.513 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "exclude_perf_counters", 2024-09-16T00:35:22.513 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.513 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.513 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.513 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.513 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.513 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.513 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.513 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-16T00:35:22.514 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-16T00:35:22.514 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.514 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.514 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.514 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.514 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.514 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.514 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.514 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.514 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.514 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.515 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.515 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.515 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.515 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.515 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.515 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.515 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.515 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.515 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.515 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.515 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.515 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.515 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.516 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.516 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.516 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.516 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.516 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.516 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.516 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.516 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.516 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.516 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.516 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.516 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.516 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.516 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.517 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.517 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.517 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.517 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.517 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.517 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.517 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.517 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.517 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.517 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.517 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.517 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.518 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.518 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.518 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.518 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.518 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.518 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.518 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.518 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.518 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.518 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.518 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.518 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.518 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.519 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.519 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.519 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.519 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.519 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.519 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.519 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.519 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.519 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.519 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.519 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.519 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_stats_pools": { 2024-09-16T00:35:22.519 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rbd_stats_pools", 2024-09-16T00:35:22.519 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.520 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.520 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.520 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.520 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.520 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.520 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.520 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.520 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.520 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.520 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.520 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.520 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-16T00:35:22.520 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-16T00:35:22.521 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.521 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.521 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.521 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "300", 2024-09-16T00:35:22.521 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.521 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.521 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.521 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.521 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.521 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.522 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.522 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.522 INFO:teuthology.orchestra.run.smithi038.stdout: "scrape_interval": { 2024-09-16T00:35:22.522 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "scrape_interval", 2024-09-16T00:35:22.522 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-16T00:35:22.522 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.522 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.522 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "15.0", 2024-09-16T00:35:22.522 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.522 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.522 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.522 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.522 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.522 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.523 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.523 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.523 INFO:teuthology.orchestra.run.smithi038.stdout: "server_addr": { 2024-09-16T00:35:22.523 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_addr", 2024-09-16T00:35:22.523 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.523 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.523 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.523 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "::", 2024-09-16T00:35:22.523 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.523 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.523 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.524 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-16T00:35:22.524 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.524 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.524 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.524 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.524 INFO:teuthology.orchestra.run.smithi038.stdout: "server_port": { 2024-09-16T00:35:22.524 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_port", 2024-09-16T00:35:22.524 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.524 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.524 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.524 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "9283", 2024-09-16T00:35:22.524 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.525 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.525 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.525 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-16T00:35:22.525 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.525 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.525 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.525 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.525 INFO:teuthology.orchestra.run.smithi038.stdout: "stale_cache_strategy": { 2024-09-16T00:35:22.525 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "stale_cache_strategy", 2024-09-16T00:35:22.525 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.526 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.526 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.526 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "log", 2024-09-16T00:35:22.526 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.526 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.526 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.526 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.526 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.526 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.526 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.526 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.526 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_behaviour": { 2024-09-16T00:35:22.526 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_behaviour", 2024-09-16T00:35:22.527 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.527 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.527 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.527 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "default", 2024-09-16T00:35:22.527 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.527 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.527 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.527 INFO:teuthology.orchestra.run.smithi038.stdout: "default", 2024-09-16T00:35:22.527 INFO:teuthology.orchestra.run.smithi038.stdout: "error" 2024-09-16T00:35:22.527 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.528 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.528 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.528 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.528 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.528 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.528 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_error_status_code": { 2024-09-16T00:35:22.528 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_error_status_code", 2024-09-16T00:35:22.528 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.528 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.528 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.528 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "500", 2024-09-16T00:35:22.529 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "400", 2024-09-16T00:35:22.529 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "599", 2024-09-16T00:35:22.529 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.529 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.529 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.529 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.529 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.529 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.529 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.529 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.530 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.530 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rbd_support", 2024-09-16T00:35:22.530 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.530 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.530 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.530 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.530 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.530 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.530 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.530 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.531 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.531 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.531 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.531 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.531 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.531 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.531 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.531 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.531 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.532 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.532 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.532 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.532 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.532 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.532 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.532 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.532 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.532 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.532 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.532 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.532 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.533 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.533 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.533 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.533 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.533 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.533 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.533 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.533 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.533 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.533 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.533 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.533 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.533 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.534 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.534 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.534 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.534 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.534 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.534 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.534 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.534 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.534 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.534 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.534 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.534 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.535 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.535 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.535 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.535 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.535 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.535 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.535 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.535 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.535 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.535 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.535 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.535 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.535 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.536 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.536 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.536 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.536 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.536 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.536 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.536 INFO:teuthology.orchestra.run.smithi038.stdout: "max_concurrent_snap_create": { 2024-09-16T00:35:22.536 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_concurrent_snap_create", 2024-09-16T00:35:22.536 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.536 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.536 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.536 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-16T00:35:22.536 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.536 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.537 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.537 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.537 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.537 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.537 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.537 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.537 INFO:teuthology.orchestra.run.smithi038.stdout: "mirror_snapshot_schedule": { 2024-09-16T00:35:22.537 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mirror_snapshot_schedule", 2024-09-16T00:35:22.537 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.537 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.538 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.538 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.538 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.538 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.538 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.538 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.538 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.538 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.538 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.538 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.538 INFO:teuthology.orchestra.run.smithi038.stdout: "trash_purge_schedule": { 2024-09-16T00:35:22.538 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "trash_purge_schedule", 2024-09-16T00:35:22.539 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.539 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.539 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.539 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.539 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.539 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.539 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.539 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.539 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.539 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.539 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.539 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.539 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.540 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.540 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.540 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "restful", 2024-09-16T00:35:22.540 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.540 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.540 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.540 INFO:teuthology.orchestra.run.smithi038.stdout: "enable_auth": { 2024-09-16T00:35:22.540 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enable_auth", 2024-09-16T00:35:22.540 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.540 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.540 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.540 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.541 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.541 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.541 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.541 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.541 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.541 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.541 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.541 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.541 INFO:teuthology.orchestra.run.smithi038.stdout: "key_file": { 2024-09-16T00:35:22.541 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "key_file", 2024-09-16T00:35:22.541 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.541 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.542 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.542 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.542 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.542 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.542 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.542 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.542 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.542 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.542 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.542 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.542 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.543 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.543 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.543 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.543 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.543 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.543 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.543 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.543 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.543 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.543 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.543 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.543 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.544 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.544 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.544 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.544 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.544 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.544 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.544 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.544 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.544 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.544 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.545 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.545 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.545 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.545 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.545 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.545 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.545 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.545 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.545 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.545 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.545 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.546 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.546 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.546 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.546 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.546 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.546 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.546 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.546 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.546 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.546 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.546 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.546 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.546 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.547 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.547 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.547 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.547 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.547 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.547 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.547 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.547 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.547 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.547 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.547 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.547 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.547 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.547 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.548 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.548 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.548 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.548 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.548 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.548 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.548 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.548 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.548 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.548 INFO:teuthology.orchestra.run.smithi038.stdout: "server_addr": { 2024-09-16T00:35:22.548 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_addr", 2024-09-16T00:35:22.548 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.548 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.549 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.549 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.549 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.549 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.549 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.549 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.549 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.549 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.549 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.549 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.549 INFO:teuthology.orchestra.run.smithi038.stdout: "server_port": { 2024-09-16T00:35:22.549 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_port", 2024-09-16T00:35:22.549 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.549 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.550 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.550 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.550 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.550 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.550 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.550 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.550 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.550 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.550 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.550 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.550 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.550 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.550 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.551 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rgw", 2024-09-16T00:35:22.551 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.551 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.551 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.551 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.551 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.551 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.551 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.551 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.551 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.551 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.551 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.552 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.552 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.552 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.552 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.552 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.552 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.552 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.552 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.552 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.552 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.552 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.552 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.552 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.552 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.553 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.553 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.553 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.553 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.553 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.553 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.553 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.553 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.553 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.553 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.553 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.553 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.553 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.553 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.554 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.554 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.554 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.554 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.554 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.554 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.554 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.554 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.554 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.554 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.554 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.554 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.554 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.555 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.555 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.555 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.555 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.555 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.555 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.555 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.555 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.555 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.555 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.555 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.556 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.556 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.556 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.556 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.556 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.556 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.556 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.556 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.556 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.556 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.556 INFO:teuthology.orchestra.run.smithi038.stdout: "secondary_zone_period_retry_limit": { 2024-09-16T00:35:22.556 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-16T00:35:22.556 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.556 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.557 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.557 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-09-16T00:35:22.557 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.557 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.557 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.557 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-16T00:35:22.557 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.557 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.558 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.560 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.560 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.560 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.561 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.561 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rook", 2024-09-16T00:35:22.561 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.562 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.562 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.562 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.562 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.562 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.562 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.562 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.562 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.563 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.563 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.563 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.563 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.563 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.563 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.564 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.564 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.564 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.564 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.564 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.564 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.564 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.564 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.565 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.565 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.565 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.565 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.565 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.565 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.566 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.566 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.566 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.566 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.566 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.566 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.566 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.567 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.567 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.567 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.567 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.567 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.567 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.567 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.568 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.568 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.568 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.568 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.568 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.568 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.568 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.568 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.569 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.569 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.569 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.569 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.569 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.569 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.569 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.570 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.570 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.570 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.570 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.570 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.570 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.570 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.570 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.571 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.571 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.571 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.571 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.571 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.571 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.571 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.571 INFO:teuthology.orchestra.run.smithi038.stdout: "storage_class": { 2024-09-16T00:35:22.572 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "storage_class", 2024-09-16T00:35:22.572 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.572 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.572 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.572 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "local", 2024-09-16T00:35:22.573 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.573 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.573 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.573 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-16T00:35:22.573 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.573 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.574 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.574 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.574 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.574 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.574 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.574 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "selftest", 2024-09-16T00:35:22.574 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.574 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.575 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.575 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.575 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.575 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.575 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.575 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.575 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.576 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.576 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.576 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.576 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.576 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.576 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.576 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.576 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.577 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.577 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.577 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.577 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.577 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.577 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.577 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.578 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.578 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.578 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.578 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.578 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.578 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.579 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.579 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.579 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.579 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.579 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.579 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.580 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.580 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.580 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.580 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.580 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.580 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.580 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.581 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.581 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.581 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.581 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.581 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.581 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.581 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.582 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.582 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.582 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.582 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.582 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.582 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.583 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.583 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.583 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.583 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.583 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.583 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.583 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.584 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.584 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.584 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.584 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.584 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.584 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.584 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.584 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.585 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.585 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.585 INFO:teuthology.orchestra.run.smithi038.stdout: "roption1": { 2024-09-16T00:35:22.585 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "roption1", 2024-09-16T00:35:22.585 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.585 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.585 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.586 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.586 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.586 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.586 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.586 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.586 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.586 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.586 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.587 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.587 INFO:teuthology.orchestra.run.smithi038.stdout: "roption2": { 2024-09-16T00:35:22.587 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "roption2", 2024-09-16T00:35:22.587 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.587 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.587 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.588 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "xyz", 2024-09-16T00:35:22.588 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.588 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.588 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.588 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.588 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.588 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.589 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.589 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.589 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption1": { 2024-09-16T00:35:22.589 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption1", 2024-09-16T00:35:22.589 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.589 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.590 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.590 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.590 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.590 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.590 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.590 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.590 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.590 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.591 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.591 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.591 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption2": { 2024-09-16T00:35:22.591 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption2", 2024-09-16T00:35:22.591 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.591 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.591 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.592 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.592 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.592 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.592 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.592 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.592 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.592 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.593 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.593 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.593 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption3": { 2024-09-16T00:35:22.593 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption3", 2024-09-16T00:35:22.593 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-16T00:35:22.593 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.593 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.594 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.594 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.594 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.594 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.595 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.595 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.595 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.595 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.595 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.595 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption4": { 2024-09-16T00:35:22.595 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption4", 2024-09-16T00:35:22.596 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.596 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.596 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.596 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.597 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.597 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.597 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.597 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.597 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.597 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.597 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.597 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.598 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption5": { 2024-09-16T00:35:22.598 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption5", 2024-09-16T00:35:22.598 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.598 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.599 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.599 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.599 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.599 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.599 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.599 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.600 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.600 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.600 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.600 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.601 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption6": { 2024-09-16T00:35:22.601 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption6", 2024-09-16T00:35:22.601 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.601 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.601 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.601 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.602 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.602 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.602 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.602 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.602 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.602 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.602 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.602 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.603 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption7": { 2024-09-16T00:35:22.603 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption7", 2024-09-16T00:35:22.603 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.603 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.604 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.604 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.604 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-09-16T00:35:22.604 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "42", 2024-09-16T00:35:22.604 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.605 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.605 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.605 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.605 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.605 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.605 INFO:teuthology.orchestra.run.smithi038.stdout: "testkey": { 2024-09-16T00:35:22.605 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "testkey", 2024-09-16T00:35:22.606 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.606 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.606 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.606 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.606 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.606 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.606 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.606 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.607 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.607 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.607 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.607 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.607 INFO:teuthology.orchestra.run.smithi038.stdout: "testlkey": { 2024-09-16T00:35:22.607 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "testlkey", 2024-09-16T00:35:22.607 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.607 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.607 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.608 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.608 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.608 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.608 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.608 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.608 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.609 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.609 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.609 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.609 INFO:teuthology.orchestra.run.smithi038.stdout: "testnewline": { 2024-09-16T00:35:22.609 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "testnewline", 2024-09-16T00:35:22.609 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.609 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.609 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.609 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.609 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.609 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.610 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.610 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.610 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.610 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.610 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.610 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.610 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.610 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.610 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.610 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "snap_schedule", 2024-09-16T00:35:22.610 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.611 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.611 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.611 INFO:teuthology.orchestra.run.smithi038.stdout: "allow_m_granularity": { 2024-09-16T00:35:22.611 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "allow_m_granularity", 2024-09-16T00:35:22.611 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.612 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.612 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.612 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.612 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.612 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.612 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.612 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "allow minute scheduled snapshots", 2024-09-16T00:35:22.613 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.613 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.613 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.613 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.613 INFO:teuthology.orchestra.run.smithi038.stdout: "dump_on_update": { 2024-09-16T00:35:22.613 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "dump_on_update", 2024-09-16T00:35:22.613 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.614 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.614 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.614 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.614 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.614 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.614 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.614 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "dump database to debug log on update", 2024-09-16T00:35:22.614 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.615 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.615 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.615 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.615 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.615 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.615 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.615 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.616 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.616 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.616 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.616 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.616 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.616 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.616 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.617 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.617 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.617 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.617 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.617 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.617 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.618 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.618 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.618 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.618 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.618 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.618 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.618 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.619 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.619 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.619 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.619 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.619 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.619 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.619 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.620 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.620 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.620 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.620 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.620 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.620 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.620 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.621 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.621 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.621 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.621 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.621 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.621 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.622 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.622 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.622 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.622 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.622 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.622 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.622 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.622 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.623 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.623 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.623 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.623 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.623 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.623 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.623 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.624 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.624 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.624 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.624 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.624 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.624 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.624 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.625 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.625 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.625 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.625 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.625 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.625 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.625 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.626 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "stats", 2024-09-16T00:35:22.626 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.626 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.626 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.626 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.626 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.627 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.627 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.627 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.627 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.627 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.628 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.628 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.628 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.628 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.628 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.628 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.628 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.629 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.629 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.629 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.629 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.629 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.629 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.630 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.630 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.630 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.630 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.630 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.630 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.631 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.631 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.631 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.631 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.631 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.632 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.632 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.632 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.632 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.632 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.632 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.632 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.632 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.632 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.632 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.633 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.633 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.633 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.633 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.633 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.633 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.634 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.634 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.634 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.634 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.634 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.634 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.634 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.634 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.634 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.635 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.635 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.635 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.635 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.635 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.635 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.635 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.635 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.635 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.636 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.636 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.636 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.636 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.636 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.636 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.636 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.636 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.637 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "status", 2024-09-16T00:35:22.637 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.637 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.637 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.637 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.637 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.637 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.637 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.638 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.638 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.638 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.638 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.638 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.638 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.638 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.638 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.639 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.639 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.639 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.639 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.639 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.639 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.639 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.639 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.640 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.640 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.640 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.640 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.640 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.640 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.640 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.640 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.640 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.640 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.640 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.641 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.641 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.641 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.641 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.641 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.641 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.641 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.641 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.641 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.641 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.641 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.642 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.642 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.642 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.642 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.642 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.642 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.642 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.642 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.642 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.642 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.643 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.643 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.643 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.643 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.643 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.643 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.643 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.643 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.643 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.644 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.644 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.644 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.644 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.644 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.644 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.644 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.644 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.644 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.644 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.644 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.645 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.645 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "telegraf", 2024-09-16T00:35:22.645 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.645 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.645 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.645 INFO:teuthology.orchestra.run.smithi038.stdout: "address": { 2024-09-16T00:35:22.645 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "address", 2024-09-16T00:35:22.645 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.645 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.645 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.646 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-16T00:35:22.646 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.646 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.646 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.646 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.646 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.646 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.646 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.646 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.646 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-09-16T00:35:22.646 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-09-16T00:35:22.647 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:22.647 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.647 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.647 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "15", 2024-09-16T00:35:22.647 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.647 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.647 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.647 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.647 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.648 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.648 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.648 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.648 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.648 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.648 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.648 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.649 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.649 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.649 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.649 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.649 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.649 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.649 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.650 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.650 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.650 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.650 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.650 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.650 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.650 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.651 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.651 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.651 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.651 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.651 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.651 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.651 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.652 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.652 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.652 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.652 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.652 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.652 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.653 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.653 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.653 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.653 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.653 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.653 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.653 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.654 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.654 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.654 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.654 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.654 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.654 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.654 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.655 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.655 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.655 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.655 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.655 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.655 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.655 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.655 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.656 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.656 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.656 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.656 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.656 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.656 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.656 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.657 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.657 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.657 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.657 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.657 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.657 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.657 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.657 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.658 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.658 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.658 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.658 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.658 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.658 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "telemetry", 2024-09-16T00:35:22.658 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.659 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.659 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.659 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_basic": { 2024-09-16T00:35:22.659 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_basic", 2024-09-16T00:35:22.659 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.659 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.659 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.659 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.660 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.660 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.660 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.660 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-16T00:35:22.660 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.660 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.660 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.661 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.661 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_crash": { 2024-09-16T00:35:22.661 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_crash", 2024-09-16T00:35:22.661 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.661 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.662 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.662 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.662 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.662 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.662 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.662 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-16T00:35:22.662 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.663 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.663 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.663 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.663 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_device": { 2024-09-16T00:35:22.663 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_device", 2024-09-16T00:35:22.663 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.663 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.664 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.664 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.664 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.664 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.664 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.664 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-16T00:35:22.664 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.664 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.665 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.665 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.665 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_ident": { 2024-09-16T00:35:22.665 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_ident", 2024-09-16T00:35:22.665 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.665 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.665 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.666 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.666 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.666 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.666 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.666 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-16T00:35:22.667 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.667 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.667 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.667 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.667 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_perf": { 2024-09-16T00:35:22.667 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_perf", 2024-09-16T00:35:22.667 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.668 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.668 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.668 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.668 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.669 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.669 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.669 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-16T00:35:22.669 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.669 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.669 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.669 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.669 INFO:teuthology.orchestra.run.smithi038.stdout: "contact": { 2024-09-16T00:35:22.669 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "contact", 2024-09-16T00:35:22.669 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.670 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.670 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.670 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.670 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.670 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.671 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.671 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.671 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.671 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.671 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.671 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.671 INFO:teuthology.orchestra.run.smithi038.stdout: "description": { 2024-09-16T00:35:22.672 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "description", 2024-09-16T00:35:22.672 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.672 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.672 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.673 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.673 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.673 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.673 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.673 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.674 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.674 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.674 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.674 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.674 INFO:teuthology.orchestra.run.smithi038.stdout: "device_url": { 2024-09-16T00:35:22.675 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "device_url", 2024-09-16T00:35:22.675 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.675 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.675 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.675 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-16T00:35:22.676 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.676 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.676 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.676 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.676 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.676 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.676 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.676 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.677 INFO:teuthology.orchestra.run.smithi038.stdout: "enabled": { 2024-09-16T00:35:22.677 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enabled", 2024-09-16T00:35:22.677 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.677 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.677 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.677 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.677 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.677 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.677 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.677 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.677 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.678 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.678 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.678 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.678 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-09-16T00:35:22.678 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-09-16T00:35:22.678 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.678 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.678 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.679 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "24", 2024-09-16T00:35:22.679 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "8", 2024-09-16T00:35:22.679 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.679 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.679 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.679 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.679 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.679 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.679 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.679 INFO:teuthology.orchestra.run.smithi038.stdout: "last_opt_revision": { 2024-09-16T00:35:22.679 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "last_opt_revision", 2024-09-16T00:35:22.680 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.680 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.680 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.680 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "1", 2024-09-16T00:35:22.680 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.680 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.680 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.680 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.680 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.681 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.681 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.681 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.681 INFO:teuthology.orchestra.run.smithi038.stdout: "leaderboard": { 2024-09-16T00:35:22.681 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "leaderboard", 2024-09-16T00:35:22.681 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.681 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.681 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.681 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.681 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.681 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.682 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.682 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.682 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.682 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.682 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.682 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.682 INFO:teuthology.orchestra.run.smithi038.stdout: "leaderboard_description": { 2024-09-16T00:35:22.682 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "leaderboard_description", 2024-09-16T00:35:22.683 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.683 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.683 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.683 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.683 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.683 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.683 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.683 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.683 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.683 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.683 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.684 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.684 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.684 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.684 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.684 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.684 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.684 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.684 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.685 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.685 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.685 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.685 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.685 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.685 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.685 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.685 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.685 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.686 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.686 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.686 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.686 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.686 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.687 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.687 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.687 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.687 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.687 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.687 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.687 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.687 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.688 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.688 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.688 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.688 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.688 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.688 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.689 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.689 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.689 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.689 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.689 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.689 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.689 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.689 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.690 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.690 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.690 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.690 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.690 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.690 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.690 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.691 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.691 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.691 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.691 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.691 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.691 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.691 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.691 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.692 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.692 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.692 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.692 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.692 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.692 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.692 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.692 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.692 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.693 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.693 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.693 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.693 INFO:teuthology.orchestra.run.smithi038.stdout: "organization": { 2024-09-16T00:35:22.693 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "organization", 2024-09-16T00:35:22.693 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.693 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.693 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.693 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.693 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.694 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.694 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.694 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.694 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.694 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.694 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.694 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.694 INFO:teuthology.orchestra.run.smithi038.stdout: "proxy": { 2024-09-16T00:35:22.694 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "proxy", 2024-09-16T00:35:22.695 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.695 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.695 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.695 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.695 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.695 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.695 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.695 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.695 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.695 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.696 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.696 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.696 INFO:teuthology.orchestra.run.smithi038.stdout: "url": { 2024-09-16T00:35:22.696 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "url", 2024-09-16T00:35:22.696 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.696 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.696 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.696 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-16T00:35:22.696 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.696 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.697 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.697 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.697 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.697 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.697 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.697 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.697 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.697 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.697 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.698 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "test_orchestrator", 2024-09-16T00:35:22.698 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.698 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.698 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.698 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.698 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.698 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.698 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.699 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.699 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.699 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.699 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.699 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.699 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.699 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.700 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.700 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.700 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.700 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.700 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.700 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.700 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.700 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.700 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.700 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.701 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.701 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.701 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.701 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.701 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.701 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.701 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.701 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.701 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.701 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.701 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.701 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.702 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.702 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.702 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.702 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.702 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.702 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.702 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.703 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.703 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.703 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.703 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.703 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.703 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.703 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.703 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.703 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.703 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.704 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.704 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.704 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.704 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.704 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.704 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.704 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.705 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.705 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.705 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.705 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.705 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.705 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.705 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.705 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.705 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.706 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.706 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.706 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.706 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.706 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.706 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.706 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.706 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "volumes", 2024-09-16T00:35:22.707 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.707 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.707 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.707 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.707 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.707 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.707 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.708 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.708 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.708 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.708 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.708 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.708 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.709 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.709 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.709 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.709 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.709 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.709 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.710 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.710 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.710 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.710 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.710 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.710 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.710 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.710 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.711 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.711 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.711 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.711 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.711 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.711 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.711 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.711 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.711 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.711 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.711 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.712 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.712 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.712 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.712 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.712 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.712 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.712 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.712 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.712 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.713 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.713 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.713 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.713 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.713 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.713 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.713 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.713 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.713 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.714 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.714 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.714 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.714 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.714 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.714 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.714 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.714 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.714 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.714 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.714 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.714 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.715 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.715 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.715 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.715 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.715 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.715 INFO:teuthology.orchestra.run.smithi038.stdout: "max_concurrent_clones": { 2024-09-16T00:35:22.715 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_concurrent_clones", 2024-09-16T00:35:22.716 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.716 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.716 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.716 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "4", 2024-09-16T00:35:22.716 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.716 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.716 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.717 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-16T00:35:22.717 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.717 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.717 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.717 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.717 INFO:teuthology.orchestra.run.smithi038.stdout: "periodic_async_work": { 2024-09-16T00:35:22.717 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "periodic_async_work", 2024-09-16T00:35:22.717 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.718 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.718 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.718 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.718 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.718 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.718 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.718 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Periodically check for async work", 2024-09-16T00:35:22.719 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.719 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.719 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.719 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.719 INFO:teuthology.orchestra.run.smithi038.stdout: "snapshot_clone_delay": { 2024-09-16T00:35:22.719 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "snapshot_clone_delay", 2024-09-16T00:35:22.719 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.720 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.720 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.720 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-09-16T00:35:22.720 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.720 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.720 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.720 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-16T00:35:22.721 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.721 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.721 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.721 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.721 INFO:teuthology.orchestra.run.smithi038.stdout: "snapshot_clone_no_wait": { 2024-09-16T00:35:22.721 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "snapshot_clone_no_wait", 2024-09-16T00:35:22.721 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.722 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.722 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.722 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.722 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.722 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.722 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.722 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-16T00:35:22.723 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.723 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.723 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.723 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.723 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.723 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.723 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.724 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix", 2024-09-16T00:35:22.724 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.724 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.724 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.724 INFO:teuthology.orchestra.run.smithi038.stdout: "discovery_interval": { 2024-09-16T00:35:22.724 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "discovery_interval", 2024-09-16T00:35:22.724 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-09-16T00:35:22.724 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.725 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.725 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "100", 2024-09-16T00:35:22.725 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.725 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.725 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.725 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.725 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.726 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.726 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.726 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.726 INFO:teuthology.orchestra.run.smithi038.stdout: "identifier": { 2024-09-16T00:35:22.726 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "identifier", 2024-09-16T00:35:22.726 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.726 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.727 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.727 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.727 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.727 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.727 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.727 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.728 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.728 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.728 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.728 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.728 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-09-16T00:35:22.728 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-09-16T00:35:22.728 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:22.728 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.729 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.729 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-09-16T00:35:22.729 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.729 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.729 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.729 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.729 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.729 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.729 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.729 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.729 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.729 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.730 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.730 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.730 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.730 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.730 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.730 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.730 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.730 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.730 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.730 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.730 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.730 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.730 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.731 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.731 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.731 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.731 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.731 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.731 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.731 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.732 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.732 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.732 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.732 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.732 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.732 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.732 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.733 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.733 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.733 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.733 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.733 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.733 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.733 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.733 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.733 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.733 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.734 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.734 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.734 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.734 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.734 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.734 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.734 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.734 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.734 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.735 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.735 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.735 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.735 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.735 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.735 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.735 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.735 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.735 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.735 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.735 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.735 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.735 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.736 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.736 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.736 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.736 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.736 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.736 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.736 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.736 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.736 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.736 INFO:teuthology.orchestra.run.smithi038.stdout: "zabbix_host": { 2024-09-16T00:35:22.736 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix_host", 2024-09-16T00:35:22.737 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.737 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.737 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.737 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.737 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.737 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.737 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.737 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.738 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.738 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.738 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.738 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.738 INFO:teuthology.orchestra.run.smithi038.stdout: "zabbix_port": { 2024-09-16T00:35:22.738 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix_port", 2024-09-16T00:35:22.738 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.738 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.738 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.738 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10051", 2024-09-16T00:35:22.738 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.739 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.739 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.739 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.739 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.739 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.739 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.739 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.740 INFO:teuthology.orchestra.run.smithi038.stdout: "zabbix_sender": { 2024-09-16T00:35:22.740 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix_sender", 2024-09-16T00:35:22.740 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.740 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.740 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.740 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-16T00:35:22.740 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.740 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.740 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.741 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.741 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.741 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.741 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.741 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.741 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.741 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.741 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-09-16T00:35:22.741 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.742 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.742 INFO:teuthology.orchestra.run.smithi038.stdout: "modules": [ 2024-09-16T00:35:22.742 INFO:teuthology.orchestra.run.smithi038.stdout: "cephadm", 2024-09-16T00:35:22.742 INFO:teuthology.orchestra.run.smithi038.stdout: "dashboard", 2024-09-16T00:35:22.742 INFO:teuthology.orchestra.run.smithi038.stdout: "iostat", 2024-09-16T00:35:22.742 INFO:teuthology.orchestra.run.smithi038.stdout: "nfs", 2024-09-16T00:35:22.742 INFO:teuthology.orchestra.run.smithi038.stdout: "restful" 2024-09-16T00:35:22.742 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.742 INFO:teuthology.orchestra.run.smithi038.stdout: "available_modules": [ 2024-09-16T00:35:22.743 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.743 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "alerts", 2024-09-16T00:35:22.743 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.743 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.743 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.743 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-09-16T00:35:22.743 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-09-16T00:35:22.744 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:22.744 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.744 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.744 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-09-16T00:35:22.744 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.744 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.744 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.744 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How frequently to reexamine health status", 2024-09-16T00:35:22.744 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.745 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.745 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.745 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.745 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.745 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.745 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.745 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.746 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.746 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.746 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.746 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.746 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.746 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.746 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.747 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.747 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.747 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.747 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.747 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.747 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.747 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.747 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.747 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.747 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.748 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.748 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.748 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.748 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.748 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.748 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.749 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.749 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.749 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.749 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.749 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.749 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.749 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.749 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.750 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.750 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.750 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.750 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.750 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.750 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.751 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.751 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.751 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.751 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.751 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.751 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.751 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.752 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.752 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.752 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.752 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.752 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.752 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.752 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.752 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.752 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.752 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.752 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.753 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.753 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.753 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.753 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.753 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.753 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.753 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.753 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.753 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.753 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.754 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.754 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_destination": { 2024-09-16T00:35:22.754 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_destination", 2024-09-16T00:35:22.754 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.754 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.754 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.754 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.754 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.754 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.754 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.754 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Email address to send alerts to", 2024-09-16T00:35:22.754 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.755 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.755 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.755 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.755 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_from_name": { 2024-09-16T00:35:22.755 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_from_name", 2024-09-16T00:35:22.755 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.755 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.755 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.755 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "Ceph", 2024-09-16T00:35:22.755 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.755 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.756 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.756 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Email From: name", 2024-09-16T00:35:22.756 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.756 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.756 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.756 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.756 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_host": { 2024-09-16T00:35:22.756 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_host", 2024-09-16T00:35:22.756 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.757 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.757 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.757 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.757 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.757 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.757 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.757 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SMTP server", 2024-09-16T00:35:22.757 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.757 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.758 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.758 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.758 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_password": { 2024-09-16T00:35:22.758 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_password", 2024-09-16T00:35:22.758 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.758 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.758 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.758 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.758 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.758 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.759 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.759 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Password to authenticate with", 2024-09-16T00:35:22.759 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.759 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.759 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.759 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.759 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_port": { 2024-09-16T00:35:22.759 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_port", 2024-09-16T00:35:22.760 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.760 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.760 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.760 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "465", 2024-09-16T00:35:22.760 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.760 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.760 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.761 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SMTP port", 2024-09-16T00:35:22.761 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.761 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.761 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.761 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.761 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_sender": { 2024-09-16T00:35:22.761 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_sender", 2024-09-16T00:35:22.761 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.761 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.761 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.762 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.762 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.762 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.762 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.762 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SMTP envelope sender", 2024-09-16T00:35:22.762 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.762 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.762 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.762 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.762 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_ssl": { 2024-09-16T00:35:22.762 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_ssl", 2024-09-16T00:35:22.764 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.764 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.764 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.764 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.764 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.764 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.764 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.764 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-16T00:35:22.764 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.764 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.765 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.765 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.765 INFO:teuthology.orchestra.run.smithi038.stdout: "smtp_user": { 2024-09-16T00:35:22.765 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "smtp_user", 2024-09-16T00:35:22.765 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.765 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.765 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.765 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.765 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.765 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.765 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.765 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "User to authenticate as", 2024-09-16T00:35:22.766 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.766 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.766 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.766 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.766 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.766 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.766 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.766 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "balancer", 2024-09-16T00:35:22.767 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.767 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.767 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.767 INFO:teuthology.orchestra.run.smithi038.stdout: "active": { 2024-09-16T00:35:22.767 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "active", 2024-09-16T00:35:22.767 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.767 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.767 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.767 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.767 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.767 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.768 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.768 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "automatically balance PGs across cluster", 2024-09-16T00:35:22.768 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.768 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.768 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.768 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.768 INFO:teuthology.orchestra.run.smithi038.stdout: "begin_time": { 2024-09-16T00:35:22.769 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "begin_time", 2024-09-16T00:35:22.769 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.769 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.769 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.769 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0000", 2024-09-16T00:35:22.769 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.769 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.770 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.770 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "beginning time of day to automatically balance", 2024-09-16T00:35:22.770 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T00:35:22.770 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.770 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.770 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.771 INFO:teuthology.orchestra.run.smithi038.stdout: "begin_weekday": { 2024-09-16T00:35:22.771 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "begin_weekday", 2024-09-16T00:35:22.771 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-09-16T00:35:22.771 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.771 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.771 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-09-16T00:35:22.771 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "0", 2024-09-16T00:35:22.771 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "6", 2024-09-16T00:35:22.771 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.772 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-16T00:35:22.772 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T00:35:22.772 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.772 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.772 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.772 INFO:teuthology.orchestra.run.smithi038.stdout: "crush_compat_max_iterations": { 2024-09-16T00:35:22.772 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crush_compat_max_iterations", 2024-09-16T00:35:22.773 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-09-16T00:35:22.773 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.773 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.773 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "25", 2024-09-16T00:35:22.773 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-09-16T00:35:22.773 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "250", 2024-09-16T00:35:22.773 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.773 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-16T00:35:22.773 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.774 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.774 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.774 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.774 INFO:teuthology.orchestra.run.smithi038.stdout: "crush_compat_metrics": { 2024-09-16T00:35:22.774 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crush_compat_metrics", 2024-09-16T00:35:22.774 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.774 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.775 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.775 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "pgs,objects,bytes", 2024-09-16T00:35:22.775 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.775 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.775 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.775 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-16T00:35:22.776 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-16T00:35:22.776 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.776 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.776 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.776 INFO:teuthology.orchestra.run.smithi038.stdout: "crush_compat_step": { 2024-09-16T00:35:22.776 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crush_compat_step", 2024-09-16T00:35:22.777 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-16T00:35:22.777 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.777 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.777 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0.5", 2024-09-16T00:35:22.777 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "0.001", 2024-09-16T00:35:22.778 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "0.999", 2024-09-16T00:35:22.778 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.778 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "aggressiveness of optimization", 2024-09-16T00:35:22.778 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-16T00:35:22.778 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.779 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.779 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.779 INFO:teuthology.orchestra.run.smithi038.stdout: "end_time": { 2024-09-16T00:35:22.779 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "end_time", 2024-09-16T00:35:22.779 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.779 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.780 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.780 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "2359", 2024-09-16T00:35:22.780 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.780 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.780 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.780 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "ending time of day to automatically balance", 2024-09-16T00:35:22.780 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-16T00:35:22.780 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.781 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.781 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.781 INFO:teuthology.orchestra.run.smithi038.stdout: "end_weekday": { 2024-09-16T00:35:22.781 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "end_weekday", 2024-09-16T00:35:22.781 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-09-16T00:35:22.782 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.782 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.782 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-09-16T00:35:22.782 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "0", 2024-09-16T00:35:22.782 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "6", 2024-09-16T00:35:22.782 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.783 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-16T00:35:22.783 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-16T00:35:22.783 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.783 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.783 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.783 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.783 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.784 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.784 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.784 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.784 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.784 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.784 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.784 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.785 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.785 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.785 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.785 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.785 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.785 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.785 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.785 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.785 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.785 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.786 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.786 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.786 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.786 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.786 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.786 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.786 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.786 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.786 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.786 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.786 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.787 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.787 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.787 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.787 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.787 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.787 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.787 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.788 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.788 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.788 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.788 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.788 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.788 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.788 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.789 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.789 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.789 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.789 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.789 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.789 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.789 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.789 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.789 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.789 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.789 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.790 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.790 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.790 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.790 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.790 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.790 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.790 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.790 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.790 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.790 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.791 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.791 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.791 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.791 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.791 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.791 INFO:teuthology.orchestra.run.smithi038.stdout: "min_score": { 2024-09-16T00:35:22.791 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "min_score", 2024-09-16T00:35:22.791 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-16T00:35:22.791 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.791 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.791 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-09-16T00:35:22.791 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.791 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.792 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.792 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-16T00:35:22.792 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.792 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.792 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.792 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.792 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": { 2024-09-16T00:35:22.792 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mode", 2024-09-16T00:35:22.792 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.792 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.792 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.793 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "upmap", 2024-09-16T00:35:22.793 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.793 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.793 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.793 INFO:teuthology.orchestra.run.smithi038.stdout: "crush-compat", 2024-09-16T00:35:22.793 INFO:teuthology.orchestra.run.smithi038.stdout: "none", 2024-09-16T00:35:22.793 INFO:teuthology.orchestra.run.smithi038.stdout: "read", 2024-09-16T00:35:22.793 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap", 2024-09-16T00:35:22.793 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap-read" 2024-09-16T00:35:22.793 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.794 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Balancer mode", 2024-09-16T00:35:22.794 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.794 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.794 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.794 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.794 INFO:teuthology.orchestra.run.smithi038.stdout: "pool_ids": { 2024-09-16T00:35:22.794 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pool_ids", 2024-09-16T00:35:22.794 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.794 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.795 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.795 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.795 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.795 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.795 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.795 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-16T00:35:22.795 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.795 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.795 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.795 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.796 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-09-16T00:35:22.796 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-09-16T00:35:22.796 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:22.796 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.796 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.796 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-09-16T00:35:22.796 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.796 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.797 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.797 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-16T00:35:22.797 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.797 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.797 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.797 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.797 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap_max_deviation": { 2024-09-16T00:35:22.797 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "upmap_max_deviation", 2024-09-16T00:35:22.797 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.797 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.798 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.798 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-09-16T00:35:22.798 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-09-16T00:35:22.798 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.798 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.798 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-16T00:35:22.798 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-16T00:35:22.798 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.799 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.799 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.799 INFO:teuthology.orchestra.run.smithi038.stdout: "upmap_max_optimizations": { 2024-09-16T00:35:22.799 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "upmap_max_optimizations", 2024-09-16T00:35:22.799 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-09-16T00:35:22.799 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.799 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.799 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-16T00:35:22.799 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.800 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.800 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.800 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-16T00:35:22.800 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.800 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.800 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.800 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.800 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.800 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.800 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.800 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cephadm", 2024-09-16T00:35:22.801 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.801 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.801 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.801 INFO:teuthology.orchestra.run.smithi038.stdout: "agent_down_multiplier": { 2024-09-16T00:35:22.801 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "agent_down_multiplier", 2024-09-16T00:35:22.801 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-16T00:35:22.801 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.801 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.801 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "3.0", 2024-09-16T00:35:22.801 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.802 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.802 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.802 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-16T00:35:22.802 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.802 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.802 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.802 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.802 INFO:teuthology.orchestra.run.smithi038.stdout: "agent_refresh_rate": { 2024-09-16T00:35:22.802 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "agent_refresh_rate", 2024-09-16T00:35:22.802 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:22.802 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.803 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.803 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "20", 2024-09-16T00:35:22.803 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.803 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.803 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.803 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-16T00:35:22.804 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.804 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.804 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.804 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.804 INFO:teuthology.orchestra.run.smithi038.stdout: "agent_starting_port": { 2024-09-16T00:35:22.804 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "agent_starting_port", 2024-09-16T00:35:22.804 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.804 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.804 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.804 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "4721", 2024-09-16T00:35:22.805 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.805 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.805 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.805 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-16T00:35:22.805 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.805 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.806 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.806 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.806 INFO:teuthology.orchestra.run.smithi038.stdout: "allow_ptrace": { 2024-09-16T00:35:22.806 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "allow_ptrace", 2024-09-16T00:35:22.806 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.806 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.806 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.807 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.807 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.807 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.807 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.807 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-16T00:35:22.808 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-16T00:35:22.808 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.808 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.808 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.809 INFO:teuthology.orchestra.run.smithi038.stdout: "autotune_interval": { 2024-09-16T00:35:22.809 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "autotune_interval", 2024-09-16T00:35:22.809 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:22.809 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.809 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.809 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-09-16T00:35:22.809 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.809 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.809 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.809 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-16T00:35:22.810 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.810 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.810 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.810 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.810 INFO:teuthology.orchestra.run.smithi038.stdout: "autotune_memory_target_ratio": { 2024-09-16T00:35:22.810 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "autotune_memory_target_ratio", 2024-09-16T00:35:22.811 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-16T00:35:22.811 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.811 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.811 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0.7", 2024-09-16T00:35:22.811 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.811 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.811 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.812 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-16T00:35:22.812 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.812 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.812 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.812 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.812 INFO:teuthology.orchestra.run.smithi038.stdout: "cephadm_log_destination": { 2024-09-16T00:35:22.812 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cephadm_log_destination", 2024-09-16T00:35:22.812 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.813 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.813 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.813 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.813 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.813 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.813 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.813 INFO:teuthology.orchestra.run.smithi038.stdout: "file", 2024-09-16T00:35:22.813 INFO:teuthology.orchestra.run.smithi038.stdout: "file,syslog", 2024-09-16T00:35:22.813 INFO:teuthology.orchestra.run.smithi038.stdout: "syslog" 2024-09-16T00:35:22.813 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.813 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-16T00:35:22.814 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.814 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.814 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.814 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.814 INFO:teuthology.orchestra.run.smithi038.stdout: "cgroups_split": { 2024-09-16T00:35:22.814 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cgroups_split", 2024-09-16T00:35:22.814 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.814 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.814 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.814 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.814 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.814 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.814 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.814 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-16T00:35:22.814 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.815 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.815 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.815 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.815 INFO:teuthology.orchestra.run.smithi038.stdout: "config_checks_enabled": { 2024-09-16T00:35:22.815 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "config_checks_enabled", 2024-09-16T00:35:22.815 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.815 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.816 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.816 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.816 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.816 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.816 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.816 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-16T00:35:22.816 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.816 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.817 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.817 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.817 INFO:teuthology.orchestra.run.smithi038.stdout: "config_dashboard": { 2024-09-16T00:35:22.817 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "config_dashboard", 2024-09-16T00:35:22.817 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.817 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.817 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.817 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.817 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.817 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.817 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.817 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-16T00:35:22.817 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.818 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.818 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.818 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.818 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_alertmanager": { 2024-09-16T00:35:22.818 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_alertmanager", 2024-09-16T00:35:22.818 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.818 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.818 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.818 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-09-16T00:35:22.818 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.819 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.819 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.819 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-09-16T00:35:22.819 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.819 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.819 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.819 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.819 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_base": { 2024-09-16T00:35:22.819 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_base", 2024-09-16T00:35:22.819 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.819 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.819 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.819 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-16T00:35:22.820 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.820 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.820 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.820 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Container image name, without the tag", 2024-09-16T00:35:22.820 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.820 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.820 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.820 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.820 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_elasticsearch": { 2024-09-16T00:35:22.820 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_elasticsearch", 2024-09-16T00:35:22.820 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.820 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.820 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.821 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-16T00:35:22.821 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.821 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.821 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.821 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "elasticsearch container image", 2024-09-16T00:35:22.821 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.821 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.821 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.821 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.821 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_grafana": { 2024-09-16T00:35:22.821 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_grafana", 2024-09-16T00:35:22.821 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.821 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.821 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.822 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-09-16T00:35:22.822 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.822 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.822 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.822 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-09-16T00:35:22.822 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.822 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.822 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.822 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.822 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_haproxy": { 2024-09-16T00:35:22.822 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_haproxy", 2024-09-16T00:35:22.822 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.822 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.822 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.823 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-16T00:35:22.823 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.823 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.823 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.823 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "HAproxy container image", 2024-09-16T00:35:22.823 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.823 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.823 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.823 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.823 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_jaeger_agent": { 2024-09-16T00:35:22.823 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_jaeger_agent", 2024-09-16T00:35:22.823 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.823 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.823 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.824 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-16T00:35:22.824 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.824 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.824 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.824 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Jaeger agent container image", 2024-09-16T00:35:22.824 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.824 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.824 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.824 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.824 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_jaeger_collector": { 2024-09-16T00:35:22.824 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_jaeger_collector", 2024-09-16T00:35:22.824 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.824 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.825 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.825 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-16T00:35:22.825 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.825 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.825 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.825 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Jaeger collector container image", 2024-09-16T00:35:22.825 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.825 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.825 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.825 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.825 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_jaeger_query": { 2024-09-16T00:35:22.825 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_jaeger_query", 2024-09-16T00:35:22.825 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.825 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.826 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.826 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-16T00:35:22.826 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.826 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.826 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.826 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Jaeger query container image", 2024-09-16T00:35:22.826 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.826 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.826 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.826 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.826 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_keepalived": { 2024-09-16T00:35:22.826 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_keepalived", 2024-09-16T00:35:22.826 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.826 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.826 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.827 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-16T00:35:22.827 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.827 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.827 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.827 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Keepalived container image", 2024-09-16T00:35:22.827 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.827 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.827 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.827 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.827 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_loki": { 2024-09-16T00:35:22.827 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_loki", 2024-09-16T00:35:22.827 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.827 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.827 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.828 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-16T00:35:22.828 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.828 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.828 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.828 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Loki container image", 2024-09-16T00:35:22.828 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.828 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.828 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.828 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.828 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_node_exporter": { 2024-09-16T00:35:22.828 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_node_exporter", 2024-09-16T00:35:22.828 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.829 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.829 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.829 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-09-16T00:35:22.829 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.829 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.829 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.829 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-09-16T00:35:22.829 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.829 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.829 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.829 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.829 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_nvmeof": { 2024-09-16T00:35:22.829 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_nvmeof", 2024-09-16T00:35:22.829 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.830 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.830 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.830 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.5", 2024-09-16T00:35:22.830 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.830 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.830 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.830 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Nvme-of container image", 2024-09-16T00:35:22.830 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.830 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.830 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.830 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.830 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_prometheus": { 2024-09-16T00:35:22.830 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_prometheus", 2024-09-16T00:35:22.830 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.831 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.831 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.831 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-09-16T00:35:22.831 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.831 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.831 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.831 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Prometheus container image", 2024-09-16T00:35:22.831 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.831 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.831 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.832 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.832 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_promtail": { 2024-09-16T00:35:22.832 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_promtail", 2024-09-16T00:35:22.832 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.832 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.832 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.832 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-16T00:35:22.832 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.832 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.832 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.832 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Promtail container image", 2024-09-16T00:35:22.832 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.832 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.833 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.833 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.833 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_samba": { 2024-09-16T00:35:22.833 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_samba", 2024-09-16T00:35:22.833 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.833 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.833 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.833 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-16T00:35:22.834 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.834 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.834 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.834 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Samba/SMB container image", 2024-09-16T00:35:22.834 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.834 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.834 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.834 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.834 INFO:teuthology.orchestra.run.smithi038.stdout: "container_image_snmp_gateway": { 2024-09-16T00:35:22.834 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_image_snmp_gateway", 2024-09-16T00:35:22.834 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.834 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.834 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.835 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-16T00:35:22.835 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.835 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.835 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.835 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "SNMP Gateway container image", 2024-09-16T00:35:22.835 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.835 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.835 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.835 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.835 INFO:teuthology.orchestra.run.smithi038.stdout: "container_init": { 2024-09-16T00:35:22.835 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "container_init", 2024-09-16T00:35:22.835 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.835 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.836 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.836 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.836 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.836 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.836 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.836 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Run podman/docker with `--init`", 2024-09-16T00:35:22.836 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.836 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.836 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.836 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.836 INFO:teuthology.orchestra.run.smithi038.stdout: "daemon_cache_timeout": { 2024-09-16T00:35:22.836 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "daemon_cache_timeout", 2024-09-16T00:35:22.836 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:22.836 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.837 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.837 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-09-16T00:35:22.837 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.837 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.837 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.837 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-16T00:35:22.837 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.837 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.837 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.837 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.837 INFO:teuthology.orchestra.run.smithi038.stdout: "default_cephadm_command_timeout": { 2024-09-16T00:35:22.837 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "default_cephadm_command_timeout", 2024-09-16T00:35:22.837 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.837 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.838 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.838 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "900", 2024-09-16T00:35:22.838 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.838 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.838 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.838 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-16T00:35:22.838 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.838 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.838 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.838 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.838 INFO:teuthology.orchestra.run.smithi038.stdout: "default_registry": { 2024-09-16T00:35:22.838 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "default_registry", 2024-09-16T00:35:22.838 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.839 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.839 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.839 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "docker.io", 2024-09-16T00:35:22.839 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.839 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.839 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.839 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-16T00:35:22.839 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.839 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.839 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.839 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.839 INFO:teuthology.orchestra.run.smithi038.stdout: "device_cache_timeout": { 2024-09-16T00:35:22.841 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "device_cache_timeout", 2024-09-16T00:35:22.841 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:22.841 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.841 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.841 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "1800", 2024-09-16T00:35:22.841 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.841 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.841 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.841 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "seconds to cache device inventory", 2024-09-16T00:35:22.841 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.842 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.842 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.842 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.842 INFO:teuthology.orchestra.run.smithi038.stdout: "device_enhanced_scan": { 2024-09-16T00:35:22.842 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "device_enhanced_scan", 2024-09-16T00:35:22.842 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.842 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.842 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.842 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.842 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.842 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.842 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.842 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-16T00:35:22.842 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.843 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.843 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.843 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.843 INFO:teuthology.orchestra.run.smithi038.stdout: "facts_cache_timeout": { 2024-09-16T00:35:22.843 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "facts_cache_timeout", 2024-09-16T00:35:22.843 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:22.843 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.843 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.843 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-09-16T00:35:22.843 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.843 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.843 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.843 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "seconds to cache host facts data", 2024-09-16T00:35:22.843 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.843 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.844 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.844 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.844 INFO:teuthology.orchestra.run.smithi038.stdout: "grafana_dashboards_path": { 2024-09-16T00:35:22.844 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "grafana_dashboards_path", 2024-09-16T00:35:22.844 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.844 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.844 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.844 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-16T00:35:22.844 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.844 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.844 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.844 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-16T00:35:22.844 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.844 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.845 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.845 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.845 INFO:teuthology.orchestra.run.smithi038.stdout: "host_check_interval": { 2024-09-16T00:35:22.845 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "host_check_interval", 2024-09-16T00:35:22.845 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:22.845 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.845 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.845 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-09-16T00:35:22.845 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.845 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.845 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.845 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to perform a host check", 2024-09-16T00:35:22.845 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.845 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.846 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.846 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.846 INFO:teuthology.orchestra.run.smithi038.stdout: "hw_monitoring": { 2024-09-16T00:35:22.846 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "hw_monitoring", 2024-09-16T00:35:22.846 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.846 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.846 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.846 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.846 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.846 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.846 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.846 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-16T00:35:22.846 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.846 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.847 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.847 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.847 INFO:teuthology.orchestra.run.smithi038.stdout: "inventory_list_all": { 2024-09-16T00:35:22.847 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "inventory_list_all", 2024-09-16T00:35:22.847 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.847 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.847 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.847 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.847 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.847 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.847 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.847 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-16T00:35:22.847 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.848 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.848 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.848 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.848 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.848 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.848 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.848 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.848 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.848 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.848 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.848 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.848 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.849 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.849 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.849 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.849 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.849 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.849 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.849 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.849 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.849 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.849 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.849 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.849 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.849 INFO:teuthology.orchestra.run.smithi038.stdout: "log_refresh_metadata": { 2024-09-16T00:35:22.850 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_refresh_metadata", 2024-09-16T00:35:22.850 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.850 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.850 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.850 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.850 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.850 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.850 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.850 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-16T00:35:22.850 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.850 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.850 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.850 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.850 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.851 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.851 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.851 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.851 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.851 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.851 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.851 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.851 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.851 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-16T00:35:22.851 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.851 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.851 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.852 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.852 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.852 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.852 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.852 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.852 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.852 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.852 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.852 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.852 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.852 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.852 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.852 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.853 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.853 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.853 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.853 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.853 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.853 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.853 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.853 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.853 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.853 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.853 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.853 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.853 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.853 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.854 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.854 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.854 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.854 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.854 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.854 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.854 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.854 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.854 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.854 INFO:teuthology.orchestra.run.smithi038.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-16T00:35:22.854 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-16T00:35:22.855 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.855 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.855 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.855 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.855 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.855 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.855 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.855 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-16T00:35:22.855 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.855 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.855 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.855 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.855 INFO:teuthology.orchestra.run.smithi038.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-16T00:35:22.855 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-16T00:35:22.856 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.856 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.856 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.856 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "*", 2024-09-16T00:35:22.856 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.856 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.856 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.857 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-16T00:35:22.857 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.857 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.857 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.857 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.857 INFO:teuthology.orchestra.run.smithi038.stdout: "max_count_per_host": { 2024-09-16T00:35:22.857 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_count_per_host", 2024-09-16T00:35:22.858 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.858 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.858 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.858 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-16T00:35:22.858 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.858 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.858 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.859 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "max number of daemons per service per host", 2024-09-16T00:35:22.859 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.859 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.859 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.859 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.859 INFO:teuthology.orchestra.run.smithi038.stdout: "max_osd_draining_count": { 2024-09-16T00:35:22.859 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_osd_draining_count", 2024-09-16T00:35:22.859 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.859 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.859 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.859 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-16T00:35:22.859 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.860 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.860 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.860 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-16T00:35:22.860 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.860 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.860 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.860 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.860 INFO:teuthology.orchestra.run.smithi038.stdout: "migration_current": { 2024-09-16T00:35:22.860 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "migration_current", 2024-09-16T00:35:22.860 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.860 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.860 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.860 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.861 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.861 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.861 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.861 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "internal - do not modify", 2024-09-16T00:35:22.861 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.861 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.861 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.861 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.861 INFO:teuthology.orchestra.run.smithi038.stdout: "mode": { 2024-09-16T00:35:22.861 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mode", 2024-09-16T00:35:22.861 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.862 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.862 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.862 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "root", 2024-09-16T00:35:22.862 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.862 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.862 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.862 INFO:teuthology.orchestra.run.smithi038.stdout: "cephadm-package", 2024-09-16T00:35:22.862 INFO:teuthology.orchestra.run.smithi038.stdout: "root" 2024-09-16T00:35:22.862 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.862 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "mode for remote execution of cephadm", 2024-09-16T00:35:22.862 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.862 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.863 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.863 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.863 INFO:teuthology.orchestra.run.smithi038.stdout: "oob_default_addr": { 2024-09-16T00:35:22.863 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "oob_default_addr", 2024-09-16T00:35:22.863 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.863 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.863 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.863 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "169.254.1.1", 2024-09-16T00:35:22.863 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.863 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.863 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.863 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-16T00:35:22.863 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.864 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.864 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.864 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.864 INFO:teuthology.orchestra.run.smithi038.stdout: "prometheus_alerts_path": { 2024-09-16T00:35:22.864 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "prometheus_alerts_path", 2024-09-16T00:35:22.864 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.864 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.864 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.864 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-16T00:35:22.864 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.864 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.864 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.864 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-16T00:35:22.864 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.865 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.865 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.865 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.865 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_insecure": { 2024-09-16T00:35:22.865 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_insecure", 2024-09-16T00:35:22.865 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.865 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.865 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.865 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.865 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.865 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.866 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.866 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-16T00:35:22.866 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.866 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.866 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.866 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.867 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_password": { 2024-09-16T00:35:22.867 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_password", 2024-09-16T00:35:22.867 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.867 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.867 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.867 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.868 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.868 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.868 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.868 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-16T00:35:22.868 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.868 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.869 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.869 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.869 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_url": { 2024-09-16T00:35:22.869 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_url", 2024-09-16T00:35:22.869 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.869 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.869 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.870 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.870 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.870 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.870 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.870 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-16T00:35:22.870 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.870 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.870 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.871 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.871 INFO:teuthology.orchestra.run.smithi038.stdout: "registry_username": { 2024-09-16T00:35:22.871 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "registry_username", 2024-09-16T00:35:22.871 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.871 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.871 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.871 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.871 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.871 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.872 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.872 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-16T00:35:22.872 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.872 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.872 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.872 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.872 INFO:teuthology.orchestra.run.smithi038.stdout: "secure_monitoring_stack": { 2024-09-16T00:35:22.872 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "secure_monitoring_stack", 2024-09-16T00:35:22.873 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.873 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.873 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.873 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.873 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.873 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.873 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.873 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-16T00:35:22.874 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.874 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.874 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.874 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.874 INFO:teuthology.orchestra.run.smithi038.stdout: "service_discovery_port": { 2024-09-16T00:35:22.874 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "service_discovery_port", 2024-09-16T00:35:22.874 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.874 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.875 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.875 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8765", 2024-09-16T00:35:22.875 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.876 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.876 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.876 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "cephadm service discovery port", 2024-09-16T00:35:22.876 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.876 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.876 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.876 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.876 INFO:teuthology.orchestra.run.smithi038.stdout: "ssh_config_file": { 2024-09-16T00:35:22.876 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssh_config_file", 2024-09-16T00:35:22.876 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.876 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.876 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.876 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.877 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.877 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.877 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.877 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-16T00:35:22.877 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.877 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.877 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.877 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.877 INFO:teuthology.orchestra.run.smithi038.stdout: "use_agent": { 2024-09-16T00:35:22.877 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "use_agent", 2024-09-16T00:35:22.877 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.877 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.877 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.878 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.878 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.878 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.878 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.878 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-16T00:35:22.878 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.878 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.878 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.878 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.878 INFO:teuthology.orchestra.run.smithi038.stdout: "use_repo_digest": { 2024-09-16T00:35:22.878 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "use_repo_digest", 2024-09-16T00:35:22.878 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.878 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.879 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.879 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.879 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.879 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.879 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.879 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-16T00:35:22.879 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.879 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.879 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.879 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.879 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_on_failed_host_check": { 2024-09-16T00:35:22.879 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_on_failed_host_check", 2024-09-16T00:35:22.879 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.879 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.880 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.880 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.880 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.880 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.880 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.880 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise a health warning if the host check fails", 2024-09-16T00:35:22.880 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.880 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.880 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.880 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.880 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_on_stray_daemons": { 2024-09-16T00:35:22.880 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_on_stray_daemons", 2024-09-16T00:35:22.880 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.881 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.881 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.881 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.881 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.881 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.882 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.882 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-16T00:35:22.882 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.882 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.882 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.882 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.882 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_on_stray_hosts": { 2024-09-16T00:35:22.882 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_on_stray_hosts", 2024-09-16T00:35:22.882 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.882 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.882 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.882 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.882 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.883 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.883 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.883 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-16T00:35:22.883 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.883 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.883 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.883 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.883 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.883 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.883 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.883 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crash", 2024-09-16T00:35:22.883 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.883 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.883 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.884 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.884 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.884 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.884 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.884 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.884 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.884 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.884 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.884 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.884 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.884 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.884 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.884 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.884 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.885 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.885 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.885 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.885 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.885 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.885 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.885 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.885 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.885 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.885 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.885 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.885 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.886 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.886 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.887 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.887 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.887 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.887 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.887 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.887 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.887 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.887 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.887 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.888 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.888 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.888 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.888 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.888 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.888 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.888 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.888 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.888 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.888 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.888 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.888 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.888 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.888 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.889 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.889 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.889 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.889 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.889 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.889 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.889 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.889 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.889 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.890 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.890 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.890 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.890 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.890 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.890 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.891 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.891 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.891 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.891 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.891 INFO:teuthology.orchestra.run.smithi038.stdout: "retain_interval": { 2024-09-16T00:35:22.891 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "retain_interval", 2024-09-16T00:35:22.891 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:22.892 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.892 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.892 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "31536000", 2024-09-16T00:35:22.892 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.892 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.892 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.892 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-16T00:35:22.892 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.893 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.893 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.893 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.893 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_recent_interval": { 2024-09-16T00:35:22.893 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_recent_interval", 2024-09-16T00:35:22.893 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:22.893 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.893 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.893 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "1209600", 2024-09-16T00:35:22.894 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.894 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.894 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.894 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-16T00:35:22.894 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.894 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.894 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.894 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.894 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.895 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.895 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.895 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "dashboard", 2024-09-16T00:35:22.895 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.895 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.895 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.895 INFO:teuthology.orchestra.run.smithi038.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-16T00:35:22.895 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-16T00:35:22.896 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.896 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.896 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.896 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-16T00:35:22.896 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.896 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.896 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.896 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.896 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.897 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.897 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.897 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.897 INFO:teuthology.orchestra.run.smithi038.stdout: "ALERTMANAGER_API_HOST": { 2024-09-16T00:35:22.897 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-16T00:35:22.897 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.897 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.897 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.898 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.898 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.898 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.898 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.898 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.898 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.898 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.898 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.898 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.898 INFO:teuthology.orchestra.run.smithi038.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-16T00:35:22.899 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-16T00:35:22.899 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.899 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.899 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.899 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.899 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.899 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.899 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.899 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.899 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.899 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.900 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.900 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.900 INFO:teuthology.orchestra.run.smithi038.stdout: "AUDIT_API_ENABLED": { 2024-09-16T00:35:22.900 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "AUDIT_API_ENABLED", 2024-09-16T00:35:22.900 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.900 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.900 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.900 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.900 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.900 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.901 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.901 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.901 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.901 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.901 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.901 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.901 INFO:teuthology.orchestra.run.smithi038.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-16T00:35:22.901 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-16T00:35:22.901 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.901 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.901 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.902 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.902 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.902 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.902 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.902 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.902 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.902 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.902 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.902 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.902 INFO:teuthology.orchestra.run.smithi038.stdout: "ENABLE_BROWSABLE_API": { 2024-09-16T00:35:22.902 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-16T00:35:22.902 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.903 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.903 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.903 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.903 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.903 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.903 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.903 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.903 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.903 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.903 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.903 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.903 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-16T00:35:22.904 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-16T00:35:22.904 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.904 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.904 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.904 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.904 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.904 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.905 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.905 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.905 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.905 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.905 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.905 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.905 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-16T00:35:22.905 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-16T00:35:22.905 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.905 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.906 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.906 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.906 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.906 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.906 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.906 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.906 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.906 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.906 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.906 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.906 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-16T00:35:22.906 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-16T00:35:22.907 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.907 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.907 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.907 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.907 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.907 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.907 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.907 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.907 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.907 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.907 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.908 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.908 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-16T00:35:22.908 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-16T00:35:22.908 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.908 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.908 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.908 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.908 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.908 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.908 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.909 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.909 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.909 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.909 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.909 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.909 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-16T00:35:22.909 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-16T00:35:22.909 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.909 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.909 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.910 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.910 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.910 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.910 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.910 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.910 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.910 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.910 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.910 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.910 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-16T00:35:22.911 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-16T00:35:22.911 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.911 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.911 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.911 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.911 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.911 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.911 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.911 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.911 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.911 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.911 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.912 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.912 INFO:teuthology.orchestra.run.smithi038.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-16T00:35:22.912 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-16T00:35:22.912 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.912 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.912 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.912 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.912 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.912 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.912 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.912 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.912 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.913 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.913 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.913 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.913 INFO:teuthology.orchestra.run.smithi038.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-16T00:35:22.913 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-16T00:35:22.913 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.913 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.913 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.913 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.913 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.914 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.914 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.914 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.914 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.914 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.914 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.914 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.914 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_PASSWORD": { 2024-09-16T00:35:22.914 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-16T00:35:22.914 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.915 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.915 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.915 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "admin", 2024-09-16T00:35:22.915 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.915 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.915 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.915 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.915 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.915 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.915 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.915 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.915 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-16T00:35:22.916 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-16T00:35:22.916 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.916 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.916 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.916 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.916 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.916 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.916 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.916 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.916 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.916 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.917 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.917 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.917 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_URL": { 2024-09-16T00:35:22.917 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_URL", 2024-09-16T00:35:22.917 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.917 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.917 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.917 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.917 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.917 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.917 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.917 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.918 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.918 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.918 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.918 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.918 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_API_USERNAME": { 2024-09-16T00:35:22.918 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-16T00:35:22.918 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.918 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.918 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.918 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "admin", 2024-09-16T00:35:22.918 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.918 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.919 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.919 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.919 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.919 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.919 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.919 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.919 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-16T00:35:22.919 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-16T00:35:22.919 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.919 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.919 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.919 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.920 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.920 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.920 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.920 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.920 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.920 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.920 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.920 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.920 INFO:teuthology.orchestra.run.smithi038.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-16T00:35:22.920 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-16T00:35:22.920 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.921 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.921 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.921 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.921 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.921 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.921 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.922 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.922 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.922 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.922 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.922 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.922 INFO:teuthology.orchestra.run.smithi038.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-16T00:35:22.922 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-16T00:35:22.922 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.922 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.923 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.923 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.923 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.923 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.923 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.923 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.923 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.923 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.923 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.923 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.923 INFO:teuthology.orchestra.run.smithi038.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-16T00:35:22.923 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-16T00:35:22.924 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.924 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.924 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.924 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.924 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.924 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.924 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.924 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.924 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.924 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.924 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.924 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.925 INFO:teuthology.orchestra.run.smithi038.stdout: "PROMETHEUS_API_HOST": { 2024-09-16T00:35:22.925 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-16T00:35:22.925 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.925 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.925 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.925 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.925 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.925 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.925 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.925 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.925 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.926 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.926 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.926 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.926 INFO:teuthology.orchestra.run.smithi038.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-16T00:35:22.926 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-16T00:35:22.926 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.926 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.926 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.926 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.926 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.926 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.926 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.927 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.927 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.927 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.927 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.927 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.927 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-16T00:35:22.927 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-16T00:35:22.927 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.927 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.927 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.927 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.927 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.928 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.928 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.928 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.928 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.928 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.928 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.928 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.928 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-16T00:35:22.928 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-16T00:35:22.928 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.928 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.928 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.929 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.929 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.929 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.929 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.929 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.929 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.929 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.929 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.929 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.929 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-16T00:35:22.929 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-16T00:35:22.929 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.930 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.930 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.930 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.930 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.930 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.930 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.930 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.930 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.930 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.930 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.930 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.930 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-16T00:35:22.931 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-16T00:35:22.931 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.931 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.931 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.931 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.931 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.931 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.931 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.931 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.931 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.931 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.931 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.932 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.932 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-16T00:35:22.932 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-16T00:35:22.932 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.932 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.932 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.932 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.932 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.932 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.932 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.932 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.932 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.933 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.933 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.933 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.933 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-16T00:35:22.933 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-16T00:35:22.933 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.933 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.933 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.933 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.933 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.933 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.934 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.934 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.934 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.934 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.934 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.934 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.934 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-16T00:35:22.934 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-16T00:35:22.934 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.934 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.934 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.934 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.935 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.935 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.935 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.935 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.935 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.935 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.935 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.935 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.935 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_ENABLED": { 2024-09-16T00:35:22.935 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-16T00:35:22.935 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.935 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.936 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.936 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.936 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.936 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.936 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.936 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.936 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.936 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.936 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.936 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.936 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-16T00:35:22.937 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-16T00:35:22.937 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.937 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.937 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.937 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-16T00:35:22.937 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.937 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.937 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.938 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.938 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.938 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.938 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.938 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.938 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-16T00:35:22.938 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-16T00:35:22.938 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.938 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.938 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.938 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-16T00:35:22.938 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.939 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.939 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.939 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.939 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.939 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.939 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.939 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.939 INFO:teuthology.orchestra.run.smithi038.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-16T00:35:22.939 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-16T00:35:22.939 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.939 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.939 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.940 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8", 2024-09-16T00:35:22.940 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.940 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.940 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.940 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.940 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.940 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.940 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.941 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.941 INFO:teuthology.orchestra.run.smithi038.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-16T00:35:22.941 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-16T00:35:22.941 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.941 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.941 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.941 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "45", 2024-09-16T00:35:22.941 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.941 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.941 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.941 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.942 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.942 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.942 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.942 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.942 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_ACCESS_KEY": { 2024-09-16T00:35:22.942 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-16T00:35:22.942 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.942 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.942 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.942 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.942 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.942 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.943 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.943 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.943 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.943 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.943 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.943 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.943 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-16T00:35:22.943 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-16T00:35:22.943 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.943 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.943 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.943 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "admin", 2024-09-16T00:35:22.944 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.944 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.944 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.944 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.944 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.944 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.944 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.944 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.944 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_SECRET_KEY": { 2024-09-16T00:35:22.944 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-16T00:35:22.944 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.944 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.945 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.945 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.945 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.945 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.945 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.945 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.945 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.945 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.945 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.945 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.945 INFO:teuthology.orchestra.run.smithi038.stdout: "RGW_API_SSL_VERIFY": { 2024-09-16T00:35:22.945 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-16T00:35:22.946 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.946 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.946 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.946 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.946 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.946 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.946 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.946 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.946 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.946 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.946 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.946 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.946 INFO:teuthology.orchestra.run.smithi038.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-16T00:35:22.947 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-16T00:35:22.947 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.947 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.947 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.947 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-09-16T00:35:22.947 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.947 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.947 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.947 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.947 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.947 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.947 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.948 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.948 INFO:teuthology.orchestra.run.smithi038.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-16T00:35:22.948 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-16T00:35:22.948 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.948 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.948 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.948 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-16T00:35:22.948 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.948 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.948 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.948 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.948 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.949 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.949 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.949 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.949 INFO:teuthology.orchestra.run.smithi038.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-16T00:35:22.949 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-16T00:35:22.949 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.949 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.949 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.949 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-09-16T00:35:22.949 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.949 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.950 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.950 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.950 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.950 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.950 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.950 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.950 INFO:teuthology.orchestra.run.smithi038.stdout: "cross_origin_url": { 2024-09-16T00:35:22.950 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cross_origin_url", 2024-09-16T00:35:22.950 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.950 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.950 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.950 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.951 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.951 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.951 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.951 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.951 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.951 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.951 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.951 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.951 INFO:teuthology.orchestra.run.smithi038.stdout: "crt_file": { 2024-09-16T00:35:22.951 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "crt_file", 2024-09-16T00:35:22.951 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.951 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.952 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.952 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.952 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.952 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.952 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.952 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.952 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.952 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.952 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.952 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.952 INFO:teuthology.orchestra.run.smithi038.stdout: "debug": { 2024-09-16T00:35:22.952 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "debug", 2024-09-16T00:35:22.953 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.953 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.953 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.953 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.953 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.953 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.953 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.953 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Enable/disable debug options", 2024-09-16T00:35:22.953 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.953 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.953 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.954 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.954 INFO:teuthology.orchestra.run.smithi038.stdout: "jwt_token_ttl": { 2024-09-16T00:35:22.954 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "jwt_token_ttl", 2024-09-16T00:35:22.954 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.954 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.954 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.954 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "28800", 2024-09-16T00:35:22.954 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.954 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.954 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.954 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.955 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.955 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.955 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.955 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.955 INFO:teuthology.orchestra.run.smithi038.stdout: "key_file": { 2024-09-16T00:35:22.955 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "key_file", 2024-09-16T00:35:22.955 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.955 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.955 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.955 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.955 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.955 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.956 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.956 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.956 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.956 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.956 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.956 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.956 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.956 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.956 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.956 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.956 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.956 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.957 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.957 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.957 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.957 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.957 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.957 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.957 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.957 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.957 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.957 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.957 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.957 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.958 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.958 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.958 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.958 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.958 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.959 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.959 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.959 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.960 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.960 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.960 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.960 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.960 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.960 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.960 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.960 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.960 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.960 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.961 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.961 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.961 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.961 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.961 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.961 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.961 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.961 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.961 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.961 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.961 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.961 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.962 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.962 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.962 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.962 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.962 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.962 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.962 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.962 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.962 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.962 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.962 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.962 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.963 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.963 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.963 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.963 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.963 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.963 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.963 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.963 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.963 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.963 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.963 INFO:teuthology.orchestra.run.smithi038.stdout: "motd": { 2024-09-16T00:35:22.964 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "motd", 2024-09-16T00:35:22.964 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.964 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.964 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.964 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.964 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.964 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.964 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.964 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "The message of the day", 2024-09-16T00:35:22.964 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.964 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.965 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.965 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.965 INFO:teuthology.orchestra.run.smithi038.stdout: "redirect_resolve_ip_addr": { 2024-09-16T00:35:22.965 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "redirect_resolve_ip_addr", 2024-09-16T00:35:22.965 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.965 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.965 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.965 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.965 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.965 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.965 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.966 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.966 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.966 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.966 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.966 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.966 INFO:teuthology.orchestra.run.smithi038.stdout: "server_addr": { 2024-09-16T00:35:22.966 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_addr", 2024-09-16T00:35:22.966 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.966 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.966 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.967 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "::", 2024-09-16T00:35:22.967 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.967 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.967 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.967 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.967 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.967 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.967 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.967 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.967 INFO:teuthology.orchestra.run.smithi038.stdout: "server_port": { 2024-09-16T00:35:22.967 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_port", 2024-09-16T00:35:22.967 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.968 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.968 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.968 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8080", 2024-09-16T00:35:22.968 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.968 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.968 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.968 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.968 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.968 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.968 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.969 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.969 INFO:teuthology.orchestra.run.smithi038.stdout: "ssl": { 2024-09-16T00:35:22.969 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssl", 2024-09-16T00:35:22.969 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.969 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.969 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.969 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.969 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.969 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.969 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.969 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.969 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.970 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.970 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.970 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.970 INFO:teuthology.orchestra.run.smithi038.stdout: "ssl_server_port": { 2024-09-16T00:35:22.970 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssl_server_port", 2024-09-16T00:35:22.970 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.970 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.970 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.970 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8443", 2024-09-16T00:35:22.970 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.970 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.970 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.970 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.971 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.971 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.971 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.971 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.971 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_behaviour": { 2024-09-16T00:35:22.971 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_behaviour", 2024-09-16T00:35:22.971 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.971 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.971 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.971 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "redirect", 2024-09-16T00:35:22.971 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.971 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.972 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.972 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.972 INFO:teuthology.orchestra.run.smithi038.stdout: "redirect" 2024-09-16T00:35:22.972 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.972 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.972 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.972 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.972 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.972 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.972 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_error_status_code": { 2024-09-16T00:35:22.972 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_error_status_code", 2024-09-16T00:35:22.972 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:22.973 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.973 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.973 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "500", 2024-09-16T00:35:22.973 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "400", 2024-09-16T00:35:22.973 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "599", 2024-09-16T00:35:22.973 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.973 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.973 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.973 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.973 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.973 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.973 INFO:teuthology.orchestra.run.smithi038.stdout: "url_prefix": { 2024-09-16T00:35:22.973 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "url_prefix", 2024-09-16T00:35:22.974 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.974 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.974 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:22.974 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.974 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.974 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.974 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.974 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.974 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.974 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.974 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.974 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.975 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.975 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.975 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.975 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "devicehealth", 2024-09-16T00:35:22.975 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.975 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.975 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.975 INFO:teuthology.orchestra.run.smithi038.stdout: "enable_monitoring": { 2024-09-16T00:35:22.975 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enable_monitoring", 2024-09-16T00:35:22.975 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.975 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.976 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.976 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.976 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.976 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.976 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.976 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "monitor device health metrics", 2024-09-16T00:35:22.976 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.976 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.976 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.976 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.976 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.976 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.977 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.977 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.977 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.977 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.977 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.977 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.977 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.977 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.977 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.977 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.977 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.977 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.978 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.978 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.978 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.978 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.978 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.978 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.978 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.978 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.978 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.978 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.978 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.978 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.979 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.979 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.979 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.979 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.979 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.979 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.979 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.979 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.979 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.979 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.979 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.980 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.980 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.980 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.980 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.980 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.980 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.980 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.980 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.980 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.980 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.980 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.981 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.981 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.981 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.981 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.981 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.981 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.981 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.981 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.981 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.981 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.982 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.982 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.982 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.982 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.982 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.982 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.982 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.982 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.982 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.983 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.983 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.983 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.983 INFO:teuthology.orchestra.run.smithi038.stdout: "mark_out_threshold": { 2024-09-16T00:35:22.983 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mark_out_threshold", 2024-09-16T00:35:22.983 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:22.983 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.983 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.983 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "2419200", 2024-09-16T00:35:22.983 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.984 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.984 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.984 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-16T00:35:22.984 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.984 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.984 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.984 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.984 INFO:teuthology.orchestra.run.smithi038.stdout: "pool_name": { 2024-09-16T00:35:22.984 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pool_name", 2024-09-16T00:35:22.984 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.985 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.985 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.985 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "device_health_metrics", 2024-09-16T00:35:22.985 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.985 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.985 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.985 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-16T00:35:22.985 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.985 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.986 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.986 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.986 INFO:teuthology.orchestra.run.smithi038.stdout: "retention_period": { 2024-09-16T00:35:22.986 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "retention_period", 2024-09-16T00:35:22.986 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:22.986 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.986 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.986 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "15552000", 2024-09-16T00:35:22.986 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.986 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.986 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.987 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how long to retain device health metrics", 2024-09-16T00:35:22.987 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.987 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.987 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.987 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.987 INFO:teuthology.orchestra.run.smithi038.stdout: "scrape_frequency": { 2024-09-16T00:35:22.987 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "scrape_frequency", 2024-09-16T00:35:22.987 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:22.987 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.988 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.988 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "86400", 2024-09-16T00:35:22.988 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.988 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.988 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.988 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-16T00:35:22.988 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.988 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.988 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.988 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.989 INFO:teuthology.orchestra.run.smithi038.stdout: "self_heal": { 2024-09-16T00:35:22.989 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "self_heal", 2024-09-16T00:35:22.989 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.989 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.989 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.989 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:22.989 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.989 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.989 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.990 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-16T00:35:22.990 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.990 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.990 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.990 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.990 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-09-16T00:35:22.990 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-09-16T00:35:22.990 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:22.990 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.990 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.991 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-09-16T00:35:22.991 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.991 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.991 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.991 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how frequently to wake up and check device health", 2024-09-16T00:35:22.991 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.991 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.991 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.991 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.991 INFO:teuthology.orchestra.run.smithi038.stdout: "warn_threshold": { 2024-09-16T00:35:22.992 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "warn_threshold", 2024-09-16T00:35:22.992 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:22.992 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.992 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.992 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "7257600", 2024-09-16T00:35:22.992 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.992 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.992 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.992 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-16T00:35:22.992 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.993 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.993 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.993 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.993 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:22.993 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.993 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:22.993 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "diskprediction_local", 2024-09-16T00:35:22.993 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:22.993 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:22.993 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:22.993 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:22.994 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:22.994 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.994 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.994 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.994 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:22.994 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.994 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.994 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.994 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.994 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.994 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.994 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.994 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.995 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.995 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.995 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.995 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.995 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.995 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.995 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.995 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:22.995 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:22.995 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.995 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.995 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.996 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.996 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.996 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.996 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.996 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.996 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.996 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.996 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.996 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.997 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:22.997 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:22.997 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:22.997 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.997 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.997 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:22.997 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.997 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.997 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:22.997 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:22.997 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:22.998 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:22.998 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:22.998 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:22.998 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:22.998 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:22.998 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.998 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.998 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.998 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:22.998 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:22.998 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:22.999 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:22.999 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:22.999 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:22.999 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:22.999 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:22.999 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:22.999 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:22.999 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:22.999 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:22.999 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:22.999 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:22.999 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.000 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.000 INFO:teuthology.orchestra.run.smithi038.stdout: "predict_interval": { 2024-09-16T00:35:23.000 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "predict_interval", 2024-09-16T00:35:23.000 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.000 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.000 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.000 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "86400", 2024-09-16T00:35:23.000 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.000 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.000 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.000 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.001 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.001 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.001 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.001 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.001 INFO:teuthology.orchestra.run.smithi038.stdout: "predictor_model": { 2024-09-16T00:35:23.001 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "predictor_model", 2024-09-16T00:35:23.001 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.001 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.001 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.001 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "prophetstor", 2024-09-16T00:35:23.001 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.001 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.002 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.002 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.002 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.002 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.002 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.002 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.002 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-09-16T00:35:23.002 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-09-16T00:35:23.002 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.002 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.002 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.002 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "600", 2024-09-16T00:35:23.003 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.003 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.003 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.003 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.003 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.003 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.003 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.003 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.003 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.003 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.003 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:23.003 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "influx", 2024-09-16T00:35:23.004 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": false, 2024-09-16T00:35:23.004 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "influxdb python module not found", 2024-09-16T00:35:23.004 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:23.004 INFO:teuthology.orchestra.run.smithi038.stdout: "batch_size": { 2024-09-16T00:35:23.004 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "batch_size", 2024-09-16T00:35:23.004 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:23.004 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.004 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.004 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5000", 2024-09-16T00:35:23.004 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.004 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.005 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.005 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-16T00:35:23.005 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.005 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.005 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.006 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.006 INFO:teuthology.orchestra.run.smithi038.stdout: "database": { 2024-09-16T00:35:23.006 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "database", 2024-09-16T00:35:23.006 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.006 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.006 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.006 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "ceph", 2024-09-16T00:35:23.006 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.006 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.006 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.006 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-16T00:35:23.006 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.007 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.007 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.007 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.007 INFO:teuthology.orchestra.run.smithi038.stdout: "hostname": { 2024-09-16T00:35:23.007 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "hostname", 2024-09-16T00:35:23.007 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.007 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.007 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.007 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.007 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.007 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.007 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.008 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "InfluxDB server hostname", 2024-09-16T00:35:23.008 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.008 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.008 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.008 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.008 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-09-16T00:35:23.008 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-09-16T00:35:23.008 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:23.008 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.008 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.008 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "30", 2024-09-16T00:35:23.008 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "5", 2024-09-16T00:35:23.009 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.009 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.009 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-16T00:35:23.009 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.009 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.009 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.009 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.009 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:23.009 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:23.009 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.009 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.010 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.010 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.010 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.010 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.010 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.010 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.010 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.010 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.010 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.010 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.010 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.010 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.011 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.011 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.011 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.011 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.011 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.011 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:23.011 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:23.011 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.011 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.011 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.011 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.011 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.012 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.012 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.012 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.012 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.012 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.012 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.012 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.012 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:23.012 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:23.012 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.012 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.012 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.013 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:23.013 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.013 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.013 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.013 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.013 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.013 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.013 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.013 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.013 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.013 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.013 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.014 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.014 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.014 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.014 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.014 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:23.014 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:23.014 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.014 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.014 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.014 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.014 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.015 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.015 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.015 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.015 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.015 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.015 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.015 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.015 INFO:teuthology.orchestra.run.smithi038.stdout: "password": { 2024-09-16T00:35:23.015 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "password", 2024-09-16T00:35:23.015 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.015 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.016 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.016 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.016 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.016 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.016 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.016 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "password of InfluxDB server user", 2024-09-16T00:35:23.016 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.016 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.016 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.016 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.016 INFO:teuthology.orchestra.run.smithi038.stdout: "port": { 2024-09-16T00:35:23.017 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "port", 2024-09-16T00:35:23.017 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:23.017 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.017 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.017 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "8086", 2024-09-16T00:35:23.017 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.017 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.017 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.017 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "InfluxDB server port", 2024-09-16T00:35:23.017 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.018 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.018 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.018 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.018 INFO:teuthology.orchestra.run.smithi038.stdout: "ssl": { 2024-09-16T00:35:23.018 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ssl", 2024-09-16T00:35:23.018 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.018 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.018 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.018 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "false", 2024-09-16T00:35:23.018 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.018 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.018 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.019 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T00:35:23.019 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.019 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.019 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.019 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.019 INFO:teuthology.orchestra.run.smithi038.stdout: "threads": { 2024-09-16T00:35:23.019 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "threads", 2024-09-16T00:35:23.019 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:23.019 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.019 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.019 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-09-16T00:35:23.020 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-09-16T00:35:23.020 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "32", 2024-09-16T00:35:23.020 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.020 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-16T00:35:23.020 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.020 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.020 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.020 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.020 INFO:teuthology.orchestra.run.smithi038.stdout: "username": { 2024-09-16T00:35:23.020 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "username", 2024-09-16T00:35:23.020 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.021 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.021 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.021 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.021 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.021 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.021 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.021 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "username of InfluxDB server user", 2024-09-16T00:35:23.021 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.021 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.021 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.021 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.021 INFO:teuthology.orchestra.run.smithi038.stdout: "verify_ssl": { 2024-09-16T00:35:23.022 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "verify_ssl", 2024-09-16T00:35:23.022 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.022 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.022 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.022 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "true", 2024-09-16T00:35:23.022 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.022 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.022 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.022 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-16T00:35:23.022 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.022 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.022 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.023 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.023 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.023 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.023 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:23.023 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "insights", 2024-09-16T00:35:23.023 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:23.023 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:23.023 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:23.023 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:23.023 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:23.023 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.024 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.024 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.024 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.024 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.024 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.024 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.024 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.024 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.024 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.024 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.024 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.024 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.025 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.025 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.025 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.025 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.025 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.025 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.025 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:23.025 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:23.025 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.025 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.025 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.026 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.026 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.026 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.026 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.026 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.026 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.026 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.026 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.026 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.026 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:23.026 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:23.026 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.027 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.027 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.027 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:23.027 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.027 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.027 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.027 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.027 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.027 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.027 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.027 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.027 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.028 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.028 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.028 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.028 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.028 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.028 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.028 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:23.028 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:23.028 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.028 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.028 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.028 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.029 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.029 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.029 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.029 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.029 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.029 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.029 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.029 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.029 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.029 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.029 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:23.029 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "iostat", 2024-09-16T00:35:23.030 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:23.030 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:23.030 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:23.030 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:23.030 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:23.030 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.030 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.030 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.030 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.030 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.030 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.030 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.031 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.031 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.031 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.031 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.031 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.031 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.031 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.031 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.031 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.031 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.031 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.031 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.031 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:23.032 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:23.032 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.032 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.032 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.032 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.032 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.032 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.032 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.032 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.032 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.032 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.033 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.033 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.033 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:23.033 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:23.033 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.033 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.033 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.033 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:23.033 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.033 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.033 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.034 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.034 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.034 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.034 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.034 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.034 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.034 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.034 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.034 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.034 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.034 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.034 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.034 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:23.035 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:23.035 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.035 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.035 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.035 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.035 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.035 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.035 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.035 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.035 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.035 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.036 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.036 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.036 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.036 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.036 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:23.036 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "k8sevents", 2024-09-16T00:35:23.036 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:23.036 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:23.036 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:23.036 INFO:teuthology.orchestra.run.smithi038.stdout: "ceph_event_retention_days": { 2024-09-16T00:35:23.036 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "ceph_event_retention_days", 2024-09-16T00:35:23.036 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:23.037 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.037 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.037 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "7", 2024-09-16T00:35:23.037 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.037 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.037 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.037 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-16T00:35:23.037 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.037 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.037 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.037 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.038 INFO:teuthology.orchestra.run.smithi038.stdout: "config_check_secs": { 2024-09-16T00:35:23.038 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "config_check_secs", 2024-09-16T00:35:23.038 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:23.038 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.038 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.038 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-16T00:35:23.038 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "10", 2024-09-16T00:35:23.038 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.038 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.038 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-16T00:35:23.038 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.039 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.039 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.039 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.039 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:23.039 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:23.039 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.039 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.039 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.039 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.039 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.039 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.040 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.040 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.040 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.040 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.040 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.040 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.040 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.040 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.040 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.040 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.040 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.041 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.041 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.041 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:23.041 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:23.041 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.041 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.041 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.041 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.041 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.041 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.041 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.042 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.042 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.042 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.042 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.042 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.042 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:23.042 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:23.042 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.042 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.042 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.042 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:23.043 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.043 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.043 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.043 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.043 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.043 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.043 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.043 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.043 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.043 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.043 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.044 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.044 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.044 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.044 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.044 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:23.044 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:23.044 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.044 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.044 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.044 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.044 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.045 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.045 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.045 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.045 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.045 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.045 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.045 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.045 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.045 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.045 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:23.045 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "localpool", 2024-09-16T00:35:23.046 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:23.046 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:23.046 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:23.046 INFO:teuthology.orchestra.run.smithi038.stdout: "failure_domain": { 2024-09-16T00:35:23.046 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "failure_domain", 2024-09-16T00:35:23.046 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.046 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.046 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.046 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "host", 2024-09-16T00:35:23.046 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.046 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.047 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.047 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "failure domain for any created local pool", 2024-09-16T00:35:23.047 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-16T00:35:23.047 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.047 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.047 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.047 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:23.047 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:23.047 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.047 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.047 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.047 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.048 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.048 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.048 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.048 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.048 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.048 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.048 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.048 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.048 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.048 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.048 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.049 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.049 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.049 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.049 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.049 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:23.049 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:23.049 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.049 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.049 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.049 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.049 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.049 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.050 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.050 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.050 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.050 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.050 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.050 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.050 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:23.050 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:23.050 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.050 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.050 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.050 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:23.051 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.051 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.051 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.051 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.051 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.051 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.051 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.051 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.051 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.051 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.051 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.051 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.052 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.052 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.052 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.052 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:23.052 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:23.052 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.052 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.052 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.052 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.052 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.052 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.053 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.053 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.053 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.053 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.053 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.053 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.053 INFO:teuthology.orchestra.run.smithi038.stdout: "min_size": { 2024-09-16T00:35:23.053 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "min_size", 2024-09-16T00:35:23.053 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:23.053 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.053 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.054 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.054 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.054 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.054 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.054 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "default min_size for any created local pool", 2024-09-16T00:35:23.054 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-16T00:35:23.054 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.054 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.054 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.054 INFO:teuthology.orchestra.run.smithi038.stdout: "num_rep": { 2024-09-16T00:35:23.055 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "num_rep", 2024-09-16T00:35:23.055 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:23.055 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.055 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.055 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "3", 2024-09-16T00:35:23.055 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.055 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.055 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.055 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "default replica count for any created local pool", 2024-09-16T00:35:23.055 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.055 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.055 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.056 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.056 INFO:teuthology.orchestra.run.smithi038.stdout: "pg_num": { 2024-09-16T00:35:23.056 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pg_num", 2024-09-16T00:35:23.056 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:23.056 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.056 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.056 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "128", 2024-09-16T00:35:23.056 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.056 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.056 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.056 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "default pg_num for any created local pool", 2024-09-16T00:35:23.056 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.057 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.057 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.057 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.057 INFO:teuthology.orchestra.run.smithi038.stdout: "prefix": { 2024-09-16T00:35:23.057 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "prefix", 2024-09-16T00:35:23.057 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.057 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.057 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.057 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.057 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.057 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.058 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.058 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "name prefix for any created local pool", 2024-09-16T00:35:23.058 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.058 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.058 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.058 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.058 INFO:teuthology.orchestra.run.smithi038.stdout: "subtree": { 2024-09-16T00:35:23.058 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "subtree", 2024-09-16T00:35:23.058 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.058 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.058 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.059 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "rack", 2024-09-16T00:35:23.059 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.059 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.059 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.059 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-16T00:35:23.059 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-16T00:35:23.059 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.060 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.060 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.060 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.060 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.060 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:23.060 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mds_autoscaler", 2024-09-16T00:35:23.060 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:23.060 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:23.060 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:23.060 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:23.060 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:23.061 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.061 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.061 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.061 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.061 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.061 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.061 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.061 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.061 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.061 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.061 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.061 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.062 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.062 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.062 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.062 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.062 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.062 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.062 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.062 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:23.062 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:23.062 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.062 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.062 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.063 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.063 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.063 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.063 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.063 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.063 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.063 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.063 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.063 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.064 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:23.064 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:23.064 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.064 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.064 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.064 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:23.064 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.064 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.065 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.065 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.065 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.065 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.065 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.065 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.065 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.065 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.066 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.066 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.066 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.066 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.066 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.066 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:23.066 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:23.066 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.067 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.067 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.067 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.067 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.067 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.067 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.067 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.067 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.067 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.067 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.068 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.068 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.068 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.068 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:23.068 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mirroring", 2024-09-16T00:35:23.068 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:23.068 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:23.068 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:23.068 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:23.068 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:23.069 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.069 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.069 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.069 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.069 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.069 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.069 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.069 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.069 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.069 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.069 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.070 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.070 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.070 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.070 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.070 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.070 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.070 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.070 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.070 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:23.070 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:23.070 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.071 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.071 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.071 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.071 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.071 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.071 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.071 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.071 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.071 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.071 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.071 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.072 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:23.072 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:23.072 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.072 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.072 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.072 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:23.072 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.072 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.072 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.072 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.072 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.072 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.073 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.073 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.073 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.073 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.073 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.073 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.073 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.073 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.073 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.073 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:23.073 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:23.074 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.074 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.074 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.074 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.074 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.074 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.074 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.074 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.074 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.074 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.074 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.074 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.074 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.075 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.075 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:23.075 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "nfs", 2024-09-16T00:35:23.075 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:23.075 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:23.075 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:23.075 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:23.075 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:23.075 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.075 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.075 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.075 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.076 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.076 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.076 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.076 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.076 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.076 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.076 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.076 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.076 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.076 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.076 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.076 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.077 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.077 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.077 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.077 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:23.077 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:23.077 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.077 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.077 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.077 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.077 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.077 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.078 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.078 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.078 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.078 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.078 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.078 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.078 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:23.078 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:23.078 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.078 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.078 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.078 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:23.079 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.079 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.079 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.079 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.079 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.079 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.079 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.079 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.079 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.079 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.079 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.079 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.080 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.080 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.080 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.080 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:23.080 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:23.080 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.080 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.080 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.080 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.080 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.080 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.081 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.081 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.081 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.081 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.081 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.081 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.081 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.081 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.081 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:23.081 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "orchestrator", 2024-09-16T00:35:23.081 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:23.082 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:23.082 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:23.082 INFO:teuthology.orchestra.run.smithi038.stdout: "fail_fs": { 2024-09-16T00:35:23.082 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "fail_fs", 2024-09-16T00:35:23.082 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.082 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.082 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.082 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.082 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.082 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.082 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.082 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-16T00:35:23.083 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.083 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.083 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.083 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.083 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:23.083 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:23.083 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.083 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.083 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.083 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.083 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.083 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.084 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.084 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.084 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.084 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.084 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.084 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.084 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.084 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.084 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.084 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.084 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.085 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.085 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.085 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:23.085 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:23.085 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.085 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.085 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.085 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.085 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.085 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.085 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.085 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.086 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.086 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.086 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.086 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.086 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:23.086 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:23.086 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.086 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.086 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.086 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:23.086 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.086 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.087 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.087 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.087 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.087 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.087 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.087 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.087 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.087 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.087 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.087 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.087 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.088 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.088 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.088 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:23.088 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:23.088 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.088 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.088 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.088 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.088 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.089 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.089 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.089 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.089 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.089 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.089 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.089 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.089 INFO:teuthology.orchestra.run.smithi038.stdout: "orchestrator": { 2024-09-16T00:35:23.089 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "orchestrator", 2024-09-16T00:35:23.089 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.090 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.090 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.090 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.090 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.090 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.090 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.090 INFO:teuthology.orchestra.run.smithi038.stdout: "cephadm", 2024-09-16T00:35:23.090 INFO:teuthology.orchestra.run.smithi038.stdout: "rook", 2024-09-16T00:35:23.090 INFO:teuthology.orchestra.run.smithi038.stdout: "test_orchestrator" 2024-09-16T00:35:23.091 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.091 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Orchestrator backend", 2024-09-16T00:35:23.091 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.091 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.091 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.091 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.091 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.091 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.091 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:23.091 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "osd_perf_query", 2024-09-16T00:35:23.092 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:23.092 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:23.092 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:23.092 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:23.092 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:23.092 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.092 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.092 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.092 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.092 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.093 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.093 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.093 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.093 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.093 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.093 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.093 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.093 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.093 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.093 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.093 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.094 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.094 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.094 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.094 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:23.094 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:23.094 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.094 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.094 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.094 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.094 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.094 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.094 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.095 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.095 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.095 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.095 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.095 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.095 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:23.095 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:23.095 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.095 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.095 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.095 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:23.095 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.096 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.096 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.096 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.096 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.096 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.096 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.096 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.096 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.096 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.096 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.096 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.096 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.097 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.097 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.097 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:23.097 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:23.097 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.097 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.097 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.097 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.097 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.097 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.097 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.098 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.098 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.098 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.098 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.098 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.098 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.098 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.098 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:23.098 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "osd_support", 2024-09-16T00:35:23.098 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:23.098 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:23.098 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:23.099 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:23.099 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:23.099 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.099 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.099 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.099 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.099 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.099 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.099 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.099 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.099 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.099 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.099 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.100 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.100 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.100 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.100 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.100 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.100 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.100 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.100 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.100 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:23.100 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:23.100 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.100 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.101 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.101 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.101 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.101 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.101 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.101 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.101 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.101 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.101 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.101 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.101 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:23.102 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:23.102 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.102 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.102 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.102 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:23.102 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.102 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.102 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.102 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.102 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.102 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.103 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.103 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.103 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.103 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.103 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.103 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.103 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.103 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.103 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.103 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:23.103 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:23.103 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.103 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.104 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.104 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.104 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.104 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.104 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.104 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.104 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.104 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.104 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.104 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.104 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.105 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.105 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:23.105 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "pg_autoscaler", 2024-09-16T00:35:23.105 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:23.105 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:23.105 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:23.105 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:23.105 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:23.106 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.106 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.106 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.106 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.106 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.106 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.106 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.106 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.106 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.106 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.106 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.107 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.107 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.107 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.107 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.107 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.107 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.107 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.107 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.107 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:23.107 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:23.107 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.108 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.108 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.108 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.108 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.108 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.108 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.108 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.108 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.108 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.108 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.108 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.109 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:23.109 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:23.109 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.109 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.109 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.109 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:23.109 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.109 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.109 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.109 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.109 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.110 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.110 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.110 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.110 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.110 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.110 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.110 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.110 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.110 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.110 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.111 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:23.111 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:23.111 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.111 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.111 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.111 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.111 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.111 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.111 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.111 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.111 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.112 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.112 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.112 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.112 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-09-16T00:35:23.112 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-09-16T00:35:23.112 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:23.112 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.112 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.112 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-09-16T00:35:23.112 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.112 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.113 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.113 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.113 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.113 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.113 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.113 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.113 INFO:teuthology.orchestra.run.smithi038.stdout: "threshold": { 2024-09-16T00:35:23.113 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "threshold", 2024-09-16T00:35:23.113 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-16T00:35:23.113 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.113 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.114 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "3.0", 2024-09-16T00:35:23.114 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1.0", 2024-09-16T00:35:23.114 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.114 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.114 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "scaling threshold", 2024-09-16T00:35:23.114 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-16T00:35:23.114 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.114 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.114 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.114 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.114 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.115 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:23.115 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "progress", 2024-09-16T00:35:23.115 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:23.115 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:23.115 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:23.115 INFO:teuthology.orchestra.run.smithi038.stdout: "allow_pg_recovery_event": { 2024-09-16T00:35:23.115 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "allow_pg_recovery_event", 2024-09-16T00:35:23.115 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.115 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.115 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.115 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.115 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.116 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.116 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.116 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-16T00:35:23.116 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.116 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.116 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.116 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.116 INFO:teuthology.orchestra.run.smithi038.stdout: "enabled": { 2024-09-16T00:35:23.116 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enabled", 2024-09-16T00:35:23.116 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.116 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.116 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.117 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:23.117 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.117 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.117 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.117 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.117 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.117 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.117 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.117 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.117 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:23.117 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:23.117 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.118 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.118 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.118 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.118 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.118 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.118 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.118 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.118 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.118 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.118 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.118 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.118 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.119 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.119 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.119 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.119 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.119 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.119 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.119 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:23.119 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:23.119 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.119 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.119 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.119 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.119 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.120 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.120 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.120 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.120 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.120 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.120 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.120 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.120 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:23.120 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:23.120 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.120 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.121 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.121 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:23.121 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.121 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.121 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.121 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.121 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.121 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.121 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.121 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.121 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.121 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.121 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.122 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.122 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.122 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.122 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.122 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:23.122 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:23.122 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.122 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.122 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.122 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.122 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.122 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.123 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.123 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.123 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.123 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.123 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.123 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.123 INFO:teuthology.orchestra.run.smithi038.stdout: "max_completed_events": { 2024-09-16T00:35:23.123 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_completed_events", 2024-09-16T00:35:23.123 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:23.123 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.123 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.123 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "50", 2024-09-16T00:35:23.124 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.124 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.124 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.124 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "number of past completed events to remember", 2024-09-16T00:35:23.124 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.124 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.124 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.124 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.124 INFO:teuthology.orchestra.run.smithi038.stdout: "sleep_interval": { 2024-09-16T00:35:23.124 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "sleep_interval", 2024-09-16T00:35:23.124 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:23.125 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.125 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.125 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-09-16T00:35:23.125 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.125 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.125 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.125 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "how long the module is going to sleep", 2024-09-16T00:35:23.125 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.125 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.125 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.125 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.125 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.126 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.126 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:23.126 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "prometheus", 2024-09-16T00:35:23.126 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:23.126 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:23.126 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:23.126 INFO:teuthology.orchestra.run.smithi038.stdout: "cache": { 2024-09-16T00:35:23.126 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "cache", 2024-09-16T00:35:23.126 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.126 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.126 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.126 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:23.126 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.127 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.127 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.127 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.127 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.127 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.127 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.127 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.127 INFO:teuthology.orchestra.run.smithi038.stdout: "exclude_perf_counters": { 2024-09-16T00:35:23.127 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "exclude_perf_counters", 2024-09-16T00:35:23.127 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.127 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.128 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.128 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:23.128 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.128 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.128 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.128 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-16T00:35:23.128 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-16T00:35:23.128 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.128 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.128 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.128 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:23.128 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:23.129 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.129 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.129 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.129 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.129 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.129 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.129 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.129 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.129 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.129 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.129 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.129 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.130 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.130 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.130 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.130 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.130 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.130 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.130 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.130 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:23.130 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:23.130 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.130 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.130 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.131 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.131 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.131 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.131 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.131 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.131 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.131 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.131 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.131 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.131 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:23.131 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:23.131 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.132 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.132 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.132 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:23.132 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.132 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.132 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.132 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.132 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.132 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.132 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.132 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.132 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.133 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.133 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.133 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.133 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.133 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.133 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.133 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:23.133 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:23.133 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.133 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.133 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.133 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.134 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.134 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.134 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.134 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.134 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.134 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.134 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.134 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.134 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_stats_pools": { 2024-09-16T00:35:23.134 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rbd_stats_pools", 2024-09-16T00:35:23.134 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.134 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.135 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.135 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.135 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.135 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.135 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.135 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.135 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.135 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.135 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.135 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.135 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-16T00:35:23.135 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-16T00:35:23.136 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:23.136 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.136 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.136 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "300", 2024-09-16T00:35:23.136 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.136 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.136 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.136 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.136 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.136 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.136 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.136 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.137 INFO:teuthology.orchestra.run.smithi038.stdout: "scrape_interval": { 2024-09-16T00:35:23.137 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "scrape_interval", 2024-09-16T00:35:23.137 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-16T00:35:23.137 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.137 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.137 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "15.0", 2024-09-16T00:35:23.137 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.137 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.137 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.137 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.137 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.137 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.138 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.138 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.138 INFO:teuthology.orchestra.run.smithi038.stdout: "server_addr": { 2024-09-16T00:35:23.138 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_addr", 2024-09-16T00:35:23.138 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.138 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.138 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.138 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "::", 2024-09-16T00:35:23.138 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.138 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.138 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.138 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-16T00:35:23.138 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.139 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.139 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.139 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.139 INFO:teuthology.orchestra.run.smithi038.stdout: "server_port": { 2024-09-16T00:35:23.139 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_port", 2024-09-16T00:35:23.139 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:23.139 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.139 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.139 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "9283", 2024-09-16T00:35:23.139 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.139 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.139 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.140 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-16T00:35:23.140 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.140 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.140 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.140 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.140 INFO:teuthology.orchestra.run.smithi038.stdout: "stale_cache_strategy": { 2024-09-16T00:35:23.140 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "stale_cache_strategy", 2024-09-16T00:35:23.140 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.140 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.140 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.140 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "log", 2024-09-16T00:35:23.140 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.141 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.141 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.141 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.141 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.141 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.141 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.141 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.141 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_behaviour": { 2024-09-16T00:35:23.141 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_behaviour", 2024-09-16T00:35:23.141 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.141 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.141 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.142 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "default", 2024-09-16T00:35:23.142 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.142 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.142 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.142 INFO:teuthology.orchestra.run.smithi038.stdout: "default", 2024-09-16T00:35:23.142 INFO:teuthology.orchestra.run.smithi038.stdout: "error" 2024-09-16T00:35:23.142 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.142 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.142 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.142 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.142 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.142 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.143 INFO:teuthology.orchestra.run.smithi038.stdout: "standby_error_status_code": { 2024-09-16T00:35:23.143 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "standby_error_status_code", 2024-09-16T00:35:23.143 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:23.143 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.143 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.143 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "500", 2024-09-16T00:35:23.143 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "400", 2024-09-16T00:35:23.143 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "599", 2024-09-16T00:35:23.143 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.143 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.143 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.143 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.144 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.144 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.144 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.144 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.144 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:23.144 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rbd_support", 2024-09-16T00:35:23.144 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:23.144 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:23.144 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:23.144 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:23.144 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:23.144 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.145 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.145 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.145 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.145 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.145 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.145 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.145 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.145 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.145 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.145 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.145 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.146 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.146 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.146 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.146 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.146 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.146 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.146 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.146 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:23.146 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:23.146 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.146 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.146 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.147 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.147 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.147 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.147 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.147 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.147 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.147 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.147 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.147 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.147 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:23.147 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:23.148 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.148 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.148 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.148 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:23.148 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.148 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.148 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.148 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.148 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.148 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.148 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.148 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.149 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.149 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.149 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.149 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.149 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.149 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.149 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.149 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:23.149 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:23.149 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.149 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.149 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.150 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.150 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.150 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.150 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.150 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.150 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.150 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.150 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.150 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.150 INFO:teuthology.orchestra.run.smithi038.stdout: "max_concurrent_snap_create": { 2024-09-16T00:35:23.150 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_concurrent_snap_create", 2024-09-16T00:35:23.150 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:23.151 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.151 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.151 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10", 2024-09-16T00:35:23.151 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.151 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.151 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.151 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.151 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.151 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.151 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.151 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.151 INFO:teuthology.orchestra.run.smithi038.stdout: "mirror_snapshot_schedule": { 2024-09-16T00:35:23.152 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "mirror_snapshot_schedule", 2024-09-16T00:35:23.152 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.152 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.152 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.152 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.152 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.152 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.152 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.152 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.152 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.152 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.152 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.153 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.153 INFO:teuthology.orchestra.run.smithi038.stdout: "trash_purge_schedule": { 2024-09-16T00:35:23.153 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "trash_purge_schedule", 2024-09-16T00:35:23.153 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.153 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.153 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.153 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.153 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.153 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.153 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.153 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.153 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.154 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.154 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.154 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.154 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.154 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.154 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:23.154 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "restful", 2024-09-16T00:35:23.154 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:23.154 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:23.154 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:23.154 INFO:teuthology.orchestra.run.smithi038.stdout: "enable_auth": { 2024-09-16T00:35:23.154 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enable_auth", 2024-09-16T00:35:23.154 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.155 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.155 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.155 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:23.155 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.155 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.155 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.155 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.155 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.155 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.155 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.155 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.155 INFO:teuthology.orchestra.run.smithi038.stdout: "key_file": { 2024-09-16T00:35:23.156 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "key_file", 2024-09-16T00:35:23.156 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.156 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.156 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.156 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.156 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.156 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.156 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.156 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.156 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.156 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.156 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.157 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.157 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:23.157 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:23.157 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.157 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.157 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.157 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.157 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.157 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.157 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.157 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.157 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.158 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.158 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.158 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.158 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.158 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.158 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.158 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.158 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.158 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.158 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.158 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:23.158 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:23.159 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.159 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.159 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.159 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.159 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.159 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.159 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.159 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.160 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.160 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.160 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.160 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.160 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:23.160 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:23.160 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.160 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.160 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.160 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:23.160 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.160 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.161 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.161 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.161 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.161 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.161 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.161 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.161 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.161 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.161 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.161 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.161 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.161 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.162 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.162 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:23.162 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:23.162 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.162 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.162 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.162 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.162 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.162 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.162 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.162 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.162 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.163 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.163 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.163 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.163 INFO:teuthology.orchestra.run.smithi038.stdout: "server_addr": { 2024-09-16T00:35:23.163 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_addr", 2024-09-16T00:35:23.163 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.163 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.163 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.163 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.163 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.163 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.164 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.164 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.164 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.164 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.164 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.164 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.164 INFO:teuthology.orchestra.run.smithi038.stdout: "server_port": { 2024-09-16T00:35:23.164 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "server_port", 2024-09-16T00:35:23.164 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.164 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.164 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.164 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.165 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.165 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.165 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.165 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.165 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.165 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.165 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.165 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.165 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.165 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.166 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:23.166 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rgw", 2024-09-16T00:35:23.166 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:23.166 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:23.166 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:23.166 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:23.166 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:23.166 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.166 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.166 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.166 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.166 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.167 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.167 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.167 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.167 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.167 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.167 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.167 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.167 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.167 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.167 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.167 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.167 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.168 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.168 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.168 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:23.168 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:23.168 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.168 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.168 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.168 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.168 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.168 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.168 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.169 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.169 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.169 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.169 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.169 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.169 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:23.169 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:23.169 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.169 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.169 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.169 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:23.169 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.170 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.170 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.170 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.170 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.170 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.170 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.170 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.170 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.170 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.170 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.170 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.170 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.171 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.171 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.171 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:23.171 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:23.171 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.171 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.171 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.171 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.171 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.171 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.171 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.171 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.172 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.172 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.172 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.172 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.172 INFO:teuthology.orchestra.run.smithi038.stdout: "secondary_zone_period_retry_limit": { 2024-09-16T00:35:23.172 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-16T00:35:23.172 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:23.172 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.172 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.172 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "5", 2024-09-16T00:35:23.172 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.172 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.173 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.173 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-16T00:35:23.173 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.173 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.173 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.173 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.173 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.173 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.173 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:23.173 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rook", 2024-09-16T00:35:23.173 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:23.173 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:23.174 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:23.174 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:23.174 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:23.174 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.174 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.174 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.174 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.174 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.174 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.174 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.174 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.174 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.175 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.175 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.175 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.175 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.175 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.175 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.175 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.175 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.175 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.175 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.175 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:23.175 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:23.175 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.176 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.176 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.176 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.176 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.176 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.176 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.176 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.176 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.176 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.176 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.176 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.177 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:23.177 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:23.177 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.177 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.177 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.177 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:23.177 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.177 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.177 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.177 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.177 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.177 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.177 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.178 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.178 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.178 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.178 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.178 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.178 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.178 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.178 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.178 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:23.178 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:23.178 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.178 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.179 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.179 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.179 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.179 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.179 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.179 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.179 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.179 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.179 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.179 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.179 INFO:teuthology.orchestra.run.smithi038.stdout: "storage_class": { 2024-09-16T00:35:23.179 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "storage_class", 2024-09-16T00:35:23.180 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.180 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.180 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.180 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "local", 2024-09-16T00:35:23.180 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.180 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.180 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.180 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-16T00:35:23.180 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.180 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.180 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.181 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.181 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.181 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.181 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:23.181 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "selftest", 2024-09-16T00:35:23.181 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:23.181 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:23.181 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:23.181 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:23.181 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:23.181 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.181 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.181 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.182 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.182 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.182 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.182 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.182 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.182 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.182 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.182 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.182 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.182 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.182 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.182 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.183 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.183 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.183 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.183 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.183 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:23.183 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:23.183 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.183 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.183 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.183 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.183 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.184 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.184 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.184 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.184 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.184 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.184 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.184 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.184 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:23.184 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:23.184 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.184 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.184 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.185 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:23.185 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.185 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.185 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.185 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.185 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.185 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.185 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.185 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.185 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.185 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.185 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.186 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.186 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.186 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.186 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.186 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:23.186 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:23.186 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.186 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.186 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.186 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.186 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.186 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.187 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.187 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.187 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.187 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.187 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.187 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.187 INFO:teuthology.orchestra.run.smithi038.stdout: "roption1": { 2024-09-16T00:35:23.187 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "roption1", 2024-09-16T00:35:23.187 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.187 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.187 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.188 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.188 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.188 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.188 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.188 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.188 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.188 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.188 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.188 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.188 INFO:teuthology.orchestra.run.smithi038.stdout: "roption2": { 2024-09-16T00:35:23.188 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "roption2", 2024-09-16T00:35:23.188 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.189 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.189 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.189 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "xyz", 2024-09-16T00:35:23.189 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.189 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.189 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.189 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.189 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.189 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.189 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.189 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.190 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption1": { 2024-09-16T00:35:23.190 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption1", 2024-09-16T00:35:23.190 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.190 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.190 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.190 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.190 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.190 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.190 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.190 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.190 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.191 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.191 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.191 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.191 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption2": { 2024-09-16T00:35:23.191 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption2", 2024-09-16T00:35:23.191 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:23.191 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.191 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.191 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.191 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.191 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.192 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.192 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.192 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.192 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.192 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.192 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.192 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption3": { 2024-09-16T00:35:23.192 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption3", 2024-09-16T00:35:23.192 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "float", 2024-09-16T00:35:23.192 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.192 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.192 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.193 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.193 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.193 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.193 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.193 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.193 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.193 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.193 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.193 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption4": { 2024-09-16T00:35:23.193 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption4", 2024-09-16T00:35:23.193 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.193 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.194 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.194 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.194 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.194 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.194 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.194 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.194 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.194 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.194 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.194 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.194 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption5": { 2024-09-16T00:35:23.194 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption5", 2024-09-16T00:35:23.195 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.195 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.195 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.195 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.195 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.195 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.195 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.195 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.195 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.195 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.195 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.195 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.196 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption6": { 2024-09-16T00:35:23.196 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption6", 2024-09-16T00:35:23.196 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.196 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.196 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.196 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:23.196 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.196 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.196 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.196 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.196 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.196 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.197 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.197 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.197 INFO:teuthology.orchestra.run.smithi038.stdout: "rwoption7": { 2024-09-16T00:35:23.197 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rwoption7", 2024-09-16T00:35:23.197 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:23.197 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.197 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.197 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.197 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "1", 2024-09-16T00:35:23.197 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "42", 2024-09-16T00:35:23.197 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.197 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.198 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.198 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.198 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.198 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.198 INFO:teuthology.orchestra.run.smithi038.stdout: "testkey": { 2024-09-16T00:35:23.198 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "testkey", 2024-09-16T00:35:23.198 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.198 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.198 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.198 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.198 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.198 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.199 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.199 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.199 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.199 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.199 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.199 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.199 INFO:teuthology.orchestra.run.smithi038.stdout: "testlkey": { 2024-09-16T00:35:23.199 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "testlkey", 2024-09-16T00:35:23.199 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.199 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.199 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.200 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.200 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.200 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.200 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.200 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.200 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.200 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.200 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.200 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.200 INFO:teuthology.orchestra.run.smithi038.stdout: "testnewline": { 2024-09-16T00:35:23.200 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "testnewline", 2024-09-16T00:35:23.200 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.201 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.201 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.201 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.201 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.201 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.201 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.201 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.201 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.201 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.201 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.201 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.201 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.202 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.202 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:23.202 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "snap_schedule", 2024-09-16T00:35:23.202 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:23.202 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:23.202 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:23.202 INFO:teuthology.orchestra.run.smithi038.stdout: "allow_m_granularity": { 2024-09-16T00:35:23.202 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "allow_m_granularity", 2024-09-16T00:35:23.202 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.202 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.202 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.202 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.203 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.203 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.203 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.203 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "allow minute scheduled snapshots", 2024-09-16T00:35:23.203 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.203 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.203 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.203 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.203 INFO:teuthology.orchestra.run.smithi038.stdout: "dump_on_update": { 2024-09-16T00:35:23.203 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "dump_on_update", 2024-09-16T00:35:23.203 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.204 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.204 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.204 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.204 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.204 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.204 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.204 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "dump database to debug log on update", 2024-09-16T00:35:23.204 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.204 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.204 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.204 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.204 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:23.205 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:23.205 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.205 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.205 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.205 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.205 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.205 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.205 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.205 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.205 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.206 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.206 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.206 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.206 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.206 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.206 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.206 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.206 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.206 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.206 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.207 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:23.207 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:23.207 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.207 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.207 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.207 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.207 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.207 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.207 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.207 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.207 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.208 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.208 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.208 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.208 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:23.208 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:23.208 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.208 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.208 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.208 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:23.208 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.208 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.208 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.209 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.209 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.209 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.209 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.209 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.209 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.209 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.209 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.209 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.209 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.209 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.210 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.210 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:23.210 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:23.210 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.210 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.210 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.210 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.210 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.210 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.210 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.211 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.211 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.211 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.211 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.211 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.211 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.211 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.211 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:23.211 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "stats", 2024-09-16T00:35:23.211 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:23.212 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:23.212 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:23.212 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:23.212 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:23.212 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.212 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.212 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.212 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.212 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.212 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.212 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.213 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.213 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.213 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.213 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.213 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.213 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.213 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.213 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.213 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.213 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.213 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.213 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.214 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:23.214 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:23.214 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.214 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.214 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.214 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.214 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.214 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.214 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.214 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.214 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.215 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.215 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.215 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.215 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:23.215 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:23.215 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.215 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.215 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.215 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:23.215 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.215 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.215 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.216 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.216 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.216 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.216 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.216 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.216 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.216 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.216 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.216 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.216 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.216 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.217 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.217 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:23.217 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:23.217 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.217 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.217 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.217 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.217 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.217 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.217 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.217 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.217 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.218 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.218 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.218 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.218 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.218 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.218 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:23.218 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "status", 2024-09-16T00:35:23.218 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:23.218 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:23.218 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:23.218 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:23.219 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:23.219 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.219 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.219 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.219 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.219 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.219 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.219 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.219 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.219 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.219 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.219 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.220 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.220 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.220 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.220 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.220 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.220 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.220 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.220 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.220 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:23.220 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:23.220 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.221 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.221 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.221 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.221 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.221 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.221 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.221 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.221 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.221 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.221 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.221 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.221 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:23.222 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:23.222 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.222 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.222 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.222 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:23.222 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.222 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.222 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.222 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.222 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.222 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.222 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.223 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.223 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.223 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.223 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.223 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.223 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.223 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.223 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.223 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:23.223 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:23.223 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.223 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.224 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.224 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.224 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.224 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.224 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.224 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.224 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.224 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.224 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.224 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.224 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.224 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.225 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:23.225 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "telegraf", 2024-09-16T00:35:23.225 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:23.225 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:23.225 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:23.225 INFO:teuthology.orchestra.run.smithi038.stdout: "address": { 2024-09-16T00:35:23.225 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "address", 2024-09-16T00:35:23.225 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.225 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.225 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.225 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-16T00:35:23.226 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.226 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.226 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.226 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.226 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.226 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.226 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.226 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.226 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-09-16T00:35:23.226 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-09-16T00:35:23.226 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:23.226 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.227 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.227 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "15", 2024-09-16T00:35:23.227 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.227 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.227 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.227 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.227 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.227 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.227 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.227 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.227 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:23.227 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:23.228 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.228 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.228 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.228 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.228 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.228 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.228 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.228 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.228 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.228 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.228 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.228 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.229 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.229 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.229 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.229 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.229 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.229 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.229 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.229 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:23.229 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:23.229 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.229 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.230 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.230 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.230 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.230 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.230 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.230 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.230 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.230 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.230 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.230 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.230 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:23.230 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:23.231 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.231 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.231 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.231 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:23.231 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.231 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.231 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.231 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.231 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.231 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.231 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.231 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.232 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.232 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.232 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.232 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.232 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.232 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.232 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.232 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:23.232 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:23.232 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.232 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.232 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.233 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.233 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.233 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.233 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.233 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.233 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.233 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.233 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.233 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.233 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.233 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.233 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:23.234 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "telemetry", 2024-09-16T00:35:23.234 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:23.234 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:23.234 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:23.234 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_basic": { 2024-09-16T00:35:23.234 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_basic", 2024-09-16T00:35:23.234 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.234 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.234 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.234 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:23.234 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.234 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.235 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.235 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-16T00:35:23.235 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.235 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.235 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.235 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.235 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_crash": { 2024-09-16T00:35:23.235 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_crash", 2024-09-16T00:35:23.235 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.235 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.235 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.235 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:23.236 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.236 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.236 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.236 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-16T00:35:23.236 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.236 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.236 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.236 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.236 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_device": { 2024-09-16T00:35:23.236 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_device", 2024-09-16T00:35:23.236 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.236 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.237 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.237 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:23.237 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.237 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.237 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.237 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-16T00:35:23.237 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.237 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.237 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.237 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.237 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_ident": { 2024-09-16T00:35:23.237 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_ident", 2024-09-16T00:35:23.238 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.238 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.238 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.238 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.238 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.238 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.238 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.238 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-16T00:35:23.238 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.238 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.238 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.238 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.239 INFO:teuthology.orchestra.run.smithi038.stdout: "channel_perf": { 2024-09-16T00:35:23.239 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "channel_perf", 2024-09-16T00:35:23.239 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.239 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.239 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.239 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.239 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.239 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.239 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.239 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-16T00:35:23.239 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.239 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.239 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.240 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.240 INFO:teuthology.orchestra.run.smithi038.stdout: "contact": { 2024-09-16T00:35:23.240 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "contact", 2024-09-16T00:35:23.240 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.240 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.240 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.240 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.240 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.240 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.240 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.240 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.240 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.241 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.241 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.241 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.241 INFO:teuthology.orchestra.run.smithi038.stdout: "description": { 2024-09-16T00:35:23.241 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "description", 2024-09-16T00:35:23.241 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.241 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.241 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.241 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.241 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.241 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.241 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.242 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.242 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.242 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.242 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.242 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.242 INFO:teuthology.orchestra.run.smithi038.stdout: "device_url": { 2024-09-16T00:35:23.242 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "device_url", 2024-09-16T00:35:23.242 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.242 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.242 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.242 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-16T00:35:23.242 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.242 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.243 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.243 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.243 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.243 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.243 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.243 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.243 INFO:teuthology.orchestra.run.smithi038.stdout: "enabled": { 2024-09-16T00:35:23.243 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "enabled", 2024-09-16T00:35:23.243 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.243 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.243 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.243 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.244 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.244 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.244 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.244 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.244 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.244 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.244 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.244 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.244 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-09-16T00:35:23.244 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-09-16T00:35:23.244 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:23.245 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.245 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.245 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "24", 2024-09-16T00:35:23.245 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "8", 2024-09-16T00:35:23.245 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.245 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.245 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.245 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.245 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.245 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.245 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.245 INFO:teuthology.orchestra.run.smithi038.stdout: "last_opt_revision": { 2024-09-16T00:35:23.246 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "last_opt_revision", 2024-09-16T00:35:23.246 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:23.246 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.246 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.246 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "1", 2024-09-16T00:35:23.246 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.246 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.246 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.246 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.246 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.246 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.246 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.247 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.247 INFO:teuthology.orchestra.run.smithi038.stdout: "leaderboard": { 2024-09-16T00:35:23.247 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "leaderboard", 2024-09-16T00:35:23.247 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.247 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.247 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.247 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.247 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.247 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.247 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.247 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.247 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.247 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.248 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.248 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.248 INFO:teuthology.orchestra.run.smithi038.stdout: "leaderboard_description": { 2024-09-16T00:35:23.248 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "leaderboard_description", 2024-09-16T00:35:23.248 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.248 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.248 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.248 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.248 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.248 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.248 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.248 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.249 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.249 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.249 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.249 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.249 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:23.249 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:23.249 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.249 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.249 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.249 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.249 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.250 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.250 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.250 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.250 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.250 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.250 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.250 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.250 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.250 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.250 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.250 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.250 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.250 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.251 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.251 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:23.251 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:23.251 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.251 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.251 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.251 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.251 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.251 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.251 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.251 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.252 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.252 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.252 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.252 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.252 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:23.252 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:23.252 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.252 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.252 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.252 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:23.252 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.252 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.253 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.253 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.253 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.253 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.253 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.253 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.253 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.253 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.253 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.253 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.253 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.253 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.254 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.254 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:23.254 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:23.254 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.254 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.254 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.254 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.254 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.254 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.254 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.254 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.254 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.254 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.255 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.255 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.255 INFO:teuthology.orchestra.run.smithi038.stdout: "organization": { 2024-09-16T00:35:23.255 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "organization", 2024-09-16T00:35:23.255 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.255 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.255 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.255 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.255 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.255 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.255 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.255 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.256 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.256 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.256 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.256 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.256 INFO:teuthology.orchestra.run.smithi038.stdout: "proxy": { 2024-09-16T00:35:23.256 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "proxy", 2024-09-16T00:35:23.256 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.256 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.256 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.256 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.256 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.256 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.257 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.257 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.257 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.257 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.257 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.257 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.257 INFO:teuthology.orchestra.run.smithi038.stdout: "url": { 2024-09-16T00:35:23.257 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "url", 2024-09-16T00:35:23.257 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.257 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.257 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.257 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-16T00:35:23.258 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.258 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.258 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.258 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.258 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.258 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.258 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.258 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.258 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.258 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.258 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:23.258 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "test_orchestrator", 2024-09-16T00:35:23.258 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:23.259 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:23.259 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:23.259 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:23.259 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:23.259 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.259 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.259 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.259 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.259 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.260 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.260 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.260 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.260 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.260 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.260 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.260 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.260 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.260 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.260 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.261 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.261 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.261 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.261 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.261 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:23.261 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:23.261 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.261 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.261 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.261 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.261 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.261 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.261 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.262 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.262 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.262 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.262 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.262 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.262 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:23.262 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:23.262 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.262 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.262 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.262 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:23.262 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.263 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.263 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.263 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.263 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.263 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.263 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.263 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.263 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.263 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.263 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.263 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.263 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.264 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.264 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.264 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:23.264 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:23.264 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.264 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.264 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.264 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.264 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.264 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.264 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.265 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.265 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.265 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.265 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.265 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.265 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.265 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.265 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:23.265 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "volumes", 2024-09-16T00:35:23.265 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:23.265 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:23.265 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:23.265 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:23.266 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:23.266 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.266 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.266 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.266 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.266 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.266 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.266 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.266 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.266 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.266 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.266 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.267 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.267 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.267 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.267 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.267 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.267 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.267 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.267 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.267 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:23.267 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:23.267 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.267 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.268 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.268 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.268 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.268 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.268 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.268 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.268 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.268 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.268 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.268 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.268 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:23.268 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:23.269 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.269 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.269 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.269 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:23.269 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.269 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.269 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.269 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.269 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.269 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.269 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.270 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.270 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.270 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.270 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.270 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.270 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.270 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.270 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.270 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:23.270 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:23.270 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.270 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.271 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.271 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.271 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.271 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.271 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.271 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.271 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.271 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.271 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.271 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.271 INFO:teuthology.orchestra.run.smithi038.stdout: "max_concurrent_clones": { 2024-09-16T00:35:23.271 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "max_concurrent_clones", 2024-09-16T00:35:23.272 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:23.272 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.272 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.272 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "4", 2024-09-16T00:35:23.272 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.272 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.272 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.272 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-16T00:35:23.272 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.272 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.272 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.272 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.273 INFO:teuthology.orchestra.run.smithi038.stdout: "periodic_async_work": { 2024-09-16T00:35:23.273 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "periodic_async_work", 2024-09-16T00:35:23.273 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.273 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.273 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.273 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.273 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.273 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.273 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.273 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Periodically check for async work", 2024-09-16T00:35:23.273 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.273 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.274 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.274 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.274 INFO:teuthology.orchestra.run.smithi038.stdout: "snapshot_clone_delay": { 2024-09-16T00:35:23.274 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "snapshot_clone_delay", 2024-09-16T00:35:23.274 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:23.274 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.274 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.274 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "0", 2024-09-16T00:35:23.274 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.274 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.274 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.274 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-16T00:35:23.275 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.275 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.275 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.275 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.275 INFO:teuthology.orchestra.run.smithi038.stdout: "snapshot_clone_no_wait": { 2024-09-16T00:35:23.275 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "snapshot_clone_no_wait", 2024-09-16T00:35:23.275 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.275 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.275 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.275 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "True", 2024-09-16T00:35:23.275 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.275 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.276 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.276 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-16T00:35:23.276 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.276 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.276 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.276 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.276 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.276 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.276 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:23.276 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix", 2024-09-16T00:35:23.276 INFO:teuthology.orchestra.run.smithi038.stdout: "can_run": true, 2024-09-16T00:35:23.276 INFO:teuthology.orchestra.run.smithi038.stdout: "error_string": "", 2024-09-16T00:35:23.277 INFO:teuthology.orchestra.run.smithi038.stdout: "module_options": { 2024-09-16T00:35:23.277 INFO:teuthology.orchestra.run.smithi038.stdout: "discovery_interval": { 2024-09-16T00:35:23.277 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "discovery_interval", 2024-09-16T00:35:23.277 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "uint", 2024-09-16T00:35:23.277 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.277 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.277 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "100", 2024-09-16T00:35:23.277 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.277 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.277 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.277 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.277 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.278 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.278 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.278 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.278 INFO:teuthology.orchestra.run.smithi038.stdout: "identifier": { 2024-09-16T00:35:23.278 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "identifier", 2024-09-16T00:35:23.278 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.278 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.278 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.278 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.278 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.278 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.278 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.279 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.279 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.279 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.279 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.279 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.279 INFO:teuthology.orchestra.run.smithi038.stdout: "interval": { 2024-09-16T00:35:23.279 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "interval", 2024-09-16T00:35:23.279 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "secs", 2024-09-16T00:35:23.279 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.279 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.279 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "60", 2024-09-16T00:35:23.279 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.280 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.280 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.280 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.280 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.280 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.280 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.280 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.280 INFO:teuthology.orchestra.run.smithi038.stdout: "log_level": { 2024-09-16T00:35:23.280 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_level", 2024-09-16T00:35:23.280 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.280 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.281 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.281 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.281 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.281 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.281 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.281 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.281 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.281 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.281 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.281 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.281 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.281 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.282 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.282 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.282 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.282 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.282 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.282 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster": { 2024-09-16T00:35:23.282 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster", 2024-09-16T00:35:23.282 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.282 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.282 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.282 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.283 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.283 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.283 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.283 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.283 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.283 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.283 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.283 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.283 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_cluster_level": { 2024-09-16T00:35:23.283 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_cluster_level", 2024-09-16T00:35:23.283 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.284 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.284 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.284 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "info", 2024-09-16T00:35:23.284 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.284 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.284 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [ 2024-09-16T00:35:23.284 INFO:teuthology.orchestra.run.smithi038.stdout: "", 2024-09-16T00:35:23.284 INFO:teuthology.orchestra.run.smithi038.stdout: "critical", 2024-09-16T00:35:23.284 INFO:teuthology.orchestra.run.smithi038.stdout: "debug", 2024-09-16T00:35:23.284 INFO:teuthology.orchestra.run.smithi038.stdout: "error", 2024-09-16T00:35:23.284 INFO:teuthology.orchestra.run.smithi038.stdout: "info", 2024-09-16T00:35:23.285 INFO:teuthology.orchestra.run.smithi038.stdout: "warning" 2024-09-16T00:35:23.285 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.285 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.285 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.285 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.285 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.285 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.285 INFO:teuthology.orchestra.run.smithi038.stdout: "log_to_file": { 2024-09-16T00:35:23.285 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "log_to_file", 2024-09-16T00:35:23.285 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "bool", 2024-09-16T00:35:23.285 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.285 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 1, 2024-09-16T00:35:23.286 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "False", 2024-09-16T00:35:23.286 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.286 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.286 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.286 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.286 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.286 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.286 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.286 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.286 INFO:teuthology.orchestra.run.smithi038.stdout: "zabbix_host": { 2024-09-16T00:35:23.286 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix_host", 2024-09-16T00:35:23.287 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.287 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.287 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.287 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "", 2024-09-16T00:35:23.287 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.287 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.287 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.287 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.287 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.287 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.287 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.287 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.288 INFO:teuthology.orchestra.run.smithi038.stdout: "zabbix_port": { 2024-09-16T00:35:23.288 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix_port", 2024-09-16T00:35:23.288 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "int", 2024-09-16T00:35:23.288 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.288 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.288 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "10051", 2024-09-16T00:35:23.288 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.288 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.288 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.288 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.288 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.289 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.289 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.289 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.289 INFO:teuthology.orchestra.run.smithi038.stdout: "zabbix_sender": { 2024-09-16T00:35:23.289 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "zabbix_sender", 2024-09-16T00:35:23.289 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "str", 2024-09-16T00:35:23.289 INFO:teuthology.orchestra.run.smithi038.stdout: "level": "advanced", 2024-09-16T00:35:23.289 INFO:teuthology.orchestra.run.smithi038.stdout: "flags": 0, 2024-09-16T00:35:23.289 INFO:teuthology.orchestra.run.smithi038.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-16T00:35:23.289 INFO:teuthology.orchestra.run.smithi038.stdout: "min": "", 2024-09-16T00:35:23.289 INFO:teuthology.orchestra.run.smithi038.stdout: "max": "", 2024-09-16T00:35:23.289 INFO:teuthology.orchestra.run.smithi038.stdout: "enum_allowed": [], 2024-09-16T00:35:23.290 INFO:teuthology.orchestra.run.smithi038.stdout: "desc": "", 2024-09-16T00:35:23.290 INFO:teuthology.orchestra.run.smithi038.stdout: "long_desc": "", 2024-09-16T00:35:23.290 INFO:teuthology.orchestra.run.smithi038.stdout: "tags": [], 2024-09-16T00:35:23.290 INFO:teuthology.orchestra.run.smithi038.stdout: "see_also": [] 2024-09-16T00:35:23.290 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.290 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.290 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.290 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.290 INFO:teuthology.orchestra.run.smithi038.stdout: "services": { 2024-09-16T00:35:23.290 INFO:teuthology.orchestra.run.smithi038.stdout: "dashboard": "https://172.21.15.38:8443/" 2024-09-16T00:35:23.290 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.290 INFO:teuthology.orchestra.run.smithi038.stdout: "always_on_modules": { 2024-09-16T00:35:23.291 INFO:teuthology.orchestra.run.smithi038.stdout: "octopus": [ 2024-09-16T00:35:23.291 INFO:teuthology.orchestra.run.smithi038.stdout: "balancer", 2024-09-16T00:35:23.291 INFO:teuthology.orchestra.run.smithi038.stdout: "crash", 2024-09-16T00:35:23.291 INFO:teuthology.orchestra.run.smithi038.stdout: "devicehealth", 2024-09-16T00:35:23.291 INFO:teuthology.orchestra.run.smithi038.stdout: "orchestrator", 2024-09-16T00:35:23.291 INFO:teuthology.orchestra.run.smithi038.stdout: "pg_autoscaler", 2024-09-16T00:35:23.291 INFO:teuthology.orchestra.run.smithi038.stdout: "progress", 2024-09-16T00:35:23.291 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_support", 2024-09-16T00:35:23.291 INFO:teuthology.orchestra.run.smithi038.stdout: "status", 2024-09-16T00:35:23.291 INFO:teuthology.orchestra.run.smithi038.stdout: "telemetry", 2024-09-16T00:35:23.291 INFO:teuthology.orchestra.run.smithi038.stdout: "volumes" 2024-09-16T00:35:23.291 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.292 INFO:teuthology.orchestra.run.smithi038.stdout: "pacific": [ 2024-09-16T00:35:23.292 INFO:teuthology.orchestra.run.smithi038.stdout: "balancer", 2024-09-16T00:35:23.292 INFO:teuthology.orchestra.run.smithi038.stdout: "crash", 2024-09-16T00:35:23.292 INFO:teuthology.orchestra.run.smithi038.stdout: "devicehealth", 2024-09-16T00:35:23.292 INFO:teuthology.orchestra.run.smithi038.stdout: "orchestrator", 2024-09-16T00:35:23.292 INFO:teuthology.orchestra.run.smithi038.stdout: "pg_autoscaler", 2024-09-16T00:35:23.292 INFO:teuthology.orchestra.run.smithi038.stdout: "progress", 2024-09-16T00:35:23.292 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_support", 2024-09-16T00:35:23.292 INFO:teuthology.orchestra.run.smithi038.stdout: "status", 2024-09-16T00:35:23.292 INFO:teuthology.orchestra.run.smithi038.stdout: "telemetry", 2024-09-16T00:35:23.292 INFO:teuthology.orchestra.run.smithi038.stdout: "volumes" 2024-09-16T00:35:23.292 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.293 INFO:teuthology.orchestra.run.smithi038.stdout: "quincy": [ 2024-09-16T00:35:23.293 INFO:teuthology.orchestra.run.smithi038.stdout: "balancer", 2024-09-16T00:35:23.293 INFO:teuthology.orchestra.run.smithi038.stdout: "crash", 2024-09-16T00:35:23.293 INFO:teuthology.orchestra.run.smithi038.stdout: "devicehealth", 2024-09-16T00:35:23.293 INFO:teuthology.orchestra.run.smithi038.stdout: "orchestrator", 2024-09-16T00:35:23.293 INFO:teuthology.orchestra.run.smithi038.stdout: "pg_autoscaler", 2024-09-16T00:35:23.293 INFO:teuthology.orchestra.run.smithi038.stdout: "progress", 2024-09-16T00:35:23.293 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_support", 2024-09-16T00:35:23.293 INFO:teuthology.orchestra.run.smithi038.stdout: "status", 2024-09-16T00:35:23.293 INFO:teuthology.orchestra.run.smithi038.stdout: "telemetry", 2024-09-16T00:35:23.293 INFO:teuthology.orchestra.run.smithi038.stdout: "volumes" 2024-09-16T00:35:23.293 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.294 INFO:teuthology.orchestra.run.smithi038.stdout: "reef": [ 2024-09-16T00:35:23.294 INFO:teuthology.orchestra.run.smithi038.stdout: "balancer", 2024-09-16T00:35:23.294 INFO:teuthology.orchestra.run.smithi038.stdout: "crash", 2024-09-16T00:35:23.294 INFO:teuthology.orchestra.run.smithi038.stdout: "devicehealth", 2024-09-16T00:35:23.294 INFO:teuthology.orchestra.run.smithi038.stdout: "orchestrator", 2024-09-16T00:35:23.294 INFO:teuthology.orchestra.run.smithi038.stdout: "pg_autoscaler", 2024-09-16T00:35:23.294 INFO:teuthology.orchestra.run.smithi038.stdout: "progress", 2024-09-16T00:35:23.294 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_support", 2024-09-16T00:35:23.294 INFO:teuthology.orchestra.run.smithi038.stdout: "status", 2024-09-16T00:35:23.294 INFO:teuthology.orchestra.run.smithi038.stdout: "telemetry", 2024-09-16T00:35:23.294 INFO:teuthology.orchestra.run.smithi038.stdout: "volumes" 2024-09-16T00:35:23.294 INFO:teuthology.orchestra.run.smithi038.stdout: ], 2024-09-16T00:35:23.294 INFO:teuthology.orchestra.run.smithi038.stdout: "squid": [ 2024-09-16T00:35:23.295 INFO:teuthology.orchestra.run.smithi038.stdout: "balancer", 2024-09-16T00:35:23.295 INFO:teuthology.orchestra.run.smithi038.stdout: "crash", 2024-09-16T00:35:23.295 INFO:teuthology.orchestra.run.smithi038.stdout: "devicehealth", 2024-09-16T00:35:23.295 INFO:teuthology.orchestra.run.smithi038.stdout: "orchestrator", 2024-09-16T00:35:23.295 INFO:teuthology.orchestra.run.smithi038.stdout: "pg_autoscaler", 2024-09-16T00:35:23.295 INFO:teuthology.orchestra.run.smithi038.stdout: "progress", 2024-09-16T00:35:23.295 INFO:teuthology.orchestra.run.smithi038.stdout: "rbd_support", 2024-09-16T00:35:23.295 INFO:teuthology.orchestra.run.smithi038.stdout: "status", 2024-09-16T00:35:23.295 INFO:teuthology.orchestra.run.smithi038.stdout: "telemetry", 2024-09-16T00:35:23.295 INFO:teuthology.orchestra.run.smithi038.stdout: "volumes" 2024-09-16T00:35:23.295 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-09-16T00:35:23.295 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.296 INFO:teuthology.orchestra.run.smithi038.stdout: "last_failure_osd_epoch": 3, 2024-09-16T00:35:23.296 INFO:teuthology.orchestra.run.smithi038.stdout: "active_clients": [ 2024-09-16T00:35:23.296 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:23.296 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "devicehealth", 2024-09-16T00:35:23.296 INFO:teuthology.orchestra.run.smithi038.stdout: "addrvec": [ 2024-09-16T00:35:23.296 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:23.296 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v2", 2024-09-16T00:35:23.296 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "172.21.15.38:0", 2024-09-16T00:35:23.296 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 2896512449 2024-09-16T00:35:23.296 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.296 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-09-16T00:35:23.296 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.297 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:23.297 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "libcephsqlite", 2024-09-16T00:35:23.297 INFO:teuthology.orchestra.run.smithi038.stdout: "addrvec": [ 2024-09-16T00:35:23.297 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:23.297 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v2", 2024-09-16T00:35:23.297 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "172.21.15.38:0", 2024-09-16T00:35:23.297 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 3857151775 2024-09-16T00:35:23.297 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.297 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-09-16T00:35:23.297 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.297 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:23.297 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "rbd_support", 2024-09-16T00:35:23.297 INFO:teuthology.orchestra.run.smithi038.stdout: "addrvec": [ 2024-09-16T00:35:23.298 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:23.298 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v2", 2024-09-16T00:35:23.298 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "172.21.15.38:0", 2024-09-16T00:35:23.298 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 3842703753 2024-09-16T00:35:23.298 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.298 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-09-16T00:35:23.298 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:23.298 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:23.298 INFO:teuthology.orchestra.run.smithi038.stdout: "name": "volumes", 2024-09-16T00:35:23.298 INFO:teuthology.orchestra.run.smithi038.stdout: "addrvec": [ 2024-09-16T00:35:23.298 INFO:teuthology.orchestra.run.smithi038.stdout: { 2024-09-16T00:35:23.298 INFO:teuthology.orchestra.run.smithi038.stdout: "type": "v2", 2024-09-16T00:35:23.299 INFO:teuthology.orchestra.run.smithi038.stdout: "addr": "172.21.15.38:0", 2024-09-16T00:35:23.299 INFO:teuthology.orchestra.run.smithi038.stdout: "nonce": 1658697649 2024-09-16T00:35:23.299 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.299 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-09-16T00:35:23.299 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:23.299 INFO:teuthology.orchestra.run.smithi038.stdout: ] 2024-09-16T00:35:23.299 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-16T00:35:23.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.696+0000 7f5693cf1640 1 -- 172.21.15.38:0/4278011359 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f56740a32d0 msgr2=0x7f56740a5790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:23.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.696+0000 7f5693cf1640 1 --2- 172.21.15.38:0/4278011359 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f56740a32d0 0x7f56740a5790 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f568400bbe0 tx=0x7f568400b390 comp rx=0 tx=0).stop 2024-09-16T00:35:23.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.696+0000 7f5693cf1640 1 -- 172.21.15.38:0/4278011359 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f568c104ec0 msgr2=0x7f568c1983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:23.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.696+0000 7f5693cf1640 1 --2- 172.21.15.38:0/4278011359 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f568c104ec0 0x7f568c1983b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f568000d740 tx=0x7f568000dc00 comp rx=0 tx=0).stop 2024-09-16T00:35:23.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.696+0000 7f5693cf1640 1 -- 172.21.15.38:0/4278011359 shutdown_connections 2024-09-16T00:35:23.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.696+0000 7f5693cf1640 1 --2- 172.21.15.38:0/4278011359 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f56740a32d0 0x7f56740a5790 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:23.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.696+0000 7f5693cf1640 1 --2- 172.21.15.38:0/4278011359 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f568c1077f0 0x7f568c19f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:23.301 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.696+0000 7f5693cf1640 1 --2- 172.21.15.38:0/4278011359 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f568c104ec0 0x7f568c1983b0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:23.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.696+0000 7f5693cf1640 1 --2- 172.21.15.38:0/4278011359 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f568c102590 0x7f568c197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:23.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.696+0000 7f5693cf1640 1 -- 172.21.15.38:0/4278011359 >> 172.21.15.38:0/4278011359 conn(0x7f568c0fc170 msgr2=0x7f568c102e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:23.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.696+0000 7f5693cf1640 1 -- 172.21.15.38:0/4278011359 shutdown_connections 2024-09-16T00:35:23.302 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:21.696+0000 7f5693cf1640 1 -- 172.21.15.38:0/4278011359 wait complete. 2024-09-16T00:35:23.302 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:21 smithi122 ceph-mon[35412]: pgmap v441: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-16T00:35:23.302 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:21 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/3337608494' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T00:35:23.302 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:21 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/4184445693' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T00:35:23.302 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:21 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/3949833680' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-16T00:35:23.302 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:22 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/4278011359' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-16T00:35:23.303 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:21 smithi073 ceph-mon[35254]: pgmap v441: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-16T00:35:23.303 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:21 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/3337608494' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T00:35:23.303 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:21 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/4184445693' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T00:35:23.303 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:21 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/3949833680' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-16T00:35:23.303 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:22 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/4278011359' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-16T00:35:23.303 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:21 smithi038 ceph-mon[30945]: pgmap v441: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 426 B/s wr, 7 op/s 2024-09-16T00:35:23.303 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:21 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/3337608494' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-09-16T00:35:23.303 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:21 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/4184445693' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-09-16T00:35:23.303 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:21 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/3949833680' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-09-16T00:35:23.373 DEBUG:tasks.check_counter:Getting stats from a 2024-09-16T00:35:23.373 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph daemon mds.a perf dump 2024-09-16T00:35:23.373 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:22 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/4278011359' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-09-16T00:35:23.642 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:35:23.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:23 smithi038 ceph-mon[30945]: pgmap v442: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-16T00:35:23.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:23 smithi073 ceph-mon[35254]: pgmap v442: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-16T00:35:23.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:23 smithi122 ceph-mon[35412]: pgmap v442: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 426 B/s wr, 5 op/s 2024-09-16T00:35:24.186 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-16T00:35:24.186 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:35:24.186 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 0, 2024-09-16T00:35:24.186 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 2, 2024-09-16T00:35:24.186 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 0, 2024-09-16T00:35:24.186 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 0, 2024-09-16T00:35:24.187 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 4, 2024-09-16T00:35:24.187 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 0, 2024-09-16T00:35:24.187 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.001529626, 2024-09-16T00:35:24.187 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.000000000, 2024-09-16T00:35:24.187 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.000220906, 2024-09-16T00:35:24.187 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:35:24.187 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:35:24.187 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-16T00:35:24.187 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-16T00:35:24.187 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-16T00:35:24.187 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:24.187 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:35:24.187 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-16T00:35:24.187 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-16T00:35:24.188 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-16T00:35:24.188 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:24.188 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T00:35:24.188 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T00:35:24.188 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:24.188 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:35:24.188 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 92, 2024-09-16T00:35:24.188 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 87, 2024-09-16T00:35:24.188 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 719760, 2024-09-16T00:35:24.188 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 52192, 2024-09-16T00:35:24.188 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 2, 2024-09-16T00:35:24.188 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 1, 2024-09-16T00:35:24.188 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.015689238, 2024-09-16T00:35:24.189 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.005581376, 2024-09-16T00:35:24.189 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.014887669, 2024-09-16T00:35:24.189 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:35:24.189 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:35:24.189 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 86, 2024-09-16T00:35:24.189 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.004259921, 2024-09-16T00:35:24.189 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000049533 2024-09-16T00:35:24.189 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:24.189 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:35:24.189 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-16T00:35:24.189 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-16T00:35:24.189 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-16T00:35:24.189 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:24.190 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 719760, 2024-09-16T00:35:24.190 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 52192 2024-09-16T00:35:24.190 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:24.190 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:35:24.190 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 5, 2024-09-16T00:35:24.190 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 67, 2024-09-16T00:35:24.190 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 3232, 2024-09-16T00:35:24.190 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 90480, 2024-09-16T00:35:24.190 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 3, 2024-09-16T00:35:24.191 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 2, 2024-09-16T00:35:24.191 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.007352604, 2024-09-16T00:35:24.191 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.003712942, 2024-09-16T00:35:24.191 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.003073952, 2024-09-16T00:35:24.191 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:35:24.191 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:35:24.191 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 66, 2024-09-16T00:35:24.191 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.007062853, 2024-09-16T00:35:24.191 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000107012 2024-09-16T00:35:24.191 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:24.191 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:35:24.191 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-16T00:35:24.191 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-16T00:35:24.191 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-16T00:35:24.192 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:24.192 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-09-16T00:35:24.192 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 90480 2024-09-16T00:35:24.192 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:24.192 INFO:teuthology.orchestra.run.smithi038.stdout: "cct": { 2024-09-16T00:35:24.192 INFO:teuthology.orchestra.run.smithi038.stdout: "total_workers": 0, 2024-09-16T00:35:24.192 INFO:teuthology.orchestra.run.smithi038.stdout: "unhealthy_workers": 0 2024-09-16T00:35:24.192 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:24.192 INFO:teuthology.orchestra.run.smithi038.stdout: "mempool": { 2024-09-16T00:35:24.192 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:35:24.192 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_items": 0, 2024-09-16T00:35:24.193 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:35:24.193 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:35:24.193 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:35:24.193 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:35:24.193 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:35:24.193 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:35:24.193 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:35:24.193 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:35:24.193 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:35:24.193 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:35:24.193 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:35:24.193 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:35:24.193 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:35:24.194 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_items": 0, 2024-09-16T00:35:24.194 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:35:24.194 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_items": 0, 2024-09-16T00:35:24.194 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:35:24.194 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:35:24.194 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:35:24.194 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:35:24.194 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:35:24.194 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:35:24.194 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:35:24.194 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_items": 0, 2024-09-16T00:35:24.194 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:35:24.194 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:35:24.194 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:35:24.195 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_items": 0, 2024-09-16T00:35:24.195 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_bytes": 0, 2024-09-16T00:35:24.195 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_items": 0, 2024-09-16T00:35:24.195 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:35:24.195 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:35:24.195 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:35:24.195 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:35:24.195 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_bytes": 148801, 2024-09-16T00:35:24.195 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_items": 27, 2024-09-16T00:35:24.195 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:35:24.195 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_items": 0, 2024-09-16T00:35:24.195 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_bytes": 0, 2024-09-16T00:35:24.195 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_items": 0, 2024-09-16T00:35:24.195 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:35:24.196 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_items": 0, 2024-09-16T00:35:24.196 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:35:24.196 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_items": 0, 2024-09-16T00:35:24.196 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_bytes": 0, 2024-09-16T00:35:24.196 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_items": 0, 2024-09-16T00:35:24.196 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:35:24.196 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:35:24.196 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_bytes": 0, 2024-09-16T00:35:24.196 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_items": 0, 2024-09-16T00:35:24.196 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_bytes": 928, 2024-09-16T00:35:24.196 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_items": 2, 2024-09-16T00:35:24.196 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_bytes": 0, 2024-09-16T00:35:24.196 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_items": 0, 2024-09-16T00:35:24.196 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_bytes": 0, 2024-09-16T00:35:24.197 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_items": 0 2024-09-16T00:35:24.197 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:24.197 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:35:24.197 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-09-16T00:35:24.197 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-09-16T00:35:24.197 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-09-16T00:35:24.197 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 93, 2024-09-16T00:35:24.197 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 706793, 2024-09-16T00:35:24.197 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-09-16T00:35:24.197 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 93, 2024-09-16T00:35:24.197 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-09-16T00:35:24.197 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-09-16T00:35:24.197 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 93, 2024-09-16T00:35:24.197 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 706793, 2024-09-16T00:35:24.198 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-09-16T00:35:24.198 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-16T00:35:24.198 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-16T00:35:24.198 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-16T00:35:24.198 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:24.198 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:24.198 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-16T00:35:25.043 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2024-09-16T00:35:25.043 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2024-09-16T00:35:25.043 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2024-09-16T00:35:25.043 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2024-09-16T00:35:25.043 DEBUG:tasks.check_counter:Getting stats from d 2024-09-16T00:35:25.044 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph daemon mds.d perf dump 2024-09-16T00:35:25.304 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:35:25.713 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:25 smithi038 ceph-mon[30945]: pgmap v443: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-16T00:35:25.793 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:25 smithi073 ceph-mon[35254]: pgmap v443: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-16T00:35:25.830 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-16T00:35:25.830 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:35:25.830 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 0, 2024-09-16T00:35:25.830 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 2, 2024-09-16T00:35:25.830 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 0, 2024-09-16T00:35:25.830 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 0, 2024-09-16T00:35:25.830 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 4, 2024-09-16T00:35:25.830 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 0, 2024-09-16T00:35:25.830 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.001271177, 2024-09-16T00:35:25.830 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.000000000, 2024-09-16T00:35:25.830 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.000240408, 2024-09-16T00:35:25.831 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:35:25.831 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:35:25.831 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-16T00:35:25.831 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-16T00:35:25.831 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-16T00:35:25.831 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:25.831 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:35:25.831 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-16T00:35:25.831 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-16T00:35:25.831 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-16T00:35:25.831 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:25.831 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T00:35:25.831 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T00:35:25.831 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:25.832 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:35:25.832 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 1, 2024-09-16T00:35:25.832 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 70, 2024-09-16T00:35:25.832 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 160, 2024-09-16T00:35:25.832 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 90912, 2024-09-16T00:35:25.832 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 3, 2024-09-16T00:35:25.832 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 2, 2024-09-16T00:35:25.832 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.005801036, 2024-09-16T00:35:25.832 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.003972848, 2024-09-16T00:35:25.832 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.001006187, 2024-09-16T00:35:25.832 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:35:25.832 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:35:25.832 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 68, 2024-09-16T00:35:25.832 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.003198412, 2024-09-16T00:35:25.833 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000047035 2024-09-16T00:35:25.833 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:25.833 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:35:25.833 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-16T00:35:25.833 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-16T00:35:25.833 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-16T00:35:25.833 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:25.833 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-16T00:35:25.833 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 90912 2024-09-16T00:35:25.833 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:25.833 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:35:25.833 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 101, 2024-09-16T00:35:25.833 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 94, 2024-09-16T00:35:25.833 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 724192, 2024-09-16T00:35:25.834 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 55840, 2024-09-16T00:35:25.834 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 2, 2024-09-16T00:35:25.834 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 1, 2024-09-16T00:35:25.834 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.018809253, 2024-09-16T00:35:25.834 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.006396941, 2024-09-16T00:35:25.834 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.017904714, 2024-09-16T00:35:25.834 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:35:25.834 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:35:25.834 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 94, 2024-09-16T00:35:25.834 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.008807722, 2024-09-16T00:35:25.834 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000093699 2024-09-16T00:35:25.834 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:25.834 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:35:25.836 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-16T00:35:25.836 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-16T00:35:25.836 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-16T00:35:25.836 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:25.837 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 724192, 2024-09-16T00:35:25.837 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 55840 2024-09-16T00:35:25.837 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:25.837 INFO:teuthology.orchestra.run.smithi038.stdout: "cct": { 2024-09-16T00:35:25.837 INFO:teuthology.orchestra.run.smithi038.stdout: "total_workers": 0, 2024-09-16T00:35:25.837 INFO:teuthology.orchestra.run.smithi038.stdout: "unhealthy_workers": 0 2024-09-16T00:35:25.837 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:25.837 INFO:teuthology.orchestra.run.smithi038.stdout: "mempool": { 2024-09-16T00:35:25.837 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:35:25.837 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_items": 0, 2024-09-16T00:35:25.838 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:35:25.838 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:35:25.838 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:35:25.838 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:35:25.838 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:35:25.838 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:35:25.838 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:35:25.838 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:35:25.838 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:35:25.838 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:35:25.838 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:35:25.838 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:35:25.838 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:35:25.838 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_items": 0, 2024-09-16T00:35:25.839 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:35:25.839 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_items": 0, 2024-09-16T00:35:25.839 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:35:25.839 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:35:25.839 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:35:25.839 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:35:25.839 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:35:25.839 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:35:25.839 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:35:25.839 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_items": 0, 2024-09-16T00:35:25.839 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:35:25.839 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:35:25.839 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:35:25.839 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_items": 0, 2024-09-16T00:35:25.840 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_bytes": 0, 2024-09-16T00:35:25.840 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_items": 0, 2024-09-16T00:35:25.840 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:35:25.840 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:35:25.840 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:35:25.840 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:35:25.840 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_bytes": 148841, 2024-09-16T00:35:25.840 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_items": 28, 2024-09-16T00:35:25.840 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:35:25.840 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_items": 0, 2024-09-16T00:35:25.840 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_bytes": 0, 2024-09-16T00:35:25.840 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_items": 0, 2024-09-16T00:35:25.840 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:35:25.840 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_items": 0, 2024-09-16T00:35:25.841 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:35:25.841 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_items": 0, 2024-09-16T00:35:25.841 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_bytes": 0, 2024-09-16T00:35:25.841 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_items": 0, 2024-09-16T00:35:25.841 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:35:25.841 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:35:25.841 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_bytes": 0, 2024-09-16T00:35:25.841 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_items": 0, 2024-09-16T00:35:25.841 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_bytes": 928, 2024-09-16T00:35:25.841 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_items": 2, 2024-09-16T00:35:25.841 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_bytes": 0, 2024-09-16T00:35:25.841 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_items": 0, 2024-09-16T00:35:25.841 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_bytes": 0, 2024-09-16T00:35:25.841 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_items": 0 2024-09-16T00:35:25.841 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:25.842 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:35:25.842 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-09-16T00:35:25.842 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-09-16T00:35:25.842 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-09-16T00:35:25.842 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 98, 2024-09-16T00:35:25.842 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 707443, 2024-09-16T00:35:25.842 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-09-16T00:35:25.842 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 98, 2024-09-16T00:35:25.842 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-09-16T00:35:25.842 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-09-16T00:35:25.842 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 98, 2024-09-16T00:35:25.842 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 707443, 2024-09-16T00:35:25.842 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-09-16T00:35:25.842 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-16T00:35:25.843 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-16T00:35:25.843 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-16T00:35:25.843 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:25.843 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:25.843 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-16T00:35:25.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:25 smithi122 ceph-mon[35412]: pgmap v443: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 6 op/s 2024-09-16T00:35:26.664 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-09-16T00:35:26.664 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-09-16T00:35:26.664 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-09-16T00:35:26.664 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-09-16T00:35:26.664 DEBUG:tasks.check_counter:Getting stats from g 2024-09-16T00:35:26.664 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph daemon mds.g perf dump 2024-09-16T00:35:26.931 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:35:27.452 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-16T00:35:27.452 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:35:27.452 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 4, 2024-09-16T00:35:27.452 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 4, 2024-09-16T00:35:27.453 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 3072, 2024-09-16T00:35:27.453 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 448, 2024-09-16T00:35:27.453 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 4, 2024-09-16T00:35:27.453 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 0, 2024-09-16T00:35:27.453 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.003792405, 2024-09-16T00:35:27.453 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.000052848, 2024-09-16T00:35:27.453 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.002564749, 2024-09-16T00:35:27.453 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:35:27.453 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:35:27.453 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 3, 2024-09-16T00:35:27.453 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.004669172, 2024-09-16T00:35:27.453 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.001556390 2024-09-16T00:35:27.454 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:27.454 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:35:27.454 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-16T00:35:27.454 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-16T00:35:27.454 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-16T00:35:27.454 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:27.454 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-16T00:35:27.454 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 448 2024-09-16T00:35:27.454 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:27.454 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:35:27.454 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 94, 2024-09-16T00:35:27.454 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 89, 2024-09-16T00:35:27.454 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 720304, 2024-09-16T00:35:27.455 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 53472, 2024-09-16T00:35:27.455 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 2, 2024-09-16T00:35:27.455 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 1, 2024-09-16T00:35:27.455 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.017213746, 2024-09-16T00:35:27.455 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.005878225, 2024-09-16T00:35:27.455 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.016824114, 2024-09-16T00:35:27.455 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:35:27.455 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:35:27.455 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 88, 2024-09-16T00:35:27.455 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.004892201, 2024-09-16T00:35:27.455 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000055593 2024-09-16T00:35:27.455 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:27.455 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:35:27.456 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-16T00:35:27.456 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-16T00:35:27.456 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-16T00:35:27.456 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:27.456 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 720304, 2024-09-16T00:35:27.456 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 53472 2024-09-16T00:35:27.456 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:27.456 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:35:27.456 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 1, 2024-09-16T00:35:27.456 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 67, 2024-09-16T00:35:27.456 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 160, 2024-09-16T00:35:27.456 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 90384, 2024-09-16T00:35:27.456 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 3, 2024-09-16T00:35:27.456 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 2, 2024-09-16T00:35:27.457 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.006096963, 2024-09-16T00:35:27.457 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.004119225, 2024-09-16T00:35:27.457 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.001255089, 2024-09-16T00:35:27.457 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:35:27.457 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:35:27.457 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 65, 2024-09-16T00:35:27.457 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.002995980, 2024-09-16T00:35:27.457 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000046092 2024-09-16T00:35:27.457 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:27.457 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:35:27.457 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-16T00:35:27.457 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-16T00:35:27.457 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-16T00:35:27.457 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:27.458 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-16T00:35:27.458 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 90384 2024-09-16T00:35:27.458 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:27.458 INFO:teuthology.orchestra.run.smithi038.stdout: "cct": { 2024-09-16T00:35:27.458 INFO:teuthology.orchestra.run.smithi038.stdout: "total_workers": 0, 2024-09-16T00:35:27.458 INFO:teuthology.orchestra.run.smithi038.stdout: "unhealthy_workers": 0 2024-09-16T00:35:27.458 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:27.458 INFO:teuthology.orchestra.run.smithi038.stdout: "mempool": { 2024-09-16T00:35:27.458 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:35:27.458 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_items": 0, 2024-09-16T00:35:27.458 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:35:27.458 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:35:27.458 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:35:27.458 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:35:27.459 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:35:27.459 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:35:27.459 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:35:27.459 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:35:27.459 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:35:27.459 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:35:27.459 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:35:27.459 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:35:27.459 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:35:27.459 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_items": 0, 2024-09-16T00:35:27.459 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:35:27.459 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_items": 0, 2024-09-16T00:35:27.459 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:35:27.459 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:35:27.460 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:35:27.460 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:35:27.460 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:35:27.460 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:35:27.460 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:35:27.460 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_items": 0, 2024-09-16T00:35:27.460 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:35:27.460 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:35:27.460 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:35:27.460 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_items": 0, 2024-09-16T00:35:27.460 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_bytes": 0, 2024-09-16T00:35:27.460 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_items": 0, 2024-09-16T00:35:27.460 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:35:27.461 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:35:27.461 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:35:27.461 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:35:27.461 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_bytes": 149854, 2024-09-16T00:35:27.461 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_items": 28, 2024-09-16T00:35:27.461 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:35:27.461 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_items": 0, 2024-09-16T00:35:27.461 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_bytes": 0, 2024-09-16T00:35:27.461 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_items": 0, 2024-09-16T00:35:27.461 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:35:27.461 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_items": 0, 2024-09-16T00:35:27.461 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:35:27.461 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_items": 0, 2024-09-16T00:35:27.461 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_bytes": 0, 2024-09-16T00:35:27.462 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_items": 0, 2024-09-16T00:35:27.462 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:35:27.462 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:35:27.462 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_bytes": 0, 2024-09-16T00:35:27.462 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_items": 0, 2024-09-16T00:35:27.462 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_bytes": 928, 2024-09-16T00:35:27.462 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_items": 2, 2024-09-16T00:35:27.462 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_bytes": 0, 2024-09-16T00:35:27.462 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_items": 0, 2024-09-16T00:35:27.462 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_bytes": 0, 2024-09-16T00:35:27.462 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_items": 0 2024-09-16T00:35:27.462 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:27.462 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:35:27.462 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-09-16T00:35:27.462 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-09-16T00:35:27.463 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-09-16T00:35:27.463 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 95, 2024-09-16T00:35:27.463 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 707053, 2024-09-16T00:35:27.463 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-09-16T00:35:27.463 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 95, 2024-09-16T00:35:27.463 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-09-16T00:35:27.463 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-09-16T00:35:27.463 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 95, 2024-09-16T00:35:27.463 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 707053, 2024-09-16T00:35:27.463 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-09-16T00:35:27.463 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-16T00:35:27.463 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-16T00:35:27.463 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-16T00:35:27.463 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:27.464 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:27.464 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-16T00:35:27.715 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:27 smithi038 ceph-mon[30945]: pgmap v444: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-16T00:35:27.793 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:27 smithi073 ceph-mon[35254]: pgmap v444: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-16T00:35:27.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:27 smithi122 ceph-mon[35412]: pgmap v444: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-16T00:35:28.271 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2024-09-16T00:35:28.272 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2024-09-16T00:35:28.272 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2024-09-16T00:35:28.272 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2024-09-16T00:35:28.272 DEBUG:tasks.check_counter:Getting stats from j 2024-09-16T00:35:28.272 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph daemon mds.j perf dump 2024-09-16T00:35:28.534 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:35:29.047 INFO:teuthology.orchestra.run.smithi038.stdout:{ 2024-09-16T00:35:29.047 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:35:29.047 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 103, 2024-09-16T00:35:29.048 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 96, 2024-09-16T00:35:29.048 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 724736, 2024-09-16T00:35:29.048 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 57120, 2024-09-16T00:35:29.048 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 4, 2024-09-16T00:35:29.048 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 0, 2024-09-16T00:35:29.048 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.021607382, 2024-09-16T00:35:29.048 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.005903358, 2024-09-16T00:35:29.048 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.021561654, 2024-09-16T00:35:29.048 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:35:29.048 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:35:29.048 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 96, 2024-09-16T00:35:29.048 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.008399197, 2024-09-16T00:35:29.048 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000087491 2024-09-16T00:35:29.048 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:29.049 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:35:29.049 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-16T00:35:29.049 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-16T00:35:29.049 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-16T00:35:29.049 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:29.049 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 724736, 2024-09-16T00:35:29.049 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 57120 2024-09-16T00:35:29.049 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:29.049 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:35:29.049 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 1, 2024-09-16T00:35:29.049 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 71, 2024-09-16T00:35:29.049 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 160, 2024-09-16T00:35:29.049 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 91088, 2024-09-16T00:35:29.050 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 3, 2024-09-16T00:35:29.050 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 2, 2024-09-16T00:35:29.050 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.005521121, 2024-09-16T00:35:29.050 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.004068754, 2024-09-16T00:35:29.050 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.000718519, 2024-09-16T00:35:29.050 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:35:29.050 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:35:29.050 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 69, 2024-09-16T00:35:29.050 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.004531579, 2024-09-16T00:35:29.050 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000065675 2024-09-16T00:35:29.050 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:29.050 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:35:29.050 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-16T00:35:29.051 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-16T00:35:29.051 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-16T00:35:29.051 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:29.051 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-16T00:35:29.051 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 91088 2024-09-16T00:35:29.051 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:29.051 INFO:teuthology.orchestra.run.smithi038.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:35:29.051 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_messages": 0, 2024-09-16T00:35:29.051 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages": 2, 2024-09-16T00:35:29.051 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_bytes": 0, 2024-09-16T00:35:29.051 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_bytes": 0, 2024-09-16T00:35:29.051 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_created_connections": 2, 2024-09-16T00:35:29.051 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_active_connections": 1, 2024-09-16T00:35:29.051 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_total_time": 0.000295873, 2024-09-16T00:35:29.052 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_send_time": 0.000000000, 2024-09-16T00:35:29.052 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_recv_time": 0.000216673, 2024-09-16T00:35:29.052 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:35:29.052 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:35:29.052 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-16T00:35:29.052 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-16T00:35:29.052 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-16T00:35:29.052 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:29.052 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:35:29.052 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-16T00:35:29.052 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-16T00:35:29.052 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-16T00:35:29.052 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:29.052 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T00:35:29.053 INFO:teuthology.orchestra.run.smithi038.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T00:35:29.053 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:29.053 INFO:teuthology.orchestra.run.smithi038.stdout: "cct": { 2024-09-16T00:35:29.053 INFO:teuthology.orchestra.run.smithi038.stdout: "total_workers": 0, 2024-09-16T00:35:29.053 INFO:teuthology.orchestra.run.smithi038.stdout: "unhealthy_workers": 0 2024-09-16T00:35:29.053 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:29.053 INFO:teuthology.orchestra.run.smithi038.stdout: "mempool": { 2024-09-16T00:35:29.053 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:35:29.053 INFO:teuthology.orchestra.run.smithi038.stdout: "bloom_filter_items": 0, 2024-09-16T00:35:29.053 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:35:29.053 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:35:29.053 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:35:29.053 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:35:29.053 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:35:29.053 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:35:29.054 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:35:29.054 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:35:29.054 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:35:29.054 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:35:29.054 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:35:29.054 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:35:29.054 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:35:29.054 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_extent_items": 0, 2024-09-16T00:35:29.054 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:35:29.054 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_blob_items": 0, 2024-09-16T00:35:29.054 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:35:29.054 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:35:29.054 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:35:29.055 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:35:29.055 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:35:29.055 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:35:29.055 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:35:29.055 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_txc_items": 0, 2024-09-16T00:35:29.055 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:35:29.055 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:35:29.055 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:35:29.055 INFO:teuthology.orchestra.run.smithi038.stdout: "bluestore_writing_items": 0, 2024-09-16T00:35:29.055 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_bytes": 0, 2024-09-16T00:35:29.055 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_items": 0, 2024-09-16T00:35:29.055 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:35:29.056 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:35:29.056 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:35:29.056 INFO:teuthology.orchestra.run.smithi038.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:35:29.056 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_bytes": 148840, 2024-09-16T00:35:29.056 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_anon_items": 28, 2024-09-16T00:35:29.056 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:35:29.056 INFO:teuthology.orchestra.run.smithi038.stdout: "buffer_meta_items": 0, 2024-09-16T00:35:29.056 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_bytes": 0, 2024-09-16T00:35:29.056 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_items": 0, 2024-09-16T00:35:29.056 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:35:29.056 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_mapbl_items": 0, 2024-09-16T00:35:29.056 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:35:29.056 INFO:teuthology.orchestra.run.smithi038.stdout: "osd_pglog_items": 0, 2024-09-16T00:35:29.057 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_bytes": 0, 2024-09-16T00:35:29.057 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_items": 0, 2024-09-16T00:35:29.057 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:35:29.057 INFO:teuthology.orchestra.run.smithi038.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:35:29.057 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_bytes": 0, 2024-09-16T00:35:29.057 INFO:teuthology.orchestra.run.smithi038.stdout: "pgmap_items": 0, 2024-09-16T00:35:29.057 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_bytes": 928, 2024-09-16T00:35:29.057 INFO:teuthology.orchestra.run.smithi038.stdout: "mds_co_items": 2, 2024-09-16T00:35:29.057 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_bytes": 0, 2024-09-16T00:35:29.057 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_1_items": 0, 2024-09-16T00:35:29.057 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_bytes": 0, 2024-09-16T00:35:29.057 INFO:teuthology.orchestra.run.smithi038.stdout: "unittest_2_items": 0 2024-09-16T00:35:29.057 INFO:teuthology.orchestra.run.smithi038.stdout: }, 2024-09-16T00:35:29.057 INFO:teuthology.orchestra.run.smithi038.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:35:29.058 INFO:teuthology.orchestra.run.smithi038.stdout: "val": 0, 2024-09-16T00:35:29.058 INFO:teuthology.orchestra.run.smithi038.stdout: "max": 104857600, 2024-09-16T00:35:29.058 INFO:teuthology.orchestra.run.smithi038.stdout: "get_started": 0, 2024-09-16T00:35:29.058 INFO:teuthology.orchestra.run.smithi038.stdout: "get": 100, 2024-09-16T00:35:29.058 INFO:teuthology.orchestra.run.smithi038.stdout: "get_sum": 707703, 2024-09-16T00:35:29.058 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_fail": 0, 2024-09-16T00:35:29.058 INFO:teuthology.orchestra.run.smithi038.stdout: "get_or_fail_success": 100, 2024-09-16T00:35:29.058 INFO:teuthology.orchestra.run.smithi038.stdout: "take": 0, 2024-09-16T00:35:29.058 INFO:teuthology.orchestra.run.smithi038.stdout: "take_sum": 0, 2024-09-16T00:35:29.058 INFO:teuthology.orchestra.run.smithi038.stdout: "put": 100, 2024-09-16T00:35:29.058 INFO:teuthology.orchestra.run.smithi038.stdout: "put_sum": 707703, 2024-09-16T00:35:29.058 INFO:teuthology.orchestra.run.smithi038.stdout: "wait": { 2024-09-16T00:35:29.058 INFO:teuthology.orchestra.run.smithi038.stdout: "avgcount": 0, 2024-09-16T00:35:29.058 INFO:teuthology.orchestra.run.smithi038.stdout: "sum": 0.000000000, 2024-09-16T00:35:29.059 INFO:teuthology.orchestra.run.smithi038.stdout: "avgtime": 0.000000000 2024-09-16T00:35:29.059 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:29.059 INFO:teuthology.orchestra.run.smithi038.stdout: } 2024-09-16T00:35:29.059 INFO:teuthology.orchestra.run.smithi038.stdout:} 2024-09-16T00:35:29.714 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:29 smithi038 ceph-mon[30945]: pgmap v445: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:35:29.793 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:29 smithi073 ceph-mon[35254]: pgmap v445: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:35:29.862 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-09-16T00:35:29.862 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-09-16T00:35:29.862 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-09-16T00:35:29.862 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-09-16T00:35:29.863 DEBUG:tasks.check_counter:Getting stats from b 2024-09-16T00:35:29.863 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph daemon mds.b perf dump 2024-09-16T00:35:29.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:29 smithi122 ceph-mon[35412]: pgmap v445: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:35:30.121 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:35:30.619 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-16T00:35:30.620 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:35:30.620 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 3884, 2024-09-16T00:35:30.620 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 4094, 2024-09-16T00:35:30.620 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 968804, 2024-09-16T00:35:30.620 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 9934307, 2024-09-16T00:35:30.620 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 31, 2024-09-16T00:35:30.620 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 4, 2024-09-16T00:35:30.620 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 0.291866818, 2024-09-16T00:35:30.620 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 0.105527577, 2024-09-16T00:35:30.620 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 0.088898702, 2024-09-16T00:35:30.620 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.079678877, 2024-09-16T00:35:30.620 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:35:30.620 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 4092, 2024-09-16T00:35:30.621 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.176170369, 2024-09-16T00:35:30.621 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000043052 2024-09-16T00:35:30.621 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.621 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:35:30.621 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 1304, 2024-09-16T00:35:30.621 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000300321, 2024-09-16T00:35:30.621 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000230 2024-09-16T00:35:30.621 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.621 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 35392, 2024-09-16T00:35:30.621 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 5136 2024-09-16T00:35:30.621 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.622 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:35:30.622 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 7740, 2024-09-16T00:35:30.622 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 12417, 2024-09-16T00:35:30.622 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 2577095, 2024-09-16T00:35:30.622 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 9530593, 2024-09-16T00:35:30.622 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 30, 2024-09-16T00:35:30.622 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 5, 2024-09-16T00:35:30.622 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 0.494727489, 2024-09-16T00:35:30.622 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 0.180206214, 2024-09-16T00:35:30.622 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 0.296592545, 2024-09-16T00:35:30.622 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.075123340, 2024-09-16T00:35:30.622 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:35:30.622 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 12416, 2024-09-16T00:35:30.623 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.340336344, 2024-09-16T00:35:30.623 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000027411 2024-09-16T00:35:30.623 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.623 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:35:30.623 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 13110, 2024-09-16T00:35:30.623 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.001968199, 2024-09-16T00:35:30.623 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000150 2024-09-16T00:35:30.623 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.623 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 53968, 2024-09-16T00:35:30.623 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 229504 2024-09-16T00:35:30.623 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.623 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:35:30.623 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 2713, 2024-09-16T00:35:30.623 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 2971, 2024-09-16T00:35:30.624 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 1435863, 2024-09-16T00:35:30.624 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 1202781, 2024-09-16T00:35:30.624 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 12, 2024-09-16T00:35:30.624 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 6, 2024-09-16T00:35:30.624 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 0.208705775, 2024-09-16T00:35:30.624 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 0.074983862, 2024-09-16T00:35:30.624 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 0.088519476, 2024-09-16T00:35:30.624 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.049554779, 2024-09-16T00:35:30.624 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:35:30.624 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 2970, 2024-09-16T00:35:30.624 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.198477546, 2024-09-16T00:35:30.625 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000066827 2024-09-16T00:35:30.625 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.625 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:35:30.625 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 1253, 2024-09-16T00:35:30.625 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000264729, 2024-09-16T00:35:30.625 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000211 2024-09-16T00:35:30.625 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.625 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 771776, 2024-09-16T00:35:30.625 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 65296 2024-09-16T00:35:30.625 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.626 INFO:teuthology.orchestra.run.smithi073.stdout: "cct": { 2024-09-16T00:35:30.626 INFO:teuthology.orchestra.run.smithi073.stdout: "total_workers": 1, 2024-09-16T00:35:30.626 INFO:teuthology.orchestra.run.smithi073.stdout: "unhealthy_workers": 0 2024-09-16T00:35:30.626 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.626 INFO:teuthology.orchestra.run.smithi073.stdout: "finisher-MDSRank": { 2024-09-16T00:35:30.626 INFO:teuthology.orchestra.run.smithi073.stdout: "queue_len": 0, 2024-09-16T00:35:30.626 INFO:teuthology.orchestra.run.smithi073.stdout: "complete_latency": { 2024-09-16T00:35:30.626 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 4351, 2024-09-16T00:35:30.626 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 2.037007315, 2024-09-16T00:35:30.626 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000468169 2024-09-16T00:35:30.626 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-16T00:35:30.626 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.626 INFO:teuthology.orchestra.run.smithi073.stdout: "finisher-PurgeQueue": { 2024-09-16T00:35:30.626 INFO:teuthology.orchestra.run.smithi073.stdout: "queue_len": 0, 2024-09-16T00:35:30.627 INFO:teuthology.orchestra.run.smithi073.stdout: "complete_latency": { 2024-09-16T00:35:30.627 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 5321, 2024-09-16T00:35:30.627 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.219740731, 2024-09-16T00:35:30.627 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000041296 2024-09-16T00:35:30.627 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-16T00:35:30.627 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.627 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": { 2024-09-16T00:35:30.627 INFO:teuthology.orchestra.run.smithi073.stdout: "request": 2487, 2024-09-16T00:35:30.627 INFO:teuthology.orchestra.run.smithi073.stdout: "reply": 2487, 2024-09-16T00:35:30.627 INFO:teuthology.orchestra.run.smithi073.stdout: "reply_latency": { 2024-09-16T00:35:30.628 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 2487, 2024-09-16T00:35:30.628 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 2.651654576, 2024-09-16T00:35:30.628 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.001066206 2024-09-16T00:35:30.628 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.628 INFO:teuthology.orchestra.run.smithi073.stdout: "slow_reply": 0, 2024-09-16T00:35:30.628 INFO:teuthology.orchestra.run.smithi073.stdout: "forward": 0, 2024-09-16T00:35:30.629 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_fetch_complete": 0, 2024-09-16T00:35:30.629 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_fetch_keys": 0, 2024-09-16T00:35:30.629 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_commit": 12, 2024-09-16T00:35:30.629 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_split": 0, 2024-09-16T00:35:30.629 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_merge": 0, 2024-09-16T00:35:30.629 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes": 24, 2024-09-16T00:35:30.629 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_top": 0, 2024-09-16T00:35:30.629 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_bottom": 24, 2024-09-16T00:35:30.629 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_pin_tail": 0, 2024-09-16T00:35:30.629 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_pinned": 23, 2024-09-16T00:35:30.629 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_expired": 0, 2024-09-16T00:35:30.629 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_with_caps": 2, 2024-09-16T00:35:30.629 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": 2, 2024-09-16T00:35:30.629 INFO:teuthology.orchestra.run.smithi073.stdout: "subtrees": 2, 2024-09-16T00:35:30.630 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse": 2533, 2024-09-16T00:35:30.630 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_hit": 2515, 2024-09-16T00:35:30.630 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_forward": 0, 2024-09-16T00:35:30.630 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_discover": 0, 2024-09-16T00:35:30.630 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_dir_fetch": 0, 2024-09-16T00:35:30.630 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_remote_ino": 0, 2024-09-16T00:35:30.630 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_lock": 0, 2024-09-16T00:35:30.630 INFO:teuthology.orchestra.run.smithi073.stdout: "load_cent": 0, 2024-09-16T00:35:30.630 INFO:teuthology.orchestra.run.smithi073.stdout: "q": 0, 2024-09-16T00:35:30.630 INFO:teuthology.orchestra.run.smithi073.stdout: "exported": 0, 2024-09-16T00:35:30.630 INFO:teuthology.orchestra.run.smithi073.stdout: "exported_inodes": 0, 2024-09-16T00:35:30.630 INFO:teuthology.orchestra.run.smithi073.stdout: "imported": 0, 2024-09-16T00:35:30.630 INFO:teuthology.orchestra.run.smithi073.stdout: "imported_inodes": 0, 2024-09-16T00:35:30.630 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_dir_fetch": 0, 2024-09-16T00:35:30.631 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_backtrace_fetch": 49, 2024-09-16T00:35:30.631 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_peer_discover": 0, 2024-09-16T00:35:30.631 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rfiles": 4, 2024-09-16T00:35:30.631 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rbytes": 234, 2024-09-16T00:35:30.631 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rsnaps": 0, 2024-09-16T00:35:30.631 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_backtrace_fetch": 49, 2024-09-16T00:35:30.631 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_set_tag": 0, 2024-09-16T00:35:30.631 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T00:35:30.631 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_inotable_repaired": 0, 2024-09-16T00:35:30.631 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dir_inodes": 32, 2024-09-16T00:35:30.631 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dir_base_inodes": 4, 2024-09-16T00:35:30.631 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dirfrag_rstats": 28, 2024-09-16T00:35:30.631 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_file_inodes": 17, 2024-09-16T00:35:30.631 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_inode_file_caps": 2, 2024-09-16T00:35:30.632 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_revoke": 26, 2024-09-16T00:35:30.632 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_grant": 3641, 2024-09-16T00:35:30.632 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T00:35:30.632 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T00:35:30.632 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T00:35:30.632 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_caps": 2443, 2024-09-16T00:35:30.632 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_caps_dirty": 1212, 2024-09-16T00:35:30.632 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_cap_release": 14, 2024-09-16T00:35:30.632 INFO:teuthology.orchestra.run.smithi073.stdout: "process_request_cap_release": 1241 2024-09-16T00:35:30.632 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.632 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache": { 2024-09-16T00:35:30.632 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_update": 13, 2024-09-16T00:35:30.632 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_update_receipt": 0, 2024-09-16T00:35:30.632 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_try_discover": 0, 2024-09-16T00:35:30.633 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_send_discover": 0, 2024-09-16T00:35:30.633 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_handle_discover": 24, 2024-09-16T00:35:30.633 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays": 0, 2024-09-16T00:35:30.633 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays_delayed": 0, 2024-09-16T00:35:30.633 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays_enqueuing": 0, 2024-09-16T00:35:30.633 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_created": 1212, 2024-09-16T00:35:30.633 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_enqueued": 1212, 2024-09-16T00:35:30.633 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_reintegrated": 0, 2024-09-16T00:35:30.633 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_migrated": 0, 2024-09-16T00:35:30.633 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_processing": 0, 2024-09-16T00:35:30.633 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_enqueued": 0, 2024-09-16T00:35:30.633 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_prioritized": 0, 2024-09-16T00:35:30.633 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_started": 0, 2024-09-16T00:35:30.633 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_completed": 0, 2024-09-16T00:35:30.634 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_quiesce_path": 1, 2024-09-16T00:35:30.634 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_quiesce_inode": 12, 2024-09-16T00:35:30.634 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_enqueue_scrub": 4, 2024-09-16T00:35:30.634 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_exportdir": 0, 2024-09-16T00:35:30.634 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_flush": 0, 2024-09-16T00:35:30.634 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_fragmentdir": 0, 2024-09-16T00:35:30.634 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_fragstats": 0, 2024-09-16T00:35:30.634 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_inodestats": 0 2024-09-16T00:35:30.634 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.634 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log": { 2024-09-16T00:35:30.634 INFO:teuthology.orchestra.run.smithi073.stdout: "evlrg": 0, 2024-09-16T00:35:30.634 INFO:teuthology.orchestra.run.smithi073.stdout: "evadd": 6119, 2024-09-16T00:35:30.634 INFO:teuthology.orchestra.run.smithi073.stdout: "evex": 0, 2024-09-16T00:35:30.634 INFO:teuthology.orchestra.run.smithi073.stdout: "evtrm": 0, 2024-09-16T00:35:30.635 INFO:teuthology.orchestra.run.smithi073.stdout: "ev": 6119, 2024-09-16T00:35:30.635 INFO:teuthology.orchestra.run.smithi073.stdout: "evexg": 0, 2024-09-16T00:35:30.635 INFO:teuthology.orchestra.run.smithi073.stdout: "evexd": 0, 2024-09-16T00:35:30.635 INFO:teuthology.orchestra.run.smithi073.stdout: "segadd": 9, 2024-09-16T00:35:30.635 INFO:teuthology.orchestra.run.smithi073.stdout: "segex": 0, 2024-09-16T00:35:30.635 INFO:teuthology.orchestra.run.smithi073.stdout: "segtrm": 0, 2024-09-16T00:35:30.635 INFO:teuthology.orchestra.run.smithi073.stdout: "seg": 9, 2024-09-16T00:35:30.635 INFO:teuthology.orchestra.run.smithi073.stdout: "segmjr": 2, 2024-09-16T00:35:30.635 INFO:teuthology.orchestra.run.smithi073.stdout: "segexg": 0, 2024-09-16T00:35:30.635 INFO:teuthology.orchestra.run.smithi073.stdout: "segexd": 0, 2024-09-16T00:35:30.635 INFO:teuthology.orchestra.run.smithi073.stdout: "expos": 4194304, 2024-09-16T00:35:30.635 INFO:teuthology.orchestra.run.smithi073.stdout: "wrpos": 16858830, 2024-09-16T00:35:30.635 INFO:teuthology.orchestra.run.smithi073.stdout: "rdpos": 0, 2024-09-16T00:35:30.635 INFO:teuthology.orchestra.run.smithi073.stdout: "jlat": { 2024-09-16T00:35:30.636 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 2447, 2024-09-16T00:35:30.636 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 3.753151336, 2024-09-16T00:35:30.636 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.001533776 2024-09-16T00:35:30.636 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.636 INFO:teuthology.orchestra.run.smithi073.stdout: "replayed": 0 2024-09-16T00:35:30.636 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.636 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_mem": { 2024-09-16T00:35:30.636 INFO:teuthology.orchestra.run.smithi073.stdout: "ino": 23, 2024-09-16T00:35:30.636 INFO:teuthology.orchestra.run.smithi073.stdout: "ino+": 1239, 2024-09-16T00:35:30.636 INFO:teuthology.orchestra.run.smithi073.stdout: "ino-": 1216, 2024-09-16T00:35:30.636 INFO:teuthology.orchestra.run.smithi073.stdout: "dir": 18, 2024-09-16T00:35:30.636 INFO:teuthology.orchestra.run.smithi073.stdout: "dir+": 24, 2024-09-16T00:35:30.636 INFO:teuthology.orchestra.run.smithi073.stdout: "dir-": 6, 2024-09-16T00:35:30.636 INFO:teuthology.orchestra.run.smithi073.stdout: "dn": 24, 2024-09-16T00:35:30.636 INFO:teuthology.orchestra.run.smithi073.stdout: "dn+": 1243, 2024-09-16T00:35:30.637 INFO:teuthology.orchestra.run.smithi073.stdout: "dn-": 1219, 2024-09-16T00:35:30.637 INFO:teuthology.orchestra.run.smithi073.stdout: "cap": 2, 2024-09-16T00:35:30.637 INFO:teuthology.orchestra.run.smithi073.stdout: "cap+": 1230, 2024-09-16T00:35:30.637 INFO:teuthology.orchestra.run.smithi073.stdout: "cap-": 1228, 2024-09-16T00:35:30.637 INFO:teuthology.orchestra.run.smithi073.stdout: "rss": 59996, 2024-09-16T00:35:30.637 INFO:teuthology.orchestra.run.smithi073.stdout: "heap": 231712 2024-09-16T00:35:30.637 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.637 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_server": { 2024-09-16T00:35:30.637 INFO:teuthology.orchestra.run.smithi073.stdout: "dispatch_client_request": 2742, 2024-09-16T00:35:30.637 INFO:teuthology.orchestra.run.smithi073.stdout: "dispatch_server_request": 0, 2024-09-16T00:35:30.637 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_request": 2487, 2024-09-16T00:35:30.637 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_session": 18, 2024-09-16T00:35:30.637 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_peer_request": 0, 2024-09-16T00:35:30.637 INFO:teuthology.orchestra.run.smithi073.stdout: "req_create_latency": { 2024-09-16T00:35:30.638 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 1210, 2024-09-16T00:35:30.638 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 1.033831455, 2024-09-16T00:35:30.638 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000854406 2024-09-16T00:35:30.638 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.638 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getattr_latency": { 2024-09-16T00:35:30.638 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 29, 2024-09-16T00:35:30.638 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.008246724, 2024-09-16T00:35:30.638 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000284369 2024-09-16T00:35:30.638 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.638 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getfilelock_latency": { 2024-09-16T00:35:30.638 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:30.638 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:30.638 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:30.638 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.639 INFO:teuthology.orchestra.run.smithi073.stdout: "req_link_latency": { 2024-09-16T00:35:30.639 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:30.639 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:30.639 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:30.639 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.639 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookup_latency": { 2024-09-16T00:35:30.639 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 3, 2024-09-16T00:35:30.639 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.001249948, 2024-09-16T00:35:30.639 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000416649 2024-09-16T00:35:30.639 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.639 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookuphash_latency": { 2024-09-16T00:35:30.639 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:30.639 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:30.639 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:30.640 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.640 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupino_latency": { 2024-09-16T00:35:30.640 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 1, 2024-09-16T00:35:30.640 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000181423, 2024-09-16T00:35:30.640 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000181423 2024-09-16T00:35:30.640 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.640 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupname_latency": { 2024-09-16T00:35:30.640 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:30.640 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:30.640 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:30.640 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.640 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupparent_latency": { 2024-09-16T00:35:30.640 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:30.640 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:30.640 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:30.641 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.641 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupsnap_latency": { 2024-09-16T00:35:30.641 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:30.641 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:30.641 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:30.641 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.641 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lssnap_latency": { 2024-09-16T00:35:30.641 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:30.641 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:30.642 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:30.642 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.642 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mkdir_latency": { 2024-09-16T00:35:30.642 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 12, 2024-09-16T00:35:30.642 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.020033043, 2024-09-16T00:35:30.642 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.001669420 2024-09-16T00:35:30.643 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.643 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mknod_latency": { 2024-09-16T00:35:30.643 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:30.643 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:30.643 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:30.643 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.643 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mksnap_latency": { 2024-09-16T00:35:30.643 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:30.643 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:30.643 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:30.643 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.643 INFO:teuthology.orchestra.run.smithi073.stdout: "req_open_latency": { 2024-09-16T00:35:30.643 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 1, 2024-09-16T00:35:30.643 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000328790, 2024-09-16T00:35:30.644 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000328790 2024-09-16T00:35:30.644 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.644 INFO:teuthology.orchestra.run.smithi073.stdout: "req_readdir_latency": { 2024-09-16T00:35:30.644 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 6, 2024-09-16T00:35:30.644 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.005228592, 2024-09-16T00:35:30.644 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000871432 2024-09-16T00:35:30.644 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.644 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rename_latency": { 2024-09-16T00:35:30.644 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 2, 2024-09-16T00:35:30.645 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.002054957, 2024-09-16T00:35:30.645 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.001027478 2024-09-16T00:35:30.645 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.645 INFO:teuthology.orchestra.run.smithi073.stdout: "req_renamesnap_latency": { 2024-09-16T00:35:30.645 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:30.645 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:30.645 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:30.645 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.645 INFO:teuthology.orchestra.run.smithi073.stdout: "req_snapdiff_latency": { 2024-09-16T00:35:30.645 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:30.645 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:30.645 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:30.646 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.646 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmdir_latency": { 2024-09-16T00:35:30.646 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 6, 2024-09-16T00:35:30.646 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.029248305, 2024-09-16T00:35:30.646 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.004874717 2024-09-16T00:35:30.646 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.646 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmsnap_latency": { 2024-09-16T00:35:30.646 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:30.646 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:30.646 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:30.646 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.646 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmxattr_latency": { 2024-09-16T00:35:30.646 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:30.646 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:30.647 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:30.647 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.647 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setattr_latency": { 2024-09-16T00:35:30.647 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 2, 2024-09-16T00:35:30.647 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.001464355, 2024-09-16T00:35:30.647 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000732177 2024-09-16T00:35:30.647 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.647 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setdirlayout_latency": { 2024-09-16T00:35:30.647 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:30.647 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:30.647 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:30.647 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.647 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setfilelock_latency": { 2024-09-16T00:35:30.647 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:30.648 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:30.648 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:30.648 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.648 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setlayout_latency": { 2024-09-16T00:35:30.648 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:30.648 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:30.648 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:30.648 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.648 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setxattr_latency": { 2024-09-16T00:35:30.648 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 9, 2024-09-16T00:35:30.648 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.021375747, 2024-09-16T00:35:30.648 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.002375083 2024-09-16T00:35:30.648 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.648 INFO:teuthology.orchestra.run.smithi073.stdout: "req_symlink_latency": { 2024-09-16T00:35:30.649 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:30.649 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:30.649 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:30.649 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.649 INFO:teuthology.orchestra.run.smithi073.stdout: "req_unlink_latency": { 2024-09-16T00:35:30.649 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 1206, 2024-09-16T00:35:30.649 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 1.528411237, 2024-09-16T00:35:30.649 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.001267339 2024-09-16T00:35:30.649 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.649 INFO:teuthology.orchestra.run.smithi073.stdout: "cap_revoke_eviction": 0, 2024-09-16T00:35:30.649 INFO:teuthology.orchestra.run.smithi073.stdout: "cap_acquisition_throttle": 0, 2024-09-16T00:35:30.649 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getvxattr_latency": { 2024-09-16T00:35:30.649 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:30.650 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:30.650 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:30.650 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-16T00:35:30.650 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.650 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_sessions": { 2024-09-16T00:35:30.650 INFO:teuthology.orchestra.run.smithi073.stdout: "session_count": 1, 2024-09-16T00:35:30.650 INFO:teuthology.orchestra.run.smithi073.stdout: "session_add": 4, 2024-09-16T00:35:30.650 INFO:teuthology.orchestra.run.smithi073.stdout: "session_remove": 3, 2024-09-16T00:35:30.650 INFO:teuthology.orchestra.run.smithi073.stdout: "sessions_open": 1, 2024-09-16T00:35:30.650 INFO:teuthology.orchestra.run.smithi073.stdout: "sessions_stale": 0, 2024-09-16T00:35:30.650 INFO:teuthology.orchestra.run.smithi073.stdout: "total_load": 2134, 2024-09-16T00:35:30.650 INFO:teuthology.orchestra.run.smithi073.stdout: "average_load": 2134, 2024-09-16T00:35:30.650 INFO:teuthology.orchestra.run.smithi073.stdout: "avg_session_uptime": 166, 2024-09-16T00:35:30.651 INFO:teuthology.orchestra.run.smithi073.stdout: "mdthresh_evicted": 0 2024-09-16T00:35:30.651 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.651 INFO:teuthology.orchestra.run.smithi073.stdout: "mempool": { 2024-09-16T00:35:30.651 INFO:teuthology.orchestra.run.smithi073.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:35:30.651 INFO:teuthology.orchestra.run.smithi073.stdout: "bloom_filter_items": 0, 2024-09-16T00:35:30.651 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:35:30.651 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:35:30.651 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:35:30.651 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:35:30.651 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:35:30.651 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:35:30.651 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:35:30.651 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:35:30.651 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:35:30.652 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:35:30.652 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:35:30.652 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:35:30.652 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:35:30.652 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_items": 0, 2024-09-16T00:35:30.652 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:35:30.652 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blob_items": 0, 2024-09-16T00:35:30.652 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:35:30.652 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:35:30.652 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:35:30.652 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:35:30.652 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:35:30.652 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:35:30.652 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:35:30.653 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_txc_items": 0, 2024-09-16T00:35:30.653 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:35:30.653 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:35:30.653 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:35:30.653 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_items": 0, 2024-09-16T00:35:30.653 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_bytes": 0, 2024-09-16T00:35:30.653 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_items": 0, 2024-09-16T00:35:30.653 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:35:30.653 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:35:30.653 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:35:30.653 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:35:30.653 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_anon_bytes": 304448, 2024-09-16T00:35:30.653 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_anon_items": 112, 2024-09-16T00:35:30.653 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_meta_bytes": 264, 2024-09-16T00:35:30.654 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_meta_items": 3, 2024-09-16T00:35:30.654 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_bytes": 0, 2024-09-16T00:35:30.654 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_items": 0, 2024-09-16T00:35:30.654 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:35:30.654 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mapbl_items": 0, 2024-09-16T00:35:30.654 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:35:30.654 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pglog_items": 0, 2024-09-16T00:35:30.654 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_bytes": 8768, 2024-09-16T00:35:30.654 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_items": 165, 2024-09-16T00:35:30.654 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:35:30.654 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:35:30.654 INFO:teuthology.orchestra.run.smithi073.stdout: "pgmap_bytes": 0, 2024-09-16T00:35:30.654 INFO:teuthology.orchestra.run.smithi073.stdout: "pgmap_items": 0, 2024-09-16T00:35:30.654 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_co_bytes": 96634, 2024-09-16T00:35:30.655 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_co_items": 713, 2024-09-16T00:35:30.655 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_1_bytes": 0, 2024-09-16T00:35:30.655 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_1_items": 0, 2024-09-16T00:35:30.655 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_2_bytes": 0, 2024-09-16T00:35:30.655 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_2_items": 0 2024-09-16T00:35:30.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.655 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter": { 2024-09-16T00:35:30.655 INFO:teuthology.orchestra.run.smithi073.stdout: "op_active": 0, 2024-09-16T00:35:30.655 INFO:teuthology.orchestra.run.smithi073.stdout: "op_laggy": 0, 2024-09-16T00:35:30.655 INFO:teuthology.orchestra.run.smithi073.stdout: "op_send": 7903, 2024-09-16T00:35:30.655 INFO:teuthology.orchestra.run.smithi073.stdout: "op_send_bytes": 12917587, 2024-09-16T00:35:30.655 INFO:teuthology.orchestra.run.smithi073.stdout: "op_resend": 37, 2024-09-16T00:35:30.655 INFO:teuthology.orchestra.run.smithi073.stdout: "op_reply": 7903, 2024-09-16T00:35:30.655 INFO:teuthology.orchestra.run.smithi073.stdout: "op_latency": { 2024-09-16T00:35:30.655 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 7903, 2024-09-16T00:35:30.656 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 39.103098845, 2024-09-16T00:35:30.656 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.004947880 2024-09-16T00:35:30.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.656 INFO:teuthology.orchestra.run.smithi073.stdout: "op_inflight": 0, 2024-09-16T00:35:30.656 INFO:teuthology.orchestra.run.smithi073.stdout: "oplen_avg": { 2024-09-16T00:35:30.656 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 7903, 2024-09-16T00:35:30.656 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 7996 2024-09-16T00:35:30.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.656 INFO:teuthology.orchestra.run.smithi073.stdout: "op": 7903, 2024-09-16T00:35:30.656 INFO:teuthology.orchestra.run.smithi073.stdout: "op_r": 91, 2024-09-16T00:35:30.656 INFO:teuthology.orchestra.run.smithi073.stdout: "op_w": 7812, 2024-09-16T00:35:30.656 INFO:teuthology.orchestra.run.smithi073.stdout: "op_rmw": 0, 2024-09-16T00:35:30.656 INFO:teuthology.orchestra.run.smithi073.stdout: "op_pg": 0, 2024-09-16T00:35:30.657 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_stat": 0, 2024-09-16T00:35:30.657 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_create": 11, 2024-09-16T00:35:30.657 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_read": 38, 2024-09-16T00:35:30.657 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_write": 2481, 2024-09-16T00:35:30.657 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_writefull": 35, 2024-09-16T00:35:30.657 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_writesame": 0, 2024-09-16T00:35:30.657 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_append": 0, 2024-09-16T00:35:30.657 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_zero": 0, 2024-09-16T00:35:30.657 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_truncate": 0, 2024-09-16T00:35:30.657 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_delete": 5219, 2024-09-16T00:35:30.657 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_mapext": 0, 2024-09-16T00:35:30.657 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_sparse_read": 0, 2024-09-16T00:35:30.657 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_clonerange": 0, 2024-09-16T00:35:30.657 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_getxattr": 53, 2024-09-16T00:35:30.657 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_setxattr": 22, 2024-09-16T00:35:30.658 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_cmpxattr": 0, 2024-09-16T00:35:30.658 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_rmxattr": 0, 2024-09-16T00:35:30.658 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_resetxattrs": 0, 2024-09-16T00:35:30.658 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_call": 0, 2024-09-16T00:35:30.658 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_watch": 0, 2024-09-16T00:35:30.658 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_notify": 0, 2024-09-16T00:35:30.658 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T00:35:30.658 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_pgls": 0, 2024-09-16T00:35:30.658 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_pgls_filter": 0, 2024-09-16T00:35:30.658 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_other": 0, 2024-09-16T00:35:30.658 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_active": 0, 2024-09-16T00:35:30.659 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_send": 0, 2024-09-16T00:35:30.659 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_resend": 0, 2024-09-16T00:35:30.659 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_ping": 0, 2024-09-16T00:35:30.659 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_active": 0, 2024-09-16T00:35:30.659 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_send": 0, 2024-09-16T00:35:30.659 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_resend": 0, 2024-09-16T00:35:30.659 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_active": 0, 2024-09-16T00:35:30.659 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_send": 0, 2024-09-16T00:35:30.659 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_resend": 0, 2024-09-16T00:35:30.659 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_active": 0, 2024-09-16T00:35:30.659 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_send": 0, 2024-09-16T00:35:30.659 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_resend": 0, 2024-09-16T00:35:30.660 INFO:teuthology.orchestra.run.smithi073.stdout: "command_active": 0, 2024-09-16T00:35:30.660 INFO:teuthology.orchestra.run.smithi073.stdout: "command_send": 0, 2024-09-16T00:35:30.660 INFO:teuthology.orchestra.run.smithi073.stdout: "command_resend": 0, 2024-09-16T00:35:30.660 INFO:teuthology.orchestra.run.smithi073.stdout: "map_epoch": 92, 2024-09-16T00:35:30.660 INFO:teuthology.orchestra.run.smithi073.stdout: "map_full": 0, 2024-09-16T00:35:30.660 INFO:teuthology.orchestra.run.smithi073.stdout: "map_inc": 10, 2024-09-16T00:35:30.660 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_sessions": 12, 2024-09-16T00:35:30.660 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_session_open": 12, 2024-09-16T00:35:30.660 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_session_close": 0, 2024-09-16T00:35:30.660 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_laggy": 0, 2024-09-16T00:35:30.660 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_wr": 112, 2024-09-16T00:35:30.660 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_rd": 0, 2024-09-16T00:35:30.660 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_del": 25 2024-09-16T00:35:30.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.661 INFO:teuthology.orchestra.run.smithi073.stdout: "oft": { 2024-09-16T00:35:30.661 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_objs": 1, 2024-09-16T00:35:30.661 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_kv_pairs": 0, 2024-09-16T00:35:30.661 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_updates": 0, 2024-09-16T00:35:30.661 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_removes": 0 2024-09-16T00:35:30.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.661 INFO:teuthology.orchestra.run.smithi073.stdout: "osd-slow-ops": { 2024-09-16T00:35:30.661 INFO:teuthology.orchestra.run.smithi073.stdout: "slow_ops_count": 13 2024-09-16T00:35:30.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.661 INFO:teuthology.orchestra.run.smithi073.stdout: "purge_queue": { 2024-09-16T00:35:30.661 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_ops": 0, 2024-09-16T00:35:30.662 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_ops_high_water": 19, 2024-09-16T00:35:30.662 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing": 0, 2024-09-16T00:35:30.662 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_high_water": 5, 2024-09-16T00:35:30.662 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executed_ops": 4704, 2024-09-16T00:35:30.662 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executed": 1212, 2024-09-16T00:35:30.662 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_item_in_journal": 0 2024-09-16T00:35:30.662 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.662 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:35:30.662 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-09-16T00:35:30.662 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 104857600, 2024-09-16T00:35:30.662 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-09-16T00:35:30.662 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 14333, 2024-09-16T00:35:30.662 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 3671549, 2024-09-16T00:35:30.662 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-09-16T00:35:30.663 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 14333, 2024-09-16T00:35:30.663 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-09-16T00:35:30.663 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-09-16T00:35:30.663 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 14333, 2024-09-16T00:35:30.663 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 3671549, 2024-09-16T00:35:30.663 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-09-16T00:35:30.663 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:30.663 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:30.663 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:30.663 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-16T00:35:30.663 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.663 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-objecter_bytes": { 2024-09-16T00:35:30.663 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-09-16T00:35:30.663 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 104857600, 2024-09-16T00:35:30.663 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-09-16T00:35:30.664 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-09-16T00:35:30.664 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-09-16T00:35:30.664 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-09-16T00:35:30.664 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-09-16T00:35:30.664 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 7903, 2024-09-16T00:35:30.664 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 13049647, 2024-09-16T00:35:30.664 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 7903, 2024-09-16T00:35:30.665 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 13049647, 2024-09-16T00:35:30.665 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-09-16T00:35:30.665 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:30.665 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:30.665 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:30.665 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-16T00:35:30.665 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.665 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-objecter_ops": { 2024-09-16T00:35:30.665 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-09-16T00:35:30.665 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 1024, 2024-09-16T00:35:30.665 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-09-16T00:35:30.665 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-09-16T00:35:30.665 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-09-16T00:35:30.665 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-09-16T00:35:30.665 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-09-16T00:35:30.666 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 7903, 2024-09-16T00:35:30.666 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 7903, 2024-09-16T00:35:30.666 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 7903, 2024-09-16T00:35:30.666 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 7903, 2024-09-16T00:35:30.666 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-09-16T00:35:30.666 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:30.666 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:30.666 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:30.666 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-16T00:35:30.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.666 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-write_buf_throttle": { 2024-09-16T00:35:30.666 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-09-16T00:35:30.666 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 3758096384, 2024-09-16T00:35:30.666 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-09-16T00:35:30.667 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 1212, 2024-09-16T00:35:30.667 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 132060, 2024-09-16T00:35:30.667 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-09-16T00:35:30.667 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 1212, 2024-09-16T00:35:30.667 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-09-16T00:35:30.667 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-09-16T00:35:30.667 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 34, 2024-09-16T00:35:30.667 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 132060, 2024-09-16T00:35:30.667 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-09-16T00:35:30.667 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:30.667 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:30.667 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:30.667 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-16T00:35:30.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:30.667 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-write_buf_throttle-0x55de2057c960": { 2024-09-16T00:35:30.668 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-09-16T00:35:30.668 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 3758096384, 2024-09-16T00:35:30.668 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-09-16T00:35:30.668 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 6119, 2024-09-16T00:35:30.668 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 12664526, 2024-09-16T00:35:30.668 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-09-16T00:35:30.668 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 6119, 2024-09-16T00:35:30.668 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-09-16T00:35:30.668 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-09-16T00:35:30.668 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 2447, 2024-09-16T00:35:30.668 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 12664526, 2024-09-16T00:35:30.668 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-09-16T00:35:30.668 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:30.668 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:30.668 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:30.669 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-16T00:35:30.669 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-16T00:35:30.669 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-16T00:35:31.444 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-09-16T00:35:31.444 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-09-16T00:35:31.444 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=13 2024-09-16T00:35:31.444 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-09-16T00:35:31.444 DEBUG:tasks.check_counter:Getting stats from e 2024-09-16T00:35:31.445 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph daemon mds.e perf dump 2024-09-16T00:35:31.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:31 smithi073 ceph-mon[35254]: pgmap v446: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-16T00:35:31.694 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:35:31.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:31 smithi038 ceph-mon[30945]: pgmap v446: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-16T00:35:31.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:31 smithi122 ceph-mon[35412]: pgmap v446: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2024-09-16T00:35:32.212 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-16T00:35:32.213 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:35:32.213 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 764, 2024-09-16T00:35:32.213 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 489, 2024-09-16T00:35:32.213 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 223479, 2024-09-16T00:35:32.213 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 109108, 2024-09-16T00:35:32.213 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 8, 2024-09-16T00:35:32.213 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 2, 2024-09-16T00:35:32.213 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 0.057957215, 2024-09-16T00:35:32.213 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 0.024801594, 2024-09-16T00:35:32.213 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 0.036111084, 2024-09-16T00:35:32.213 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.001651241, 2024-09-16T00:35:32.214 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:35:32.214 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 489, 2024-09-16T00:35:32.214 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.036544534, 2024-09-16T00:35:32.214 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000074733 2024-09-16T00:35:32.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.214 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:35:32.214 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 1200, 2024-09-16T00:35:32.214 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000211823, 2024-09-16T00:35:32.214 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000176 2024-09-16T00:35:32.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.214 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T00:35:32.214 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T00:35:32.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.215 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:35:32.215 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 19, 2024-09-16T00:35:32.215 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 89, 2024-09-16T00:35:32.215 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 4528, 2024-09-16T00:35:32.215 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 248476, 2024-09-16T00:35:32.215 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 10, 2024-09-16T00:35:32.215 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 6, 2024-09-16T00:35:32.215 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 0.007567267, 2024-09-16T00:35:32.215 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 0.003295045, 2024-09-16T00:35:32.215 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 0.003034048, 2024-09-16T00:35:32.215 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.000405559, 2024-09-16T00:35:32.215 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:35:32.215 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 87, 2024-09-16T00:35:32.215 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.017034865, 2024-09-16T00:35:32.216 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000195803 2024-09-16T00:35:32.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.216 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:35:32.216 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 4, 2024-09-16T00:35:32.216 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000259, 2024-09-16T00:35:32.216 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000064 2024-09-16T00:35:32.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.216 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-16T00:35:32.216 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 187632 2024-09-16T00:35:32.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.216 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:35:32.216 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 163, 2024-09-16T00:35:32.216 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 154, 2024-09-16T00:35:32.217 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 762551, 2024-09-16T00:35:32.217 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 189102, 2024-09-16T00:35:32.217 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 16, 2024-09-16T00:35:32.217 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 5, 2024-09-16T00:35:32.217 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 0.028446734, 2024-09-16T00:35:32.217 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 0.007626210, 2024-09-16T00:35:32.217 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 0.029014230, 2024-09-16T00:35:32.217 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.000132733, 2024-09-16T00:35:32.217 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:35:32.217 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 152, 2024-09-16T00:35:32.217 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.015973799, 2024-09-16T00:35:32.217 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000105090 2024-09-16T00:35:32.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.218 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:35:32.218 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 14, 2024-09-16T00:35:32.218 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000001633, 2024-09-16T00:35:32.218 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000116 2024-09-16T00:35:32.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.218 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 759296, 2024-09-16T00:35:32.218 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 63456 2024-09-16T00:35:32.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.218 INFO:teuthology.orchestra.run.smithi073.stdout: "cct": { 2024-09-16T00:35:32.218 INFO:teuthology.orchestra.run.smithi073.stdout: "total_workers": 1, 2024-09-16T00:35:32.218 INFO:teuthology.orchestra.run.smithi073.stdout: "unhealthy_workers": 0 2024-09-16T00:35:32.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.218 INFO:teuthology.orchestra.run.smithi073.stdout: "finisher-MDSRank": { 2024-09-16T00:35:32.218 INFO:teuthology.orchestra.run.smithi073.stdout: "queue_len": 0, 2024-09-16T00:35:32.218 INFO:teuthology.orchestra.run.smithi073.stdout: "complete_latency": { 2024-09-16T00:35:32.219 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 39, 2024-09-16T00:35:32.219 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.001284189, 2024-09-16T00:35:32.219 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000032927 2024-09-16T00:35:32.219 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-16T00:35:32.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.219 INFO:teuthology.orchestra.run.smithi073.stdout: "finisher-PurgeQueue": { 2024-09-16T00:35:32.219 INFO:teuthology.orchestra.run.smithi073.stdout: "queue_len": 0, 2024-09-16T00:35:32.219 INFO:teuthology.orchestra.run.smithi073.stdout: "complete_latency": { 2024-09-16T00:35:32.219 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 2, 2024-09-16T00:35:32.219 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000013262, 2024-09-16T00:35:32.219 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000006631 2024-09-16T00:35:32.219 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-16T00:35:32.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.220 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": { 2024-09-16T00:35:32.220 INFO:teuthology.orchestra.run.smithi073.stdout: "request": 0, 2024-09-16T00:35:32.220 INFO:teuthology.orchestra.run.smithi073.stdout: "reply": 0, 2024-09-16T00:35:32.220 INFO:teuthology.orchestra.run.smithi073.stdout: "reply_latency": { 2024-09-16T00:35:32.220 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:32.220 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:32.220 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:32.220 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.220 INFO:teuthology.orchestra.run.smithi073.stdout: "slow_reply": 0, 2024-09-16T00:35:32.220 INFO:teuthology.orchestra.run.smithi073.stdout: "forward": 0, 2024-09-16T00:35:32.220 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_fetch_complete": 0, 2024-09-16T00:35:32.220 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_fetch_keys": 0, 2024-09-16T00:35:32.220 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_commit": 11, 2024-09-16T00:35:32.221 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_split": 0, 2024-09-16T00:35:32.221 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_merge": 0, 2024-09-16T00:35:32.221 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes": 18, 2024-09-16T00:35:32.221 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_top": 2, 2024-09-16T00:35:32.221 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_bottom": 16, 2024-09-16T00:35:32.221 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_pin_tail": 0, 2024-09-16T00:35:32.221 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_pinned": 13, 2024-09-16T00:35:32.221 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_expired": 8, 2024-09-16T00:35:32.221 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_with_caps": 0, 2024-09-16T00:35:32.221 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": 0, 2024-09-16T00:35:32.221 INFO:teuthology.orchestra.run.smithi073.stdout: "subtrees": 3, 2024-09-16T00:35:32.221 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse": 13, 2024-09-16T00:35:32.221 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_hit": 9, 2024-09-16T00:35:32.221 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_forward": 0, 2024-09-16T00:35:32.222 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_discover": 4, 2024-09-16T00:35:32.222 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_dir_fetch": 0, 2024-09-16T00:35:32.222 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_remote_ino": 0, 2024-09-16T00:35:32.222 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_lock": 0, 2024-09-16T00:35:32.222 INFO:teuthology.orchestra.run.smithi073.stdout: "load_cent": 0, 2024-09-16T00:35:32.222 INFO:teuthology.orchestra.run.smithi073.stdout: "q": 0, 2024-09-16T00:35:32.222 INFO:teuthology.orchestra.run.smithi073.stdout: "exported": 0, 2024-09-16T00:35:32.222 INFO:teuthology.orchestra.run.smithi073.stdout: "exported_inodes": 0, 2024-09-16T00:35:32.222 INFO:teuthology.orchestra.run.smithi073.stdout: "imported": 0, 2024-09-16T00:35:32.222 INFO:teuthology.orchestra.run.smithi073.stdout: "imported_inodes": 0, 2024-09-16T00:35:32.222 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_dir_fetch": 0, 2024-09-16T00:35:32.222 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_backtrace_fetch": 0, 2024-09-16T00:35:32.223 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_peer_discover": 0, 2024-09-16T00:35:32.223 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rfiles": 4, 2024-09-16T00:35:32.223 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rbytes": 234, 2024-09-16T00:35:32.223 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rsnaps": 0, 2024-09-16T00:35:32.223 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T00:35:32.223 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_set_tag": 0, 2024-09-16T00:35:32.223 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T00:35:32.223 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_inotable_repaired": 0, 2024-09-16T00:35:32.223 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dir_inodes": 0, 2024-09-16T00:35:32.223 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T00:35:32.223 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T00:35:32.223 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_file_inodes": 0, 2024-09-16T00:35:32.223 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_inode_file_caps": 0, 2024-09-16T00:35:32.223 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T00:35:32.224 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_grant": 0, 2024-09-16T00:35:32.224 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T00:35:32.224 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T00:35:32.224 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T00:35:32.224 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_caps": 0, 2024-09-16T00:35:32.224 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_caps_dirty": 0, 2024-09-16T00:35:32.224 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_cap_release": 0, 2024-09-16T00:35:32.224 INFO:teuthology.orchestra.run.smithi073.stdout: "process_request_cap_release": 0 2024-09-16T00:35:32.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.224 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache": { 2024-09-16T00:35:32.224 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_update": 0, 2024-09-16T00:35:32.225 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_update_receipt": 18, 2024-09-16T00:35:32.225 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_try_discover": 11, 2024-09-16T00:35:32.225 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_send_discover": 12, 2024-09-16T00:35:32.225 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_handle_discover": 0, 2024-09-16T00:35:32.225 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays": 0, 2024-09-16T00:35:32.225 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays_delayed": 0, 2024-09-16T00:35:32.225 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays_enqueuing": 0, 2024-09-16T00:35:32.225 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_created": 0, 2024-09-16T00:35:32.226 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_enqueued": 0, 2024-09-16T00:35:32.226 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_reintegrated": 0, 2024-09-16T00:35:32.226 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_migrated": 0, 2024-09-16T00:35:32.226 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_processing": 0, 2024-09-16T00:35:32.226 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_enqueued": 0, 2024-09-16T00:35:32.226 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_prioritized": 0, 2024-09-16T00:35:32.226 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_started": 0, 2024-09-16T00:35:32.226 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_completed": 0, 2024-09-16T00:35:32.226 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_quiesce_path": 1, 2024-09-16T00:35:32.226 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_quiesce_inode": 5, 2024-09-16T00:35:32.226 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T00:35:32.226 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_exportdir": 0, 2024-09-16T00:35:32.226 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_flush": 0, 2024-09-16T00:35:32.226 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_fragmentdir": 0, 2024-09-16T00:35:32.227 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_fragstats": 0, 2024-09-16T00:35:32.227 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_inodestats": 0 2024-09-16T00:35:32.227 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.227 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log": { 2024-09-16T00:35:32.227 INFO:teuthology.orchestra.run.smithi073.stdout: "evlrg": 0, 2024-09-16T00:35:32.227 INFO:teuthology.orchestra.run.smithi073.stdout: "evadd": 3, 2024-09-16T00:35:32.227 INFO:teuthology.orchestra.run.smithi073.stdout: "evex": 0, 2024-09-16T00:35:32.227 INFO:teuthology.orchestra.run.smithi073.stdout: "evtrm": 0, 2024-09-16T00:35:32.227 INFO:teuthology.orchestra.run.smithi073.stdout: "ev": 3, 2024-09-16T00:35:32.227 INFO:teuthology.orchestra.run.smithi073.stdout: "evexg": 0, 2024-09-16T00:35:32.227 INFO:teuthology.orchestra.run.smithi073.stdout: "evexd": 0, 2024-09-16T00:35:32.228 INFO:teuthology.orchestra.run.smithi073.stdout: "segadd": 2, 2024-09-16T00:35:32.228 INFO:teuthology.orchestra.run.smithi073.stdout: "segex": 0, 2024-09-16T00:35:32.228 INFO:teuthology.orchestra.run.smithi073.stdout: "segtrm": 0, 2024-09-16T00:35:32.228 INFO:teuthology.orchestra.run.smithi073.stdout: "seg": 2, 2024-09-16T00:35:32.228 INFO:teuthology.orchestra.run.smithi073.stdout: "segmjr": 2, 2024-09-16T00:35:32.228 INFO:teuthology.orchestra.run.smithi073.stdout: "segexg": 0, 2024-09-16T00:35:32.228 INFO:teuthology.orchestra.run.smithi073.stdout: "segexd": 0, 2024-09-16T00:35:32.228 INFO:teuthology.orchestra.run.smithi073.stdout: "expos": 4194304, 2024-09-16T00:35:32.228 INFO:teuthology.orchestra.run.smithi073.stdout: "wrpos": 4195200, 2024-09-16T00:35:32.228 INFO:teuthology.orchestra.run.smithi073.stdout: "rdpos": 0, 2024-09-16T00:35:32.228 INFO:teuthology.orchestra.run.smithi073.stdout: "jlat": { 2024-09-16T00:35:32.229 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 2, 2024-09-16T00:35:32.229 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.003310003, 2024-09-16T00:35:32.229 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.001655001 2024-09-16T00:35:32.229 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.229 INFO:teuthology.orchestra.run.smithi073.stdout: "replayed": 0 2024-09-16T00:35:32.229 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.229 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_mem": { 2024-09-16T00:35:32.229 INFO:teuthology.orchestra.run.smithi073.stdout: "ino": 22, 2024-09-16T00:35:32.229 INFO:teuthology.orchestra.run.smithi073.stdout: "ino+": 28, 2024-09-16T00:35:32.229 INFO:teuthology.orchestra.run.smithi073.stdout: "ino-": 6, 2024-09-16T00:35:32.229 INFO:teuthology.orchestra.run.smithi073.stdout: "dir": 21, 2024-09-16T00:35:32.229 INFO:teuthology.orchestra.run.smithi073.stdout: "dir+": 27, 2024-09-16T00:35:32.229 INFO:teuthology.orchestra.run.smithi073.stdout: "dir-": 6, 2024-09-16T00:35:32.229 INFO:teuthology.orchestra.run.smithi073.stdout: "dn": 18, 2024-09-16T00:35:32.230 INFO:teuthology.orchestra.run.smithi073.stdout: "dn+": 26, 2024-09-16T00:35:32.230 INFO:teuthology.orchestra.run.smithi073.stdout: "dn-": 8, 2024-09-16T00:35:32.230 INFO:teuthology.orchestra.run.smithi073.stdout: "cap": 0, 2024-09-16T00:35:32.230 INFO:teuthology.orchestra.run.smithi073.stdout: "cap+": 0, 2024-09-16T00:35:32.230 INFO:teuthology.orchestra.run.smithi073.stdout: "cap-": 0, 2024-09-16T00:35:32.230 INFO:teuthology.orchestra.run.smithi073.stdout: "rss": 49056, 2024-09-16T00:35:32.230 INFO:teuthology.orchestra.run.smithi073.stdout: "heap": 223520 2024-09-16T00:35:32.230 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.230 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_server": { 2024-09-16T00:35:32.230 INFO:teuthology.orchestra.run.smithi073.stdout: "dispatch_client_request": 0, 2024-09-16T00:35:32.230 INFO:teuthology.orchestra.run.smithi073.stdout: "dispatch_server_request": 0, 2024-09-16T00:35:32.230 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_request": 0, 2024-09-16T00:35:32.230 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_session": 2, 2024-09-16T00:35:32.230 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_peer_request": 0, 2024-09-16T00:35:32.231 INFO:teuthology.orchestra.run.smithi073.stdout: "req_create_latency": { 2024-09-16T00:35:32.231 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:32.231 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:32.231 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:32.231 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.231 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getattr_latency": { 2024-09-16T00:35:32.231 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:32.231 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:32.231 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:32.231 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.231 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getfilelock_latency": { 2024-09-16T00:35:32.231 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:32.231 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:32.231 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:32.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.232 INFO:teuthology.orchestra.run.smithi073.stdout: "req_link_latency": { 2024-09-16T00:35:32.232 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:32.232 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:32.232 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:32.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.232 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookup_latency": { 2024-09-16T00:35:32.232 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:32.232 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:32.232 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:32.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.232 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookuphash_latency": { 2024-09-16T00:35:32.232 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:32.233 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:32.233 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:32.233 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.233 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupino_latency": { 2024-09-16T00:35:32.233 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:32.233 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:32.233 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:32.233 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.233 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupname_latency": { 2024-09-16T00:35:32.233 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:32.233 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:32.233 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:32.233 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.233 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupparent_latency": { 2024-09-16T00:35:32.234 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:32.234 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:32.234 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:32.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.234 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupsnap_latency": { 2024-09-16T00:35:32.234 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:32.234 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:32.234 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:32.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.234 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lssnap_latency": { 2024-09-16T00:35:32.234 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:32.234 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:32.234 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:32.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.235 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mkdir_latency": { 2024-09-16T00:35:32.235 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:32.235 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:32.235 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:32.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.236 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mknod_latency": { 2024-09-16T00:35:32.236 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:32.236 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:32.236 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:32.236 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.236 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mksnap_latency": { 2024-09-16T00:35:32.236 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:32.236 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:32.236 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:32.236 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.236 INFO:teuthology.orchestra.run.smithi073.stdout: "req_open_latency": { 2024-09-16T00:35:32.236 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:32.236 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:32.237 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:32.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.237 INFO:teuthology.orchestra.run.smithi073.stdout: "req_readdir_latency": { 2024-09-16T00:35:32.237 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:32.237 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:32.237 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:32.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.237 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rename_latency": { 2024-09-16T00:35:32.237 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:32.237 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:32.237 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:32.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.237 INFO:teuthology.orchestra.run.smithi073.stdout: "req_renamesnap_latency": { 2024-09-16T00:35:32.237 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:32.238 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:32.238 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:32.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.238 INFO:teuthology.orchestra.run.smithi073.stdout: "req_snapdiff_latency": { 2024-09-16T00:35:32.238 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:32.238 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:32.238 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:32.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.238 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmdir_latency": { 2024-09-16T00:35:32.238 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:32.238 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:32.238 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:32.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.238 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmsnap_latency": { 2024-09-16T00:35:32.239 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:32.239 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:32.239 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:32.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.239 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmxattr_latency": { 2024-09-16T00:35:32.239 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:32.239 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:32.239 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:32.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.239 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setattr_latency": { 2024-09-16T00:35:32.239 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:32.239 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:32.239 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:32.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.240 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setdirlayout_latency": { 2024-09-16T00:35:32.240 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:32.240 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:32.240 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:32.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.240 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setfilelock_latency": { 2024-09-16T00:35:32.240 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:32.240 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:32.240 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:32.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.240 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setlayout_latency": { 2024-09-16T00:35:32.240 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:32.241 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:32.241 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:32.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.241 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setxattr_latency": { 2024-09-16T00:35:32.241 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:32.241 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:32.241 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:32.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.241 INFO:teuthology.orchestra.run.smithi073.stdout: "req_symlink_latency": { 2024-09-16T00:35:32.241 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:32.241 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:32.241 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:32.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.241 INFO:teuthology.orchestra.run.smithi073.stdout: "req_unlink_latency": { 2024-09-16T00:35:32.242 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:32.242 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:32.242 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:32.242 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.242 INFO:teuthology.orchestra.run.smithi073.stdout: "cap_revoke_eviction": 0, 2024-09-16T00:35:32.242 INFO:teuthology.orchestra.run.smithi073.stdout: "cap_acquisition_throttle": 0, 2024-09-16T00:35:32.242 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getvxattr_latency": { 2024-09-16T00:35:32.242 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:32.242 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:32.242 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:32.242 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-16T00:35:32.242 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.242 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_sessions": { 2024-09-16T00:35:32.242 INFO:teuthology.orchestra.run.smithi073.stdout: "session_count": 1, 2024-09-16T00:35:32.243 INFO:teuthology.orchestra.run.smithi073.stdout: "session_add": 1, 2024-09-16T00:35:32.243 INFO:teuthology.orchestra.run.smithi073.stdout: "session_remove": 0, 2024-09-16T00:35:32.243 INFO:teuthology.orchestra.run.smithi073.stdout: "sessions_open": 1, 2024-09-16T00:35:32.243 INFO:teuthology.orchestra.run.smithi073.stdout: "sessions_stale": 0, 2024-09-16T00:35:32.243 INFO:teuthology.orchestra.run.smithi073.stdout: "total_load": 0, 2024-09-16T00:35:32.243 INFO:teuthology.orchestra.run.smithi073.stdout: "average_load": 0, 2024-09-16T00:35:32.243 INFO:teuthology.orchestra.run.smithi073.stdout: "avg_session_uptime": 86, 2024-09-16T00:35:32.243 INFO:teuthology.orchestra.run.smithi073.stdout: "mdthresh_evicted": 0 2024-09-16T00:35:32.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.243 INFO:teuthology.orchestra.run.smithi073.stdout: "mempool": { 2024-09-16T00:35:32.243 INFO:teuthology.orchestra.run.smithi073.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:35:32.243 INFO:teuthology.orchestra.run.smithi073.stdout: "bloom_filter_items": 0, 2024-09-16T00:35:32.243 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:35:32.244 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:35:32.244 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:35:32.244 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:35:32.244 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:35:32.244 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:35:32.244 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:35:32.244 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:35:32.244 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:35:32.244 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:35:32.244 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:35:32.244 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:35:32.244 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:35:32.244 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_items": 0, 2024-09-16T00:35:32.244 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:35:32.245 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blob_items": 0, 2024-09-16T00:35:32.245 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:35:32.245 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:35:32.245 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:35:32.245 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:35:32.245 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:35:32.245 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:35:32.245 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:35:32.245 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_txc_items": 0, 2024-09-16T00:35:32.245 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:35:32.245 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:35:32.245 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:35:32.245 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_items": 0, 2024-09-16T00:35:32.245 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_bytes": 0, 2024-09-16T00:35:32.246 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_items": 0, 2024-09-16T00:35:32.246 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:35:32.246 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:35:32.246 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:35:32.246 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:35:32.246 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_anon_bytes": 221911, 2024-09-16T00:35:32.246 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_anon_items": 68, 2024-09-16T00:35:32.246 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:35:32.246 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_meta_items": 0, 2024-09-16T00:35:32.246 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_bytes": 0, 2024-09-16T00:35:32.246 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_items": 0, 2024-09-16T00:35:32.246 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:35:32.246 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mapbl_items": 0, 2024-09-16T00:35:32.247 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:35:32.247 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pglog_items": 0, 2024-09-16T00:35:32.247 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_bytes": 8768, 2024-09-16T00:35:32.247 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_items": 165, 2024-09-16T00:35:32.247 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:35:32.247 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:35:32.247 INFO:teuthology.orchestra.run.smithi073.stdout: "pgmap_bytes": 0, 2024-09-16T00:35:32.247 INFO:teuthology.orchestra.run.smithi073.stdout: "pgmap_items": 0, 2024-09-16T00:35:32.247 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_co_bytes": 89285, 2024-09-16T00:35:32.247 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_co_items": 345, 2024-09-16T00:35:32.247 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_1_bytes": 0, 2024-09-16T00:35:32.247 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_1_items": 0, 2024-09-16T00:35:32.247 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_2_bytes": 0, 2024-09-16T00:35:32.247 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_2_items": 0 2024-09-16T00:35:32.248 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.248 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter": { 2024-09-16T00:35:32.248 INFO:teuthology.orchestra.run.smithi073.stdout: "op_active": 0, 2024-09-16T00:35:32.248 INFO:teuthology.orchestra.run.smithi073.stdout: "op_laggy": 0, 2024-09-16T00:35:32.248 INFO:teuthology.orchestra.run.smithi073.stdout: "op_send": 37, 2024-09-16T00:35:32.248 INFO:teuthology.orchestra.run.smithi073.stdout: "op_send_bytes": 11108, 2024-09-16T00:35:32.248 INFO:teuthology.orchestra.run.smithi073.stdout: "op_resend": 33, 2024-09-16T00:35:32.248 INFO:teuthology.orchestra.run.smithi073.stdout: "op_reply": 37, 2024-09-16T00:35:32.248 INFO:teuthology.orchestra.run.smithi073.stdout: "op_latency": { 2024-09-16T00:35:32.248 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 37, 2024-09-16T00:35:32.248 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.214995089, 2024-09-16T00:35:32.248 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.005810678 2024-09-16T00:35:32.248 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.249 INFO:teuthology.orchestra.run.smithi073.stdout: "op_inflight": 0, 2024-09-16T00:35:32.249 INFO:teuthology.orchestra.run.smithi073.stdout: "oplen_avg": { 2024-09-16T00:35:32.249 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 37, 2024-09-16T00:35:32.249 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 59 2024-09-16T00:35:32.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.249 INFO:teuthology.orchestra.run.smithi073.stdout: "op": 37, 2024-09-16T00:35:32.249 INFO:teuthology.orchestra.run.smithi073.stdout: "op_r": 0, 2024-09-16T00:35:32.249 INFO:teuthology.orchestra.run.smithi073.stdout: "op_w": 37, 2024-09-16T00:35:32.249 INFO:teuthology.orchestra.run.smithi073.stdout: "op_rmw": 0, 2024-09-16T00:35:32.249 INFO:teuthology.orchestra.run.smithi073.stdout: "op_pg": 0, 2024-09-16T00:35:32.249 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_stat": 0, 2024-09-16T00:35:32.249 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_create": 10, 2024-09-16T00:35:32.249 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_read": 0, 2024-09-16T00:35:32.249 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_write": 2, 2024-09-16T00:35:32.250 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_writefull": 6, 2024-09-16T00:35:32.250 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_writesame": 0, 2024-09-16T00:35:32.250 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_append": 0, 2024-09-16T00:35:32.250 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_zero": 0, 2024-09-16T00:35:32.250 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_truncate": 0, 2024-09-16T00:35:32.250 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_delete": 6, 2024-09-16T00:35:32.250 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_mapext": 0, 2024-09-16T00:35:32.250 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_sparse_read": 0, 2024-09-16T00:35:32.250 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_clonerange": 0, 2024-09-16T00:35:32.250 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_getxattr": 0, 2024-09-16T00:35:32.250 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_setxattr": 20, 2024-09-16T00:35:32.251 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_cmpxattr": 0, 2024-09-16T00:35:32.251 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_rmxattr": 0, 2024-09-16T00:35:32.251 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_resetxattrs": 0, 2024-09-16T00:35:32.251 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_call": 0, 2024-09-16T00:35:32.251 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_watch": 0, 2024-09-16T00:35:32.251 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_notify": 0, 2024-09-16T00:35:32.251 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T00:35:32.251 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_pgls": 0, 2024-09-16T00:35:32.252 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_pgls_filter": 0, 2024-09-16T00:35:32.252 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_other": 0, 2024-09-16T00:35:32.252 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_active": 0, 2024-09-16T00:35:32.252 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_send": 0, 2024-09-16T00:35:32.252 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_resend": 0, 2024-09-16T00:35:32.252 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_ping": 0, 2024-09-16T00:35:32.252 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_active": 0, 2024-09-16T00:35:32.252 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_send": 0, 2024-09-16T00:35:32.252 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_resend": 0, 2024-09-16T00:35:32.252 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_active": 0, 2024-09-16T00:35:32.252 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_send": 0, 2024-09-16T00:35:32.252 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_resend": 0, 2024-09-16T00:35:32.252 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_active": 0, 2024-09-16T00:35:32.253 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_send": 0, 2024-09-16T00:35:32.253 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_resend": 0, 2024-09-16T00:35:32.253 INFO:teuthology.orchestra.run.smithi073.stdout: "command_active": 0, 2024-09-16T00:35:32.253 INFO:teuthology.orchestra.run.smithi073.stdout: "command_send": 0, 2024-09-16T00:35:32.253 INFO:teuthology.orchestra.run.smithi073.stdout: "command_resend": 0, 2024-09-16T00:35:32.253 INFO:teuthology.orchestra.run.smithi073.stdout: "map_epoch": 92, 2024-09-16T00:35:32.253 INFO:teuthology.orchestra.run.smithi073.stdout: "map_full": 0, 2024-09-16T00:35:32.253 INFO:teuthology.orchestra.run.smithi073.stdout: "map_inc": 4, 2024-09-16T00:35:32.253 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_sessions": 8, 2024-09-16T00:35:32.253 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_session_open": 8, 2024-09-16T00:35:32.253 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_session_close": 0, 2024-09-16T00:35:32.253 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_laggy": 0, 2024-09-16T00:35:32.253 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_wr": 14, 2024-09-16T00:35:32.253 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_rd": 0, 2024-09-16T00:35:32.254 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_del": 1 2024-09-16T00:35:32.254 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.254 INFO:teuthology.orchestra.run.smithi073.stdout: "oft": { 2024-09-16T00:35:32.254 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_objs": 1, 2024-09-16T00:35:32.254 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_kv_pairs": 0, 2024-09-16T00:35:32.254 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_updates": 0, 2024-09-16T00:35:32.254 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_removes": 0 2024-09-16T00:35:32.254 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.254 INFO:teuthology.orchestra.run.smithi073.stdout: "osd-slow-ops": { 2024-09-16T00:35:32.254 INFO:teuthology.orchestra.run.smithi073.stdout: "slow_ops_count": 6 2024-09-16T00:35:32.254 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.254 INFO:teuthology.orchestra.run.smithi073.stdout: "purge_queue": { 2024-09-16T00:35:32.254 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_ops": 0, 2024-09-16T00:35:32.254 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T00:35:32.255 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing": 0, 2024-09-16T00:35:32.255 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_high_water": 0, 2024-09-16T00:35:32.255 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executed_ops": 0, 2024-09-16T00:35:32.255 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executed": 0, 2024-09-16T00:35:32.255 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_item_in_journal": 0 2024-09-16T00:35:32.255 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.255 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:35:32.255 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-09-16T00:35:32.255 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 104857600, 2024-09-16T00:35:32.255 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-09-16T00:35:32.255 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 942, 2024-09-16T00:35:32.255 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 895623, 2024-09-16T00:35:32.255 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-09-16T00:35:32.255 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 942, 2024-09-16T00:35:32.256 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-09-16T00:35:32.256 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-09-16T00:35:32.256 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 942, 2024-09-16T00:35:32.256 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 895623, 2024-09-16T00:35:32.256 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-09-16T00:35:32.256 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:32.256 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:32.256 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:32.256 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-16T00:35:32.256 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.256 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-objecter_bytes": { 2024-09-16T00:35:32.256 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-09-16T00:35:32.256 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 104857600, 2024-09-16T00:35:32.256 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-09-16T00:35:32.257 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-09-16T00:35:32.257 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-09-16T00:35:32.257 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-09-16T00:35:32.257 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-09-16T00:35:32.257 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 37, 2024-09-16T00:35:32.257 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 11108, 2024-09-16T00:35:32.257 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 37, 2024-09-16T00:35:32.257 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 11108, 2024-09-16T00:35:32.257 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-09-16T00:35:32.257 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:32.257 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:32.257 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:32.257 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-16T00:35:32.257 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.258 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-objecter_ops": { 2024-09-16T00:35:32.258 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-09-16T00:35:32.258 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 1024, 2024-09-16T00:35:32.258 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-09-16T00:35:32.258 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-09-16T00:35:32.258 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-09-16T00:35:32.258 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-09-16T00:35:32.258 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-09-16T00:35:32.258 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 37, 2024-09-16T00:35:32.258 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 37, 2024-09-16T00:35:32.258 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 37, 2024-09-16T00:35:32.258 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 37, 2024-09-16T00:35:32.258 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-09-16T00:35:32.258 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:32.259 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:32.259 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:32.259 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-16T00:35:32.259 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.259 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-write_buf_throttle": { 2024-09-16T00:35:32.259 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-09-16T00:35:32.259 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 3758096384, 2024-09-16T00:35:32.259 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-09-16T00:35:32.259 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-09-16T00:35:32.259 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-09-16T00:35:32.259 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-09-16T00:35:32.259 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-09-16T00:35:32.259 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-09-16T00:35:32.259 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-09-16T00:35:32.260 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 0, 2024-09-16T00:35:32.260 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 0, 2024-09-16T00:35:32.260 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-09-16T00:35:32.260 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:32.260 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:32.260 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:32.260 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-16T00:35:32.260 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:32.260 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-write_buf_throttle-0x559150a6c960": { 2024-09-16T00:35:32.260 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-09-16T00:35:32.260 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 3758096384, 2024-09-16T00:35:32.260 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-09-16T00:35:32.260 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 3, 2024-09-16T00:35:32.260 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 896, 2024-09-16T00:35:32.261 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-09-16T00:35:32.261 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 3, 2024-09-16T00:35:32.261 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-09-16T00:35:32.261 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-09-16T00:35:32.261 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 2, 2024-09-16T00:35:32.261 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 896, 2024-09-16T00:35:32.261 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-09-16T00:35:32.261 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:32.261 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:32.261 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:32.261 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-16T00:35:32.261 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-16T00:35:32.261 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-16T00:35:33.104 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-09-16T00:35:33.104 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-09-16T00:35:33.104 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-09-16T00:35:33.104 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=18 2024-09-16T00:35:33.104 DEBUG:tasks.check_counter:Getting stats from h 2024-09-16T00:35:33.104 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph daemon mds.h perf dump 2024-09-16T00:35:33.351 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:35:33.771 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:33 smithi073 ceph-mon[35254]: pgmap v447: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T00:35:33.874 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-16T00:35:33.874 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:35:33.874 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 48, 2024-09-16T00:35:33.874 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 52, 2024-09-16T00:35:33.874 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 10979, 2024-09-16T00:35:33.874 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 162259, 2024-09-16T00:35:33.874 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 22, 2024-09-16T00:35:33.874 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 7, 2024-09-16T00:35:33.874 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 0.011346933, 2024-09-16T00:35:33.875 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 0.001020403, 2024-09-16T00:35:33.875 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 0.007428153, 2024-09-16T00:35:33.875 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.000593992, 2024-09-16T00:35:33.875 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:35:33.875 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 48, 2024-09-16T00:35:33.875 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.024162869, 2024-09-16T00:35:33.875 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000503393 2024-09-16T00:35:33.875 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.875 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:35:33.875 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 30, 2024-09-16T00:35:33.875 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000003840, 2024-09-16T00:35:33.876 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000128 2024-09-16T00:35:33.876 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.876 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 2512, 2024-09-16T00:35:33.876 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 2176 2024-09-16T00:35:33.876 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.876 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:35:33.876 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 904, 2024-09-16T00:35:33.876 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 614, 2024-09-16T00:35:33.876 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 981447, 2024-09-16T00:35:33.876 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 200608, 2024-09-16T00:35:33.876 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 6, 2024-09-16T00:35:33.877 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 4, 2024-09-16T00:35:33.877 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 0.075106282, 2024-09-16T00:35:33.877 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 0.031843199, 2024-09-16T00:35:33.877 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 0.055435624, 2024-09-16T00:35:33.877 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.001552026, 2024-09-16T00:35:33.877 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:35:33.877 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 615, 2024-09-16T00:35:33.877 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.034374416, 2024-09-16T00:35:33.877 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000055893 2024-09-16T00:35:33.877 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.877 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:35:33.878 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 1214, 2024-09-16T00:35:33.878 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000251597, 2024-09-16T00:35:33.878 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000207 2024-09-16T00:35:33.878 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.878 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 759296, 2024-09-16T00:35:33.878 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 61728 2024-09-16T00:35:33.878 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.878 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:35:33.878 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 34, 2024-09-16T00:35:33.878 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 100, 2024-09-16T00:35:33.878 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 10182, 2024-09-16T00:35:33.878 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 228715, 2024-09-16T00:35:33.879 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 9, 2024-09-16T00:35:33.879 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 5, 2024-09-16T00:35:33.879 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 0.012153192, 2024-09-16T00:35:33.879 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 0.004296030, 2024-09-16T00:35:33.879 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 0.006574165, 2024-09-16T00:35:33.879 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.000511261, 2024-09-16T00:35:33.879 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:35:33.879 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 99, 2024-09-16T00:35:33.879 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.021647997, 2024-09-16T00:35:33.879 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000218666 2024-09-16T00:35:33.879 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.880 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:35:33.880 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 25, 2024-09-16T00:35:33.880 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000005319, 2024-09-16T00:35:33.880 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000212 2024-09-16T00:35:33.880 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.880 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-09-16T00:35:33.880 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 187728 2024-09-16T00:35:33.880 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.880 INFO:teuthology.orchestra.run.smithi073.stdout: "cct": { 2024-09-16T00:35:33.880 INFO:teuthology.orchestra.run.smithi073.stdout: "total_workers": 1, 2024-09-16T00:35:33.880 INFO:teuthology.orchestra.run.smithi073.stdout: "unhealthy_workers": 0 2024-09-16T00:35:33.880 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.881 INFO:teuthology.orchestra.run.smithi073.stdout: "finisher-MDSRank": { 2024-09-16T00:35:33.881 INFO:teuthology.orchestra.run.smithi073.stdout: "queue_len": 0, 2024-09-16T00:35:33.881 INFO:teuthology.orchestra.run.smithi073.stdout: "complete_latency": { 2024-09-16T00:35:33.881 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 56, 2024-09-16T00:35:33.881 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.001786792, 2024-09-16T00:35:33.881 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000031907 2024-09-16T00:35:33.881 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-16T00:35:33.881 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.881 INFO:teuthology.orchestra.run.smithi073.stdout: "finisher-PurgeQueue": { 2024-09-16T00:35:33.881 INFO:teuthology.orchestra.run.smithi073.stdout: "queue_len": 0, 2024-09-16T00:35:33.881 INFO:teuthology.orchestra.run.smithi073.stdout: "complete_latency": { 2024-09-16T00:35:33.882 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 2, 2024-09-16T00:35:33.882 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000015884, 2024-09-16T00:35:33.882 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000007942 2024-09-16T00:35:33.882 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-16T00:35:33.882 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.882 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": { 2024-09-16T00:35:33.882 INFO:teuthology.orchestra.run.smithi073.stdout: "request": 7, 2024-09-16T00:35:33.882 INFO:teuthology.orchestra.run.smithi073.stdout: "reply": 6, 2024-09-16T00:35:33.882 INFO:teuthology.orchestra.run.smithi073.stdout: "reply_latency": { 2024-09-16T00:35:33.882 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 6, 2024-09-16T00:35:33.882 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 11.438256027, 2024-09-16T00:35:33.883 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 1.906376004 2024-09-16T00:35:33.883 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.883 INFO:teuthology.orchestra.run.smithi073.stdout: "slow_reply": 0, 2024-09-16T00:35:33.883 INFO:teuthology.orchestra.run.smithi073.stdout: "forward": 1, 2024-09-16T00:35:33.883 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_fetch_complete": 0, 2024-09-16T00:35:33.883 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_fetch_keys": 0, 2024-09-16T00:35:33.883 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_commit": 11, 2024-09-16T00:35:33.883 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_split": 0, 2024-09-16T00:35:33.883 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_merge": 0, 2024-09-16T00:35:33.883 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes": 18, 2024-09-16T00:35:33.883 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_top": 2, 2024-09-16T00:35:33.883 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_bottom": 16, 2024-09-16T00:35:33.884 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_pin_tail": 0, 2024-09-16T00:35:33.884 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_pinned": 13, 2024-09-16T00:35:33.884 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_expired": 8, 2024-09-16T00:35:33.884 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_with_caps": 1, 2024-09-16T00:35:33.884 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": 1, 2024-09-16T00:35:33.884 INFO:teuthology.orchestra.run.smithi073.stdout: "subtrees": 3, 2024-09-16T00:35:33.884 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse": 28, 2024-09-16T00:35:33.884 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_hit": 10, 2024-09-16T00:35:33.884 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_forward": 0, 2024-09-16T00:35:33.884 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_discover": 5, 2024-09-16T00:35:33.884 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_dir_fetch": 0, 2024-09-16T00:35:33.884 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_remote_ino": 0, 2024-09-16T00:35:33.885 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_lock": 4, 2024-09-16T00:35:33.885 INFO:teuthology.orchestra.run.smithi073.stdout: "load_cent": 0, 2024-09-16T00:35:33.885 INFO:teuthology.orchestra.run.smithi073.stdout: "q": 0, 2024-09-16T00:35:33.885 INFO:teuthology.orchestra.run.smithi073.stdout: "exported": 0, 2024-09-16T00:35:33.885 INFO:teuthology.orchestra.run.smithi073.stdout: "exported_inodes": 0, 2024-09-16T00:35:33.885 INFO:teuthology.orchestra.run.smithi073.stdout: "imported": 0, 2024-09-16T00:35:33.885 INFO:teuthology.orchestra.run.smithi073.stdout: "imported_inodes": 0, 2024-09-16T00:35:33.885 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_dir_fetch": 0, 2024-09-16T00:35:33.885 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_backtrace_fetch": 0, 2024-09-16T00:35:33.885 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_peer_discover": 0, 2024-09-16T00:35:33.885 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rfiles": 4, 2024-09-16T00:35:33.886 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rbytes": 234, 2024-09-16T00:35:33.886 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rsnaps": 0, 2024-09-16T00:35:33.886 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T00:35:33.886 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_set_tag": 0, 2024-09-16T00:35:33.886 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T00:35:33.886 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_inotable_repaired": 0, 2024-09-16T00:35:33.886 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dir_inodes": 0, 2024-09-16T00:35:33.886 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T00:35:33.886 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T00:35:33.886 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_file_inodes": 0, 2024-09-16T00:35:33.886 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_inode_file_caps": 0, 2024-09-16T00:35:33.886 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_revoke": 2, 2024-09-16T00:35:33.887 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_grant": 1, 2024-09-16T00:35:33.887 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T00:35:33.887 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T00:35:33.887 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T00:35:33.887 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_caps": 2, 2024-09-16T00:35:33.887 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_caps_dirty": 0, 2024-09-16T00:35:33.887 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_cap_release": 1, 2024-09-16T00:35:33.887 INFO:teuthology.orchestra.run.smithi073.stdout: "process_request_cap_release": 1 2024-09-16T00:35:33.887 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.887 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache": { 2024-09-16T00:35:33.887 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_update": 0, 2024-09-16T00:35:33.888 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_update_receipt": 16, 2024-09-16T00:35:33.888 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_try_discover": 10, 2024-09-16T00:35:33.888 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_send_discover": 12, 2024-09-16T00:35:33.888 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_handle_discover": 0, 2024-09-16T00:35:33.888 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays": 0, 2024-09-16T00:35:33.888 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays_delayed": 0, 2024-09-16T00:35:33.888 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays_enqueuing": 0, 2024-09-16T00:35:33.888 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_created": 0, 2024-09-16T00:35:33.889 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_enqueued": 0, 2024-09-16T00:35:33.889 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_reintegrated": 0, 2024-09-16T00:35:33.889 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_migrated": 0, 2024-09-16T00:35:33.889 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_processing": 0, 2024-09-16T00:35:33.889 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_enqueued": 0, 2024-09-16T00:35:33.889 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_prioritized": 0, 2024-09-16T00:35:33.889 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_started": 0, 2024-09-16T00:35:33.889 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_completed": 0, 2024-09-16T00:35:33.889 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_quiesce_path": 1, 2024-09-16T00:35:33.889 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_quiesce_inode": 5, 2024-09-16T00:35:33.890 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T00:35:33.890 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_exportdir": 0, 2024-09-16T00:35:33.890 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_flush": 0, 2024-09-16T00:35:33.890 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_fragmentdir": 0, 2024-09-16T00:35:33.890 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_fragstats": 0, 2024-09-16T00:35:33.890 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_inodestats": 0 2024-09-16T00:35:33.890 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.890 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log": { 2024-09-16T00:35:33.890 INFO:teuthology.orchestra.run.smithi073.stdout: "evlrg": 0, 2024-09-16T00:35:33.890 INFO:teuthology.orchestra.run.smithi073.stdout: "evadd": 7, 2024-09-16T00:35:33.890 INFO:teuthology.orchestra.run.smithi073.stdout: "evex": 0, 2024-09-16T00:35:33.891 INFO:teuthology.orchestra.run.smithi073.stdout: "evtrm": 0, 2024-09-16T00:35:33.891 INFO:teuthology.orchestra.run.smithi073.stdout: "ev": 7, 2024-09-16T00:35:33.891 INFO:teuthology.orchestra.run.smithi073.stdout: "evexg": 0, 2024-09-16T00:35:33.891 INFO:teuthology.orchestra.run.smithi073.stdout: "evexd": 0, 2024-09-16T00:35:33.891 INFO:teuthology.orchestra.run.smithi073.stdout: "segadd": 2, 2024-09-16T00:35:33.891 INFO:teuthology.orchestra.run.smithi073.stdout: "segex": 0, 2024-09-16T00:35:33.891 INFO:teuthology.orchestra.run.smithi073.stdout: "segtrm": 0, 2024-09-16T00:35:33.891 INFO:teuthology.orchestra.run.smithi073.stdout: "seg": 2, 2024-09-16T00:35:33.891 INFO:teuthology.orchestra.run.smithi073.stdout: "segmjr": 2, 2024-09-16T00:35:33.891 INFO:teuthology.orchestra.run.smithi073.stdout: "segexg": 0, 2024-09-16T00:35:33.892 INFO:teuthology.orchestra.run.smithi073.stdout: "segexd": 0, 2024-09-16T00:35:33.892 INFO:teuthology.orchestra.run.smithi073.stdout: "expos": 4194304, 2024-09-16T00:35:33.892 INFO:teuthology.orchestra.run.smithi073.stdout: "wrpos": 4196268, 2024-09-16T00:35:33.892 INFO:teuthology.orchestra.run.smithi073.stdout: "rdpos": 0, 2024-09-16T00:35:33.892 INFO:teuthology.orchestra.run.smithi073.stdout: "jlat": { 2024-09-16T00:35:33.892 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 6, 2024-09-16T00:35:33.892 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.011146481, 2024-09-16T00:35:33.892 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.001857746 2024-09-16T00:35:33.892 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.892 INFO:teuthology.orchestra.run.smithi073.stdout: "replayed": 0 2024-09-16T00:35:33.892 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.893 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_mem": { 2024-09-16T00:35:33.893 INFO:teuthology.orchestra.run.smithi073.stdout: "ino": 22, 2024-09-16T00:35:33.893 INFO:teuthology.orchestra.run.smithi073.stdout: "ino+": 28, 2024-09-16T00:35:33.893 INFO:teuthology.orchestra.run.smithi073.stdout: "ino-": 6, 2024-09-16T00:35:33.893 INFO:teuthology.orchestra.run.smithi073.stdout: "dir": 21, 2024-09-16T00:35:33.893 INFO:teuthology.orchestra.run.smithi073.stdout: "dir+": 26, 2024-09-16T00:35:33.893 INFO:teuthology.orchestra.run.smithi073.stdout: "dir-": 5, 2024-09-16T00:35:33.893 INFO:teuthology.orchestra.run.smithi073.stdout: "dn": 18, 2024-09-16T00:35:33.893 INFO:teuthology.orchestra.run.smithi073.stdout: "dn+": 26, 2024-09-16T00:35:33.893 INFO:teuthology.orchestra.run.smithi073.stdout: "dn-": 8, 2024-09-16T00:35:33.894 INFO:teuthology.orchestra.run.smithi073.stdout: "cap": 1, 2024-09-16T00:35:33.894 INFO:teuthology.orchestra.run.smithi073.stdout: "cap+": 3, 2024-09-16T00:35:33.894 INFO:teuthology.orchestra.run.smithi073.stdout: "cap-": 2, 2024-09-16T00:35:33.894 INFO:teuthology.orchestra.run.smithi073.stdout: "rss": 50292, 2024-09-16T00:35:33.894 INFO:teuthology.orchestra.run.smithi073.stdout: "heap": 223520 2024-09-16T00:35:33.894 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.894 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_server": { 2024-09-16T00:35:33.894 INFO:teuthology.orchestra.run.smithi073.stdout: "dispatch_client_request": 12, 2024-09-16T00:35:33.894 INFO:teuthology.orchestra.run.smithi073.stdout: "dispatch_server_request": 0, 2024-09-16T00:35:33.894 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_request": 7, 2024-09-16T00:35:33.894 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_session": 11, 2024-09-16T00:35:33.895 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_peer_request": 0, 2024-09-16T00:35:33.895 INFO:teuthology.orchestra.run.smithi073.stdout: "req_create_latency": { 2024-09-16T00:35:33.895 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:33.895 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:33.895 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:33.895 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.895 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getattr_latency": { 2024-09-16T00:35:33.895 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 1, 2024-09-16T00:35:33.895 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000174359, 2024-09-16T00:35:33.895 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000174359 2024-09-16T00:35:33.895 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.895 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getfilelock_latency": { 2024-09-16T00:35:33.896 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:33.896 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:33.896 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:33.896 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.896 INFO:teuthology.orchestra.run.smithi073.stdout: "req_link_latency": { 2024-09-16T00:35:33.896 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:33.896 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:33.896 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:33.896 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.896 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookup_latency": { 2024-09-16T00:35:33.896 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 4, 2024-09-16T00:35:33.897 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 11.437971048, 2024-09-16T00:35:33.897 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 2.859492762 2024-09-16T00:35:33.897 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.897 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookuphash_latency": { 2024-09-16T00:35:33.897 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:33.897 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:33.897 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:33.897 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.897 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupino_latency": { 2024-09-16T00:35:33.897 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 1, 2024-09-16T00:35:33.897 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000110620, 2024-09-16T00:35:33.897 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000110620 2024-09-16T00:35:33.898 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.898 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupname_latency": { 2024-09-16T00:35:33.898 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:33.898 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:33.898 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:33.898 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.898 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupparent_latency": { 2024-09-16T00:35:33.898 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:33.898 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:33.898 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:33.898 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.898 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupsnap_latency": { 2024-09-16T00:35:33.899 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:33.899 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:33.899 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:33.899 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.899 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lssnap_latency": { 2024-09-16T00:35:33.899 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:33.899 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:33.899 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:33.899 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.899 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mkdir_latency": { 2024-09-16T00:35:33.899 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:33.900 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:33.900 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:33.900 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.901 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mknod_latency": { 2024-09-16T00:35:33.901 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:33.901 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:33.901 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:33.901 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.901 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mksnap_latency": { 2024-09-16T00:35:33.901 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:33.901 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:33.901 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:33.902 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.902 INFO:teuthology.orchestra.run.smithi073.stdout: "req_open_latency": { 2024-09-16T00:35:33.902 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:33.902 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:33.902 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:33.902 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.902 INFO:teuthology.orchestra.run.smithi073.stdout: "req_readdir_latency": { 2024-09-16T00:35:33.902 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:33.902 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:33.902 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:33.902 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.902 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rename_latency": { 2024-09-16T00:35:33.902 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:33.902 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:33.903 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:33.903 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.903 INFO:teuthology.orchestra.run.smithi073.stdout: "req_renamesnap_latency": { 2024-09-16T00:35:33.903 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:33.903 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:33.903 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:33.903 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.903 INFO:teuthology.orchestra.run.smithi073.stdout: "req_snapdiff_latency": { 2024-09-16T00:35:33.903 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:33.903 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:33.903 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:33.903 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.903 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmdir_latency": { 2024-09-16T00:35:33.904 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:33.904 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:33.904 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:33.904 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.904 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmsnap_latency": { 2024-09-16T00:35:33.904 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:33.904 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:33.904 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:33.904 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.904 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmxattr_latency": { 2024-09-16T00:35:33.904 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:33.904 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:33.904 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:33.904 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.905 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setattr_latency": { 2024-09-16T00:35:33.905 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:33.905 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:33.905 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:33.905 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.905 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setdirlayout_latency": { 2024-09-16T00:35:33.905 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:33.905 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:33.905 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:33.905 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.905 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setfilelock_latency": { 2024-09-16T00:35:33.905 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:33.905 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:33.905 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:33.906 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.906 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setlayout_latency": { 2024-09-16T00:35:33.906 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:33.906 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:33.906 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:33.906 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.906 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setxattr_latency": { 2024-09-16T00:35:33.906 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:33.906 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:33.906 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:33.906 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.906 INFO:teuthology.orchestra.run.smithi073.stdout: "req_symlink_latency": { 2024-09-16T00:35:33.906 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:33.906 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:33.907 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:33.907 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.907 INFO:teuthology.orchestra.run.smithi073.stdout: "req_unlink_latency": { 2024-09-16T00:35:33.907 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:33.907 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:33.907 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:33.907 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.907 INFO:teuthology.orchestra.run.smithi073.stdout: "cap_revoke_eviction": 0, 2024-09-16T00:35:33.907 INFO:teuthology.orchestra.run.smithi073.stdout: "cap_acquisition_throttle": 0, 2024-09-16T00:35:33.907 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getvxattr_latency": { 2024-09-16T00:35:33.907 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:33.907 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:33.907 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:33.907 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-16T00:35:33.908 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.908 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_sessions": { 2024-09-16T00:35:33.908 INFO:teuthology.orchestra.run.smithi073.stdout: "session_count": 1, 2024-09-16T00:35:33.908 INFO:teuthology.orchestra.run.smithi073.stdout: "session_add": 3, 2024-09-16T00:35:33.908 INFO:teuthology.orchestra.run.smithi073.stdout: "session_remove": 2, 2024-09-16T00:35:33.908 INFO:teuthology.orchestra.run.smithi073.stdout: "sessions_open": 1, 2024-09-16T00:35:33.908 INFO:teuthology.orchestra.run.smithi073.stdout: "sessions_stale": 0, 2024-09-16T00:35:33.908 INFO:teuthology.orchestra.run.smithi073.stdout: "total_load": 4, 2024-09-16T00:35:33.908 INFO:teuthology.orchestra.run.smithi073.stdout: "average_load": 4, 2024-09-16T00:35:33.908 INFO:teuthology.orchestra.run.smithi073.stdout: "avg_session_uptime": 146, 2024-09-16T00:35:33.908 INFO:teuthology.orchestra.run.smithi073.stdout: "mdthresh_evicted": 0 2024-09-16T00:35:33.908 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.908 INFO:teuthology.orchestra.run.smithi073.stdout: "mempool": { 2024-09-16T00:35:33.908 INFO:teuthology.orchestra.run.smithi073.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:35:33.909 INFO:teuthology.orchestra.run.smithi073.stdout: "bloom_filter_items": 0, 2024-09-16T00:35:33.909 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:35:33.909 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:35:33.909 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:35:33.909 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:35:33.909 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:35:33.909 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:35:33.909 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:35:33.909 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:35:33.909 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:35:33.909 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:35:33.909 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:35:33.909 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:35:33.909 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:35:33.910 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_items": 0, 2024-09-16T00:35:33.910 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:35:33.910 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blob_items": 0, 2024-09-16T00:35:33.910 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:35:33.910 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:35:33.910 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:35:33.910 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:35:33.910 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:35:33.910 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:35:33.910 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:35:33.910 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_txc_items": 0, 2024-09-16T00:35:33.910 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:35:33.910 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:35:33.910 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:35:33.911 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_items": 0, 2024-09-16T00:35:33.911 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_bytes": 0, 2024-09-16T00:35:33.911 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_items": 0, 2024-09-16T00:35:33.911 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:35:33.911 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:35:33.911 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:35:33.911 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:35:33.911 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_anon_bytes": 247807, 2024-09-16T00:35:33.911 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_anon_items": 89, 2024-09-16T00:35:33.911 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:35:33.911 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_meta_items": 0, 2024-09-16T00:35:33.911 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_bytes": 0, 2024-09-16T00:35:33.911 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_items": 0, 2024-09-16T00:35:33.912 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:35:33.912 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mapbl_items": 0, 2024-09-16T00:35:33.912 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:35:33.912 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pglog_items": 0, 2024-09-16T00:35:33.912 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_bytes": 8768, 2024-09-16T00:35:33.912 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_items": 165, 2024-09-16T00:35:33.912 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:35:33.912 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:35:33.912 INFO:teuthology.orchestra.run.smithi073.stdout: "pgmap_bytes": 0, 2024-09-16T00:35:33.912 INFO:teuthology.orchestra.run.smithi073.stdout: "pgmap_items": 0, 2024-09-16T00:35:33.912 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_co_bytes": 89685, 2024-09-16T00:35:33.912 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_co_items": 351, 2024-09-16T00:35:33.912 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_1_bytes": 0, 2024-09-16T00:35:33.912 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_1_items": 0, 2024-09-16T00:35:33.913 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_2_bytes": 0, 2024-09-16T00:35:33.913 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_2_items": 0 2024-09-16T00:35:33.913 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.913 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter": { 2024-09-16T00:35:33.913 INFO:teuthology.orchestra.run.smithi073.stdout: "op_active": 0, 2024-09-16T00:35:33.913 INFO:teuthology.orchestra.run.smithi073.stdout: "op_laggy": 0, 2024-09-16T00:35:33.913 INFO:teuthology.orchestra.run.smithi073.stdout: "op_send": 48, 2024-09-16T00:35:33.913 INFO:teuthology.orchestra.run.smithi073.stdout: "op_send_bytes": 13270, 2024-09-16T00:35:33.913 INFO:teuthology.orchestra.run.smithi073.stdout: "op_resend": 29, 2024-09-16T00:35:33.913 INFO:teuthology.orchestra.run.smithi073.stdout: "op_reply": 48, 2024-09-16T00:35:33.913 INFO:teuthology.orchestra.run.smithi073.stdout: "op_latency": { 2024-09-16T00:35:33.913 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 48, 2024-09-16T00:35:33.913 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.235994631, 2024-09-16T00:35:33.913 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.004916554 2024-09-16T00:35:33.913 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.914 INFO:teuthology.orchestra.run.smithi073.stdout: "op_inflight": 0, 2024-09-16T00:35:33.914 INFO:teuthology.orchestra.run.smithi073.stdout: "oplen_avg": { 2024-09-16T00:35:33.914 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 48, 2024-09-16T00:35:33.914 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 75 2024-09-16T00:35:33.914 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.914 INFO:teuthology.orchestra.run.smithi073.stdout: "op": 48, 2024-09-16T00:35:33.914 INFO:teuthology.orchestra.run.smithi073.stdout: "op_r": 0, 2024-09-16T00:35:33.914 INFO:teuthology.orchestra.run.smithi073.stdout: "op_w": 48, 2024-09-16T00:35:33.914 INFO:teuthology.orchestra.run.smithi073.stdout: "op_rmw": 0, 2024-09-16T00:35:33.914 INFO:teuthology.orchestra.run.smithi073.stdout: "op_pg": 0, 2024-09-16T00:35:33.914 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_stat": 0, 2024-09-16T00:35:33.914 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_create": 10, 2024-09-16T00:35:33.914 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_read": 0, 2024-09-16T00:35:33.915 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_write": 6, 2024-09-16T00:35:33.915 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_writefull": 9, 2024-09-16T00:35:33.915 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_writesame": 0, 2024-09-16T00:35:33.915 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_append": 0, 2024-09-16T00:35:33.915 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_zero": 0, 2024-09-16T00:35:33.915 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_truncate": 0, 2024-09-16T00:35:33.915 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_delete": 6, 2024-09-16T00:35:33.915 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_mapext": 0, 2024-09-16T00:35:33.915 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_sparse_read": 0, 2024-09-16T00:35:33.915 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_clonerange": 0, 2024-09-16T00:35:33.915 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_getxattr": 0, 2024-09-16T00:35:33.915 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_setxattr": 20, 2024-09-16T00:35:33.915 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_cmpxattr": 0, 2024-09-16T00:35:33.915 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_rmxattr": 0, 2024-09-16T00:35:33.915 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_resetxattrs": 0, 2024-09-16T00:35:33.916 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_call": 0, 2024-09-16T00:35:33.916 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_watch": 0, 2024-09-16T00:35:33.916 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_notify": 0, 2024-09-16T00:35:33.916 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T00:35:33.916 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_pgls": 0, 2024-09-16T00:35:33.916 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_pgls_filter": 0, 2024-09-16T00:35:33.916 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_other": 0, 2024-09-16T00:35:33.916 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_active": 0, 2024-09-16T00:35:33.916 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_send": 0, 2024-09-16T00:35:33.916 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_resend": 0, 2024-09-16T00:35:33.916 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_ping": 0, 2024-09-16T00:35:33.916 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_active": 0, 2024-09-16T00:35:33.916 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_send": 0, 2024-09-16T00:35:33.916 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_resend": 0, 2024-09-16T00:35:33.917 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_active": 0, 2024-09-16T00:35:33.917 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_send": 0, 2024-09-16T00:35:33.917 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_resend": 0, 2024-09-16T00:35:33.917 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_active": 0, 2024-09-16T00:35:33.917 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_send": 0, 2024-09-16T00:35:33.917 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_resend": 0, 2024-09-16T00:35:33.917 INFO:teuthology.orchestra.run.smithi073.stdout: "command_active": 0, 2024-09-16T00:35:33.917 INFO:teuthology.orchestra.run.smithi073.stdout: "command_send": 0, 2024-09-16T00:35:33.917 INFO:teuthology.orchestra.run.smithi073.stdout: "command_resend": 0, 2024-09-16T00:35:33.917 INFO:teuthology.orchestra.run.smithi073.stdout: "map_epoch": 92, 2024-09-16T00:35:33.917 INFO:teuthology.orchestra.run.smithi073.stdout: "map_full": 0, 2024-09-16T00:35:33.917 INFO:teuthology.orchestra.run.smithi073.stdout: "map_inc": 4, 2024-09-16T00:35:33.917 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_sessions": 9, 2024-09-16T00:35:33.917 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_session_open": 9, 2024-09-16T00:35:33.918 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_session_close": 0, 2024-09-16T00:35:33.918 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_laggy": 0, 2024-09-16T00:35:33.918 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_wr": 21, 2024-09-16T00:35:33.918 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_rd": 0, 2024-09-16T00:35:33.918 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_del": 3 2024-09-16T00:35:33.918 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.918 INFO:teuthology.orchestra.run.smithi073.stdout: "oft": { 2024-09-16T00:35:33.918 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_objs": 1, 2024-09-16T00:35:33.918 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_kv_pairs": 0, 2024-09-16T00:35:33.918 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_updates": 0, 2024-09-16T00:35:33.918 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_removes": 0 2024-09-16T00:35:33.918 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.918 INFO:teuthology.orchestra.run.smithi073.stdout: "osd-slow-ops": { 2024-09-16T00:35:33.918 INFO:teuthology.orchestra.run.smithi073.stdout: "slow_ops_count": 6 2024-09-16T00:35:33.918 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.919 INFO:teuthology.orchestra.run.smithi073.stdout: "purge_queue": { 2024-09-16T00:35:33.919 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_ops": 0, 2024-09-16T00:35:33.919 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T00:35:33.919 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing": 0, 2024-09-16T00:35:33.919 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_high_water": 0, 2024-09-16T00:35:33.919 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executed_ops": 0, 2024-09-16T00:35:33.919 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executed": 0, 2024-09-16T00:35:33.919 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_item_in_journal": 0 2024-09-16T00:35:33.919 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.919 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:35:33.919 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-09-16T00:35:33.919 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 104857600, 2024-09-16T00:35:33.919 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-09-16T00:35:33.919 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 982, 2024-09-16T00:35:33.919 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 903790, 2024-09-16T00:35:33.920 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-09-16T00:35:33.920 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 982, 2024-09-16T00:35:33.920 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-09-16T00:35:33.920 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-09-16T00:35:33.920 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 982, 2024-09-16T00:35:33.920 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 903790, 2024-09-16T00:35:33.920 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-09-16T00:35:33.920 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:33.920 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:33.920 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:33.920 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-16T00:35:33.920 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.920 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-objecter_bytes": { 2024-09-16T00:35:33.920 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-09-16T00:35:33.920 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 104857600, 2024-09-16T00:35:33.921 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-09-16T00:35:33.921 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-09-16T00:35:33.921 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-09-16T00:35:33.921 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-09-16T00:35:33.921 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-09-16T00:35:33.921 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 48, 2024-09-16T00:35:33.921 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 13270, 2024-09-16T00:35:33.921 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 48, 2024-09-16T00:35:33.921 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 13270, 2024-09-16T00:35:33.921 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-09-16T00:35:33.921 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:33.921 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:33.921 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:33.921 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-16T00:35:33.922 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.922 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-objecter_ops": { 2024-09-16T00:35:33.922 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-09-16T00:35:33.922 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 1024, 2024-09-16T00:35:33.922 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-09-16T00:35:33.922 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-09-16T00:35:33.922 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-09-16T00:35:33.922 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-09-16T00:35:33.922 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-09-16T00:35:33.922 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 48, 2024-09-16T00:35:33.922 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 48, 2024-09-16T00:35:33.922 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 48, 2024-09-16T00:35:33.922 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 48, 2024-09-16T00:35:33.922 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-09-16T00:35:33.923 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:33.923 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:33.923 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:33.923 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-16T00:35:33.923 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.923 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-write_buf_throttle": { 2024-09-16T00:35:33.923 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-09-16T00:35:33.923 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 3758096384, 2024-09-16T00:35:33.923 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-09-16T00:35:33.923 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-09-16T00:35:33.923 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-09-16T00:35:33.923 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-09-16T00:35:33.923 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-09-16T00:35:33.924 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-09-16T00:35:33.924 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-09-16T00:35:33.924 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 0, 2024-09-16T00:35:33.924 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 0, 2024-09-16T00:35:33.924 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-09-16T00:35:33.924 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:33.924 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:33.924 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:33.924 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-16T00:35:33.924 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:33.924 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-write_buf_throttle-0x5569c5428960": { 2024-09-16T00:35:33.924 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-09-16T00:35:33.924 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 3758096384, 2024-09-16T00:35:33.924 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-09-16T00:35:33.925 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 7, 2024-09-16T00:35:33.925 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 1964, 2024-09-16T00:35:33.925 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-09-16T00:35:33.925 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 7, 2024-09-16T00:35:33.925 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-09-16T00:35:33.925 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-09-16T00:35:33.925 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 6, 2024-09-16T00:35:33.925 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 1964, 2024-09-16T00:35:33.925 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-09-16T00:35:33.925 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:33.925 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:33.925 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:33.925 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-16T00:35:33.925 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-16T00:35:33.925 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-16T00:35:33.964 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:33 smithi038 ceph-mon[30945]: pgmap v447: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T00:35:33.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:33 smithi122 ceph-mon[35412]: pgmap v447: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T00:35:34.612 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-09-16T00:35:34.612 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-09-16T00:35:34.612 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-09-16T00:35:34.612 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=16 2024-09-16T00:35:34.612 DEBUG:tasks.check_counter:Getting stats from k 2024-09-16T00:35:34.613 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph daemon mds.k perf dump 2024-09-16T00:35:34.793 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:34 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:35:34.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:34 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:35:34.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:34 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:35:34.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:34 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:35:34.857 INFO:teuthology.orchestra.run.smithi073.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.b/config 2024-09-16T00:35:34.964 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:34 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:35:34.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:34 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:35:34.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:34 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:35:34.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:34 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:35:34.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:34 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-16T00:35:34.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:34 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-16T00:35:34.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:34 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-16T00:35:34.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:34 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' 2024-09-16T00:35:35.383 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-09-16T00:35:35.383 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:35:35.383 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 277, 2024-09-16T00:35:35.383 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 279, 2024-09-16T00:35:35.383 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 74077, 2024-09-16T00:35:35.383 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 88150, 2024-09-16T00:35:35.383 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 6, 2024-09-16T00:35:35.383 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 2, 2024-09-16T00:35:35.383 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 0.026513214, 2024-09-16T00:35:35.384 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 0.010404910, 2024-09-16T00:35:35.384 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 0.006540892, 2024-09-16T00:35:35.384 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.006269871, 2024-09-16T00:35:35.384 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:35:35.384 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 277, 2024-09-16T00:35:35.384 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.011887292, 2024-09-16T00:35:35.384 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000042914 2024-09-16T00:35:35.384 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.384 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:35:35.384 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:35.384 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:35.384 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:35.384 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.384 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T00:35:35.385 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T00:35:35.385 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.385 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:35:35.385 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 97, 2024-09-16T00:35:35.385 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 167, 2024-09-16T00:35:35.385 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 25886, 2024-09-16T00:35:35.385 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 217693, 2024-09-16T00:35:35.385 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 5, 2024-09-16T00:35:35.385 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 4, 2024-09-16T00:35:35.385 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 0.014849638, 2024-09-16T00:35:35.385 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 0.005466606, 2024-09-16T00:35:35.385 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 0.005018383, 2024-09-16T00:35:35.385 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.002849046, 2024-09-16T00:35:35.386 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:35:35.386 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 166, 2024-09-16T00:35:35.386 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.012012401, 2024-09-16T00:35:35.386 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000072363 2024-09-16T00:35:35.386 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.386 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:35:35.386 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:35.386 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:35.386 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:35.386 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.386 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-09-16T00:35:35.386 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 188432 2024-09-16T00:35:35.386 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.386 INFO:teuthology.orchestra.run.smithi073.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:35:35.387 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_messages": 385, 2024-09-16T00:35:35.387 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages": 378, 2024-09-16T00:35:35.387 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_bytes": 838416, 2024-09-16T00:35:35.387 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_bytes": 146624, 2024-09-16T00:35:35.387 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_created_connections": 3, 2024-09-16T00:35:35.387 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_active_connections": 2, 2024-09-16T00:35:35.387 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_total_time": 0.051655636, 2024-09-16T00:35:35.387 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_send_time": 0.016953232, 2024-09-16T00:35:35.387 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_recv_time": 0.026860510, 2024-09-16T00:35:35.387 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_running_fast_dispatch_time": 0.010980475, 2024-09-16T00:35:35.387 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:35:35.387 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 377, 2024-09-16T00:35:35.387 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.022876635, 2024-09-16T00:35:35.387 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000060680 2024-09-16T00:35:35.388 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.388 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:35:35.388 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:35.388 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:35.388 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:35.388 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.388 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_recv_encrypted_bytes": 747024, 2024-09-16T00:35:35.388 INFO:teuthology.orchestra.run.smithi073.stdout: "msgr_send_encrypted_bytes": 60128 2024-09-16T00:35:35.388 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.388 INFO:teuthology.orchestra.run.smithi073.stdout: "cct": { 2024-09-16T00:35:35.389 INFO:teuthology.orchestra.run.smithi073.stdout: "total_workers": 1, 2024-09-16T00:35:35.389 INFO:teuthology.orchestra.run.smithi073.stdout: "unhealthy_workers": 0 2024-09-16T00:35:35.389 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.389 INFO:teuthology.orchestra.run.smithi073.stdout: "finisher-MDSRank": { 2024-09-16T00:35:35.389 INFO:teuthology.orchestra.run.smithi073.stdout: "queue_len": 0, 2024-09-16T00:35:35.389 INFO:teuthology.orchestra.run.smithi073.stdout: "complete_latency": { 2024-09-16T00:35:35.389 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 1172, 2024-09-16T00:35:35.389 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.034363037, 2024-09-16T00:35:35.389 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000029319 2024-09-16T00:35:35.389 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-16T00:35:35.389 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.389 INFO:teuthology.orchestra.run.smithi073.stdout: "finisher-PurgeQueue": { 2024-09-16T00:35:35.389 INFO:teuthology.orchestra.run.smithi073.stdout: "queue_len": 0, 2024-09-16T00:35:35.390 INFO:teuthology.orchestra.run.smithi073.stdout: "complete_latency": { 2024-09-16T00:35:35.390 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:35.390 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:35.390 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:35.390 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-16T00:35:35.390 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.390 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": { 2024-09-16T00:35:35.390 INFO:teuthology.orchestra.run.smithi073.stdout: "request": 0, 2024-09-16T00:35:35.390 INFO:teuthology.orchestra.run.smithi073.stdout: "reply": 0, 2024-09-16T00:35:35.390 INFO:teuthology.orchestra.run.smithi073.stdout: "reply_latency": { 2024-09-16T00:35:35.390 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:35.390 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:35.390 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:35.390 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.391 INFO:teuthology.orchestra.run.smithi073.stdout: "slow_reply": 0, 2024-09-16T00:35:35.391 INFO:teuthology.orchestra.run.smithi073.stdout: "forward": 0, 2024-09-16T00:35:35.391 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_fetch_complete": 0, 2024-09-16T00:35:35.391 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_fetch_keys": 0, 2024-09-16T00:35:35.391 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_commit": 0, 2024-09-16T00:35:35.391 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_split": 0, 2024-09-16T00:35:35.391 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_merge": 0, 2024-09-16T00:35:35.391 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes": 0, 2024-09-16T00:35:35.391 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_top": 0, 2024-09-16T00:35:35.391 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_bottom": 0, 2024-09-16T00:35:35.391 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_pin_tail": 0, 2024-09-16T00:35:35.391 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_pinned": 0, 2024-09-16T00:35:35.391 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_expired": 0, 2024-09-16T00:35:35.392 INFO:teuthology.orchestra.run.smithi073.stdout: "inodes_with_caps": 0, 2024-09-16T00:35:35.392 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": 0, 2024-09-16T00:35:35.392 INFO:teuthology.orchestra.run.smithi073.stdout: "subtrees": 1, 2024-09-16T00:35:35.392 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse": 0, 2024-09-16T00:35:35.392 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_hit": 0, 2024-09-16T00:35:35.392 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_forward": 0, 2024-09-16T00:35:35.392 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_discover": 0, 2024-09-16T00:35:35.392 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_dir_fetch": 0, 2024-09-16T00:35:35.392 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_remote_ino": 0, 2024-09-16T00:35:35.392 INFO:teuthology.orchestra.run.smithi073.stdout: "traverse_lock": 0, 2024-09-16T00:35:35.392 INFO:teuthology.orchestra.run.smithi073.stdout: "load_cent": 0, 2024-09-16T00:35:35.392 INFO:teuthology.orchestra.run.smithi073.stdout: "q": 0, 2024-09-16T00:35:35.392 INFO:teuthology.orchestra.run.smithi073.stdout: "exported": 0, 2024-09-16T00:35:35.392 INFO:teuthology.orchestra.run.smithi073.stdout: "exported_inodes": 0, 2024-09-16T00:35:35.393 INFO:teuthology.orchestra.run.smithi073.stdout: "imported": 0, 2024-09-16T00:35:35.393 INFO:teuthology.orchestra.run.smithi073.stdout: "imported_inodes": 0, 2024-09-16T00:35:35.393 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_dir_fetch": 0, 2024-09-16T00:35:35.393 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_backtrace_fetch": 0, 2024-09-16T00:35:35.393 INFO:teuthology.orchestra.run.smithi073.stdout: "openino_peer_discover": 0, 2024-09-16T00:35:35.393 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rfiles": 0, 2024-09-16T00:35:35.393 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rbytes": 0, 2024-09-16T00:35:35.393 INFO:teuthology.orchestra.run.smithi073.stdout: "root_rsnaps": 0, 2024-09-16T00:35:35.393 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T00:35:35.393 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_set_tag": 0, 2024-09-16T00:35:35.393 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T00:35:35.393 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_inotable_repaired": 0, 2024-09-16T00:35:35.393 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dir_inodes": 0, 2024-09-16T00:35:35.393 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T00:35:35.394 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T00:35:35.394 INFO:teuthology.orchestra.run.smithi073.stdout: "scrub_file_inodes": 0, 2024-09-16T00:35:35.394 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_inode_file_caps": 0, 2024-09-16T00:35:35.394 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T00:35:35.394 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_grant": 0, 2024-09-16T00:35:35.394 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T00:35:35.394 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T00:35:35.394 INFO:teuthology.orchestra.run.smithi073.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T00:35:35.394 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_caps": 0, 2024-09-16T00:35:35.394 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_caps_dirty": 0, 2024-09-16T00:35:35.394 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_cap_release": 0, 2024-09-16T00:35:35.394 INFO:teuthology.orchestra.run.smithi073.stdout: "process_request_cap_release": 0 2024-09-16T00:35:35.394 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.394 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache": { 2024-09-16T00:35:35.395 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_update": 0, 2024-09-16T00:35:35.395 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_update_receipt": 0, 2024-09-16T00:35:35.395 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_try_discover": 0, 2024-09-16T00:35:35.395 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_send_discover": 0, 2024-09-16T00:35:35.395 INFO:teuthology.orchestra.run.smithi073.stdout: "dir_handle_discover": 0, 2024-09-16T00:35:35.395 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays": 0, 2024-09-16T00:35:35.395 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays_delayed": 0, 2024-09-16T00:35:35.395 INFO:teuthology.orchestra.run.smithi073.stdout: "num_strays_enqueuing": 0, 2024-09-16T00:35:35.395 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_created": 0, 2024-09-16T00:35:35.395 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_enqueued": 0, 2024-09-16T00:35:35.395 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_reintegrated": 0, 2024-09-16T00:35:35.395 INFO:teuthology.orchestra.run.smithi073.stdout: "strays_migrated": 0, 2024-09-16T00:35:35.395 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_processing": 0, 2024-09-16T00:35:35.395 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_enqueued": 0, 2024-09-16T00:35:35.396 INFO:teuthology.orchestra.run.smithi073.stdout: "num_recovering_prioritized": 0, 2024-09-16T00:35:35.396 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_started": 0, 2024-09-16T00:35:35.396 INFO:teuthology.orchestra.run.smithi073.stdout: "recovery_completed": 0, 2024-09-16T00:35:35.396 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_quiesce_path": 0, 2024-09-16T00:35:35.396 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_quiesce_inode": 0, 2024-09-16T00:35:35.396 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T00:35:35.396 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_exportdir": 0, 2024-09-16T00:35:35.396 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_flush": 0, 2024-09-16T00:35:35.396 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_fragmentdir": 0, 2024-09-16T00:35:35.396 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_fragstats": 0, 2024-09-16T00:35:35.396 INFO:teuthology.orchestra.run.smithi073.stdout: "ireq_inodestats": 0 2024-09-16T00:35:35.396 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.396 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log": { 2024-09-16T00:35:35.396 INFO:teuthology.orchestra.run.smithi073.stdout: "evlrg": 0, 2024-09-16T00:35:35.397 INFO:teuthology.orchestra.run.smithi073.stdout: "evadd": 0, 2024-09-16T00:35:35.397 INFO:teuthology.orchestra.run.smithi073.stdout: "evex": 0, 2024-09-16T00:35:35.397 INFO:teuthology.orchestra.run.smithi073.stdout: "evtrm": 0, 2024-09-16T00:35:35.397 INFO:teuthology.orchestra.run.smithi073.stdout: "ev": 3, 2024-09-16T00:35:35.397 INFO:teuthology.orchestra.run.smithi073.stdout: "evexg": 0, 2024-09-16T00:35:35.397 INFO:teuthology.orchestra.run.smithi073.stdout: "evexd": 0, 2024-09-16T00:35:35.397 INFO:teuthology.orchestra.run.smithi073.stdout: "segadd": 0, 2024-09-16T00:35:35.397 INFO:teuthology.orchestra.run.smithi073.stdout: "segex": 0, 2024-09-16T00:35:35.397 INFO:teuthology.orchestra.run.smithi073.stdout: "segtrm": 0, 2024-09-16T00:35:35.397 INFO:teuthology.orchestra.run.smithi073.stdout: "seg": 2, 2024-09-16T00:35:35.397 INFO:teuthology.orchestra.run.smithi073.stdout: "segmjr": 2, 2024-09-16T00:35:35.397 INFO:teuthology.orchestra.run.smithi073.stdout: "segexg": 0, 2024-09-16T00:35:35.397 INFO:teuthology.orchestra.run.smithi073.stdout: "segexd": 0, 2024-09-16T00:35:35.397 INFO:teuthology.orchestra.run.smithi073.stdout: "expos": 4194304, 2024-09-16T00:35:35.398 INFO:teuthology.orchestra.run.smithi073.stdout: "wrpos": 4195200, 2024-09-16T00:35:35.398 INFO:teuthology.orchestra.run.smithi073.stdout: "rdpos": 4194892, 2024-09-16T00:35:35.398 INFO:teuthology.orchestra.run.smithi073.stdout: "jlat": { 2024-09-16T00:35:35.398 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:35.398 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:35.398 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:35.399 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.399 INFO:teuthology.orchestra.run.smithi073.stdout: "replayed": 3 2024-09-16T00:35:35.399 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.399 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_mem": { 2024-09-16T00:35:35.399 INFO:teuthology.orchestra.run.smithi073.stdout: "ino": 3, 2024-09-16T00:35:35.399 INFO:teuthology.orchestra.run.smithi073.stdout: "ino+": 3, 2024-09-16T00:35:35.399 INFO:teuthology.orchestra.run.smithi073.stdout: "ino-": 0, 2024-09-16T00:35:35.399 INFO:teuthology.orchestra.run.smithi073.stdout: "dir": 1, 2024-09-16T00:35:35.399 INFO:teuthology.orchestra.run.smithi073.stdout: "dir+": 1, 2024-09-16T00:35:35.399 INFO:teuthology.orchestra.run.smithi073.stdout: "dir-": 0, 2024-09-16T00:35:35.399 INFO:teuthology.orchestra.run.smithi073.stdout: "dn": 0, 2024-09-16T00:35:35.399 INFO:teuthology.orchestra.run.smithi073.stdout: "dn+": 0, 2024-09-16T00:35:35.399 INFO:teuthology.orchestra.run.smithi073.stdout: "dn-": 0, 2024-09-16T00:35:35.400 INFO:teuthology.orchestra.run.smithi073.stdout: "cap": 0, 2024-09-16T00:35:35.400 INFO:teuthology.orchestra.run.smithi073.stdout: "cap+": 0, 2024-09-16T00:35:35.400 INFO:teuthology.orchestra.run.smithi073.stdout: "cap-": 0, 2024-09-16T00:35:35.400 INFO:teuthology.orchestra.run.smithi073.stdout: "rss": 47812, 2024-09-16T00:35:35.400 INFO:teuthology.orchestra.run.smithi073.stdout: "heap": 215328 2024-09-16T00:35:35.400 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.400 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_server": { 2024-09-16T00:35:35.400 INFO:teuthology.orchestra.run.smithi073.stdout: "dispatch_client_request": 0, 2024-09-16T00:35:35.400 INFO:teuthology.orchestra.run.smithi073.stdout: "dispatch_server_request": 0, 2024-09-16T00:35:35.400 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_request": 0, 2024-09-16T00:35:35.400 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_client_session": 0, 2024-09-16T00:35:35.400 INFO:teuthology.orchestra.run.smithi073.stdout: "handle_peer_request": 0, 2024-09-16T00:35:35.401 INFO:teuthology.orchestra.run.smithi073.stdout: "req_create_latency": { 2024-09-16T00:35:35.401 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:35.401 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:35.401 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:35.401 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.401 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getattr_latency": { 2024-09-16T00:35:35.401 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:35.401 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:35.401 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:35.401 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.401 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getfilelock_latency": { 2024-09-16T00:35:35.401 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:35.401 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:35.402 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:35.402 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.402 INFO:teuthology.orchestra.run.smithi073.stdout: "req_link_latency": { 2024-09-16T00:35:35.402 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:35.402 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:35.402 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:35.402 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.402 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookup_latency": { 2024-09-16T00:35:35.402 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:35.402 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:35.402 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:35.402 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.402 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookuphash_latency": { 2024-09-16T00:35:35.402 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:35.403 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:35.403 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:35.403 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.403 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupino_latency": { 2024-09-16T00:35:35.403 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:35.403 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:35.403 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:35.403 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.403 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupname_latency": { 2024-09-16T00:35:35.403 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:35.403 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:35.403 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:35.403 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.403 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupparent_latency": { 2024-09-16T00:35:35.403 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:35.404 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:35.404 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:35.404 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.404 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lookupsnap_latency": { 2024-09-16T00:35:35.404 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:35.404 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:35.404 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:35.404 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.404 INFO:teuthology.orchestra.run.smithi073.stdout: "req_lssnap_latency": { 2024-09-16T00:35:35.404 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:35.404 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:35.404 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:35.404 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.404 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mkdir_latency": { 2024-09-16T00:35:35.404 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:35.405 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:35.405 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:35.405 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.405 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mknod_latency": { 2024-09-16T00:35:35.405 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:35.405 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:35.405 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:35.405 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.405 INFO:teuthology.orchestra.run.smithi073.stdout: "req_mksnap_latency": { 2024-09-16T00:35:35.405 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:35.405 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:35.405 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:35.405 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.406 INFO:teuthology.orchestra.run.smithi073.stdout: "req_open_latency": { 2024-09-16T00:35:35.406 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:35.406 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:35.406 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:35.406 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.406 INFO:teuthology.orchestra.run.smithi073.stdout: "req_readdir_latency": { 2024-09-16T00:35:35.406 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:35.406 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:35.406 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:35.406 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.406 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rename_latency": { 2024-09-16T00:35:35.406 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:35.406 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:35.406 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:35.407 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.407 INFO:teuthology.orchestra.run.smithi073.stdout: "req_renamesnap_latency": { 2024-09-16T00:35:35.407 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:35.407 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:35.407 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:35.407 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.407 INFO:teuthology.orchestra.run.smithi073.stdout: "req_snapdiff_latency": { 2024-09-16T00:35:35.407 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:35.407 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:35.407 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:35.407 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.407 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmdir_latency": { 2024-09-16T00:35:35.407 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:35.407 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:35.408 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:35.408 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.408 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmsnap_latency": { 2024-09-16T00:35:35.408 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:35.408 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:35.408 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:35.408 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.408 INFO:teuthology.orchestra.run.smithi073.stdout: "req_rmxattr_latency": { 2024-09-16T00:35:35.408 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:35.408 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:35.408 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:35.408 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.408 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setattr_latency": { 2024-09-16T00:35:35.409 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:35.409 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:35.409 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:35.409 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.409 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setdirlayout_latency": { 2024-09-16T00:35:35.409 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:35.409 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:35.409 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:35.409 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.409 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setfilelock_latency": { 2024-09-16T00:35:35.409 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:35.409 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:35.409 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:35.410 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.410 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setlayout_latency": { 2024-09-16T00:35:35.410 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:35.410 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:35.410 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:35.410 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.410 INFO:teuthology.orchestra.run.smithi073.stdout: "req_setxattr_latency": { 2024-09-16T00:35:35.410 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:35.410 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:35.411 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:35.411 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.411 INFO:teuthology.orchestra.run.smithi073.stdout: "req_symlink_latency": { 2024-09-16T00:35:35.411 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:35.411 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:35.411 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:35.411 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.411 INFO:teuthology.orchestra.run.smithi073.stdout: "req_unlink_latency": { 2024-09-16T00:35:35.411 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:35.411 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:35.411 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:35.411 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.411 INFO:teuthology.orchestra.run.smithi073.stdout: "cap_revoke_eviction": 0, 2024-09-16T00:35:35.412 INFO:teuthology.orchestra.run.smithi073.stdout: "cap_acquisition_throttle": 0, 2024-09-16T00:35:35.412 INFO:teuthology.orchestra.run.smithi073.stdout: "req_getvxattr_latency": { 2024-09-16T00:35:35.412 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:35.412 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:35.412 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:35.412 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-16T00:35:35.412 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.412 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_sessions": { 2024-09-16T00:35:35.412 INFO:teuthology.orchestra.run.smithi073.stdout: "session_count": 1, 2024-09-16T00:35:35.412 INFO:teuthology.orchestra.run.smithi073.stdout: "session_add": 1, 2024-09-16T00:35:35.412 INFO:teuthology.orchestra.run.smithi073.stdout: "session_remove": 0, 2024-09-16T00:35:35.412 INFO:teuthology.orchestra.run.smithi073.stdout: "sessions_open": 1, 2024-09-16T00:35:35.412 INFO:teuthology.orchestra.run.smithi073.stdout: "sessions_stale": 0, 2024-09-16T00:35:35.413 INFO:teuthology.orchestra.run.smithi073.stdout: "total_load": 0, 2024-09-16T00:35:35.413 INFO:teuthology.orchestra.run.smithi073.stdout: "average_load": 0, 2024-09-16T00:35:35.413 INFO:teuthology.orchestra.run.smithi073.stdout: "avg_session_uptime": 1487, 2024-09-16T00:35:35.413 INFO:teuthology.orchestra.run.smithi073.stdout: "mdthresh_evicted": 0 2024-09-16T00:35:35.413 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.413 INFO:teuthology.orchestra.run.smithi073.stdout: "mempool": { 2024-09-16T00:35:35.413 INFO:teuthology.orchestra.run.smithi073.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:35:35.413 INFO:teuthology.orchestra.run.smithi073.stdout: "bloom_filter_items": 0, 2024-09-16T00:35:35.413 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:35:35.413 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:35:35.413 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:35:35.413 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:35:35.413 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:35:35.414 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:35:35.414 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:35:35.414 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:35:35.414 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:35:35.414 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:35:35.414 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:35:35.414 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:35:35.414 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:35:35.414 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_items": 0, 2024-09-16T00:35:35.414 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:35:35.414 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blob_items": 0, 2024-09-16T00:35:35.414 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:35:35.414 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:35:35.414 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:35:35.415 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:35:35.415 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:35:35.415 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:35:35.415 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:35:35.415 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_txc_items": 0, 2024-09-16T00:35:35.415 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:35:35.415 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:35:35.415 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:35:35.415 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_writing_items": 0, 2024-09-16T00:35:35.415 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_bytes": 0, 2024-09-16T00:35:35.415 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_items": 0, 2024-09-16T00:35:35.415 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:35:35.415 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:35:35.416 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:35:35.416 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:35:35.416 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_anon_bytes": 189239, 2024-09-16T00:35:35.416 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_anon_items": 48, 2024-09-16T00:35:35.416 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:35:35.416 INFO:teuthology.orchestra.run.smithi073.stdout: "buffer_meta_items": 0, 2024-09-16T00:35:35.416 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_bytes": 0, 2024-09-16T00:35:35.416 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_items": 0, 2024-09-16T00:35:35.416 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:35:35.416 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mapbl_items": 0, 2024-09-16T00:35:35.416 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:35:35.416 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pglog_items": 0, 2024-09-16T00:35:35.416 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_bytes": 8768, 2024-09-16T00:35:35.416 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_items": 165, 2024-09-16T00:35:35.417 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:35:35.417 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:35:35.417 INFO:teuthology.orchestra.run.smithi073.stdout: "pgmap_bytes": 0, 2024-09-16T00:35:35.417 INFO:teuthology.orchestra.run.smithi073.stdout: "pgmap_items": 0, 2024-09-16T00:35:35.417 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_co_bytes": 8512, 2024-09-16T00:35:35.417 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_co_items": 13, 2024-09-16T00:35:35.417 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_1_bytes": 0, 2024-09-16T00:35:35.417 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_1_items": 0, 2024-09-16T00:35:35.417 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_2_bytes": 0, 2024-09-16T00:35:35.417 INFO:teuthology.orchestra.run.smithi073.stdout: "unittest_2_items": 0 2024-09-16T00:35:35.417 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.417 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter": { 2024-09-16T00:35:35.417 INFO:teuthology.orchestra.run.smithi073.stdout: "op_active": 0, 2024-09-16T00:35:35.418 INFO:teuthology.orchestra.run.smithi073.stdout: "op_laggy": 0, 2024-09-16T00:35:35.418 INFO:teuthology.orchestra.run.smithi073.stdout: "op_send": 641, 2024-09-16T00:35:35.418 INFO:teuthology.orchestra.run.smithi073.stdout: "op_send_bytes": 21, 2024-09-16T00:35:35.418 INFO:teuthology.orchestra.run.smithi073.stdout: "op_resend": 3, 2024-09-16T00:35:35.418 INFO:teuthology.orchestra.run.smithi073.stdout: "op_reply": 641, 2024-09-16T00:35:35.418 INFO:teuthology.orchestra.run.smithi073.stdout: "op_latency": { 2024-09-16T00:35:35.418 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 641, 2024-09-16T00:35:35.418 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.307992882, 2024-09-16T00:35:35.418 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000480488 2024-09-16T00:35:35.418 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.418 INFO:teuthology.orchestra.run.smithi073.stdout: "op_inflight": 0, 2024-09-16T00:35:35.418 INFO:teuthology.orchestra.run.smithi073.stdout: "oplen_avg": { 2024-09-16T00:35:35.418 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 641, 2024-09-16T00:35:35.418 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 642 2024-09-16T00:35:35.419 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.419 INFO:teuthology.orchestra.run.smithi073.stdout: "op": 641, 2024-09-16T00:35:35.419 INFO:teuthology.orchestra.run.smithi073.stdout: "op_r": 641, 2024-09-16T00:35:35.419 INFO:teuthology.orchestra.run.smithi073.stdout: "op_w": 0, 2024-09-16T00:35:35.419 INFO:teuthology.orchestra.run.smithi073.stdout: "op_rmw": 0, 2024-09-16T00:35:35.419 INFO:teuthology.orchestra.run.smithi073.stdout: "op_pg": 0, 2024-09-16T00:35:35.419 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_stat": 362, 2024-09-16T00:35:35.419 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_create": 0, 2024-09-16T00:35:35.419 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_read": 277, 2024-09-16T00:35:35.419 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_write": 0, 2024-09-16T00:35:35.419 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_writefull": 0, 2024-09-16T00:35:35.419 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_writesame": 0, 2024-09-16T00:35:35.419 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_append": 0, 2024-09-16T00:35:35.420 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_zero": 0, 2024-09-16T00:35:35.420 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_truncate": 0, 2024-09-16T00:35:35.420 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_delete": 0, 2024-09-16T00:35:35.420 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_mapext": 0, 2024-09-16T00:35:35.420 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_sparse_read": 0, 2024-09-16T00:35:35.420 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_clonerange": 0, 2024-09-16T00:35:35.420 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_getxattr": 1, 2024-09-16T00:35:35.420 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_setxattr": 0, 2024-09-16T00:35:35.420 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_cmpxattr": 0, 2024-09-16T00:35:35.420 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_rmxattr": 0, 2024-09-16T00:35:35.420 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_resetxattrs": 0, 2024-09-16T00:35:35.420 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_call": 0, 2024-09-16T00:35:35.420 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_watch": 0, 2024-09-16T00:35:35.420 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_notify": 0, 2024-09-16T00:35:35.421 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T00:35:35.421 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_pgls": 0, 2024-09-16T00:35:35.421 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_pgls_filter": 0, 2024-09-16T00:35:35.421 INFO:teuthology.orchestra.run.smithi073.stdout: "osdop_other": 0, 2024-09-16T00:35:35.421 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_active": 0, 2024-09-16T00:35:35.421 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_send": 0, 2024-09-16T00:35:35.421 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_resend": 0, 2024-09-16T00:35:35.421 INFO:teuthology.orchestra.run.smithi073.stdout: "linger_ping": 0, 2024-09-16T00:35:35.421 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_active": 0, 2024-09-16T00:35:35.421 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_send": 0, 2024-09-16T00:35:35.421 INFO:teuthology.orchestra.run.smithi073.stdout: "poolop_resend": 0, 2024-09-16T00:35:35.421 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_active": 0, 2024-09-16T00:35:35.421 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_send": 0, 2024-09-16T00:35:35.422 INFO:teuthology.orchestra.run.smithi073.stdout: "poolstat_resend": 0, 2024-09-16T00:35:35.422 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_active": 0, 2024-09-16T00:35:35.422 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_send": 0, 2024-09-16T00:35:35.422 INFO:teuthology.orchestra.run.smithi073.stdout: "statfs_resend": 0, 2024-09-16T00:35:35.422 INFO:teuthology.orchestra.run.smithi073.stdout: "command_active": 0, 2024-09-16T00:35:35.422 INFO:teuthology.orchestra.run.smithi073.stdout: "command_send": 0, 2024-09-16T00:35:35.422 INFO:teuthology.orchestra.run.smithi073.stdout: "command_resend": 0, 2024-09-16T00:35:35.422 INFO:teuthology.orchestra.run.smithi073.stdout: "map_epoch": 92, 2024-09-16T00:35:35.422 INFO:teuthology.orchestra.run.smithi073.stdout: "map_full": 0, 2024-09-16T00:35:35.422 INFO:teuthology.orchestra.run.smithi073.stdout: "map_inc": 4, 2024-09-16T00:35:35.422 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_sessions": 5, 2024-09-16T00:35:35.422 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_session_open": 5, 2024-09-16T00:35:35.422 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_session_close": 0, 2024-09-16T00:35:35.422 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_laggy": 0, 2024-09-16T00:35:35.422 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_wr": 0, 2024-09-16T00:35:35.423 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_rd": 2, 2024-09-16T00:35:35.423 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_del": 0 2024-09-16T00:35:35.423 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.423 INFO:teuthology.orchestra.run.smithi073.stdout: "oft": { 2024-09-16T00:35:35.423 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_objs": 0, 2024-09-16T00:35:35.423 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_kv_pairs": 0, 2024-09-16T00:35:35.423 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_updates": 0, 2024-09-16T00:35:35.423 INFO:teuthology.orchestra.run.smithi073.stdout: "omap_total_removes": 0 2024-09-16T00:35:35.423 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.423 INFO:teuthology.orchestra.run.smithi073.stdout: "osd-slow-ops": { 2024-09-16T00:35:35.423 INFO:teuthology.orchestra.run.smithi073.stdout: "slow_ops_count": 0 2024-09-16T00:35:35.423 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.423 INFO:teuthology.orchestra.run.smithi073.stdout: "purge_queue": { 2024-09-16T00:35:35.424 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_ops": 0, 2024-09-16T00:35:35.424 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T00:35:35.424 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing": 0, 2024-09-16T00:35:35.424 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executing_high_water": 0, 2024-09-16T00:35:35.424 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executed_ops": 0, 2024-09-16T00:35:35.424 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_executed": 0, 2024-09-16T00:35:35.424 INFO:teuthology.orchestra.run.smithi073.stdout: "pq_item_in_journal": 0 2024-09-16T00:35:35.424 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.424 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:35:35.424 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-09-16T00:35:35.424 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 104857600, 2024-09-16T00:35:35.424 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-09-16T00:35:35.424 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 755, 2024-09-16T00:35:35.424 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 861594, 2024-09-16T00:35:35.425 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-09-16T00:35:35.425 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 755, 2024-09-16T00:35:35.425 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-09-16T00:35:35.425 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-09-16T00:35:35.425 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 755, 2024-09-16T00:35:35.425 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 861594, 2024-09-16T00:35:35.425 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-09-16T00:35:35.425 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:35.425 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:35.425 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:35.425 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-16T00:35:35.425 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.425 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-objecter_bytes": { 2024-09-16T00:35:35.425 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-09-16T00:35:35.426 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 104857600, 2024-09-16T00:35:35.426 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-09-16T00:35:35.426 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-09-16T00:35:35.426 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-09-16T00:35:35.426 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-09-16T00:35:35.426 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-09-16T00:35:35.426 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 641, 2024-09-16T00:35:35.426 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 901, 2024-09-16T00:35:35.426 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 641, 2024-09-16T00:35:35.426 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 901, 2024-09-16T00:35:35.426 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-09-16T00:35:35.426 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:35.426 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:35.427 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:35.427 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-16T00:35:35.427 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.427 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-objecter_ops": { 2024-09-16T00:35:35.427 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-09-16T00:35:35.427 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 1024, 2024-09-16T00:35:35.427 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-09-16T00:35:35.427 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-09-16T00:35:35.427 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-09-16T00:35:35.427 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-09-16T00:35:35.427 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-09-16T00:35:35.427 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 641, 2024-09-16T00:35:35.427 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 641, 2024-09-16T00:35:35.427 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 641, 2024-09-16T00:35:35.428 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 641, 2024-09-16T00:35:35.428 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-09-16T00:35:35.428 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:35.428 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:35.428 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:35.428 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-16T00:35:35.428 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.428 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-write_buf_throttle": { 2024-09-16T00:35:35.428 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-09-16T00:35:35.428 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 3758096384, 2024-09-16T00:35:35.428 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-09-16T00:35:35.428 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-09-16T00:35:35.428 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-09-16T00:35:35.429 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-09-16T00:35:35.429 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-09-16T00:35:35.429 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-09-16T00:35:35.429 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-09-16T00:35:35.429 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 0, 2024-09-16T00:35:35.429 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 0, 2024-09-16T00:35:35.429 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-09-16T00:35:35.429 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:35.429 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:35.429 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:35.429 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-16T00:35:35.430 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-09-16T00:35:35.430 INFO:teuthology.orchestra.run.smithi073.stdout: "throttle-write_buf_throttle-0x55dfc824eb40": { 2024-09-16T00:35:35.430 INFO:teuthology.orchestra.run.smithi073.stdout: "val": 0, 2024-09-16T00:35:35.430 INFO:teuthology.orchestra.run.smithi073.stdout: "max": 3758096384, 2024-09-16T00:35:35.430 INFO:teuthology.orchestra.run.smithi073.stdout: "get_started": 0, 2024-09-16T00:35:35.430 INFO:teuthology.orchestra.run.smithi073.stdout: "get": 0, 2024-09-16T00:35:35.430 INFO:teuthology.orchestra.run.smithi073.stdout: "get_sum": 0, 2024-09-16T00:35:35.430 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_fail": 0, 2024-09-16T00:35:35.430 INFO:teuthology.orchestra.run.smithi073.stdout: "get_or_fail_success": 0, 2024-09-16T00:35:35.430 INFO:teuthology.orchestra.run.smithi073.stdout: "take": 0, 2024-09-16T00:35:35.430 INFO:teuthology.orchestra.run.smithi073.stdout: "take_sum": 0, 2024-09-16T00:35:35.430 INFO:teuthology.orchestra.run.smithi073.stdout: "put": 0, 2024-09-16T00:35:35.430 INFO:teuthology.orchestra.run.smithi073.stdout: "put_sum": 0, 2024-09-16T00:35:35.431 INFO:teuthology.orchestra.run.smithi073.stdout: "wait": { 2024-09-16T00:35:35.431 INFO:teuthology.orchestra.run.smithi073.stdout: "avgcount": 0, 2024-09-16T00:35:35.431 INFO:teuthology.orchestra.run.smithi073.stdout: "sum": 0.000000000, 2024-09-16T00:35:35.431 INFO:teuthology.orchestra.run.smithi073.stdout: "avgtime": 0.000000000 2024-09-16T00:35:35.431 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-16T00:35:35.431 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-09-16T00:35:35.431 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-09-16T00:35:35.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:35 smithi073 ceph-mon[35254]: pgmap v448: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-09-16T00:35:35.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:35 smithi038 ceph-mon[30945]: pgmap v448: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-09-16T00:35:35.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:35 smithi122 ceph-mon[35412]: pgmap v448: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-09-16T00:35:36.166 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-09-16T00:35:36.166 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-09-16T00:35:36.166 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-09-16T00:35:36.166 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2024-09-16T00:35:36.166 DEBUG:tasks.check_counter:Getting stats from c 2024-09-16T00:35:36.166 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph daemon mds.c perf dump 2024-09-16T00:35:36.422 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:35:36.925 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-16T00:35:36.925 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:35:36.925 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 383, 2024-09-16T00:35:36.925 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 376, 2024-09-16T00:35:36.925 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 838000, 2024-09-16T00:35:36.925 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 144700, 2024-09-16T00:35:36.925 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 5, 2024-09-16T00:35:36.925 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 1, 2024-09-16T00:35:36.925 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.049966605, 2024-09-16T00:35:36.926 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.015782798, 2024-09-16T00:35:36.926 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.026689274, 2024-09-16T00:35:36.926 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.010192154, 2024-09-16T00:35:36.926 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:35:36.926 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 375, 2024-09-16T00:35:36.926 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.019855784, 2024-09-16T00:35:36.926 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000052948 2024-09-16T00:35:36.926 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.926 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:35:36.926 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:36.926 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:36.926 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:36.926 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.926 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 745936, 2024-09-16T00:35:36.927 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 57568 2024-09-16T00:35:36.927 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.927 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:35:36.927 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 3, 2024-09-16T00:35:36.927 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 72, 2024-09-16T00:35:36.927 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 716, 2024-09-16T00:35:36.927 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 188154, 2024-09-16T00:35:36.927 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 4, 2024-09-16T00:35:36.927 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 3, 2024-09-16T00:35:36.927 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.005378280, 2024-09-16T00:35:36.927 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.003015251, 2024-09-16T00:35:36.927 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.001471983, 2024-09-16T00:35:36.927 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.000084671, 2024-09-16T00:35:36.928 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:35:36.928 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 70, 2024-09-16T00:35:36.928 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.005047659, 2024-09-16T00:35:36.928 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000072109 2024-09-16T00:35:36.928 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.928 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:35:36.928 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:36.928 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:36.928 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:36.928 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.928 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-16T00:35:36.928 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 187456 2024-09-16T00:35:36.928 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.929 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:35:36.929 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 482, 2024-09-16T00:35:36.929 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 482, 2024-09-16T00:35:36.929 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 127653, 2024-09-16T00:35:36.929 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 152459, 2024-09-16T00:35:36.929 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 4, 2024-09-16T00:35:36.929 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 3, 2024-09-16T00:35:36.929 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.036609703, 2024-09-16T00:35:36.929 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.011787511, 2024-09-16T00:35:36.929 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.011152511, 2024-09-16T00:35:36.929 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.011329226, 2024-09-16T00:35:36.929 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:35:36.929 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 481, 2024-09-16T00:35:36.930 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.021897493, 2024-09-16T00:35:36.930 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000045524 2024-09-16T00:35:36.930 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.930 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:35:36.930 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:36.930 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:36.930 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:36.930 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.930 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-16T00:35:36.930 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 448 2024-09-16T00:35:36.930 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.930 INFO:teuthology.orchestra.run.smithi122.stdout: "cct": { 2024-09-16T00:35:36.930 INFO:teuthology.orchestra.run.smithi122.stdout: "total_workers": 1, 2024-09-16T00:35:36.930 INFO:teuthology.orchestra.run.smithi122.stdout: "unhealthy_workers": 0 2024-09-16T00:35:36.931 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.931 INFO:teuthology.orchestra.run.smithi122.stdout: "finisher-MDSRank": { 2024-09-16T00:35:36.931 INFO:teuthology.orchestra.run.smithi122.stdout: "queue_len": 0, 2024-09-16T00:35:36.931 INFO:teuthology.orchestra.run.smithi122.stdout: "complete_latency": { 2024-09-16T00:35:36.931 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 1306, 2024-09-16T00:35:36.931 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.034476617, 2024-09-16T00:35:36.931 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000026398 2024-09-16T00:35:36.931 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-16T00:35:36.931 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.931 INFO:teuthology.orchestra.run.smithi122.stdout: "finisher-PurgeQueue": { 2024-09-16T00:35:36.931 INFO:teuthology.orchestra.run.smithi122.stdout: "queue_len": 0, 2024-09-16T00:35:36.931 INFO:teuthology.orchestra.run.smithi122.stdout: "complete_latency": { 2024-09-16T00:35:36.931 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:36.931 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:36.932 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:36.932 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-16T00:35:36.932 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.932 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": { 2024-09-16T00:35:36.932 INFO:teuthology.orchestra.run.smithi122.stdout: "request": 0, 2024-09-16T00:35:36.932 INFO:teuthology.orchestra.run.smithi122.stdout: "reply": 0, 2024-09-16T00:35:36.932 INFO:teuthology.orchestra.run.smithi122.stdout: "reply_latency": { 2024-09-16T00:35:36.932 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:36.932 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:36.932 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:36.932 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.932 INFO:teuthology.orchestra.run.smithi122.stdout: "slow_reply": 0, 2024-09-16T00:35:36.932 INFO:teuthology.orchestra.run.smithi122.stdout: "forward": 0, 2024-09-16T00:35:36.933 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_fetch_complete": 0, 2024-09-16T00:35:36.933 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_fetch_keys": 0, 2024-09-16T00:35:36.933 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_commit": 0, 2024-09-16T00:35:36.933 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_split": 0, 2024-09-16T00:35:36.933 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_merge": 0, 2024-09-16T00:35:36.933 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes": 0, 2024-09-16T00:35:36.933 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_top": 0, 2024-09-16T00:35:36.933 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_bottom": 0, 2024-09-16T00:35:36.933 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_pin_tail": 0, 2024-09-16T00:35:36.933 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_pinned": 0, 2024-09-16T00:35:36.933 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_expired": 0, 2024-09-16T00:35:36.933 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_with_caps": 0, 2024-09-16T00:35:36.933 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": 0, 2024-09-16T00:35:36.933 INFO:teuthology.orchestra.run.smithi122.stdout: "subtrees": 1, 2024-09-16T00:35:36.933 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse": 0, 2024-09-16T00:35:36.934 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_hit": 0, 2024-09-16T00:35:36.934 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_forward": 0, 2024-09-16T00:35:36.934 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_discover": 0, 2024-09-16T00:35:36.934 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_dir_fetch": 0, 2024-09-16T00:35:36.934 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_remote_ino": 0, 2024-09-16T00:35:36.934 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_lock": 0, 2024-09-16T00:35:36.934 INFO:teuthology.orchestra.run.smithi122.stdout: "load_cent": 0, 2024-09-16T00:35:36.934 INFO:teuthology.orchestra.run.smithi122.stdout: "q": 0, 2024-09-16T00:35:36.934 INFO:teuthology.orchestra.run.smithi122.stdout: "exported": 0, 2024-09-16T00:35:36.934 INFO:teuthology.orchestra.run.smithi122.stdout: "exported_inodes": 0, 2024-09-16T00:35:36.934 INFO:teuthology.orchestra.run.smithi122.stdout: "imported": 0, 2024-09-16T00:35:36.934 INFO:teuthology.orchestra.run.smithi122.stdout: "imported_inodes": 0, 2024-09-16T00:35:36.934 INFO:teuthology.orchestra.run.smithi122.stdout: "openino_dir_fetch": 0, 2024-09-16T00:35:36.934 INFO:teuthology.orchestra.run.smithi122.stdout: "openino_backtrace_fetch": 0, 2024-09-16T00:35:36.935 INFO:teuthology.orchestra.run.smithi122.stdout: "openino_peer_discover": 0, 2024-09-16T00:35:36.935 INFO:teuthology.orchestra.run.smithi122.stdout: "root_rfiles": 0, 2024-09-16T00:35:36.935 INFO:teuthology.orchestra.run.smithi122.stdout: "root_rbytes": 0, 2024-09-16T00:35:36.935 INFO:teuthology.orchestra.run.smithi122.stdout: "root_rsnaps": 0, 2024-09-16T00:35:36.935 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T00:35:36.935 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_set_tag": 0, 2024-09-16T00:35:36.935 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T00:35:36.935 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_inotable_repaired": 0, 2024-09-16T00:35:36.935 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_dir_inodes": 0, 2024-09-16T00:35:36.935 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T00:35:36.935 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T00:35:36.935 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_file_inodes": 0, 2024-09-16T00:35:36.935 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_inode_file_caps": 0, 2024-09-16T00:35:36.935 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T00:35:36.936 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_grant": 0, 2024-09-16T00:35:36.936 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T00:35:36.936 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T00:35:36.936 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T00:35:36.936 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_caps": 0, 2024-09-16T00:35:36.936 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_caps_dirty": 0, 2024-09-16T00:35:36.936 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_cap_release": 0, 2024-09-16T00:35:36.936 INFO:teuthology.orchestra.run.smithi122.stdout: "process_request_cap_release": 0 2024-09-16T00:35:36.936 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.936 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_cache": { 2024-09-16T00:35:36.936 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_update": 0, 2024-09-16T00:35:36.936 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_update_receipt": 0, 2024-09-16T00:35:36.936 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_try_discover": 0, 2024-09-16T00:35:36.937 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_send_discover": 0, 2024-09-16T00:35:36.937 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_handle_discover": 0, 2024-09-16T00:35:36.937 INFO:teuthology.orchestra.run.smithi122.stdout: "num_strays": 0, 2024-09-16T00:35:36.937 INFO:teuthology.orchestra.run.smithi122.stdout: "num_strays_delayed": 0, 2024-09-16T00:35:36.937 INFO:teuthology.orchestra.run.smithi122.stdout: "num_strays_enqueuing": 0, 2024-09-16T00:35:36.937 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_created": 0, 2024-09-16T00:35:36.937 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_enqueued": 0, 2024-09-16T00:35:36.937 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_reintegrated": 0, 2024-09-16T00:35:36.937 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_migrated": 0, 2024-09-16T00:35:36.937 INFO:teuthology.orchestra.run.smithi122.stdout: "num_recovering_processing": 0, 2024-09-16T00:35:36.937 INFO:teuthology.orchestra.run.smithi122.stdout: "num_recovering_enqueued": 0, 2024-09-16T00:35:36.937 INFO:teuthology.orchestra.run.smithi122.stdout: "num_recovering_prioritized": 0, 2024-09-16T00:35:36.938 INFO:teuthology.orchestra.run.smithi122.stdout: "recovery_started": 0, 2024-09-16T00:35:36.938 INFO:teuthology.orchestra.run.smithi122.stdout: "recovery_completed": 0, 2024-09-16T00:35:36.938 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_quiesce_path": 0, 2024-09-16T00:35:36.938 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_quiesce_inode": 0, 2024-09-16T00:35:36.938 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T00:35:36.938 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_exportdir": 0, 2024-09-16T00:35:36.938 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_flush": 0, 2024-09-16T00:35:36.938 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_fragmentdir": 0, 2024-09-16T00:35:36.938 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_fragstats": 0, 2024-09-16T00:35:36.938 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_inodestats": 0 2024-09-16T00:35:36.938 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.938 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_log": { 2024-09-16T00:35:36.938 INFO:teuthology.orchestra.run.smithi122.stdout: "evlrg": 0, 2024-09-16T00:35:36.938 INFO:teuthology.orchestra.run.smithi122.stdout: "evadd": 0, 2024-09-16T00:35:36.939 INFO:teuthology.orchestra.run.smithi122.stdout: "evex": 0, 2024-09-16T00:35:36.939 INFO:teuthology.orchestra.run.smithi122.stdout: "evtrm": 0, 2024-09-16T00:35:36.939 INFO:teuthology.orchestra.run.smithi122.stdout: "ev": 7, 2024-09-16T00:35:36.939 INFO:teuthology.orchestra.run.smithi122.stdout: "evexg": 0, 2024-09-16T00:35:36.939 INFO:teuthology.orchestra.run.smithi122.stdout: "evexd": 0, 2024-09-16T00:35:36.939 INFO:teuthology.orchestra.run.smithi122.stdout: "segadd": 0, 2024-09-16T00:35:36.939 INFO:teuthology.orchestra.run.smithi122.stdout: "segex": 0, 2024-09-16T00:35:36.939 INFO:teuthology.orchestra.run.smithi122.stdout: "segtrm": 0, 2024-09-16T00:35:36.939 INFO:teuthology.orchestra.run.smithi122.stdout: "seg": 2, 2024-09-16T00:35:36.939 INFO:teuthology.orchestra.run.smithi122.stdout: "segmjr": 2, 2024-09-16T00:35:36.939 INFO:teuthology.orchestra.run.smithi122.stdout: "segexg": 0, 2024-09-16T00:35:36.939 INFO:teuthology.orchestra.run.smithi122.stdout: "segexd": 0, 2024-09-16T00:35:36.939 INFO:teuthology.orchestra.run.smithi122.stdout: "expos": 4194304, 2024-09-16T00:35:36.939 INFO:teuthology.orchestra.run.smithi122.stdout: "wrpos": 4196268, 2024-09-16T00:35:36.940 INFO:teuthology.orchestra.run.smithi122.stdout: "rdpos": 4196127, 2024-09-16T00:35:36.940 INFO:teuthology.orchestra.run.smithi122.stdout: "jlat": { 2024-09-16T00:35:36.940 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:36.940 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:36.940 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:36.940 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.940 INFO:teuthology.orchestra.run.smithi122.stdout: "replayed": 7 2024-09-16T00:35:36.940 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.940 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_mem": { 2024-09-16T00:35:36.940 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 3, 2024-09-16T00:35:36.940 INFO:teuthology.orchestra.run.smithi122.stdout: "ino+": 3, 2024-09-16T00:35:36.940 INFO:teuthology.orchestra.run.smithi122.stdout: "ino-": 0, 2024-09-16T00:35:36.940 INFO:teuthology.orchestra.run.smithi122.stdout: "dir": 1, 2024-09-16T00:35:36.941 INFO:teuthology.orchestra.run.smithi122.stdout: "dir+": 1, 2024-09-16T00:35:36.941 INFO:teuthology.orchestra.run.smithi122.stdout: "dir-": 0, 2024-09-16T00:35:36.941 INFO:teuthology.orchestra.run.smithi122.stdout: "dn": 0, 2024-09-16T00:35:36.941 INFO:teuthology.orchestra.run.smithi122.stdout: "dn+": 0, 2024-09-16T00:35:36.941 INFO:teuthology.orchestra.run.smithi122.stdout: "dn-": 0, 2024-09-16T00:35:36.941 INFO:teuthology.orchestra.run.smithi122.stdout: "cap": 0, 2024-09-16T00:35:36.941 INFO:teuthology.orchestra.run.smithi122.stdout: "cap+": 0, 2024-09-16T00:35:36.941 INFO:teuthology.orchestra.run.smithi122.stdout: "cap-": 0, 2024-09-16T00:35:36.941 INFO:teuthology.orchestra.run.smithi122.stdout: "rss": 47464, 2024-09-16T00:35:36.941 INFO:teuthology.orchestra.run.smithi122.stdout: "heap": 215328 2024-09-16T00:35:36.941 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.941 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_server": { 2024-09-16T00:35:36.941 INFO:teuthology.orchestra.run.smithi122.stdout: "dispatch_client_request": 0, 2024-09-16T00:35:36.941 INFO:teuthology.orchestra.run.smithi122.stdout: "dispatch_server_request": 0, 2024-09-16T00:35:36.942 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_request": 0, 2024-09-16T00:35:36.942 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_session": 0, 2024-09-16T00:35:36.942 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_peer_request": 0, 2024-09-16T00:35:36.942 INFO:teuthology.orchestra.run.smithi122.stdout: "req_create_latency": { 2024-09-16T00:35:36.942 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:36.942 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:36.942 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:36.942 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.942 INFO:teuthology.orchestra.run.smithi122.stdout: "req_getattr_latency": { 2024-09-16T00:35:36.942 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:36.942 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:36.942 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:36.942 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.942 INFO:teuthology.orchestra.run.smithi122.stdout: "req_getfilelock_latency": { 2024-09-16T00:35:36.943 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:36.943 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:36.943 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:36.943 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.943 INFO:teuthology.orchestra.run.smithi122.stdout: "req_link_latency": { 2024-09-16T00:35:36.943 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:36.943 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:36.943 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:36.943 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.943 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookup_latency": { 2024-09-16T00:35:36.943 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:36.943 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:36.943 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:36.943 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.943 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookuphash_latency": { 2024-09-16T00:35:36.944 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:36.944 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:36.944 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:36.944 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.944 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupino_latency": { 2024-09-16T00:35:36.944 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:36.944 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:36.944 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:36.944 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.944 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupname_latency": { 2024-09-16T00:35:36.944 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:36.944 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:36.944 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:36.944 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.945 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupparent_latency": { 2024-09-16T00:35:36.945 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:36.945 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:36.945 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:36.945 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.945 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupsnap_latency": { 2024-09-16T00:35:36.945 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:36.945 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:36.945 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:36.945 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.945 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lssnap_latency": { 2024-09-16T00:35:36.945 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:36.945 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:36.945 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:36.946 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.946 INFO:teuthology.orchestra.run.smithi122.stdout: "req_mkdir_latency": { 2024-09-16T00:35:36.946 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:36.946 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:36.946 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:36.946 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.946 INFO:teuthology.orchestra.run.smithi122.stdout: "req_mknod_latency": { 2024-09-16T00:35:36.946 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:36.946 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:36.946 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:36.946 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.946 INFO:teuthology.orchestra.run.smithi122.stdout: "req_mksnap_latency": { 2024-09-16T00:35:36.946 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:36.946 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:36.947 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:36.947 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.947 INFO:teuthology.orchestra.run.smithi122.stdout: "req_open_latency": { 2024-09-16T00:35:36.947 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:36.947 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:36.947 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:36.947 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.947 INFO:teuthology.orchestra.run.smithi122.stdout: "req_readdir_latency": { 2024-09-16T00:35:36.947 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:36.947 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:36.947 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:36.947 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.947 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rename_latency": { 2024-09-16T00:35:36.948 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:36.948 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:36.948 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:36.948 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.948 INFO:teuthology.orchestra.run.smithi122.stdout: "req_renamesnap_latency": { 2024-09-16T00:35:36.948 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:36.948 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:36.948 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:36.948 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.948 INFO:teuthology.orchestra.run.smithi122.stdout: "req_snapdiff_latency": { 2024-09-16T00:35:36.948 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:36.948 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:36.948 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:36.948 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.949 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rmdir_latency": { 2024-09-16T00:35:36.949 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:36.949 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:36.949 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:36.949 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.949 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rmsnap_latency": { 2024-09-16T00:35:36.949 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:36.949 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:36.949 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:36.949 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.949 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rmxattr_latency": { 2024-09-16T00:35:36.949 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:36.949 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:36.949 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:36.950 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.950 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setattr_latency": { 2024-09-16T00:35:36.950 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:36.950 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:36.950 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:36.950 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.950 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setdirlayout_latency": { 2024-09-16T00:35:36.950 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:36.950 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:36.950 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:36.950 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.950 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setfilelock_latency": { 2024-09-16T00:35:36.950 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:36.950 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:36.951 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:36.951 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.951 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setlayout_latency": { 2024-09-16T00:35:36.951 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:36.951 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:36.951 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:36.951 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.951 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setxattr_latency": { 2024-09-16T00:35:36.951 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:36.951 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:36.951 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:36.951 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.951 INFO:teuthology.orchestra.run.smithi122.stdout: "req_symlink_latency": { 2024-09-16T00:35:36.951 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:36.952 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:36.952 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:36.952 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.952 INFO:teuthology.orchestra.run.smithi122.stdout: "req_unlink_latency": { 2024-09-16T00:35:36.952 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:36.952 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:36.952 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:36.952 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.952 INFO:teuthology.orchestra.run.smithi122.stdout: "cap_revoke_eviction": 0, 2024-09-16T00:35:36.952 INFO:teuthology.orchestra.run.smithi122.stdout: "cap_acquisition_throttle": 0, 2024-09-16T00:35:36.952 INFO:teuthology.orchestra.run.smithi122.stdout: "req_getvxattr_latency": { 2024-09-16T00:35:36.952 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:36.952 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:36.952 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:36.953 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-16T00:35:36.953 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.953 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_sessions": { 2024-09-16T00:35:36.953 INFO:teuthology.orchestra.run.smithi122.stdout: "session_count": 1, 2024-09-16T00:35:36.953 INFO:teuthology.orchestra.run.smithi122.stdout: "session_add": 3, 2024-09-16T00:35:36.953 INFO:teuthology.orchestra.run.smithi122.stdout: "session_remove": 2, 2024-09-16T00:35:36.953 INFO:teuthology.orchestra.run.smithi122.stdout: "sessions_open": 2, 2024-09-16T00:35:36.953 INFO:teuthology.orchestra.run.smithi122.stdout: "sessions_stale": 0, 2024-09-16T00:35:36.953 INFO:teuthology.orchestra.run.smithi122.stdout: "total_load": 0, 2024-09-16T00:35:36.953 INFO:teuthology.orchestra.run.smithi122.stdout: "average_load": 0, 2024-09-16T00:35:36.953 INFO:teuthology.orchestra.run.smithi122.stdout: "avg_session_uptime": 2796, 2024-09-16T00:35:36.953 INFO:teuthology.orchestra.run.smithi122.stdout: "mdthresh_evicted": 0 2024-09-16T00:35:36.953 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.953 INFO:teuthology.orchestra.run.smithi122.stdout: "mempool": { 2024-09-16T00:35:36.954 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:35:36.954 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_items": 0, 2024-09-16T00:35:36.954 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:35:36.954 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:35:36.954 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:35:36.954 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:35:36.954 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:35:36.954 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:35:36.954 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:35:36.954 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:35:36.954 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:35:36.954 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:35:36.954 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:35:36.954 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:35:36.955 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:35:36.955 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_items": 0, 2024-09-16T00:35:36.955 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:35:36.955 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_items": 0, 2024-09-16T00:35:36.955 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:35:36.955 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:35:36.955 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:35:36.955 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:35:36.955 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:35:36.955 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:35:36.955 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:35:36.955 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_items": 0, 2024-09-16T00:35:36.955 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:35:36.955 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:35:36.956 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:35:36.956 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_items": 0, 2024-09-16T00:35:36.956 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_bytes": 0, 2024-09-16T00:35:36.956 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_items": 0, 2024-09-16T00:35:36.956 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:35:36.956 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:35:36.956 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:35:36.956 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:35:36.956 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_bytes": 181159, 2024-09-16T00:35:36.956 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_items": 44, 2024-09-16T00:35:36.956 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:35:36.956 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_items": 0, 2024-09-16T00:35:36.956 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_bytes": 0, 2024-09-16T00:35:36.956 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_items": 0, 2024-09-16T00:35:36.957 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:35:36.957 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_items": 0, 2024-09-16T00:35:36.957 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:35:36.957 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_items": 0, 2024-09-16T00:35:36.957 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_bytes": 8768, 2024-09-16T00:35:36.957 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_items": 165, 2024-09-16T00:35:36.957 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:35:36.957 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:35:36.957 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_bytes": 0, 2024-09-16T00:35:36.957 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_items": 0, 2024-09-16T00:35:36.957 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_bytes": 8512, 2024-09-16T00:35:36.957 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_items": 13, 2024-09-16T00:35:36.957 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_bytes": 0, 2024-09-16T00:35:36.957 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_items": 0, 2024-09-16T00:35:36.958 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_bytes": 0, 2024-09-16T00:35:36.958 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_items": 0 2024-09-16T00:35:36.958 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.958 INFO:teuthology.orchestra.run.smithi122.stdout: "objecter": { 2024-09-16T00:35:36.958 INFO:teuthology.orchestra.run.smithi122.stdout: "op_active": 0, 2024-09-16T00:35:36.958 INFO:teuthology.orchestra.run.smithi122.stdout: "op_laggy": 0, 2024-09-16T00:35:36.958 INFO:teuthology.orchestra.run.smithi122.stdout: "op_send": 754, 2024-09-16T00:35:36.958 INFO:teuthology.orchestra.run.smithi122.stdout: "op_send_bytes": 21, 2024-09-16T00:35:36.958 INFO:teuthology.orchestra.run.smithi122.stdout: "op_resend": 3, 2024-09-16T00:35:36.959 INFO:teuthology.orchestra.run.smithi122.stdout: "op_reply": 754, 2024-09-16T00:35:36.959 INFO:teuthology.orchestra.run.smithi122.stdout: "op_latency": { 2024-09-16T00:35:36.959 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 754, 2024-09-16T00:35:36.959 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.370991136, 2024-09-16T00:35:36.959 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000492030 2024-09-16T00:35:36.959 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.959 INFO:teuthology.orchestra.run.smithi122.stdout: "op_inflight": 0, 2024-09-16T00:35:36.959 INFO:teuthology.orchestra.run.smithi122.stdout: "oplen_avg": { 2024-09-16T00:35:36.959 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 754, 2024-09-16T00:35:36.959 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 755 2024-09-16T00:35:36.959 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.959 INFO:teuthology.orchestra.run.smithi122.stdout: "op": 754, 2024-09-16T00:35:36.959 INFO:teuthology.orchestra.run.smithi122.stdout: "op_r": 754, 2024-09-16T00:35:36.959 INFO:teuthology.orchestra.run.smithi122.stdout: "op_w": 0, 2024-09-16T00:35:36.960 INFO:teuthology.orchestra.run.smithi122.stdout: "op_rmw": 0, 2024-09-16T00:35:36.960 INFO:teuthology.orchestra.run.smithi122.stdout: "op_pg": 0, 2024-09-16T00:35:36.960 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_stat": 469, 2024-09-16T00:35:36.960 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_create": 0, 2024-09-16T00:35:36.960 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_read": 283, 2024-09-16T00:35:36.960 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_write": 0, 2024-09-16T00:35:36.960 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_writefull": 0, 2024-09-16T00:35:36.960 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_writesame": 0, 2024-09-16T00:35:36.960 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_append": 0, 2024-09-16T00:35:36.960 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_zero": 0, 2024-09-16T00:35:36.960 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_truncate": 0, 2024-09-16T00:35:36.960 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_delete": 0, 2024-09-16T00:35:36.960 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_mapext": 0, 2024-09-16T00:35:36.960 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_sparse_read": 0, 2024-09-16T00:35:36.961 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_clonerange": 0, 2024-09-16T00:35:36.961 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_getxattr": 1, 2024-09-16T00:35:36.961 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_setxattr": 0, 2024-09-16T00:35:36.961 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_cmpxattr": 0, 2024-09-16T00:35:36.961 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_rmxattr": 0, 2024-09-16T00:35:36.961 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_resetxattrs": 0, 2024-09-16T00:35:36.961 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_call": 0, 2024-09-16T00:35:36.961 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_watch": 0, 2024-09-16T00:35:36.961 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_notify": 0, 2024-09-16T00:35:36.961 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T00:35:36.961 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_pgls": 0, 2024-09-16T00:35:36.961 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_pgls_filter": 0, 2024-09-16T00:35:36.961 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_other": 0, 2024-09-16T00:35:36.961 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_active": 0, 2024-09-16T00:35:36.961 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_send": 0, 2024-09-16T00:35:36.962 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_resend": 0, 2024-09-16T00:35:36.962 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_ping": 0, 2024-09-16T00:35:36.962 INFO:teuthology.orchestra.run.smithi122.stdout: "poolop_active": 0, 2024-09-16T00:35:36.962 INFO:teuthology.orchestra.run.smithi122.stdout: "poolop_send": 0, 2024-09-16T00:35:36.962 INFO:teuthology.orchestra.run.smithi122.stdout: "poolop_resend": 0, 2024-09-16T00:35:36.962 INFO:teuthology.orchestra.run.smithi122.stdout: "poolstat_active": 0, 2024-09-16T00:35:36.962 INFO:teuthology.orchestra.run.smithi122.stdout: "poolstat_send": 0, 2024-09-16T00:35:36.962 INFO:teuthology.orchestra.run.smithi122.stdout: "poolstat_resend": 0, 2024-09-16T00:35:36.962 INFO:teuthology.orchestra.run.smithi122.stdout: "statfs_active": 0, 2024-09-16T00:35:36.962 INFO:teuthology.orchestra.run.smithi122.stdout: "statfs_send": 0, 2024-09-16T00:35:36.962 INFO:teuthology.orchestra.run.smithi122.stdout: "statfs_resend": 0, 2024-09-16T00:35:36.962 INFO:teuthology.orchestra.run.smithi122.stdout: "command_active": 0, 2024-09-16T00:35:36.962 INFO:teuthology.orchestra.run.smithi122.stdout: "command_send": 0, 2024-09-16T00:35:36.962 INFO:teuthology.orchestra.run.smithi122.stdout: "command_resend": 0, 2024-09-16T00:35:36.963 INFO:teuthology.orchestra.run.smithi122.stdout: "map_epoch": 92, 2024-09-16T00:35:36.963 INFO:teuthology.orchestra.run.smithi122.stdout: "map_full": 0, 2024-09-16T00:35:36.963 INFO:teuthology.orchestra.run.smithi122.stdout: "map_inc": 4, 2024-09-16T00:35:36.963 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_sessions": 4, 2024-09-16T00:35:36.963 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_session_open": 4, 2024-09-16T00:35:36.963 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_session_close": 0, 2024-09-16T00:35:36.963 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_laggy": 0, 2024-09-16T00:35:36.963 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_wr": 0, 2024-09-16T00:35:36.963 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_rd": 2, 2024-09-16T00:35:36.963 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_del": 0 2024-09-16T00:35:36.963 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.963 INFO:teuthology.orchestra.run.smithi122.stdout: "oft": { 2024-09-16T00:35:36.963 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_objs": 0, 2024-09-16T00:35:36.963 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_kv_pairs": 0, 2024-09-16T00:35:36.964 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_updates": 0, 2024-09-16T00:35:36.964 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_removes": 0 2024-09-16T00:35:36.964 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.964 INFO:teuthology.orchestra.run.smithi122.stdout: "osd-slow-ops": { 2024-09-16T00:35:36.964 INFO:teuthology.orchestra.run.smithi122.stdout: "slow_ops_count": 0 2024-09-16T00:35:36.964 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.964 INFO:teuthology.orchestra.run.smithi122.stdout: "purge_queue": { 2024-09-16T00:35:36.964 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing_ops": 0, 2024-09-16T00:35:36.964 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T00:35:36.964 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing": 0, 2024-09-16T00:35:36.964 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing_high_water": 0, 2024-09-16T00:35:36.964 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executed_ops": 0, 2024-09-16T00:35:36.964 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executed": 0, 2024-09-16T00:35:36.965 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_item_in_journal": 0 2024-09-16T00:35:36.965 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.965 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:35:36.965 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-09-16T00:35:36.965 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 104857600, 2024-09-16T00:35:36.965 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-09-16T00:35:36.965 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 864, 2024-09-16T00:35:36.965 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 879982, 2024-09-16T00:35:36.965 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-09-16T00:35:36.965 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 864, 2024-09-16T00:35:36.965 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-09-16T00:35:36.965 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-09-16T00:35:36.966 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 864, 2024-09-16T00:35:36.966 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 879982, 2024-09-16T00:35:36.966 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-09-16T00:35:36.966 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:36.966 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:36.966 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:36.966 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-16T00:35:36.966 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.966 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-objecter_bytes": { 2024-09-16T00:35:36.966 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-09-16T00:35:36.966 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 104857600, 2024-09-16T00:35:36.966 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-09-16T00:35:36.966 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 0, 2024-09-16T00:35:36.966 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 0, 2024-09-16T00:35:36.966 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-09-16T00:35:36.967 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 0, 2024-09-16T00:35:36.967 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 754, 2024-09-16T00:35:36.967 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 1969, 2024-09-16T00:35:36.967 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 754, 2024-09-16T00:35:36.967 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 1969, 2024-09-16T00:35:36.967 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-09-16T00:35:36.967 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:36.967 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:36.967 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:36.967 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-16T00:35:36.967 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.967 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-objecter_ops": { 2024-09-16T00:35:36.967 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-09-16T00:35:36.967 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 1024, 2024-09-16T00:35:36.968 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-09-16T00:35:36.968 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 0, 2024-09-16T00:35:36.968 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 0, 2024-09-16T00:35:36.968 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-09-16T00:35:36.968 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 0, 2024-09-16T00:35:36.968 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 754, 2024-09-16T00:35:36.968 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 754, 2024-09-16T00:35:36.968 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 754, 2024-09-16T00:35:36.968 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 754, 2024-09-16T00:35:36.968 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-09-16T00:35:36.968 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:36.968 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:36.968 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:36.968 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-16T00:35:36.969 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.969 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-write_buf_throttle": { 2024-09-16T00:35:36.969 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-09-16T00:35:36.969 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 3758096384, 2024-09-16T00:35:36.969 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-09-16T00:35:36.969 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 0, 2024-09-16T00:35:36.969 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 0, 2024-09-16T00:35:36.969 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-09-16T00:35:36.969 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 0, 2024-09-16T00:35:36.969 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-09-16T00:35:36.969 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-09-16T00:35:36.969 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 0, 2024-09-16T00:35:36.969 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 0, 2024-09-16T00:35:36.970 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-09-16T00:35:36.970 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:36.970 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:36.970 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:36.970 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-16T00:35:36.970 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:36.970 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-write_buf_throttle-0x5611bdeaeb40": { 2024-09-16T00:35:36.970 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-09-16T00:35:36.970 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 3758096384, 2024-09-16T00:35:36.970 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-09-16T00:35:36.970 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 0, 2024-09-16T00:35:36.970 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 0, 2024-09-16T00:35:36.970 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-09-16T00:35:36.970 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 0, 2024-09-16T00:35:36.971 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-09-16T00:35:36.971 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-09-16T00:35:36.971 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 0, 2024-09-16T00:35:36.971 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 0, 2024-09-16T00:35:36.971 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-09-16T00:35:36.971 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:36.971 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:36.971 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:36.971 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-16T00:35:36.971 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-16T00:35:36.971 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-16T00:35:37.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:37 smithi073 ceph-mon[35254]: pgmap v449: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:35:38.376 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:37 smithi038 ceph-mon[30945]: pgmap v449: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:35:38.376 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:37 smithi122 ceph-mon[35412]: pgmap v449: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 6 op/s 2024-09-16T00:35:38.377 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-09-16T00:35:38.377 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-09-16T00:35:38.377 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-09-16T00:35:38.377 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-09-16T00:35:38.378 DEBUG:tasks.check_counter:Getting stats from f 2024-09-16T00:35:38.378 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph daemon mds.f perf dump 2024-09-16T00:35:38.625 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:35:39.138 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-16T00:35:39.138 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:35:39.138 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 0, 2024-09-16T00:35:39.138 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 2, 2024-09-16T00:35:39.138 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 0, 2024-09-16T00:35:39.138 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 0, 2024-09-16T00:35:39.138 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 4, 2024-09-16T00:35:39.138 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 0, 2024-09-16T00:35:39.139 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.001044700, 2024-09-16T00:35:39.139 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.000000000, 2024-09-16T00:35:39.139 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.000217754, 2024-09-16T00:35:39.139 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:35:39.139 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:35:39.139 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:39.139 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:39.139 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:39.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:39.139 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:35:39.139 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:39.139 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:39.139 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:39.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:39.140 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T00:35:39.140 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T00:35:39.140 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:39.140 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:35:39.140 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 99, 2024-09-16T00:35:39.140 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 94, 2024-09-16T00:35:39.140 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 721664, 2024-09-16T00:35:39.140 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 56672, 2024-09-16T00:35:39.140 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 2, 2024-09-16T00:35:39.140 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 1, 2024-09-16T00:35:39.140 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.017887934, 2024-09-16T00:35:39.140 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.006360819, 2024-09-16T00:35:39.140 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.017868324, 2024-09-16T00:35:39.141 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:35:39.141 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:35:39.141 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 93, 2024-09-16T00:35:39.141 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.006328872, 2024-09-16T00:35:39.141 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000068052 2024-09-16T00:35:39.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:39.141 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:35:39.141 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:39.141 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:39.141 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:39.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:39.141 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 721664, 2024-09-16T00:35:39.141 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 56672 2024-09-16T00:35:39.141 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:39.142 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:35:39.142 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 5, 2024-09-16T00:35:39.142 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 73, 2024-09-16T00:35:39.142 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 3232, 2024-09-16T00:35:39.142 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 91536, 2024-09-16T00:35:39.142 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 3, 2024-09-16T00:35:39.142 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 2, 2024-09-16T00:35:39.142 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.007011708, 2024-09-16T00:35:39.142 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.003273949, 2024-09-16T00:35:39.142 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.002874732, 2024-09-16T00:35:39.142 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:35:39.142 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:35:39.142 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 72, 2024-09-16T00:35:39.143 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.008960179, 2024-09-16T00:35:39.143 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000124446 2024-09-16T00:35:39.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:39.143 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:35:39.143 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:39.143 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:39.143 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:39.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:39.143 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-09-16T00:35:39.143 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 91536 2024-09-16T00:35:39.143 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:39.143 INFO:teuthology.orchestra.run.smithi122.stdout: "cct": { 2024-09-16T00:35:39.143 INFO:teuthology.orchestra.run.smithi122.stdout: "total_workers": 0, 2024-09-16T00:35:39.143 INFO:teuthology.orchestra.run.smithi122.stdout: "unhealthy_workers": 0 2024-09-16T00:35:39.144 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:39.144 INFO:teuthology.orchestra.run.smithi122.stdout: "mempool": { 2024-09-16T00:35:39.144 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:35:39.144 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_items": 0, 2024-09-16T00:35:39.144 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:35:39.144 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:35:39.144 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:35:39.144 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:35:39.144 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:35:39.144 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:35:39.144 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:35:39.144 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:35:39.144 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:35:39.144 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:35:39.145 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:35:39.145 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:35:39.145 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:35:39.145 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_items": 0, 2024-09-16T00:35:39.145 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:35:39.145 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_items": 0, 2024-09-16T00:35:39.145 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:35:39.145 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:35:39.145 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:35:39.145 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:35:39.145 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:35:39.145 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:35:39.145 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:35:39.145 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_items": 0, 2024-09-16T00:35:39.146 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:35:39.146 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:35:39.146 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:35:39.148 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_items": 0, 2024-09-16T00:35:39.148 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_bytes": 0, 2024-09-16T00:35:39.148 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_items": 0, 2024-09-16T00:35:39.148 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:35:39.148 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:35:39.148 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:35:39.148 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:35:39.148 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_bytes": 148863, 2024-09-16T00:35:39.148 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_items": 28, 2024-09-16T00:35:39.148 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:35:39.148 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_items": 0, 2024-09-16T00:35:39.148 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_bytes": 0, 2024-09-16T00:35:39.149 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_items": 0, 2024-09-16T00:35:39.149 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:35:39.149 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_items": 0, 2024-09-16T00:35:39.149 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:35:39.149 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_items": 0, 2024-09-16T00:35:39.149 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_bytes": 0, 2024-09-16T00:35:39.149 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_items": 0, 2024-09-16T00:35:39.149 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:35:39.149 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:35:39.149 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_bytes": 0, 2024-09-16T00:35:39.149 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_items": 0, 2024-09-16T00:35:39.149 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_bytes": 928, 2024-09-16T00:35:39.149 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_items": 2, 2024-09-16T00:35:39.149 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_bytes": 0, 2024-09-16T00:35:39.150 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_items": 0, 2024-09-16T00:35:39.150 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_bytes": 0, 2024-09-16T00:35:39.150 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_items": 0 2024-09-16T00:35:39.150 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:39.150 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:35:39.150 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-09-16T00:35:39.150 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 104857600, 2024-09-16T00:35:39.150 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-09-16T00:35:39.150 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 100, 2024-09-16T00:35:39.150 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 707703, 2024-09-16T00:35:39.150 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-09-16T00:35:39.150 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 100, 2024-09-16T00:35:39.150 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-09-16T00:35:39.150 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-09-16T00:35:39.151 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 100, 2024-09-16T00:35:39.151 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 707703, 2024-09-16T00:35:39.151 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-09-16T00:35:39.151 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:39.151 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:39.151 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:39.151 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-16T00:35:39.151 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-16T00:35:39.151 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-16T00:35:39.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:39 smithi073 ceph-mon[35254]: pgmap v450: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:35:39.824 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:39 smithi122 ceph-mon[35412]: pgmap v450: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:35:39.850 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.f 2024-09-16T00:35:39.850 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.f 2024-09-16T00:35:39.850 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.f 2024-09-16T00:35:39.850 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.f 2024-09-16T00:35:39.850 DEBUG:tasks.check_counter:Getting stats from i 2024-09-16T00:35:39.851 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph daemon mds.i perf dump 2024-09-16T00:35:39.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:39 smithi038 ceph-mon[30945]: pgmap v450: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2024-09-16T00:35:40.093 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:35:40.615 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-16T00:35:40.615 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:35:40.615 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 572, 2024-09-16T00:35:40.615 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 572, 2024-09-16T00:35:40.615 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 4366319, 2024-09-16T00:35:40.615 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 181082, 2024-09-16T00:35:40.615 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 6, 2024-09-16T00:35:40.615 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 2, 2024-09-16T00:35:40.615 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.056573302, 2024-09-16T00:35:40.616 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.016251582, 2024-09-16T00:35:40.616 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.018020446, 2024-09-16T00:35:40.616 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.016840712, 2024-09-16T00:35:40.616 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:35:40.616 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 571, 2024-09-16T00:35:40.616 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.024789108, 2024-09-16T00:35:40.616 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000043413 2024-09-16T00:35:40.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.616 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:35:40.616 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:40.616 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:40.616 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:40.616 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.617 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-09-16T00:35:40.617 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 448 2024-09-16T00:35:40.617 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.617 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:35:40.617 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 95, 2024-09-16T00:35:40.617 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 164, 2024-09-16T00:35:40.617 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 4301894, 2024-09-16T00:35:40.617 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 219030, 2024-09-16T00:35:40.617 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 5, 2024-09-16T00:35:40.617 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 4, 2024-09-16T00:35:40.617 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.014069515, 2024-09-16T00:35:40.617 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.005185315, 2024-09-16T00:35:40.617 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.005139760, 2024-09-16T00:35:40.617 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.002435707, 2024-09-16T00:35:40.618 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:35:40.618 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 162, 2024-09-16T00:35:40.618 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.011893740, 2024-09-16T00:35:40.618 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000073418 2024-09-16T00:35:40.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.618 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:35:40.618 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:40.618 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:40.618 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:40.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.618 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-16T00:35:40.618 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 189072 2024-09-16T00:35:40.618 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.619 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:35:40.619 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 348, 2024-09-16T00:35:40.619 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 341, 2024-09-16T00:35:40.619 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 4999243, 2024-09-16T00:35:40.619 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 133413, 2024-09-16T00:35:40.619 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 3, 2024-09-16T00:35:40.619 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 2, 2024-09-16T00:35:40.619 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.040935141, 2024-09-16T00:35:40.619 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.014214817, 2024-09-16T00:35:40.619 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.023110659, 2024-09-16T00:35:40.619 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.007057832, 2024-09-16T00:35:40.619 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:35:40.619 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 340, 2024-09-16T00:35:40.620 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.014131331, 2024-09-16T00:35:40.620 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000041562 2024-09-16T00:35:40.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.620 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:35:40.620 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:40.620 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:40.620 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:40.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.620 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 746096, 2024-09-16T00:35:40.620 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 57728 2024-09-16T00:35:40.620 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.620 INFO:teuthology.orchestra.run.smithi122.stdout: "cct": { 2024-09-16T00:35:40.620 INFO:teuthology.orchestra.run.smithi122.stdout: "total_workers": 1, 2024-09-16T00:35:40.621 INFO:teuthology.orchestra.run.smithi122.stdout: "unhealthy_workers": 0 2024-09-16T00:35:40.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.621 INFO:teuthology.orchestra.run.smithi122.stdout: "finisher-MDSRank": { 2024-09-16T00:35:40.621 INFO:teuthology.orchestra.run.smithi122.stdout: "queue_len": 0, 2024-09-16T00:35:40.621 INFO:teuthology.orchestra.run.smithi122.stdout: "complete_latency": { 2024-09-16T00:35:40.621 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 1480, 2024-09-16T00:35:40.621 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.038563655, 2024-09-16T00:35:40.621 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000026056 2024-09-16T00:35:40.621 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-16T00:35:40.621 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.621 INFO:teuthology.orchestra.run.smithi122.stdout: "finisher-PurgeQueue": { 2024-09-16T00:35:40.621 INFO:teuthology.orchestra.run.smithi122.stdout: "queue_len": 0, 2024-09-16T00:35:40.621 INFO:teuthology.orchestra.run.smithi122.stdout: "complete_latency": { 2024-09-16T00:35:40.622 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:40.622 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:40.622 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:40.622 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-16T00:35:40.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.622 INFO:teuthology.orchestra.run.smithi122.stdout: "mds": { 2024-09-16T00:35:40.622 INFO:teuthology.orchestra.run.smithi122.stdout: "request": 0, 2024-09-16T00:35:40.622 INFO:teuthology.orchestra.run.smithi122.stdout: "reply": 0, 2024-09-16T00:35:40.622 INFO:teuthology.orchestra.run.smithi122.stdout: "reply_latency": { 2024-09-16T00:35:40.622 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:40.622 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:40.622 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:40.622 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.623 INFO:teuthology.orchestra.run.smithi122.stdout: "slow_reply": 0, 2024-09-16T00:35:40.623 INFO:teuthology.orchestra.run.smithi122.stdout: "forward": 0, 2024-09-16T00:35:40.623 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_fetch_complete": 0, 2024-09-16T00:35:40.623 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_fetch_keys": 0, 2024-09-16T00:35:40.623 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_commit": 0, 2024-09-16T00:35:40.623 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_split": 0, 2024-09-16T00:35:40.623 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_merge": 0, 2024-09-16T00:35:40.623 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes": 1233, 2024-09-16T00:35:40.623 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_top": 0, 2024-09-16T00:35:40.623 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_bottom": 1233, 2024-09-16T00:35:40.623 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_pin_tail": 0, 2024-09-16T00:35:40.623 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_pinned": 1233, 2024-09-16T00:35:40.623 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_expired": 0, 2024-09-16T00:35:40.624 INFO:teuthology.orchestra.run.smithi122.stdout: "inodes_with_caps": 0, 2024-09-16T00:35:40.624 INFO:teuthology.orchestra.run.smithi122.stdout: "caps": 0, 2024-09-16T00:35:40.624 INFO:teuthology.orchestra.run.smithi122.stdout: "subtrees": 2, 2024-09-16T00:35:40.624 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse": 0, 2024-09-16T00:35:40.624 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_hit": 0, 2024-09-16T00:35:40.624 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_forward": 0, 2024-09-16T00:35:40.624 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_discover": 0, 2024-09-16T00:35:40.624 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_dir_fetch": 0, 2024-09-16T00:35:40.624 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_remote_ino": 0, 2024-09-16T00:35:40.624 INFO:teuthology.orchestra.run.smithi122.stdout: "traverse_lock": 0, 2024-09-16T00:35:40.624 INFO:teuthology.orchestra.run.smithi122.stdout: "load_cent": 0, 2024-09-16T00:35:40.624 INFO:teuthology.orchestra.run.smithi122.stdout: "q": 0, 2024-09-16T00:35:40.624 INFO:teuthology.orchestra.run.smithi122.stdout: "exported": 0, 2024-09-16T00:35:40.625 INFO:teuthology.orchestra.run.smithi122.stdout: "exported_inodes": 0, 2024-09-16T00:35:40.625 INFO:teuthology.orchestra.run.smithi122.stdout: "imported": 0, 2024-09-16T00:35:40.625 INFO:teuthology.orchestra.run.smithi122.stdout: "imported_inodes": 0, 2024-09-16T00:35:40.625 INFO:teuthology.orchestra.run.smithi122.stdout: "openino_dir_fetch": 0, 2024-09-16T00:35:40.625 INFO:teuthology.orchestra.run.smithi122.stdout: "openino_backtrace_fetch": 0, 2024-09-16T00:35:40.625 INFO:teuthology.orchestra.run.smithi122.stdout: "openino_peer_discover": 0, 2024-09-16T00:35:40.625 INFO:teuthology.orchestra.run.smithi122.stdout: "root_rfiles": 4, 2024-09-16T00:35:40.625 INFO:teuthology.orchestra.run.smithi122.stdout: "root_rbytes": 234, 2024-09-16T00:35:40.625 INFO:teuthology.orchestra.run.smithi122.stdout: "root_rsnaps": 0, 2024-09-16T00:35:40.625 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_backtrace_fetch": 0, 2024-09-16T00:35:40.625 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_set_tag": 0, 2024-09-16T00:35:40.625 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_backtrace_repaired": 0, 2024-09-16T00:35:40.625 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_inotable_repaired": 0, 2024-09-16T00:35:40.625 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_dir_inodes": 0, 2024-09-16T00:35:40.626 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_dir_base_inodes": 0, 2024-09-16T00:35:40.626 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_dirfrag_rstats": 0, 2024-09-16T00:35:40.626 INFO:teuthology.orchestra.run.smithi122.stdout: "scrub_file_inodes": 0, 2024-09-16T00:35:40.626 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_inode_file_caps": 0, 2024-09-16T00:35:40.626 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_revoke": 0, 2024-09-16T00:35:40.626 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_grant": 0, 2024-09-16T00:35:40.626 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_trunc": 0, 2024-09-16T00:35:40.626 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-16T00:35:40.626 INFO:teuthology.orchestra.run.smithi122.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-16T00:35:40.626 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_caps": 0, 2024-09-16T00:35:40.626 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_caps_dirty": 0, 2024-09-16T00:35:40.626 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_cap_release": 0, 2024-09-16T00:35:40.626 INFO:teuthology.orchestra.run.smithi122.stdout: "process_request_cap_release": 0 2024-09-16T00:35:40.627 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.627 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_cache": { 2024-09-16T00:35:40.627 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_update": 0, 2024-09-16T00:35:40.627 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_update_receipt": 0, 2024-09-16T00:35:40.627 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_try_discover": 0, 2024-09-16T00:35:40.627 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_send_discover": 0, 2024-09-16T00:35:40.627 INFO:teuthology.orchestra.run.smithi122.stdout: "dir_handle_discover": 0, 2024-09-16T00:35:40.627 INFO:teuthology.orchestra.run.smithi122.stdout: "num_strays": 0, 2024-09-16T00:35:40.627 INFO:teuthology.orchestra.run.smithi122.stdout: "num_strays_delayed": 0, 2024-09-16T00:35:40.627 INFO:teuthology.orchestra.run.smithi122.stdout: "num_strays_enqueuing": 0, 2024-09-16T00:35:40.627 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_created": 0, 2024-09-16T00:35:40.627 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_enqueued": 0, 2024-09-16T00:35:40.627 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_reintegrated": 0, 2024-09-16T00:35:40.627 INFO:teuthology.orchestra.run.smithi122.stdout: "strays_migrated": 0, 2024-09-16T00:35:40.628 INFO:teuthology.orchestra.run.smithi122.stdout: "num_recovering_processing": 0, 2024-09-16T00:35:40.628 INFO:teuthology.orchestra.run.smithi122.stdout: "num_recovering_enqueued": 0, 2024-09-16T00:35:40.628 INFO:teuthology.orchestra.run.smithi122.stdout: "num_recovering_prioritized": 0, 2024-09-16T00:35:40.628 INFO:teuthology.orchestra.run.smithi122.stdout: "recovery_started": 0, 2024-09-16T00:35:40.628 INFO:teuthology.orchestra.run.smithi122.stdout: "recovery_completed": 0, 2024-09-16T00:35:40.628 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_quiesce_path": 0, 2024-09-16T00:35:40.628 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_quiesce_inode": 0, 2024-09-16T00:35:40.628 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_enqueue_scrub": 0, 2024-09-16T00:35:40.628 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_exportdir": 0, 2024-09-16T00:35:40.628 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_flush": 0, 2024-09-16T00:35:40.628 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_fragmentdir": 0, 2024-09-16T00:35:40.628 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_fragstats": 0, 2024-09-16T00:35:40.628 INFO:teuthology.orchestra.run.smithi122.stdout: "ireq_inodestats": 0 2024-09-16T00:35:40.629 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.629 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_log": { 2024-09-16T00:35:40.629 INFO:teuthology.orchestra.run.smithi122.stdout: "evlrg": 0, 2024-09-16T00:35:40.629 INFO:teuthology.orchestra.run.smithi122.stdout: "evadd": 0, 2024-09-16T00:35:40.629 INFO:teuthology.orchestra.run.smithi122.stdout: "evex": 0, 2024-09-16T00:35:40.629 INFO:teuthology.orchestra.run.smithi122.stdout: "evtrm": 0, 2024-09-16T00:35:40.629 INFO:teuthology.orchestra.run.smithi122.stdout: "ev": 6119, 2024-09-16T00:35:40.629 INFO:teuthology.orchestra.run.smithi122.stdout: "evexg": 0, 2024-09-16T00:35:40.629 INFO:teuthology.orchestra.run.smithi122.stdout: "evexd": 0, 2024-09-16T00:35:40.629 INFO:teuthology.orchestra.run.smithi122.stdout: "segadd": 0, 2024-09-16T00:35:40.629 INFO:teuthology.orchestra.run.smithi122.stdout: "segex": 0, 2024-09-16T00:35:40.629 INFO:teuthology.orchestra.run.smithi122.stdout: "segtrm": 0, 2024-09-16T00:35:40.629 INFO:teuthology.orchestra.run.smithi122.stdout: "seg": 9, 2024-09-16T00:35:40.629 INFO:teuthology.orchestra.run.smithi122.stdout: "segmjr": 2, 2024-09-16T00:35:40.630 INFO:teuthology.orchestra.run.smithi122.stdout: "segexg": 0, 2024-09-16T00:35:40.630 INFO:teuthology.orchestra.run.smithi122.stdout: "segexd": 0, 2024-09-16T00:35:40.630 INFO:teuthology.orchestra.run.smithi122.stdout: "expos": 4194304, 2024-09-16T00:35:40.630 INFO:teuthology.orchestra.run.smithi122.stdout: "wrpos": 16858830, 2024-09-16T00:35:40.630 INFO:teuthology.orchestra.run.smithi122.stdout: "rdpos": 16855408, 2024-09-16T00:35:40.630 INFO:teuthology.orchestra.run.smithi122.stdout: "jlat": { 2024-09-16T00:35:40.630 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:40.630 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:40.630 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:40.630 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.630 INFO:teuthology.orchestra.run.smithi122.stdout: "replayed": 6119 2024-09-16T00:35:40.630 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.630 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_mem": { 2024-09-16T00:35:40.630 INFO:teuthology.orchestra.run.smithi122.stdout: "ino": 21, 2024-09-16T00:35:40.631 INFO:teuthology.orchestra.run.smithi122.stdout: "ino+": 1233, 2024-09-16T00:35:40.631 INFO:teuthology.orchestra.run.smithi122.stdout: "ino-": 1212, 2024-09-16T00:35:40.631 INFO:teuthology.orchestra.run.smithi122.stdout: "dir": 16, 2024-09-16T00:35:40.631 INFO:teuthology.orchestra.run.smithi122.stdout: "dir+": 22, 2024-09-16T00:35:40.631 INFO:teuthology.orchestra.run.smithi122.stdout: "dir-": 6, 2024-09-16T00:35:40.631 INFO:teuthology.orchestra.run.smithi122.stdout: "dn": 1233, 2024-09-16T00:35:40.631 INFO:teuthology.orchestra.run.smithi122.stdout: "dn+": 1240, 2024-09-16T00:35:40.631 INFO:teuthology.orchestra.run.smithi122.stdout: "dn-": 7, 2024-09-16T00:35:40.631 INFO:teuthology.orchestra.run.smithi122.stdout: "cap": 0, 2024-09-16T00:35:40.631 INFO:teuthology.orchestra.run.smithi122.stdout: "cap+": 0, 2024-09-16T00:35:40.631 INFO:teuthology.orchestra.run.smithi122.stdout: "cap-": 0, 2024-09-16T00:35:40.631 INFO:teuthology.orchestra.run.smithi122.stdout: "rss": 50316, 2024-09-16T00:35:40.631 INFO:teuthology.orchestra.run.smithi122.stdout: "heap": 215328 2024-09-16T00:35:40.631 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.632 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_server": { 2024-09-16T00:35:40.632 INFO:teuthology.orchestra.run.smithi122.stdout: "dispatch_client_request": 0, 2024-09-16T00:35:40.632 INFO:teuthology.orchestra.run.smithi122.stdout: "dispatch_server_request": 0, 2024-09-16T00:35:40.632 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_request": 0, 2024-09-16T00:35:40.632 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_client_session": 0, 2024-09-16T00:35:40.632 INFO:teuthology.orchestra.run.smithi122.stdout: "handle_peer_request": 0, 2024-09-16T00:35:40.632 INFO:teuthology.orchestra.run.smithi122.stdout: "req_create_latency": { 2024-09-16T00:35:40.632 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:40.632 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:40.632 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:40.632 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.632 INFO:teuthology.orchestra.run.smithi122.stdout: "req_getattr_latency": { 2024-09-16T00:35:40.632 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:40.632 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:40.633 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:40.633 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.633 INFO:teuthology.orchestra.run.smithi122.stdout: "req_getfilelock_latency": { 2024-09-16T00:35:40.633 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:40.633 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:40.633 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:40.633 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.633 INFO:teuthology.orchestra.run.smithi122.stdout: "req_link_latency": { 2024-09-16T00:35:40.633 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:40.633 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:40.633 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:40.633 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.633 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookup_latency": { 2024-09-16T00:35:40.633 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:40.634 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:40.634 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:40.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.634 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookuphash_latency": { 2024-09-16T00:35:40.634 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:40.634 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:40.634 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:40.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.634 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupino_latency": { 2024-09-16T00:35:40.634 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:40.634 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:40.634 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:40.634 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.634 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupname_latency": { 2024-09-16T00:35:40.635 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:40.635 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:40.635 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:40.635 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.635 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupparent_latency": { 2024-09-16T00:35:40.635 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:40.635 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:40.635 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:40.635 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.635 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lookupsnap_latency": { 2024-09-16T00:35:40.635 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:40.635 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:40.635 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:40.635 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.636 INFO:teuthology.orchestra.run.smithi122.stdout: "req_lssnap_latency": { 2024-09-16T00:35:40.636 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:40.636 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:40.636 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:40.636 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.636 INFO:teuthology.orchestra.run.smithi122.stdout: "req_mkdir_latency": { 2024-09-16T00:35:40.637 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:40.637 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:40.637 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:40.637 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.637 INFO:teuthology.orchestra.run.smithi122.stdout: "req_mknod_latency": { 2024-09-16T00:35:40.637 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:40.637 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:40.637 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:40.637 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.637 INFO:teuthology.orchestra.run.smithi122.stdout: "req_mksnap_latency": { 2024-09-16T00:35:40.637 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:40.637 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:40.638 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:40.638 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.638 INFO:teuthology.orchestra.run.smithi122.stdout: "req_open_latency": { 2024-09-16T00:35:40.638 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:40.638 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:40.638 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:40.638 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.638 INFO:teuthology.orchestra.run.smithi122.stdout: "req_readdir_latency": { 2024-09-16T00:35:40.638 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:40.638 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:40.638 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:40.638 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.638 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rename_latency": { 2024-09-16T00:35:40.639 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:40.639 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:40.639 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:40.639 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.639 INFO:teuthology.orchestra.run.smithi122.stdout: "req_renamesnap_latency": { 2024-09-16T00:35:40.639 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:40.639 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:40.639 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:40.639 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.639 INFO:teuthology.orchestra.run.smithi122.stdout: "req_snapdiff_latency": { 2024-09-16T00:35:40.639 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:40.639 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:40.639 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:40.639 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.640 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rmdir_latency": { 2024-09-16T00:35:40.640 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:40.640 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:40.640 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:40.640 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.640 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rmsnap_latency": { 2024-09-16T00:35:40.640 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:40.640 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:40.640 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:40.640 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.640 INFO:teuthology.orchestra.run.smithi122.stdout: "req_rmxattr_latency": { 2024-09-16T00:35:40.640 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:40.640 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:40.640 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:40.641 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.641 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setattr_latency": { 2024-09-16T00:35:40.641 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:40.641 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:40.641 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:40.641 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.641 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setdirlayout_latency": { 2024-09-16T00:35:40.641 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:40.641 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:40.641 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:40.641 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.641 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setfilelock_latency": { 2024-09-16T00:35:40.641 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:40.642 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:40.642 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:40.642 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.642 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setlayout_latency": { 2024-09-16T00:35:40.642 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:40.642 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:40.642 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:40.642 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.642 INFO:teuthology.orchestra.run.smithi122.stdout: "req_setxattr_latency": { 2024-09-16T00:35:40.642 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:40.642 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:40.642 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:40.642 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.642 INFO:teuthology.orchestra.run.smithi122.stdout: "req_symlink_latency": { 2024-09-16T00:35:40.643 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:40.643 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:40.643 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:40.643 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.643 INFO:teuthology.orchestra.run.smithi122.stdout: "req_unlink_latency": { 2024-09-16T00:35:40.643 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:40.643 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:40.643 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:40.643 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.643 INFO:teuthology.orchestra.run.smithi122.stdout: "cap_revoke_eviction": 0, 2024-09-16T00:35:40.643 INFO:teuthology.orchestra.run.smithi122.stdout: "cap_acquisition_throttle": 0, 2024-09-16T00:35:40.643 INFO:teuthology.orchestra.run.smithi122.stdout: "req_getvxattr_latency": { 2024-09-16T00:35:40.643 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:40.643 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:40.644 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:40.644 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-16T00:35:40.644 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.644 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_sessions": { 2024-09-16T00:35:40.644 INFO:teuthology.orchestra.run.smithi122.stdout: "session_count": 1, 2024-09-16T00:35:40.644 INFO:teuthology.orchestra.run.smithi122.stdout: "session_add": 4, 2024-09-16T00:35:40.644 INFO:teuthology.orchestra.run.smithi122.stdout: "session_remove": 3, 2024-09-16T00:35:40.644 INFO:teuthology.orchestra.run.smithi122.stdout: "sessions_open": 2, 2024-09-16T00:35:40.644 INFO:teuthology.orchestra.run.smithi122.stdout: "sessions_stale": 0, 2024-09-16T00:35:40.644 INFO:teuthology.orchestra.run.smithi122.stdout: "total_load": 0, 2024-09-16T00:35:40.644 INFO:teuthology.orchestra.run.smithi122.stdout: "average_load": 0, 2024-09-16T00:35:40.644 INFO:teuthology.orchestra.run.smithi122.stdout: "avg_session_uptime": 2796, 2024-09-16T00:35:40.644 INFO:teuthology.orchestra.run.smithi122.stdout: "mdthresh_evicted": 0 2024-09-16T00:35:40.645 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.645 INFO:teuthology.orchestra.run.smithi122.stdout: "mempool": { 2024-09-16T00:35:40.645 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:35:40.645 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_items": 0, 2024-09-16T00:35:40.645 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:35:40.645 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:35:40.645 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:35:40.645 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:35:40.645 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:35:40.645 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:35:40.645 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:35:40.645 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:35:40.645 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:35:40.645 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:35:40.646 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:35:40.646 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:35:40.646 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:35:40.646 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_items": 0, 2024-09-16T00:35:40.646 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:35:40.646 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_items": 0, 2024-09-16T00:35:40.646 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:35:40.646 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:35:40.646 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:35:40.646 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:35:40.646 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:35:40.646 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:35:40.646 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:35:40.647 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_items": 0, 2024-09-16T00:35:40.647 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:35:40.647 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:35:40.647 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:35:40.647 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_items": 0, 2024-09-16T00:35:40.647 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_bytes": 0, 2024-09-16T00:35:40.647 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_items": 0, 2024-09-16T00:35:40.647 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:35:40.647 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:35:40.647 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:35:40.647 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:35:40.648 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_bytes": 189239, 2024-09-16T00:35:40.648 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_items": 48, 2024-09-16T00:35:40.648 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:35:40.648 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_items": 0, 2024-09-16T00:35:40.648 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_bytes": 0, 2024-09-16T00:35:40.648 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_items": 0, 2024-09-16T00:35:40.648 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:35:40.648 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_items": 0, 2024-09-16T00:35:40.648 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:35:40.648 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_items": 0, 2024-09-16T00:35:40.648 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_bytes": 8768, 2024-09-16T00:35:40.648 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_items": 165, 2024-09-16T00:35:40.648 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:35:40.649 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:35:40.649 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_bytes": 0, 2024-09-16T00:35:40.649 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_items": 0, 2024-09-16T00:35:40.649 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_bytes": 935578, 2024-09-16T00:35:40.649 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_items": 5203, 2024-09-16T00:35:40.649 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_bytes": 0, 2024-09-16T00:35:40.649 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_items": 0, 2024-09-16T00:35:40.649 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_bytes": 0, 2024-09-16T00:35:40.649 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_items": 0 2024-09-16T00:35:40.649 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.649 INFO:teuthology.orchestra.run.smithi122.stdout: "objecter": { 2024-09-16T00:35:40.649 INFO:teuthology.orchestra.run.smithi122.stdout: "op_active": 0, 2024-09-16T00:35:40.649 INFO:teuthology.orchestra.run.smithi122.stdout: "op_laggy": 0, 2024-09-16T00:35:40.649 INFO:teuthology.orchestra.run.smithi122.stdout: "op_send": 900, 2024-09-16T00:35:40.650 INFO:teuthology.orchestra.run.smithi122.stdout: "op_send_bytes": 26, 2024-09-16T00:35:40.650 INFO:teuthology.orchestra.run.smithi122.stdout: "op_resend": 4, 2024-09-16T00:35:40.650 INFO:teuthology.orchestra.run.smithi122.stdout: "op_reply": 900, 2024-09-16T00:35:40.650 INFO:teuthology.orchestra.run.smithi122.stdout: "op_latency": { 2024-09-16T00:35:40.650 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 900, 2024-09-16T00:35:40.650 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.412990135, 2024-09-16T00:35:40.650 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000458877 2024-09-16T00:35:40.650 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.650 INFO:teuthology.orchestra.run.smithi122.stdout: "op_inflight": 0, 2024-09-16T00:35:40.650 INFO:teuthology.orchestra.run.smithi122.stdout: "oplen_avg": { 2024-09-16T00:35:40.650 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 900, 2024-09-16T00:35:40.650 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 901 2024-09-16T00:35:40.650 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.650 INFO:teuthology.orchestra.run.smithi122.stdout: "op": 900, 2024-09-16T00:35:40.651 INFO:teuthology.orchestra.run.smithi122.stdout: "op_r": 900, 2024-09-16T00:35:40.651 INFO:teuthology.orchestra.run.smithi122.stdout: "op_w": 0, 2024-09-16T00:35:40.651 INFO:teuthology.orchestra.run.smithi122.stdout: "op_rmw": 0, 2024-09-16T00:35:40.651 INFO:teuthology.orchestra.run.smithi122.stdout: "op_pg": 0, 2024-09-16T00:35:40.651 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_stat": 561, 2024-09-16T00:35:40.651 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_create": 0, 2024-09-16T00:35:40.651 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_read": 336, 2024-09-16T00:35:40.651 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_write": 0, 2024-09-16T00:35:40.651 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_writefull": 0, 2024-09-16T00:35:40.651 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_writesame": 0, 2024-09-16T00:35:40.651 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_append": 0, 2024-09-16T00:35:40.651 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_zero": 0, 2024-09-16T00:35:40.651 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_truncate": 0, 2024-09-16T00:35:40.651 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_delete": 0, 2024-09-16T00:35:40.652 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_mapext": 0, 2024-09-16T00:35:40.652 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_sparse_read": 0, 2024-09-16T00:35:40.652 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_clonerange": 0, 2024-09-16T00:35:40.652 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_getxattr": 2, 2024-09-16T00:35:40.652 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_setxattr": 0, 2024-09-16T00:35:40.652 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_cmpxattr": 0, 2024-09-16T00:35:40.652 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_rmxattr": 0, 2024-09-16T00:35:40.652 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_resetxattrs": 0, 2024-09-16T00:35:40.652 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_call": 0, 2024-09-16T00:35:40.652 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_watch": 0, 2024-09-16T00:35:40.652 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_notify": 0, 2024-09-16T00:35:40.652 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_src_cmpxattr": 0, 2024-09-16T00:35:40.652 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_pgls": 0, 2024-09-16T00:35:40.653 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_pgls_filter": 0, 2024-09-16T00:35:40.653 INFO:teuthology.orchestra.run.smithi122.stdout: "osdop_other": 0, 2024-09-16T00:35:40.653 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_active": 0, 2024-09-16T00:35:40.653 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_send": 0, 2024-09-16T00:35:40.653 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_resend": 0, 2024-09-16T00:35:40.653 INFO:teuthology.orchestra.run.smithi122.stdout: "linger_ping": 0, 2024-09-16T00:35:40.653 INFO:teuthology.orchestra.run.smithi122.stdout: "poolop_active": 0, 2024-09-16T00:35:40.653 INFO:teuthology.orchestra.run.smithi122.stdout: "poolop_send": 0, 2024-09-16T00:35:40.653 INFO:teuthology.orchestra.run.smithi122.stdout: "poolop_resend": 0, 2024-09-16T00:35:40.653 INFO:teuthology.orchestra.run.smithi122.stdout: "poolstat_active": 0, 2024-09-16T00:35:40.653 INFO:teuthology.orchestra.run.smithi122.stdout: "poolstat_send": 0, 2024-09-16T00:35:40.653 INFO:teuthology.orchestra.run.smithi122.stdout: "poolstat_resend": 0, 2024-09-16T00:35:40.653 INFO:teuthology.orchestra.run.smithi122.stdout: "statfs_active": 0, 2024-09-16T00:35:40.653 INFO:teuthology.orchestra.run.smithi122.stdout: "statfs_send": 0, 2024-09-16T00:35:40.654 INFO:teuthology.orchestra.run.smithi122.stdout: "statfs_resend": 0, 2024-09-16T00:35:40.654 INFO:teuthology.orchestra.run.smithi122.stdout: "command_active": 0, 2024-09-16T00:35:40.654 INFO:teuthology.orchestra.run.smithi122.stdout: "command_send": 0, 2024-09-16T00:35:40.654 INFO:teuthology.orchestra.run.smithi122.stdout: "command_resend": 0, 2024-09-16T00:35:40.654 INFO:teuthology.orchestra.run.smithi122.stdout: "map_epoch": 92, 2024-09-16T00:35:40.654 INFO:teuthology.orchestra.run.smithi122.stdout: "map_full": 0, 2024-09-16T00:35:40.654 INFO:teuthology.orchestra.run.smithi122.stdout: "map_inc": 4, 2024-09-16T00:35:40.654 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_sessions": 5, 2024-09-16T00:35:40.654 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_session_open": 5, 2024-09-16T00:35:40.654 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_session_close": 0, 2024-09-16T00:35:40.654 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_laggy": 0, 2024-09-16T00:35:40.654 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_wr": 0, 2024-09-16T00:35:40.654 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_rd": 2, 2024-09-16T00:35:40.654 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_del": 0 2024-09-16T00:35:40.655 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.655 INFO:teuthology.orchestra.run.smithi122.stdout: "oft": { 2024-09-16T00:35:40.655 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_objs": 0, 2024-09-16T00:35:40.655 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_kv_pairs": 0, 2024-09-16T00:35:40.655 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_updates": 0, 2024-09-16T00:35:40.655 INFO:teuthology.orchestra.run.smithi122.stdout: "omap_total_removes": 0 2024-09-16T00:35:40.655 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.655 INFO:teuthology.orchestra.run.smithi122.stdout: "osd-slow-ops": { 2024-09-16T00:35:40.655 INFO:teuthology.orchestra.run.smithi122.stdout: "slow_ops_count": 0 2024-09-16T00:35:40.655 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.655 INFO:teuthology.orchestra.run.smithi122.stdout: "purge_queue": { 2024-09-16T00:35:40.655 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing_ops": 0, 2024-09-16T00:35:40.655 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing_ops_high_water": 0, 2024-09-16T00:35:40.655 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing": 0, 2024-09-16T00:35:40.656 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executing_high_water": 0, 2024-09-16T00:35:40.656 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executed_ops": 0, 2024-09-16T00:35:40.656 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_executed": 0, 2024-09-16T00:35:40.656 INFO:teuthology.orchestra.run.smithi122.stdout: "pq_item_in_journal": 0 2024-09-16T00:35:40.656 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.656 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:35:40.656 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-09-16T00:35:40.656 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 104857600, 2024-09-16T00:35:40.656 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-09-16T00:35:40.656 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 1011, 2024-09-16T00:35:40.656 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 13567793, 2024-09-16T00:35:40.656 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-09-16T00:35:40.656 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 1011, 2024-09-16T00:35:40.656 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-09-16T00:35:40.657 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-09-16T00:35:40.657 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 1011, 2024-09-16T00:35:40.657 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 13567793, 2024-09-16T00:35:40.657 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-09-16T00:35:40.657 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:40.657 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:40.657 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:40.657 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-16T00:35:40.657 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.657 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-objecter_bytes": { 2024-09-16T00:35:40.657 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-09-16T00:35:40.657 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 104857600, 2024-09-16T00:35:40.657 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-09-16T00:35:40.658 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 0, 2024-09-16T00:35:40.658 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 0, 2024-09-16T00:35:40.658 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-09-16T00:35:40.658 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 0, 2024-09-16T00:35:40.658 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 900, 2024-09-16T00:35:40.658 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 12665436, 2024-09-16T00:35:40.658 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 900, 2024-09-16T00:35:40.658 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 12665436, 2024-09-16T00:35:40.658 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-09-16T00:35:40.658 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:40.658 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:40.658 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:40.658 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-16T00:35:40.658 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.659 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-objecter_ops": { 2024-09-16T00:35:40.659 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-09-16T00:35:40.659 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 1024, 2024-09-16T00:35:40.659 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-09-16T00:35:40.659 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 0, 2024-09-16T00:35:40.659 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 0, 2024-09-16T00:35:40.659 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-09-16T00:35:40.659 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 0, 2024-09-16T00:35:40.659 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 900, 2024-09-16T00:35:40.659 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 900, 2024-09-16T00:35:40.659 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 900, 2024-09-16T00:35:40.659 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 900, 2024-09-16T00:35:40.659 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-09-16T00:35:40.659 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:40.660 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:40.660 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:40.660 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-16T00:35:40.660 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.660 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-write_buf_throttle": { 2024-09-16T00:35:40.660 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-09-16T00:35:40.660 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 3758096384, 2024-09-16T00:35:40.660 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-09-16T00:35:40.660 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 0, 2024-09-16T00:35:40.660 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 0, 2024-09-16T00:35:40.660 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-09-16T00:35:40.660 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 0, 2024-09-16T00:35:40.660 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-09-16T00:35:40.660 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-09-16T00:35:40.661 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 0, 2024-09-16T00:35:40.661 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 0, 2024-09-16T00:35:40.661 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-09-16T00:35:40.661 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:40.661 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:40.661 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:40.661 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-16T00:35:40.661 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:40.661 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-write_buf_throttle-0x564fa76beb40": { 2024-09-16T00:35:40.661 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-09-16T00:35:40.661 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 3758096384, 2024-09-16T00:35:40.661 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-09-16T00:35:40.661 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 0, 2024-09-16T00:35:40.661 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 0, 2024-09-16T00:35:40.662 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-09-16T00:35:40.662 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 0, 2024-09-16T00:35:40.662 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-09-16T00:35:40.662 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-09-16T00:35:40.662 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 0, 2024-09-16T00:35:40.662 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 0, 2024-09-16T00:35:40.662 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-09-16T00:35:40.662 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:40.662 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:40.662 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:40.662 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-16T00:35:40.662 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-16T00:35:40.662 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-16T00:35:41.411 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-09-16T00:35:41.411 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-09-16T00:35:41.411 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-09-16T00:35:41.411 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=0 2024-09-16T00:35:41.411 DEBUG:tasks.check_counter:Getting stats from l 2024-09-16T00:35:41.411 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph daemon mds.l perf dump 2024-09-16T00:35:41.656 INFO:teuthology.orchestra.run.smithi122.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.c/config 2024-09-16T00:35:41.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:41 smithi122 ceph-mon[35412]: pgmap v451: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T00:35:41.794 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:41 smithi073 ceph-mon[35254]: pgmap v451: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T00:35:41.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:41 smithi038 ceph-mon[30945]: pgmap v451: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2024-09-16T00:35:42.129 INFO:teuthology.orchestra.run.smithi122.stdout:{ 2024-09-16T00:35:42.129 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-0": { 2024-09-16T00:35:42.129 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 0, 2024-09-16T00:35:42.129 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 2, 2024-09-16T00:35:42.129 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 0, 2024-09-16T00:35:42.129 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 0, 2024-09-16T00:35:42.130 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 4, 2024-09-16T00:35:42.130 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 0, 2024-09-16T00:35:42.130 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.000971644, 2024-09-16T00:35:42.130 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.000000000, 2024-09-16T00:35:42.130 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.000201251, 2024-09-16T00:35:42.130 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:35:42.130 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:35:42.130 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:42.130 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:42.130 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:42.130 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:42.130 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:35:42.130 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:42.130 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:42.131 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:42.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:42.131 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-16T00:35:42.131 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 0 2024-09-16T00:35:42.131 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:42.131 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-1": { 2024-09-16T00:35:42.131 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 5, 2024-09-16T00:35:42.131 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 77, 2024-09-16T00:35:42.131 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 3232, 2024-09-16T00:35:42.131 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 92240, 2024-09-16T00:35:42.131 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 3, 2024-09-16T00:35:42.131 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 2, 2024-09-16T00:35:42.132 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.006787834, 2024-09-16T00:35:42.132 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.003267972, 2024-09-16T00:35:42.132 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.002714575, 2024-09-16T00:35:42.132 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:35:42.132 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:35:42.132 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 76, 2024-09-16T00:35:42.132 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.008635736, 2024-09-16T00:35:42.132 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000113628 2024-09-16T00:35:42.132 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:42.132 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:35:42.132 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:42.132 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:42.132 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:42.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:42.133 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 3232, 2024-09-16T00:35:42.133 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 92240 2024-09-16T00:35:42.133 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:42.133 INFO:teuthology.orchestra.run.smithi122.stdout: "AsyncMessenger::Worker-2": { 2024-09-16T00:35:42.133 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_messages": 104, 2024-09-16T00:35:42.133 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages": 99, 2024-09-16T00:35:42.133 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_bytes": 723024, 2024-09-16T00:35:42.133 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_bytes": 59872, 2024-09-16T00:35:42.133 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_created_connections": 2, 2024-09-16T00:35:42.133 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_active_connections": 1, 2024-09-16T00:35:42.133 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_total_time": 0.019164596, 2024-09-16T00:35:42.133 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_send_time": 0.006528848, 2024-09-16T00:35:42.134 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_recv_time": 0.020125986, 2024-09-16T00:35:42.134 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-16T00:35:42.134 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_messages_queue_lat": { 2024-09-16T00:35:42.134 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 98, 2024-09-16T00:35:42.134 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.006152554, 2024-09-16T00:35:42.134 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000062781 2024-09-16T00:35:42.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:42.134 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_handle_ack_lat": { 2024-09-16T00:35:42.134 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:42.134 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:42.134 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:42.134 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:42.134 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_recv_encrypted_bytes": 723024, 2024-09-16T00:35:42.135 INFO:teuthology.orchestra.run.smithi122.stdout: "msgr_send_encrypted_bytes": 59872 2024-09-16T00:35:42.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:42.135 INFO:teuthology.orchestra.run.smithi122.stdout: "cct": { 2024-09-16T00:35:42.135 INFO:teuthology.orchestra.run.smithi122.stdout: "total_workers": 0, 2024-09-16T00:35:42.135 INFO:teuthology.orchestra.run.smithi122.stdout: "unhealthy_workers": 0 2024-09-16T00:35:42.135 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:42.135 INFO:teuthology.orchestra.run.smithi122.stdout: "mempool": { 2024-09-16T00:35:42.135 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_bytes": 0, 2024-09-16T00:35:42.135 INFO:teuthology.orchestra.run.smithi122.stdout: "bloom_filter_items": 0, 2024-09-16T00:35:42.135 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_bytes": 0, 2024-09-16T00:35:42.135 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_alloc_items": 0, 2024-09-16T00:35:42.135 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_bytes": 0, 2024-09-16T00:35:42.135 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_data_items": 0, 2024-09-16T00:35:42.135 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-16T00:35:42.136 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_onode_items": 0, 2024-09-16T00:35:42.136 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-16T00:35:42.136 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_meta_items": 0, 2024-09-16T00:35:42.136 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_bytes": 0, 2024-09-16T00:35:42.136 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_other_items": 0, 2024-09-16T00:35:42.136 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-16T00:35:42.136 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_cache_buffer_items": 0, 2024-09-16T00:35:42.136 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_bytes": 0, 2024-09-16T00:35:42.136 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_extent_items": 0, 2024-09-16T00:35:42.136 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_bytes": 0, 2024-09-16T00:35:42.136 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_blob_items": 0, 2024-09-16T00:35:42.136 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-16T00:35:42.136 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_shared_blob_items": 0, 2024-09-16T00:35:42.136 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-16T00:35:42.137 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_inline_bl_items": 0, 2024-09-16T00:35:42.137 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_bytes": 0, 2024-09-16T00:35:42.137 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_fsck_items": 0, 2024-09-16T00:35:42.137 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_bytes": 0, 2024-09-16T00:35:42.137 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_txc_items": 0, 2024-09-16T00:35:42.137 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-16T00:35:42.137 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_deferred_items": 0, 2024-09-16T00:35:42.137 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_bytes": 0, 2024-09-16T00:35:42.137 INFO:teuthology.orchestra.run.smithi122.stdout: "bluestore_writing_items": 0, 2024-09-16T00:35:42.137 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_bytes": 0, 2024-09-16T00:35:42.137 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_items": 0, 2024-09-16T00:35:42.137 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_bytes": 0, 2024-09-16T00:35:42.137 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_reader_items": 0, 2024-09-16T00:35:42.137 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_bytes": 0, 2024-09-16T00:35:42.138 INFO:teuthology.orchestra.run.smithi122.stdout: "bluefs_file_writer_items": 0, 2024-09-16T00:35:42.138 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_bytes": 148863, 2024-09-16T00:35:42.138 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_anon_items": 28, 2024-09-16T00:35:42.138 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_bytes": 0, 2024-09-16T00:35:42.138 INFO:teuthology.orchestra.run.smithi122.stdout: "buffer_meta_items": 0, 2024-09-16T00:35:42.138 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_bytes": 0, 2024-09-16T00:35:42.138 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_items": 0, 2024-09-16T00:35:42.138 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_bytes": 0, 2024-09-16T00:35:42.138 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_mapbl_items": 0, 2024-09-16T00:35:42.138 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_bytes": 0, 2024-09-16T00:35:42.138 INFO:teuthology.orchestra.run.smithi122.stdout: "osd_pglog_items": 0, 2024-09-16T00:35:42.138 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_bytes": 0, 2024-09-16T00:35:42.138 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_items": 0, 2024-09-16T00:35:42.138 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_bytes": 0, 2024-09-16T00:35:42.139 INFO:teuthology.orchestra.run.smithi122.stdout: "osdmap_mapping_items": 0, 2024-09-16T00:35:42.139 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_bytes": 0, 2024-09-16T00:35:42.139 INFO:teuthology.orchestra.run.smithi122.stdout: "pgmap_items": 0, 2024-09-16T00:35:42.139 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_bytes": 928, 2024-09-16T00:35:42.139 INFO:teuthology.orchestra.run.smithi122.stdout: "mds_co_items": 2, 2024-09-16T00:35:42.139 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_bytes": 0, 2024-09-16T00:35:42.139 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_1_items": 0, 2024-09-16T00:35:42.139 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_bytes": 0, 2024-09-16T00:35:42.139 INFO:teuthology.orchestra.run.smithi122.stdout: "unittest_2_items": 0 2024-09-16T00:35:42.139 INFO:teuthology.orchestra.run.smithi122.stdout: }, 2024-09-16T00:35:42.139 INFO:teuthology.orchestra.run.smithi122.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-16T00:35:42.139 INFO:teuthology.orchestra.run.smithi122.stdout: "val": 0, 2024-09-16T00:35:42.139 INFO:teuthology.orchestra.run.smithi122.stdout: "max": 104857600, 2024-09-16T00:35:42.140 INFO:teuthology.orchestra.run.smithi122.stdout: "get_started": 0, 2024-09-16T00:35:42.140 INFO:teuthology.orchestra.run.smithi122.stdout: "get": 105, 2024-09-16T00:35:42.140 INFO:teuthology.orchestra.run.smithi122.stdout: "get_sum": 708353, 2024-09-16T00:35:42.140 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_fail": 0, 2024-09-16T00:35:42.140 INFO:teuthology.orchestra.run.smithi122.stdout: "get_or_fail_success": 105, 2024-09-16T00:35:42.140 INFO:teuthology.orchestra.run.smithi122.stdout: "take": 0, 2024-09-16T00:35:42.140 INFO:teuthology.orchestra.run.smithi122.stdout: "take_sum": 0, 2024-09-16T00:35:42.140 INFO:teuthology.orchestra.run.smithi122.stdout: "put": 105, 2024-09-16T00:35:42.140 INFO:teuthology.orchestra.run.smithi122.stdout: "put_sum": 708353, 2024-09-16T00:35:42.140 INFO:teuthology.orchestra.run.smithi122.stdout: "wait": { 2024-09-16T00:35:42.140 INFO:teuthology.orchestra.run.smithi122.stdout: "avgcount": 0, 2024-09-16T00:35:42.140 INFO:teuthology.orchestra.run.smithi122.stdout: "sum": 0.000000000, 2024-09-16T00:35:42.140 INFO:teuthology.orchestra.run.smithi122.stdout: "avgtime": 0.000000000 2024-09-16T00:35:42.140 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-16T00:35:42.141 INFO:teuthology.orchestra.run.smithi122.stdout: } 2024-09-16T00:35:42.141 INFO:teuthology.orchestra.run.smithi122.stdout:} 2024-09-16T00:35:42.811 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2024-09-16T00:35:42.811 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2024-09-16T00:35:42.811 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2024-09-16T00:35:42.811 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2024-09-16T00:35:42.811 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-16T00:35:42.821 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-16T00:35:42.822 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-16T00:35:42.822 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T00:35:42.841 DEBUG:teuthology.orchestra.run.smithi038:> set -ex 2024-09-16T00:35:42.841 DEBUG:teuthology.orchestra.run.smithi038:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-16T00:35:42.901 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd blocklist ls 2024-09-16T00:35:43.204 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:35:43.628 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:43 smithi038 ceph-mon[30945]: pgmap v452: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T00:35:43.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.715+0000 7f52d0183640 1 -- 172.21.15.38:0/1513886501 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52c8102dd0 msgr2=0x7f52c81031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:43.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.715+0000 7f52d0183640 1 --2- 172.21.15.38:0/1513886501 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52c8102dd0 0x7f52c81031b0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f52c4009930 tx=0x7f52c402a420 comp rx=0 tx=0).stop 2024-09-16T00:35:43.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.715+0000 7f52d0183640 1 -- 172.21.15.38:0/1513886501 shutdown_connections 2024-09-16T00:35:43.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.715+0000 7f52d0183640 1 --2- 172.21.15.38:0/1513886501 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52c8108300 0x7f52c810ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:43.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.715+0000 7f52d0183640 1 --2- 172.21.15.38:0/1513886501 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52c8103780 0x7f52c8107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:43.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.715+0000 7f52d0183640 1 --2- 172.21.15.38:0/1513886501 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52c8102dd0 0x7f52c81031b0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:43.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.715+0000 7f52d0183640 1 -- 172.21.15.38:0/1513886501 >> 172.21.15.38:0/1513886501 conn(0x7f52c80fc7d0 msgr2=0x7f52c80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:43.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.715+0000 7f52d0183640 1 -- 172.21.15.38:0/1513886501 shutdown_connections 2024-09-16T00:35:43.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.715+0000 7f52d0183640 1 -- 172.21.15.38:0/1513886501 wait complete. 2024-09-16T00:35:43.745 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.716+0000 7f52d0183640 1 Processor -- start 2024-09-16T00:35:43.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.716+0000 7f52d0183640 1 -- start start 2024-09-16T00:35:43.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.716+0000 7f52d0183640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52c8102dd0 0x7f52c8100a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:43.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.716+0000 7f52d0183640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52c8103780 0x7f52c8100f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:43.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.716+0000 7f52d0183640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52c8108300 0x7f52c81014b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:43.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.716+0000 7f52d0183640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f52c8112090 con 0x7f52c8102dd0 2024-09-16T00:35:43.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.716+0000 7f52d0183640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f52c8111f10 con 0x7f52c8103780 2024-09-16T00:35:43.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.716+0000 7f52d0183640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f52c8112210 con 0x7f52c8108300 2024-09-16T00:35:43.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.716+0000 7f52cdf1f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52c8102dd0 0x7f52c8100a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:43.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.716+0000 7f52cdf1f640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52c8102dd0 0x7f52c8100a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43054/0 (socket says 172.21.15.38:43054) 2024-09-16T00:35:43.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.716+0000 7f52cdf1f640 1 -- 172.21.15.38:0/2336463342 learned_addr learned my addr 172.21.15.38:0/2336463342 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:35:43.746 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.716+0000 7f52cd71e640 1 --2- 172.21.15.38:0/2336463342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52c8103780 0x7f52c8100f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.716+0000 7f52ce720640 1 --2- 172.21.15.38:0/2336463342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52c8108300 0x7f52c81014b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.716+0000 7f52cdf1f640 1 -- 172.21.15.38:0/2336463342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52c8108300 msgr2=0x7f52c81014b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.716+0000 7f52cdf1f640 1 --2- 172.21.15.38:0/2336463342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52c8108300 0x7f52c81014b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.716+0000 7f52cdf1f640 1 -- 172.21.15.38:0/2336463342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52c8103780 msgr2=0x7f52c8100f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.716+0000 7f52cdf1f640 1 --2- 172.21.15.38:0/2336463342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52c8103780 0x7f52c8100f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.716+0000 7f52cdf1f640 1 -- 172.21.15.38:0/2336463342 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c81a6a00 con 0x7f52c8102dd0 2024-09-16T00:35:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.716+0000 7f52ce720640 1 --2- 172.21.15.38:0/2336463342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52c8108300 0x7f52c81014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:35:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.717+0000 7f52cdf1f640 1 --2- 172.21.15.38:0/2336463342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52c8102dd0 0x7f52c8100a30 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f52c4009a50 tx=0x7f52c4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.717+0000 7f52beffd640 1 -- 172.21.15.38:0/2336463342 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c403f030 con 0x7f52c8102dd0 2024-09-16T00:35:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.717+0000 7f52beffd640 1 -- 172.21.15.38:0/2336463342 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f52c4003ce0 con 0x7f52c8102dd0 2024-09-16T00:35:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.717+0000 7f52d0183640 1 -- 172.21.15.38:0/2336463342 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52c81a6c90 con 0x7f52c8102dd0 2024-09-16T00:35:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.717+0000 7f52beffd640 1 -- 172.21.15.38:0/2336463342 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52c4035070 con 0x7f52c8102dd0 2024-09-16T00:35:43.747 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.717+0000 7f52d0183640 1 -- 172.21.15.38:0/2336463342 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52c81a7170 con 0x7f52c8102dd0 2024-09-16T00:35:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.718+0000 7f52d0183640 1 -- 172.21.15.38:0/2336463342 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52c81031b0 con 0x7f52c8102dd0 2024-09-16T00:35:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.720+0000 7f52beffd640 1 -- 172.21.15.38:0/2336463342 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f52c4042030 con 0x7f52c8102dd0 2024-09-16T00:35:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.721+0000 7f52beffd640 1 --2- 172.21.15.38:0/2336463342 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f52b40a3240 0x7f52b40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.721+0000 7f52cd71e640 1 --2- 172.21.15.38:0/2336463342 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f52b40a3240 0x7f52b40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.721+0000 7f52beffd640 1 -- 172.21.15.38:0/2336463342 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f52c40f06f0 con 0x7f52c8102dd0 2024-09-16T00:35:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.721+0000 7f52cd71e640 1 --2- 172.21.15.38:0/2336463342 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f52b40a3240 0x7f52b40a5700 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f52b8005e10 tx=0x7f52b8005da0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:43.748 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.724+0000 7f52beffd640 1 -- 172.21.15.38:0/2336463342 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f52c40bd090 con 0x7f52c8102dd0 2024-09-16T00:35:43.913 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.913+0000 7f52d0183640 1 -- 172.21.15.38:0/2336463342 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f52c810a290 con 0x7f52c8102dd0 2024-09-16T00:35:43.914 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.913+0000 7f52beffd640 1 -- 172.21.15.38:0/2336463342 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v92) ==== 81+0+588 (secure 0 0 0) 0x7f52c40bc990 con 0x7f52c8102dd0 2024-09-16T00:35:43.914 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6801/1779324916 2024-09-17T00:23:21.417483+0000 2024-09-16T00:35:43.914 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/924150447 2024-09-17T00:23:21.417483+0000 2024-09-16T00:35:43.914 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/141237849 2024-09-17T00:23:21.417483+0000 2024-09-16T00:35:43.914 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/2302081750 2024-09-17T00:22:56.844212+0000 2024-09-16T00:35:43.914 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/2245468594 2024-09-17T00:22:56.844212+0000 2024-09-16T00:35:43.914 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6801/583006373 2024-09-17T00:22:56.844212+0000 2024-09-16T00:35:43.915 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/2435492667 2024-09-17T00:23:21.417483+0000 2024-09-16T00:35:43.915 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6800/583006373 2024-09-17T00:22:56.844212+0000 2024-09-16T00:35:43.915 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:6800/1779324916 2024-09-17T00:23:21.417483+0000 2024-09-16T00:35:43.915 INFO:teuthology.orchestra.run.smithi038.stdout:172.21.15.38:0/1103516837 2024-09-17T00:22:56.844212+0000 2024-09-16T00:35:43.915 INFO:teuthology.orchestra.run.smithi038.stderr:listed 10 entries 2024-09-16T00:35:43.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.915+0000 7f52d0183640 1 -- 172.21.15.38:0/2336463342 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f52b40a3240 msgr2=0x7f52b40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:43.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.915+0000 7f52d0183640 1 --2- 172.21.15.38:0/2336463342 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f52b40a3240 0x7f52b40a5700 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f52b8005e10 tx=0x7f52b8005da0 comp rx=0 tx=0).stop 2024-09-16T00:35:43.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.915+0000 7f52d0183640 1 -- 172.21.15.38:0/2336463342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52c8102dd0 msgr2=0x7f52c8100a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:43.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.915+0000 7f52d0183640 1 --2- 172.21.15.38:0/2336463342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52c8102dd0 0x7f52c8100a30 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f52c4009a50 tx=0x7f52c4004270 comp rx=0 tx=0).stop 2024-09-16T00:35:43.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.916+0000 7f52d0183640 1 -- 172.21.15.38:0/2336463342 shutdown_connections 2024-09-16T00:35:43.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.916+0000 7f52d0183640 1 --2- 172.21.15.38:0/2336463342 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f52b40a3240 0x7f52b40a5700 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:43.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.916+0000 7f52d0183640 1 --2- 172.21.15.38:0/2336463342 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f52c8108300 0x7f52c81014b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:43.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.916+0000 7f52d0183640 1 --2- 172.21.15.38:0/2336463342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52c8103780 0x7f52c8100f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:43.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.916+0000 7f52d0183640 1 --2- 172.21.15.38:0/2336463342 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f52c8102dd0 0x7f52c8100a30 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:43.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.916+0000 7f52d0183640 1 -- 172.21.15.38:0/2336463342 >> 172.21.15.38:0/2336463342 conn(0x7f52c80fc7d0 msgr2=0x7f52c8105d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:43.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.916+0000 7f52d0183640 1 -- 172.21.15.38:0/2336463342 shutdown_connections 2024-09-16T00:35:43.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:43.916+0000 7f52d0183640 1 -- 172.21.15.38:0/2336463342 wait complete. 2024-09-16T00:35:43.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:43 smithi122 ceph-mon[35412]: pgmap v452: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T00:35:44.045 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:43 smithi073 ceph-mon[35254]: pgmap v452: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2024-09-16T00:35:44.659 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:44 smithi038 ceph-mon[30945]: pgmap v453: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-09-16T00:35:44.659 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:44 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/2336463342' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:35:44.687 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-16T00:35:44.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:35:44.687 DEBUG:teuthology.orchestra.run.smithi038:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-16T00:35:44.729 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi038.front.sepia.ceph.com 2024-09-16T00:35:44.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:35:44.730 DEBUG:teuthology.orchestra.run.smithi038:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-16T00:35:44.787 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:35:44.788 DEBUG:teuthology.orchestra.run.smithi038:> ip netns list 2024-09-16T00:35:44.845 INFO:teuthology.orchestra.run.smithi038.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-16T00:35:44.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:35:44.846 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-16T00:35:44.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-16T00:35:44.912 DEBUG:teuthology.orchestra.run.smithi038:> sudo ip link delete ceph-brx 2024-09-16T00:35:44.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:44 smithi122 ceph-mon[35412]: pgmap v453: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-09-16T00:35:44.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:44 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/2336463342' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:35:45.003 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-16T00:35:45.015 INFO:tasks.cephadm:Teardown begin 2024-09-16T00:35:45.015 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-16T00:35:45.016 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:35:45.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:44 smithi073 ceph-mon[35254]: pgmap v453: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2024-09-16T00:35:45.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:44 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/2336463342' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-16T00:35:45.276 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:35:45.765 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.732+0000 7f253d103640 1 -- 172.21.15.38:0/1597572340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25381082a0 msgr2=0x7f253810ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:45.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.732+0000 7f253d103640 1 --2- 172.21.15.38:0/1597572340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25381082a0 0x7f253810ec40 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f2530009930 tx=0x7f253002a420 comp rx=0 tx=0).stop 2024-09-16T00:35:45.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.732+0000 7f253d103640 1 -- 172.21.15.38:0/1597572340 shutdown_connections 2024-09-16T00:35:45.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.732+0000 7f253d103640 1 --2- 172.21.15.38:0/1597572340 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f25381082a0 0x7f253810ec40 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:45.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.732+0000 7f253d103640 1 --2- 172.21.15.38:0/1597572340 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25381036e0 0x7f2538107b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:45.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.732+0000 7f253d103640 1 --2- 172.21.15.38:0/1597572340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2538102d30 0x7f2538103110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:45.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.732+0000 7f253d103640 1 -- 172.21.15.38:0/1597572340 >> 172.21.15.38:0/1597572340 conn(0x7f25380fc7d0 msgr2=0x7f25380febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:45.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.732+0000 7f253d103640 1 -- 172.21.15.38:0/1597572340 shutdown_connections 2024-09-16T00:35:45.767 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.732+0000 7f253d103640 1 -- 172.21.15.38:0/1597572340 wait complete. 2024-09-16T00:35:45.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.733+0000 7f253d103640 1 Processor -- start 2024-09-16T00:35:45.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.733+0000 7f253d103640 1 -- start start 2024-09-16T00:35:45.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.733+0000 7f253d103640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2538102d30 0x7f2538198550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:45.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.733+0000 7f253d103640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25381036e0 0x7f2538198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:45.768 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.733+0000 7f253d103640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25381082a0 0x7f253819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:45.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.733+0000 7f253d103640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f25381120a0 con 0x7f2538102d30 2024-09-16T00:35:45.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.733+0000 7f253d103640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f2538111f20 con 0x7f25381082a0 2024-09-16T00:35:45.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.733+0000 7f253d103640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f2538112220 con 0x7f25381036e0 2024-09-16T00:35:45.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.733+0000 7f2536d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2538102d30 0x7f2538198550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:45.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.733+0000 7f2536d9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2538102d30 0x7f2538198550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:43072/0 (socket says 172.21.15.38:43072) 2024-09-16T00:35:45.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.733+0000 7f2536d9d640 1 -- 172.21.15.38:0/251467460 learned_addr learned my addr 172.21.15.38:0/251467460 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:35:45.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.734+0000 7f253759e640 1 --2- 172.21.15.38:0/251467460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25381082a0 0x7f253819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:45.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.734+0000 7f253659c640 1 --2- 172.21.15.38:0/251467460 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25381036e0 0x7f2538198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:45.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.734+0000 7f2536d9d640 1 -- 172.21.15.38:0/251467460 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25381036e0 msgr2=0x7f2538198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:45.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.734+0000 7f2536d9d640 1 --2- 172.21.15.38:0/251467460 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25381036e0 0x7f2538198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:45.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.734+0000 7f2536d9d640 1 -- 172.21.15.38:0/251467460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25381082a0 msgr2=0x7f253819ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:45.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.734+0000 7f2536d9d640 1 --2- 172.21.15.38:0/251467460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25381082a0 0x7f253819ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:45.771 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.734+0000 7f2536d9d640 1 -- 172.21.15.38:0/251467460 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f253819d500 con 0x7f2538102d30 2024-09-16T00:35:45.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.734+0000 7f2536d9d640 1 --2- 172.21.15.38:0/251467460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2538102d30 0x7f2538198550 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f252800eab0 tx=0x7f252800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:45.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.734+0000 7f251ffff640 1 -- 172.21.15.38:0/251467460 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f252800ce40 con 0x7f2538102d30 2024-09-16T00:35:45.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.734+0000 7f253d103640 1 -- 172.21.15.38:0/251467460 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f253819d7f0 con 0x7f2538102d30 2024-09-16T00:35:45.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.734+0000 7f251ffff640 1 -- 172.21.15.38:0/251467460 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f2528011950 con 0x7f2538102d30 2024-09-16T00:35:45.772 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.734+0000 7f251ffff640 1 -- 172.21.15.38:0/251467460 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2528010ae0 con 0x7f2538102d30 2024-09-16T00:35:45.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.734+0000 7f253d103640 1 -- 172.21.15.38:0/251467460 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25381a50d0 con 0x7f2538102d30 2024-09-16T00:35:45.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.735+0000 7f253d103640 1 -- 172.21.15.38:0/251467460 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2538103110 con 0x7f2538102d30 2024-09-16T00:35:45.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.736+0000 7f251ffff640 1 -- 172.21.15.38:0/251467460 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f2528011b00 con 0x7f2538102d30 2024-09-16T00:35:45.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.737+0000 7f251ffff640 1 --2- 172.21.15.38:0/251467460 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f25200a3240 0x7f25200a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:45.773 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.737+0000 7f251ffff640 1 -- 172.21.15.38:0/251467460 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f25280d03e0 con 0x7f2538102d30 2024-09-16T00:35:45.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.737+0000 7f253659c640 1 --2- 172.21.15.38:0/251467460 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f25200a3240 0x7f25200a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:45.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.738+0000 7f253659c640 1 --2- 172.21.15.38:0/251467460 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f25200a3240 0x7f25200a5700 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f252c005d70 tx=0x7f252c005d00 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:45.774 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.741+0000 7f251ffff640 1 -- 172.21.15.38:0/251467460 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f2528016030 con 0x7f2538102d30 2024-09-16T00:35:45.955 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.955+0000 7f253d103640 1 -- 172.21.15.38:0/251467460 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f253810a2b0 con 0x7f2538102d30 2024-09-16T00:35:45.956 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.955+0000 7f251ffff640 1 -- 172.21.15.38:0/251467460 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f252809cd20 con 0x7f2538102d30 2024-09-16T00:35:45.956 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:35:45.956 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"btime":"2024-09-16T00:32:41:853514+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/367974527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":367974527},{"type":"v1","addr":"172.21.15.38:6835","nonce":367974527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/335077136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":335077136},{"type":"v1","addr":"172.21.15.38:6837","nonce":335077136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/3501211160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3501211160},{"type":"v1","addr":"172.21.15.38:6839","nonce":3501211160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6841/2623130915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":2623130915},{"type":"v1","addr":"172.21.15.38:6841","nonce":2623130915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/412277519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":412277519},{"type":"v1","addr":"172.21.15.122:6833","nonce":412277519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3987234757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3987234757},{"type":"v1","addr":"172.21.15.122:6835","nonce":3987234757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:30:26.764565+0000","modified":"2024-09-16T00:32:41.355828+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24440},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1210328177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1210328177},{"type":"v1","addr":"172.21.15.73:6833","nonce":1210328177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6837/1858299680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1858299680},{"type":"v1","addr":"172.21.15.122:6837","nonce":1858299680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1983975311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1983975311},{"type":"v1","addr":"172.21.15.122:6839","nonce":1983975311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.73:6835/2866363113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2866363113},{"type":"v1","addr":"172.21.15.73:6835","nonce":2866363113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.73:6837/2681081265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2681081265},{"type":"v1","addr":"172.21.15.73:6837","nonce":2681081265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.73:6839/2396224525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2396224525},{"type":"v1","addr":"172.21.15.73:6839","nonce":2396224525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24464,24440]},"id":1}]} 2024-09-16T00:35:45.957 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 26 2024-09-16T00:35:45.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.958+0000 7f253d103640 1 -- 172.21.15.38:0/251467460 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f25200a3240 msgr2=0x7f25200a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:45.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.958+0000 7f253d103640 1 --2- 172.21.15.38:0/251467460 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f25200a3240 0x7f25200a5700 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f252c005d70 tx=0x7f252c005d00 comp rx=0 tx=0).stop 2024-09-16T00:35:45.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.958+0000 7f253d103640 1 -- 172.21.15.38:0/251467460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2538102d30 msgr2=0x7f2538198550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:45.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.958+0000 7f253d103640 1 --2- 172.21.15.38:0/251467460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2538102d30 0x7f2538198550 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f252800eab0 tx=0x7f252800ef70 comp rx=0 tx=0).stop 2024-09-16T00:35:45.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.958+0000 7f253d103640 1 -- 172.21.15.38:0/251467460 shutdown_connections 2024-09-16T00:35:45.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.958+0000 7f253d103640 1 --2- 172.21.15.38:0/251467460 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f25200a3240 0x7f25200a5700 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:45.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.958+0000 7f253d103640 1 --2- 172.21.15.38:0/251467460 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25381082a0 0x7f253819ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:45.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.958+0000 7f253d103640 1 --2- 172.21.15.38:0/251467460 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f25381036e0 0x7f2538198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:45.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.958+0000 7f253d103640 1 --2- 172.21.15.38:0/251467460 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f2538102d30 0x7f2538198550 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:45.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.958+0000 7f253d103640 1 -- 172.21.15.38:0/251467460 >> 172.21.15.38:0/251467460 conn(0x7f25380fc7d0 msgr2=0x7f2538106380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:45.959 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.958+0000 7f253d103640 1 -- 172.21.15.38:0/251467460 shutdown_connections 2024-09-16T00:35:45.960 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:45.958+0000 7f253d103640 1 -- 172.21.15.38:0/251467460 wait complete. 2024-09-16T00:35:46.712 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:46 smithi038 ceph-mon[30945]: pgmap v454: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-16T00:35:46.713 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:46 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/251467460' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:35:46.744 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs dump --format=json 2024-09-16T00:35:46.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:46 smithi122 ceph-mon[35412]: pgmap v454: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-16T00:35:46.967 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:46 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/251467460' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:35:47.004 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:35:47.043 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:46 smithi073 ceph-mon[35254]: pgmap v454: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 6 op/s 2024-09-16T00:35:47.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:46 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/251467460' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:35:47.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.541+0000 7f0b6367e640 1 -- 172.21.15.38:0/1045874219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b5c10e480 msgr2=0x7f0b5c110870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:47.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.541+0000 7f0b6367e640 1 --2- 172.21.15.38:0/1045874219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b5c10e480 0x7f0b5c110870 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f0b54009900 tx=0x7f0b5402a410 comp rx=0 tx=0).stop 2024-09-16T00:35:47.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.541+0000 7f0b6367e640 1 -- 172.21.15.38:0/1045874219 shutdown_connections 2024-09-16T00:35:47.567 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.541+0000 7f0b6367e640 1 --2- 172.21.15.38:0/1045874219 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b5c10e480 0x7f0b5c110870 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:47.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.541+0000 7f0b6367e640 1 --2- 172.21.15.38:0/1045874219 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b5c101280 0x7f0b5c10df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:47.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.541+0000 7f0b6367e640 1 --2- 172.21.15.38:0/1045874219 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b5c1008d0 0x7f0b5c100cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:47.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.541+0000 7f0b6367e640 1 -- 172.21.15.38:0/1045874219 >> 172.21.15.38:0/1045874219 conn(0x7f0b5c0fc7a0 msgr2=0x7f0b5c0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:47.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.541+0000 7f0b6367e640 1 -- 172.21.15.38:0/1045874219 shutdown_connections 2024-09-16T00:35:47.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.541+0000 7f0b6367e640 1 -- 172.21.15.38:0/1045874219 wait complete. 2024-09-16T00:35:47.568 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.541+0000 7f0b6367e640 1 Processor -- start 2024-09-16T00:35:47.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.541+0000 7f0b6367e640 1 -- start start 2024-09-16T00:35:47.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.542+0000 7f0b6367e640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b5c1008d0 0x7f0b5c19a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:47.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.542+0000 7f0b6367e640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b5c101280 0x7f0b5c19ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:47.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.542+0000 7f0b6367e640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b5c10e480 0x7f0b5c19efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:47.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.542+0000 7f0b6367e640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0b5c1141c0 con 0x7f0b5c1008d0 2024-09-16T00:35:47.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.542+0000 7f0b6367e640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0b5c114040 con 0x7f0b5c10e480 2024-09-16T00:35:47.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.542+0000 7f0b6367e640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0b5c114340 con 0x7f0b5c101280 2024-09-16T00:35:47.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.542+0000 7f0b6141a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b5c1008d0 0x7f0b5c19a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:47.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.542+0000 7f0b6141a640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b5c1008d0 0x7f0b5c19a6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54022/0 (socket says 172.21.15.38:54022) 2024-09-16T00:35:47.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.542+0000 7f0b6141a640 1 -- 172.21.15.38:0/3579684568 learned_addr learned my addr 172.21.15.38:0/3579684568 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:35:47.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.542+0000 7f0b61c1b640 1 --2- 172.21.15.38:0/3579684568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b5c10e480 0x7f0b5c19efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:47.569 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.542+0000 7f0b60c19640 1 --2- 172.21.15.38:0/3579684568 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b5c101280 0x7f0b5c19ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:47.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.542+0000 7f0b6141a640 1 -- 172.21.15.38:0/3579684568 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b5c101280 msgr2=0x7f0b5c19ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:47.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.542+0000 7f0b6141a640 1 --2- 172.21.15.38:0/3579684568 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b5c101280 0x7f0b5c19ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:47.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.542+0000 7f0b6141a640 1 -- 172.21.15.38:0/3579684568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b5c10e480 msgr2=0x7f0b5c19efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:47.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.542+0000 7f0b6141a640 1 --2- 172.21.15.38:0/3579684568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b5c10e480 0x7f0b5c19efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:47.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.542+0000 7f0b6141a640 1 -- 172.21.15.38:0/3579684568 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b5c19f690 con 0x7f0b5c1008d0 2024-09-16T00:35:47.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.542+0000 7f0b61c1b640 1 --2- 172.21.15.38:0/3579684568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b5c10e480 0x7f0b5c19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:35:47.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.542+0000 7f0b60c19640 1 --2- 172.21.15.38:0/3579684568 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b5c101280 0x7f0b5c19ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:35:47.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.542+0000 7f0b6141a640 1 --2- 172.21.15.38:0/3579684568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b5c1008d0 0x7f0b5c19a6e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c00c890 tx=0x7f0b4c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:47.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.543+0000 7f0b5a7fc640 1 -- 172.21.15.38:0/3579684568 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c004220 con 0x7f0b5c1008d0 2024-09-16T00:35:47.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.543+0000 7f0b5a7fc640 1 -- 172.21.15.38:0/3579684568 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0b4c0043c0 con 0x7f0b5c1008d0 2024-09-16T00:35:47.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.543+0000 7f0b6367e640 1 -- 172.21.15.38:0/3579684568 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b5c19f980 con 0x7f0b5c1008d0 2024-09-16T00:35:47.570 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.543+0000 7f0b5a7fc640 1 -- 172.21.15.38:0/3579684568 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c0049e0 con 0x7f0b5c1008d0 2024-09-16T00:35:47.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.543+0000 7f0b6367e640 1 -- 172.21.15.38:0/3579684568 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b5c1a7260 con 0x7f0b5c1008d0 2024-09-16T00:35:47.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.544+0000 7f0b6367e640 1 -- 172.21.15.38:0/3579684568 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b5c100cb0 con 0x7f0b5c1008d0 2024-09-16T00:35:47.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.546+0000 7f0b5a7fc640 1 -- 172.21.15.38:0/3579684568 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f0b4c0079b0 con 0x7f0b5c1008d0 2024-09-16T00:35:47.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.546+0000 7f0b5a7fc640 1 --2- 172.21.15.38:0/3579684568 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f0b440a3240 0x7f0b440a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:47.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.547+0000 7f0b5a7fc640 1 -- 172.21.15.38:0/3579684568 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7f0b4c0cfdf0 con 0x7f0b5c1008d0 2024-09-16T00:35:47.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.547+0000 7f0b60c19640 1 --2- 172.21.15.38:0/3579684568 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f0b440a3240 0x7f0b440a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:47.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.547+0000 7f0b60c19640 1 --2- 172.21.15.38:0/3579684568 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f0b440a3240 0x7f0b440a5700 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c1102c0 tx=0x7f0b500073c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:47.571 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.550+0000 7f0b5a7fc640 1 -- 172.21.15.38:0/3579684568 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0b4c014030 con 0x7f0b5c1008d0 2024-09-16T00:35:47.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.765+0000 7f0b6367e640 1 -- 172.21.15.38:0/3579684568 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0b5c19ba40 con 0x7f0b5c1008d0 2024-09-16T00:35:47.766 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.766+0000 7f0b5a7fc640 1 -- 172.21.15.38:0/3579684568 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+11909 (secure 0 0 0) 0x7f0b4c09c790 con 0x7f0b5c1008d0 2024-09-16T00:35:47.767 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:35:47.767 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":26,"btime":"2024-09-16T00:32:41:853514+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14535,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6835/367974527","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6834","nonce":367974527},{"type":"v1","addr":"172.21.15.38:6835","nonce":367974527}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14547,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6837/335077136","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6836","nonce":335077136},{"type":"v1","addr":"172.21.15.38:6837","nonce":335077136}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14604,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6839/3501211160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6838","nonce":3501211160},{"type":"v1","addr":"172.21.15.38:6839","nonce":3501211160}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14616,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.38:6841/2623130915","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6840","nonce":2623130915},{"type":"v1","addr":"172.21.15.38:6841","nonce":2623130915}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24388,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6833/412277519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6832","nonce":412277519},{"type":"v1","addr":"172.21.15.122:6833","nonce":412277519}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.122:6835/3987234757","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6834","nonce":3987234757},{"type":"v1","addr":"172.21.15.122:6835","nonce":3987234757}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-16T00:30:26.764565+0000","modified":"2024-09-16T00:32:41.355828+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24494,"mds_1":24464,"mds_2":24440},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24416":{"gid":24416,"name":"k","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.73:6833/1210328177","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":1210328177},{"type":"v1","addr":"172.21.15.73:6833","nonce":1210328177}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6837/1858299680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6836","nonce":1858299680},{"type":"v1","addr":"172.21.15.122:6837","nonce":1858299680}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"i","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.122:6839/1983975311","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6838","nonce":1983975311},{"type":"v1","addr":"172.21.15.122:6839","nonce":1983975311}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24440":{"gid":24440,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":18,"addr":"172.21.15.73:6835/2866363113","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2866363113},{"type":"v1","addr":"172.21.15.73:6835","nonce":2866363113}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24464":{"gid":24464,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":16,"addr":"172.21.15.73:6837/2681081265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":2681081265},{"type":"v1","addr":"172.21.15.73:6837","nonce":2681081265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24494":{"gid":24494,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.73:6839/2396224525","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6838","nonce":2396224525},{"type":"v1","addr":"172.21.15.73:6839","nonce":2396224525}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24494,"qdb_cluster":[24494,24464,24440]},"id":1}]} 2024-09-16T00:35:47.767 INFO:teuthology.orchestra.run.smithi038.stderr:dumped fsmap epoch 26 2024-09-16T00:35:47.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.768+0000 7f0b6367e640 1 -- 172.21.15.38:0/3579684568 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f0b440a3240 msgr2=0x7f0b440a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:47.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.768+0000 7f0b6367e640 1 --2- 172.21.15.38:0/3579684568 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f0b440a3240 0x7f0b440a5700 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c1102c0 tx=0x7f0b500073c0 comp rx=0 tx=0).stop 2024-09-16T00:35:47.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.768+0000 7f0b6367e640 1 -- 172.21.15.38:0/3579684568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b5c1008d0 msgr2=0x7f0b5c19a6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:47.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.768+0000 7f0b6367e640 1 --2- 172.21.15.38:0/3579684568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b5c1008d0 0x7f0b5c19a6e0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c00c890 tx=0x7f0b4c00cd50 comp rx=0 tx=0).stop 2024-09-16T00:35:47.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.768+0000 7f0b6367e640 1 -- 172.21.15.38:0/3579684568 shutdown_connections 2024-09-16T00:35:47.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.768+0000 7f0b6367e640 1 --2- 172.21.15.38:0/3579684568 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f0b440a3240 0x7f0b440a5700 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:47.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.768+0000 7f0b6367e640 1 --2- 172.21.15.38:0/3579684568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b5c10e480 0x7f0b5c19efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:47.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.768+0000 7f0b6367e640 1 --2- 172.21.15.38:0/3579684568 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0b5c101280 0x7f0b5c19ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:47.769 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.768+0000 7f0b6367e640 1 --2- 172.21.15.38:0/3579684568 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0b5c1008d0 0x7f0b5c19a6e0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:47.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.768+0000 7f0b6367e640 1 -- 172.21.15.38:0/3579684568 >> 172.21.15.38:0/3579684568 conn(0x7f0b5c0fc7a0 msgr2=0x7f0b5c0feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:47.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.769+0000 7f0b6367e640 1 -- 172.21.15.38:0/3579684568 shutdown_connections 2024-09-16T00:35:47.770 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:47.769+0000 7f0b6367e640 1 -- 172.21.15.38:0/3579684568 wait complete. 2024-09-16T00:35:48.617 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd dump --format=json 2024-09-16T00:35:48.881 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:35:48.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:48 smithi038 ceph-mon[30945]: pgmap v455: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-16T00:35:48.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:48 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/3579684568' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:35:48.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:48 smithi122 ceph-mon[35412]: pgmap v455: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-16T00:35:48.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:48 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/3579684568' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:35:49.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:48 smithi073 ceph-mon[35254]: pgmap v455: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2024-09-16T00:35:49.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:48 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/3579684568' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-09-16T00:35:49.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.357+0000 7efec2bbb640 1 -- 172.21.15.38:0/2053323227 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efebc100a80 msgr2=0x7efebc100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:49.410 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.357+0000 7efec2bbb640 1 --2- 172.21.15.38:0/2053323227 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efebc100a80 0x7efebc100ee0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7efeb0009930 tx=0x7efeb002a420 comp rx=0 tx=0).stop 2024-09-16T00:35:49.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.358+0000 7efec2bbb640 1 -- 172.21.15.38:0/2053323227 shutdown_connections 2024-09-16T00:35:49.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.358+0000 7efec2bbb640 1 --2- 172.21.15.38:0/2053323227 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efebc101420 0x7efebc10d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:49.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.358+0000 7efec2bbb640 1 --2- 172.21.15.38:0/2053323227 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efebc100a80 0x7efebc100ee0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:49.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.358+0000 7efec2bbb640 1 --2- 172.21.15.38:0/2053323227 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efebc106a80 0x7efebc106e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:49.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.358+0000 7efec2bbb640 1 -- 172.21.15.38:0/2053323227 >> 172.21.15.38:0/2053323227 conn(0x7efebc0fc7a0 msgr2=0x7efebc0febc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:49.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.358+0000 7efec2bbb640 1 -- 172.21.15.38:0/2053323227 shutdown_connections 2024-09-16T00:35:49.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.358+0000 7efec2bbb640 1 -- 172.21.15.38:0/2053323227 wait complete. 2024-09-16T00:35:49.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.358+0000 7efec2bbb640 1 Processor -- start 2024-09-16T00:35:49.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.358+0000 7efec2bbb640 1 -- start start 2024-09-16T00:35:49.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.358+0000 7efec2bbb640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efebc100a80 0x7efebc199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:49.411 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.358+0000 7efec2bbb640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efebc101420 0x7efebc199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:49.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.358+0000 7efec2bbb640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efebc106a80 0x7efebc1a43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:49.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.358+0000 7efec2bbb640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7efebc110ea0 con 0x7efebc100a80 2024-09-16T00:35:49.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.358+0000 7efec2bbb640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7efebc110d20 con 0x7efebc106a80 2024-09-16T00:35:49.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.358+0000 7efec2bbb640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7efebc111020 con 0x7efebc101420 2024-09-16T00:35:49.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.359+0000 7efec0957640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efebc100a80 0x7efebc199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:49.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.359+0000 7efec0957640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efebc100a80 0x7efebc199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54030/0 (socket says 172.21.15.38:54030) 2024-09-16T00:35:49.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.359+0000 7efec0957640 1 -- 172.21.15.38:0/373233773 learned_addr learned my addr 172.21.15.38:0/373233773 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:35:49.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.359+0000 7efec1158640 1 --2- 172.21.15.38:0/373233773 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efebc106a80 0x7efebc1a43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:49.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.359+0000 7efebbfff640 1 --2- 172.21.15.38:0/373233773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efebc101420 0x7efebc199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:49.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.359+0000 7efec0957640 1 -- 172.21.15.38:0/373233773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efebc101420 msgr2=0x7efebc199870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:49.412 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.359+0000 7efec0957640 1 --2- 172.21.15.38:0/373233773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efebc101420 0x7efebc199870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:49.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.359+0000 7efec0957640 1 -- 172.21.15.38:0/373233773 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efebc106a80 msgr2=0x7efebc1a43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:49.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.359+0000 7efec0957640 1 --2- 172.21.15.38:0/373233773 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efebc106a80 0x7efebc1a43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:49.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.359+0000 7efec0957640 1 -- 172.21.15.38:0/373233773 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efebc1a48f0 con 0x7efebc100a80 2024-09-16T00:35:49.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.359+0000 7efec1158640 1 --2- 172.21.15.38:0/373233773 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efebc106a80 0x7efebc1a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:35:49.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.359+0000 7efebbfff640 1 --2- 172.21.15.38:0/373233773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efebc101420 0x7efebc199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:35:49.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.359+0000 7efec0957640 1 --2- 172.21.15.38:0/373233773 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efebc100a80 0x7efebc199330 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7efeac00d950 tx=0x7efeac00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:49.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.359+0000 7efeb9ffb640 1 -- 172.21.15.38:0/373233773 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac017070 con 0x7efebc100a80 2024-09-16T00:35:49.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.359+0000 7efec2bbb640 1 -- 172.21.15.38:0/373233773 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efebc1a4ac0 con 0x7efebc100a80 2024-09-16T00:35:49.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.359+0000 7efeb9ffb640 1 -- 172.21.15.38:0/373233773 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7efeac009c30 con 0x7efebc100a80 2024-09-16T00:35:49.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.359+0000 7efeb9ffb640 1 -- 172.21.15.38:0/373233773 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeac00f6b0 con 0x7efebc100a80 2024-09-16T00:35:49.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.359+0000 7efec2bbb640 1 -- 172.21.15.38:0/373233773 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efebc1a5000 con 0x7efebc100a80 2024-09-16T00:35:49.413 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.360+0000 7efec2bbb640 1 -- 172.21.15.38:0/373233773 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efebc100ee0 con 0x7efebc100a80 2024-09-16T00:35:49.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.362+0000 7efeb9ffb640 1 -- 172.21.15.38:0/373233773 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7efeac0109d0 con 0x7efebc100a80 2024-09-16T00:35:49.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.362+0000 7efeb9ffb640 1 --2- 172.21.15.38:0/373233773 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efea40a3240 0x7efea40a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:49.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.362+0000 7efebbfff640 1 --2- 172.21.15.38:0/373233773 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efea40a3240 0x7efea40a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:49.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.362+0000 7efeb9ffb640 1 -- 172.21.15.38:0/373233773 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7efeac0cf960 con 0x7efebc100a80 2024-09-16T00:35:49.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.363+0000 7efebbfff640 1 --2- 172.21.15.38:0/373233773 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efea40a3240 0x7efea40a5700 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7efeb0000c00 tx=0x7efeb0032c70 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:49.414 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.365+0000 7efeb9ffb640 1 -- 172.21.15.38:0/373233773 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7efeac009de0 con 0x7efebc100a80 2024-09-16T00:35:49.545 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.545+0000 7efec2bbb640 1 -- 172.21.15.38:0/373233773 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efebc06ad80 con 0x7efebc100a80 2024-09-16T00:35:49.546 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.546+0000 7efeb9ffb640 1 -- 172.21.15.38:0/373233773 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) ==== 74+0+24767 (secure 0 0 0) 0x7efeac09c300 con 0x7efebc100a80 2024-09-16T00:35:49.547 INFO:teuthology.orchestra.run.smithi038.stdout: 2024-09-16T00:35:49.547 INFO:teuthology.orchestra.run.smithi038.stdout:{"epoch":92,"fsid":"983b2072-73c1-11ef-bceb-c7b262605968","created":"2024-09-16T00:22:34.897053+0000","modified":"2024-09-16T00:32:24.296695+0000","last_up_change":"2024-09-16T00:29:34.503134+0000","last_in_change":"2024-09-16T00:29:10.135656+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-16T00:25:41.931977+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-16T00:30:22.741889+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-16T00:30:25.042590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-09-16T00:30:31.554350+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"3afc20e3-869f-43a9-a911-b0e8d07de86f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6802","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6803","nonce":1622028921}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6804","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6805","nonce":1622028921}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6808","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6809","nonce":1622028921}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6806","nonce":1622028921},{"type":"v1","addr":"172.21.15.38:6807","nonce":1622028921}]},"public_addr":"172.21.15.38:6803/1622028921","cluster_addr":"172.21.15.38:6805/1622028921","heartbeat_back_addr":"172.21.15.38:6809/1622028921","heartbeat_front_addr":"172.21.15.38:6807/1622028921","state":["exists","up"]},{"osd":1,"uuid":"6d9d7acb-45ed-4543-8afc-6433fc7f096f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6801","nonce":3811690336}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6803","nonce":3811690336}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6807","nonce":3811690336}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":3811690336},{"type":"v1","addr":"172.21.15.73:6805","nonce":3811690336}]},"public_addr":"172.21.15.73:6801/3811690336","cluster_addr":"172.21.15.73:6803/3811690336","heartbeat_back_addr":"172.21.15.73:6807/3811690336","heartbeat_front_addr":"172.21.15.73:6805/3811690336","state":["exists","up"]},{"osd":2,"uuid":"48690f35-99e9-4f45-9bee-487857caa8b8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6800","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6801","nonce":1754741655}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6802","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6803","nonce":1754741655}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6806","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6807","nonce":1754741655}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6804","nonce":1754741655},{"type":"v1","addr":"172.21.15.122:6805","nonce":1754741655}]},"public_addr":"172.21.15.122:6801/1754741655","cluster_addr":"172.21.15.122:6803/1754741655","heartbeat_back_addr":"172.21.15.122:6807/1754741655","heartbeat_front_addr":"172.21.15.122:6805/1754741655","state":["exists","up"]},{"osd":3,"uuid":"28ebb4a0-92e7-4f57-82a5-b9b761eca269","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6810","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6811","nonce":3523190079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6812","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6813","nonce":3523190079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6816","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6817","nonce":3523190079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6814","nonce":3523190079},{"type":"v1","addr":"172.21.15.38:6815","nonce":3523190079}]},"public_addr":"172.21.15.38:6811/3523190079","cluster_addr":"172.21.15.38:6813/3523190079","heartbeat_back_addr":"172.21.15.38:6817/3523190079","heartbeat_front_addr":"172.21.15.38:6815/3523190079","state":["exists","up"]},{"osd":4,"uuid":"5ea363f7-7ad8-45ed-b7e4-4b010fe7f8cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6809","nonce":1469927628}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6811","nonce":1469927628}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6815","nonce":1469927628}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":1469927628},{"type":"v1","addr":"172.21.15.73:6813","nonce":1469927628}]},"public_addr":"172.21.15.73:6809/1469927628","cluster_addr":"172.21.15.73:6811/1469927628","heartbeat_back_addr":"172.21.15.73:6815/1469927628","heartbeat_front_addr":"172.21.15.73:6813/1469927628","state":["exists","up"]},{"osd":5,"uuid":"e3c66d43-dd28-4bb1-a886-62a869be82cb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":37,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6808","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6809","nonce":2954413058}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6810","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6811","nonce":2954413058}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6814","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6815","nonce":2954413058}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6812","nonce":2954413058},{"type":"v1","addr":"172.21.15.122:6813","nonce":2954413058}]},"public_addr":"172.21.15.122:6809/2954413058","cluster_addr":"172.21.15.122:6811/2954413058","heartbeat_back_addr":"172.21.15.122:6815/2954413058","heartbeat_front_addr":"172.21.15.122:6813/2954413058","state":["exists","up"]},{"osd":6,"uuid":"211497ac-7c48-4095-bfdb-1d4df0149f57","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":44,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6818","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6819","nonce":1415719550}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6820","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6821","nonce":1415719550}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6824","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6825","nonce":1415719550}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6822","nonce":1415719550},{"type":"v1","addr":"172.21.15.38:6823","nonce":1415719550}]},"public_addr":"172.21.15.38:6819/1415719550","cluster_addr":"172.21.15.38:6821/1415719550","heartbeat_back_addr":"172.21.15.38:6825/1415719550","heartbeat_front_addr":"172.21.15.38:6823/1415719550","state":["exists","up"]},{"osd":7,"uuid":"06512315-3f5e-44f8-976d-d3f5f1ba00e3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":49,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6817","nonce":2634361473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6819","nonce":2634361473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6823","nonce":2634361473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":2634361473},{"type":"v1","addr":"172.21.15.73:6821","nonce":2634361473}]},"public_addr":"172.21.15.73:6817/2634361473","cluster_addr":"172.21.15.73:6819/2634361473","heartbeat_back_addr":"172.21.15.73:6823/2634361473","heartbeat_front_addr":"172.21.15.73:6821/2634361473","state":["exists","up"]},{"osd":8,"uuid":"db703c0a-1f87-46f0-be5f-f826f7f9daac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":57,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6816","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6817","nonce":1800567633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6818","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6819","nonce":1800567633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6822","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6823","nonce":1800567633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6820","nonce":1800567633},{"type":"v1","addr":"172.21.15.122:6821","nonce":1800567633}]},"public_addr":"172.21.15.122:6817/1800567633","cluster_addr":"172.21.15.122:6819/1800567633","heartbeat_back_addr":"172.21.15.122:6823/1800567633","heartbeat_front_addr":"172.21.15.122:6821/1800567633","state":["exists","up"]},{"osd":9,"uuid":"f5384200-b3b4-4936-85e1-fc57a9fe2bb8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6826","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6827","nonce":445727833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6828","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6829","nonce":445727833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6832","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6833","nonce":445727833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.38:6830","nonce":445727833},{"type":"v1","addr":"172.21.15.38:6831","nonce":445727833}]},"public_addr":"172.21.15.38:6827/445727833","cluster_addr":"172.21.15.38:6829/445727833","heartbeat_back_addr":"172.21.15.38:6833/445727833","heartbeat_front_addr":"172.21.15.38:6831/445727833","state":["exists","up"]},{"osd":10,"uuid":"c22883d9-6672-4796-a961-963d21eb37fc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6825","nonce":43297139}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6827","nonce":43297139}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6831","nonce":43297139}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":43297139},{"type":"v1","addr":"172.21.15.73:6829","nonce":43297139}]},"public_addr":"172.21.15.73:6825/43297139","cluster_addr":"172.21.15.73:6827/43297139","heartbeat_back_addr":"172.21.15.73:6831/43297139","heartbeat_front_addr":"172.21.15.73:6829/43297139","state":["exists","up"]},{"osd":11,"uuid":"9f98e8cb-1bff-4b8b-aa19-56b691387383","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":76,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6824","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6825","nonce":1886446061}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6826","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6827","nonce":1886446061}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6830","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6831","nonce":1886446061}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.122:6828","nonce":1886446061},{"type":"v1","addr":"172.21.15.122:6829","nonce":1886446061}]},"public_addr":"172.21.15.122:6825/1886446061","cluster_addr":"172.21.15.122:6827/1886446061","heartbeat_back_addr":"172.21.15.122:6831/1886446061","heartbeat_front_addr":"172.21.15.122:6829/1886446061","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:24:47.384012+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:12.834488+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:25:38.770055+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:04.800466+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:30.808076+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:26:56.646967+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:27:22.721270+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:27:48.510011+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:28:13.922944+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:28:40.514947+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:29:06.031052+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-09-16T00:29:31.882062+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.13","mappings":[{"from":2,"to":11}]},{"pgid":"2.16","mappings":[{"from":2,"to":4}]},{"pgid":"3.7","mappings":[{"from":3,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8},{"from":5,"to":9}]},{"pgid":"3.21","mappings":[{"from":4,"to":6}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.f","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.17","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.38:6801/1779324916":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/924150447":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/141237849":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/2302081750":"2024-09-17T00:22:56.844212+0000","172.21.15.38:0/2245468594":"2024-09-17T00:22:56.844212+0000","172.21.15.38:6801/583006373":"2024-09-17T00:22:56.844212+0000","172.21.15.38:0/2435492667":"2024-09-17T00:23:21.417483+0000","172.21.15.38:6800/583006373":"2024-09-17T00:22:56.844212+0000","172.21.15.38:6800/1779324916":"2024-09-17T00:23:21.417483+0000","172.21.15.38:0/1103516837":"2024-09-17T00:22:56.844212+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-16T00:35:49.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.548+0000 7efec2bbb640 1 -- 172.21.15.38:0/373233773 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efea40a3240 msgr2=0x7efea40a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:49.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.548+0000 7efec2bbb640 1 --2- 172.21.15.38:0/373233773 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efea40a3240 0x7efea40a5700 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7efeb0000c00 tx=0x7efeb0032c70 comp rx=0 tx=0).stop 2024-09-16T00:35:49.549 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.548+0000 7efec2bbb640 1 -- 172.21.15.38:0/373233773 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efebc100a80 msgr2=0x7efebc199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:49.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.548+0000 7efec2bbb640 1 --2- 172.21.15.38:0/373233773 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efebc100a80 0x7efebc199330 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7efeac00d950 tx=0x7efeac00de10 comp rx=0 tx=0).stop 2024-09-16T00:35:49.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.549+0000 7efec2bbb640 1 -- 172.21.15.38:0/373233773 shutdown_connections 2024-09-16T00:35:49.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.549+0000 7efec2bbb640 1 --2- 172.21.15.38:0/373233773 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7efea40a3240 0x7efea40a5700 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:49.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.549+0000 7efec2bbb640 1 --2- 172.21.15.38:0/373233773 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efebc106a80 0x7efebc1a43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:49.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.549+0000 7efec2bbb640 1 --2- 172.21.15.38:0/373233773 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7efebc101420 0x7efebc199870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:49.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.549+0000 7efec2bbb640 1 --2- 172.21.15.38:0/373233773 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7efebc100a80 0x7efebc199330 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:49.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.549+0000 7efec2bbb640 1 -- 172.21.15.38:0/373233773 >> 172.21.15.38:0/373233773 conn(0x7efebc0fc7a0 msgr2=0x7efebc0fcf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:49.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.549+0000 7efec2bbb640 1 -- 172.21.15.38:0/373233773 shutdown_connections 2024-09-16T00:35:49.550 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:49.549+0000 7efec2bbb640 1 -- 172.21.15.38:0/373233773 wait complete. 2024-09-16T00:35:49.964 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:49 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/373233773' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:35:49.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:49 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/373233773' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:35:50.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:49 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/373233773' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-16T00:35:50.271 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs fail cephfs 2024-09-16T00:35:50.527 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:35:50.911 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:50 smithi038 ceph-mon[30945]: pgmap v456: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-16T00:35:50.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:50 smithi122 ceph-mon[35412]: pgmap v456: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-16T00:35:51.019 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:50.995+0000 7ff889e6d640 1 -- 172.21.15.38:0/1135019251 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff884106a60 msgr2=0x7ff884106e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:51.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:50.995+0000 7ff889e6d640 1 --2- 172.21.15.38:0/1135019251 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff884106a60 0x7ff884106e40 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ff874009930 tx=0x7ff87402a420 comp rx=0 tx=0).stop 2024-09-16T00:35:51.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:50.995+0000 7ff889e6d640 1 -- 172.21.15.38:0/1135019251 shutdown_connections 2024-09-16T00:35:51.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:50.995+0000 7ff889e6d640 1 --2- 172.21.15.38:0/1135019251 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff884101400 0x7ff88410d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:51.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:50.995+0000 7ff889e6d640 1 --2- 172.21.15.38:0/1135019251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff884100a60 0x7ff884100ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:51.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:50.995+0000 7ff889e6d640 1 --2- 172.21.15.38:0/1135019251 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff884106a60 0x7ff884106e40 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:51.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:50.995+0000 7ff889e6d640 1 -- 172.21.15.38:0/1135019251 >> 172.21.15.38:0/1135019251 conn(0x7ff8840fc780 msgr2=0x7ff8840feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:51.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:50.995+0000 7ff889e6d640 1 -- 172.21.15.38:0/1135019251 shutdown_connections 2024-09-16T00:35:51.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:50.995+0000 7ff889e6d640 1 -- 172.21.15.38:0/1135019251 wait complete. 2024-09-16T00:35:51.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:50.995+0000 7ff889e6d640 1 Processor -- start 2024-09-16T00:35:51.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:50.996+0000 7ff889e6d640 1 -- start start 2024-09-16T00:35:51.020 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:50.996+0000 7ff889e6d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff884100a60 0x7ff884199280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:51.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:50.996+0000 7ff889e6d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff884101400 0x7ff8841997c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:51.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:50.996+0000 7ff889e6d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff884106a60 0x7ff8841a4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:51.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:50.996+0000 7ff889e6d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7ff884110e60 con 0x7ff884100a60 2024-09-16T00:35:51.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:50.996+0000 7ff889e6d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7ff884110ce0 con 0x7ff884106a60 2024-09-16T00:35:51.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:50.996+0000 7ff889e6d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7ff884110fe0 con 0x7ff884101400 2024-09-16T00:35:51.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:50.996+0000 7ff8837fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff884100a60 0x7ff884199280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:51.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:50.996+0000 7ff8837fe640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff884100a60 0x7ff884199280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54050/0 (socket says 172.21.15.38:54050) 2024-09-16T00:35:51.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:50.996+0000 7ff8837fe640 1 -- 172.21.15.38:0/1202983916 learned_addr learned my addr 172.21.15.38:0/1202983916 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:35:51.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:50.996+0000 7ff883fff640 1 --2- 172.21.15.38:0/1202983916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff884106a60 0x7ff8841a4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:51.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:50.996+0000 7ff8837fe640 1 -- 172.21.15.38:0/1202983916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff884101400 msgr2=0x7ff8841997c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:51.021 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:50.996+0000 7ff882ffd640 1 --2- 172.21.15.38:0/1202983916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff884101400 0x7ff8841997c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:51.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:50.996+0000 7ff8837fe640 1 --2- 172.21.15.38:0/1202983916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff884101400 0x7ff8841997c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:51.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:50.996+0000 7ff8837fe640 1 -- 172.21.15.38:0/1202983916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff884106a60 msgr2=0x7ff8841a4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:51.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:50.996+0000 7ff8837fe640 1 --2- 172.21.15.38:0/1202983916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff884106a60 0x7ff8841a4300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:51.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:50.996+0000 7ff8837fe640 1 -- 172.21.15.38:0/1202983916 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8841a4840 con 0x7ff884100a60 2024-09-16T00:35:51.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:50.996+0000 7ff883fff640 1 --2- 172.21.15.38:0/1202983916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff884106a60 0x7ff8841a4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:35:51.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:50.997+0000 7ff8837fe640 1 --2- 172.21.15.38:0/1202983916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff884100a60 0x7ff884199280 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff874009a50 tx=0x7ff874004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:51.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:50.997+0000 7ff880ff9640 1 -- 172.21.15.38:0/1202983916 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff87403f030 con 0x7ff884100a60 2024-09-16T00:35:51.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:50.997+0000 7ff880ff9640 1 -- 172.21.15.38:0/1202983916 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7ff874003ce0 con 0x7ff884100a60 2024-09-16T00:35:51.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:50.997+0000 7ff889e6d640 1 -- 172.21.15.38:0/1202983916 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8841a4a10 con 0x7ff884100a60 2024-09-16T00:35:51.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:50.997+0000 7ff880ff9640 1 -- 172.21.15.38:0/1202983916 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff874035070 con 0x7ff884100a60 2024-09-16T00:35:51.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:50.997+0000 7ff889e6d640 1 -- 172.21.15.38:0/1202983916 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8841a4ec0 con 0x7ff884100a60 2024-09-16T00:35:51.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:50.998+0000 7ff889e6d640 1 -- 172.21.15.38:0/1202983916 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff884100ec0 con 0x7ff884100a60 2024-09-16T00:35:51.022 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:50.999+0000 7ff880ff9640 1 -- 172.21.15.38:0/1202983916 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7ff874042030 con 0x7ff884100a60 2024-09-16T00:35:51.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:51.000+0000 7ff880ff9640 1 --2- 172.21.15.38:0/1202983916 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ff86c0a32c0 0x7ff86c0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:51.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:51.000+0000 7ff882ffd640 1 --2- 172.21.15.38:0/1202983916 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ff86c0a32c0 0x7ff86c0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:51.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:51.000+0000 7ff880ff9640 1 -- 172.21.15.38:0/1202983916 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9129+0+0 (secure 0 0 0) 0x7ff8740f0300 con 0x7ff884100a60 2024-09-16T00:35:51.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:51.000+0000 7ff882ffd640 1 --2- 172.21.15.38:0/1202983916 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ff86c0a32c0 0x7ff86c0a5780 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff88406b2e0 tx=0x7ff878005cc0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:51.023 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:51.004+0000 7ff880ff9640 1 -- 172.21.15.38:0/1202983916 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7ff8740bcc40 con 0x7ff884100a60 2024-09-16T00:35:51.043 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:50 smithi073 ceph-mon[35254]: pgmap v456: 129 pgs: 129 active+clean; 13 MiB data, 438 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 597 B/s wr, 8 op/s 2024-09-16T00:35:51.221 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:51.221+0000 7ff889e6d640 1 -- 172.21.15.38:0/1202983916 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ff884108fd0 con 0x7ff884100a60 2024-09-16T00:35:51.223 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:51.223+0000 7ff880ff9640 1 -- 172.21.15.38:0/1202983916 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) ==== 161+0+0 (secure 0 0 0) 0x7ff8740bc540 con 0x7ff884100a60 2024-09-16T00:35:51.223 INFO:teuthology.orchestra.run.smithi038.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-16T00:35:51.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:51.226+0000 7ff889e6d640 1 -- 172.21.15.38:0/1202983916 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ff86c0a32c0 msgr2=0x7ff86c0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:51.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:51.226+0000 7ff889e6d640 1 --2- 172.21.15.38:0/1202983916 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ff86c0a32c0 0x7ff86c0a5780 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff88406b2e0 tx=0x7ff878005cc0 comp rx=0 tx=0).stop 2024-09-16T00:35:51.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:51.226+0000 7ff889e6d640 1 -- 172.21.15.38:0/1202983916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff884100a60 msgr2=0x7ff884199280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:51.226 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:51.226+0000 7ff889e6d640 1 --2- 172.21.15.38:0/1202983916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff884100a60 0x7ff884199280 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff874009a50 tx=0x7ff874004270 comp rx=0 tx=0).stop 2024-09-16T00:35:51.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:51.226+0000 7ff889e6d640 1 -- 172.21.15.38:0/1202983916 shutdown_connections 2024-09-16T00:35:51.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:51.226+0000 7ff889e6d640 1 --2- 172.21.15.38:0/1202983916 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7ff86c0a32c0 0x7ff86c0a5780 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:51.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:51.226+0000 7ff889e6d640 1 --2- 172.21.15.38:0/1202983916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff884106a60 0x7ff8841a4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:51.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:51.226+0000 7ff889e6d640 1 --2- 172.21.15.38:0/1202983916 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7ff884101400 0x7ff8841997c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:51.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:51.226+0000 7ff889e6d640 1 --2- 172.21.15.38:0/1202983916 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7ff884100a60 0x7ff884199280 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:51.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:51.226+0000 7ff889e6d640 1 -- 172.21.15.38:0/1202983916 >> 172.21.15.38:0/1202983916 conn(0x7ff8840fc780 msgr2=0x7ff8840fd030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:51.227 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:51.227+0000 7ff889e6d640 1 -- 172.21.15.38:0/1202983916 shutdown_connections 2024-09-16T00:35:51.228 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:51.227+0000 7ff889e6d640 1 -- 172.21.15.38:0/1202983916 wait complete. 2024-09-16T00:35:51.544 INFO:journalctl@ceph.mds.b.smithi073.stdout:Sep 16 00:35:51 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-b[65406]: ignoring --setuser ceph since I am not root 2024-09-16T00:35:51.544 INFO:journalctl@ceph.mds.b.smithi073.stdout:Sep 16 00:35:51 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-b[65406]: ignoring --setgroup ceph since I am not root 2024-09-16T00:35:51.544 INFO:journalctl@ceph.mds.b.smithi073.stdout:Sep 16 00:35:51 smithi073 ceph-mds[65410]: -- 172.21.15.73:0/1195156666 <== mon.1 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562321b53c20 con 0x562321b30800 2024-09-16T00:35:51.544 INFO:journalctl@ceph.mds.b.smithi073.stdout:Sep 16 00:35:51 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-b[65406]: starting mds.b at 2024-09-16T00:35:51.544 INFO:journalctl@ceph.mds.e.smithi073.stdout:Sep 16 00:35:51 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-e[64789]: ignoring --setuser ceph since I am not root 2024-09-16T00:35:51.544 INFO:journalctl@ceph.mds.e.smithi073.stdout:Sep 16 00:35:51 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-e[64789]: ignoring --setgroup ceph since I am not root 2024-09-16T00:35:51.545 INFO:journalctl@ceph.mds.e.smithi073.stdout:Sep 16 00:35:51 smithi073 ceph-mds[64793]: -- 172.21.15.73:0/1846882013 <== mon.2 v2:172.21.15.73:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d069e8d860 con 0x55d069e6a800 2024-09-16T00:35:51.545 INFO:journalctl@ceph.mds.e.smithi073.stdout:Sep 16 00:35:51 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-e[64789]: starting mds.e at 2024-09-16T00:35:51.545 INFO:journalctl@ceph.mds.h.smithi073.stdout:Sep 16 00:35:51 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-h[65090]: ignoring --setuser ceph since I am not root 2024-09-16T00:35:51.545 INFO:journalctl@ceph.mds.h.smithi073.stdout:Sep 16 00:35:51 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-h[65090]: ignoring --setgroup ceph since I am not root 2024-09-16T00:35:51.545 INFO:journalctl@ceph.mds.h.smithi073.stdout:Sep 16 00:35:51 smithi073 ceph-mds[65094]: -- 172.21.15.73:0/664040732 <== mon.2 v2:172.21.15.73:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555dab1a9860 con 0x555dab187000 2024-09-16T00:35:51.545 INFO:journalctl@ceph.mds.h.smithi073.stdout:Sep 16 00:35:51 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-h[65090]: starting mds.h at 2024-09-16T00:35:51.545 INFO:journalctl@ceph.mds.k.smithi073.stdout:Sep 16 00:35:51 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-k[64476]: ignoring --setuser ceph since I am not root 2024-09-16T00:35:51.545 INFO:journalctl@ceph.mds.k.smithi073.stdout:Sep 16 00:35:51 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-k[64476]: ignoring --setgroup ceph since I am not root 2024-09-16T00:35:51.546 INFO:journalctl@ceph.mds.k.smithi073.stdout:Sep 16 00:35:51 smithi073 ceph-mds[64480]: -- 172.21.15.73:0/585032759 <== mon.0 v2:172.21.15.38:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dde8cfd860 con 0x55dde8cdb400 2024-09-16T00:35:51.546 INFO:journalctl@ceph.mds.k.smithi073.stdout:Sep 16 00:35:51 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-k[64476]: starting mds.k at 2024-09-16T00:35:51.616 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:51 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mon-a[30941]: 2024-09-16T00:35:51.221+0000 7fb681c0f640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T00:35:51.665 INFO:journalctl@ceph.mds.c.smithi122.stdout:Sep 16 00:35:51 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-c[65133]: ignoring --setuser ceph since I am not root 2024-09-16T00:35:51.665 INFO:journalctl@ceph.mds.c.smithi122.stdout:Sep 16 00:35:51 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-c[65133]: ignoring --setgroup ceph since I am not root 2024-09-16T00:35:51.665 INFO:journalctl@ceph.mds.c.smithi122.stdout:Sep 16 00:35:51 smithi122 ceph-mds[65137]: -- 172.21.15.122:0/3786413988 <== mon.1 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5577a312bc20 con 0x5577a3109400 2024-09-16T00:35:51.665 INFO:journalctl@ceph.mds.c.smithi122.stdout:Sep 16 00:35:51 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-c[65133]: starting mds.c at 2024-09-16T00:35:51.665 INFO:journalctl@ceph.mds.i.smithi122.stdout:Sep 16 00:35:51 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-i[65444]: ignoring --setuser ceph since I am not root 2024-09-16T00:35:51.666 INFO:journalctl@ceph.mds.i.smithi122.stdout:Sep 16 00:35:51 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-i[65444]: ignoring --setgroup ceph since I am not root 2024-09-16T00:35:51.666 INFO:journalctl@ceph.mds.i.smithi122.stdout:Sep 16 00:35:51 smithi122 ceph-mds[65448]: -- 172.21.15.122:0/3004936386 <== mon.1 v2:172.21.15.122:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561f2eb1d860 con 0x561f2eafa800 2024-09-16T00:35:51.666 INFO:journalctl@ceph.mds.i.smithi122.stdout:Sep 16 00:35:51 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-i[65444]: starting mds.i at 2024-09-16T00:35:51.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:51 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/1202983916' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-16T00:35:51.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:51 smithi038 ceph-mon[30945]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-16T00:35:51.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:51 smithi038 ceph-mon[30945]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T00:35:51.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:51 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/1202983916' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-16T00:35:51.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:51 smithi038 ceph-mon[30945]: osdmap e93: 12 total, 12 up, 12 in 2024-09-16T00:35:51.965 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:51 smithi038 ceph-mon[30945]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-16T00:35:51.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:51 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/1202983916' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-16T00:35:51.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:51 smithi122 ceph-mon[35412]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-16T00:35:51.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:51 smithi122 ceph-mon[35412]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T00:35:51.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:51 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/1202983916' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-16T00:35:51.966 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:51 smithi122 ceph-mon[35412]: osdmap e93: 12 total, 12 up, 12 in 2024-09-16T00:35:51.967 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:51 smithi122 ceph-mon[35412]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-16T00:35:52.043 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:51 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/1202983916' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-09-16T00:35:52.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:51 smithi073 ceph-mon[35254]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-16T00:35:52.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:51 smithi073 ceph-mon[35254]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-16T00:35:52.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:51 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/1202983916' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-16T00:35:52.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:51 smithi073 ceph-mon[35254]: osdmap e93: 12 total, 12 up, 12 in 2024-09-16T00:35:52.044 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:51 smithi073 ceph-mon[35254]: fsmap cephfs:0/3 6 up:standby, 3 failed 2024-09-16T00:35:52.056 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-16T00:35:52.318 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:35:52.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:52.805+0000 7f0dac2dc640 1 -- 172.21.15.38:0/3510581345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0da406a370 msgr2=0x7f0da41009c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:52.834 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:52.805+0000 7f0dac2dc640 1 --2- 172.21.15.38:0/3510581345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0da406a370 0x7f0da41009c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f0d98009ca0 tx=0x7f0d9802a8c0 comp rx=0 tx=0).stop 2024-09-16T00:35:52.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:52.805+0000 7f0dac2dc640 1 -- 172.21.15.38:0/3510581345 shutdown_connections 2024-09-16T00:35:52.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:52.805+0000 7f0dac2dc640 1 --2- 172.21.15.38:0/3510581345 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0da4104e80 0x7f0da4105330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:52.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:52.805+0000 7f0dac2dc640 1 --2- 172.21.15.38:0/3510581345 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0da406a370 0x7f0da41009c0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:52.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:52.805+0000 7f0dac2dc640 1 --2- 172.21.15.38:0/3510581345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0da4069f00 0x7f0da4100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:52.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:52.805+0000 7f0dac2dc640 1 -- 172.21.15.38:0/3510581345 >> 172.21.15.38:0/3510581345 conn(0x7f0da4077210 msgr2=0x7f0da40fe5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:52.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:52.805+0000 7f0dac2dc640 1 -- 172.21.15.38:0/3510581345 shutdown_connections 2024-09-16T00:35:52.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:52.805+0000 7f0dac2dc640 1 -- 172.21.15.38:0/3510581345 wait complete. 2024-09-16T00:35:52.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:52.806+0000 7f0dac2dc640 1 Processor -- start 2024-09-16T00:35:52.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:52.806+0000 7f0dac2dc640 1 -- start start 2024-09-16T00:35:52.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:52.806+0000 7f0dac2dc640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0da4069f00 0x7f0da4102ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:52.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:52.806+0000 7f0dac2dc640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0da406a370 0x7f0da4103000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:52.835 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:52.806+0000 7f0dac2dc640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0da4104e80 0x7f0da4103540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:52.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:52.806+0000 7f0dac2dc640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f0da4110eb0 con 0x7f0da4104e80 2024-09-16T00:35:52.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:52.806+0000 7f0dac2dc640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0da4110d30 con 0x7f0da406a370 2024-09-16T00:35:52.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:52.806+0000 7f0dac2dc640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0da4111030 con 0x7f0da4069f00 2024-09-16T00:35:52.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:52.806+0000 7f0daa879640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0da4104e80 0x7f0da4103540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:52.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:52.806+0000 7f0daa879640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0da4104e80 0x7f0da4103540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54070/0 (socket says 172.21.15.38:54070) 2024-09-16T00:35:52.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:52.806+0000 7f0daa879640 1 -- 172.21.15.38:0/2653764777 learned_addr learned my addr 172.21.15.38:0/2653764777 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:35:52.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:52.807+0000 7f0daa078640 1 --2- 172.21.15.38:0/2653764777 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0da4069f00 0x7f0da4102ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:52.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:52.807+0000 7f0da9877640 1 --2- 172.21.15.38:0/2653764777 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0da406a370 0x7f0da4103000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:52.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:52.807+0000 7f0daa879640 1 -- 172.21.15.38:0/2653764777 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0da4069f00 msgr2=0x7f0da4102ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:52.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:52.807+0000 7f0daa879640 1 --2- 172.21.15.38:0/2653764777 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0da4069f00 0x7f0da4102ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:52.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:52.807+0000 7f0daa879640 1 -- 172.21.15.38:0/2653764777 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0da406a370 msgr2=0x7f0da4103000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:52.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:52.807+0000 7f0daa879640 1 --2- 172.21.15.38:0/2653764777 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0da406a370 0x7f0da4103000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:52.836 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:52.807+0000 7f0daa879640 1 -- 172.21.15.38:0/2653764777 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da41a6950 con 0x7f0da4104e80 2024-09-16T00:35:52.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:52.807+0000 7f0da9877640 1 --2- 172.21.15.38:0/2653764777 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0da406a370 0x7f0da4103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-16T00:35:52.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:52.807+0000 7f0daa879640 1 --2- 172.21.15.38:0/2653764777 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0da4104e80 0x7f0da4103540 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c009fd0 tx=0x7f0d9c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:52.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:52.807+0000 7f0da37fe640 1 -- 172.21.15.38:0/2653764777 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c011830 con 0x7f0da4104e80 2024-09-16T00:35:52.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:52.807+0000 7f0dac2dc640 1 -- 172.21.15.38:0/2653764777 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0da41a6c40 con 0x7f0da4104e80 2024-09-16T00:35:52.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:52.807+0000 7f0da37fe640 1 -- 172.21.15.38:0/2653764777 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0d9c016070 con 0x7f0da4104e80 2024-09-16T00:35:52.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:52.807+0000 7f0da37fe640 1 -- 172.21.15.38:0/2653764777 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c00f2e0 con 0x7f0da4104e80 2024-09-16T00:35:52.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:52.807+0000 7f0dac2dc640 1 -- 172.21.15.38:0/2653764777 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0da41a7130 con 0x7f0da4104e80 2024-09-16T00:35:52.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:52.808+0000 7f0dac2dc640 1 -- 172.21.15.38:0/2653764777 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0da4105330 con 0x7f0da4104e80 2024-09-16T00:35:52.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:52.809+0000 7f0da37fe640 1 -- 172.21.15.38:0/2653764777 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f0d9c0119d0 con 0x7f0da4104e80 2024-09-16T00:35:52.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:52.810+0000 7f0da37fe640 1 --2- 172.21.15.38:0/2653764777 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f0d8c0a3240 0x7f0d8c0a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:52.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:52.810+0000 7f0da37fe640 1 -- 172.21.15.38:0/2653764777 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9387+0+0 (secure 0 0 0) 0x7f0d9c0cf450 con 0x7f0da4104e80 2024-09-16T00:35:52.837 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:52.810+0000 7f0daa078640 1 --2- 172.21.15.38:0/2653764777 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f0d8c0a3240 0x7f0d8c0a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:52.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:52.810+0000 7f0daa078640 1 --2- 172.21.15.38:0/2653764777 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f0d8c0a3240 0x7f0d8c0a5700 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0d940058c0 tx=0x7f0d9400a4c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:52.838 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:52.814+0000 7f0da37fe640 1 -- 172.21.15.38:0/2653764777 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0d9c09bd90 con 0x7f0da4104e80 2024-09-16T00:35:53.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:53.030+0000 7f0dac2dc640 1 -- 172.21.15.38:0/2653764777 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f0da41a73c0 con 0x7f0da4104e80 2024-09-16T00:35:53.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:53.238+0000 7f0da37fe640 1 -- 172.21.15.38:0/2653764777 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7f0d9c09b690 con 0x7f0da4104e80 2024-09-16T00:35:53.241 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:53.240+0000 7f0dac2dc640 1 -- 172.21.15.38:0/2653764777 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f0d8c0a3240 msgr2=0x7f0d8c0a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:53.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:53.240+0000 7f0dac2dc640 1 --2- 172.21.15.38:0/2653764777 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f0d8c0a3240 0x7f0d8c0a5700 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f0d940058c0 tx=0x7f0d9400a4c0 comp rx=0 tx=0).stop 2024-09-16T00:35:53.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:53.240+0000 7f0dac2dc640 1 -- 172.21.15.38:0/2653764777 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0da4104e80 msgr2=0x7f0da4103540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:53.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:53.240+0000 7f0dac2dc640 1 --2- 172.21.15.38:0/2653764777 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0da4104e80 0x7f0da4103540 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c009fd0 tx=0x7f0d9c00b8e0 comp rx=0 tx=0).stop 2024-09-16T00:35:53.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:53.240+0000 7f0dac2dc640 1 -- 172.21.15.38:0/2653764777 shutdown_connections 2024-09-16T00:35:53.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:53.240+0000 7f0dac2dc640 1 --2- 172.21.15.38:0/2653764777 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f0d8c0a3240 0x7f0d8c0a5700 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:53.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:53.240+0000 7f0dac2dc640 1 --2- 172.21.15.38:0/2653764777 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0da4104e80 0x7f0da4103540 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:53.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:53.240+0000 7f0dac2dc640 1 --2- 172.21.15.38:0/2653764777 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0da406a370 0x7f0da4103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:53.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:53.240+0000 7f0dac2dc640 1 --2- 172.21.15.38:0/2653764777 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0da4069f00 0x7f0da4102ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:53.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:53.240+0000 7f0dac2dc640 1 -- 172.21.15.38:0/2653764777 >> 172.21.15.38:0/2653764777 conn(0x7f0da4077210 msgr2=0x7f0da40fc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:53.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:53.240+0000 7f0dac2dc640 1 -- 172.21.15.38:0/2653764777 shutdown_connections 2024-09-16T00:35:53.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:53.240+0000 7f0dac2dc640 1 -- 172.21.15.38:0/2653764777 wait complete. 2024-09-16T00:35:53.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:53 smithi038 ceph-mon[30945]: pgmap v458: 129 pgs: 129 active+clean; 13 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 716 B/s wr, 7 op/s 2024-09-16T00:35:53.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:53 smithi038 ceph-mon[30945]: mds.? [v2:172.21.15.73:6832/924741942,v1:172.21.15.73:6833/924741942] up:boot 2024-09-16T00:35:53.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:53 smithi038 ceph-mon[30945]: mds.? [v2:172.21.15.122:6837/3452934854,v1:172.21.15.122:6838/3452934854] up:boot 2024-09-16T00:35:53.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:53 smithi038 ceph-mon[30945]: mds.? [v2:172.21.15.122:6836/1692316263,v1:172.21.15.122:6839/1692316263] up:boot 2024-09-16T00:35:53.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:53 smithi038 ceph-mon[30945]: mds.? [v2:172.21.15.73:6834/2379405889,v1:172.21.15.73:6835/2379405889] up:boot 2024-09-16T00:35:53.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:53 smithi038 ceph-mon[30945]: mds.? [v2:172.21.15.73:6836/1772129912,v1:172.21.15.73:6838/1772129912] up:boot 2024-09-16T00:35:53.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:53 smithi038 ceph-mon[30945]: mds.? [v2:172.21.15.73:6837/1775328014,v1:172.21.15.73:6839/1775328014] up:boot 2024-09-16T00:35:53.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:53 smithi038 ceph-mon[30945]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-16T00:35:53.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:53 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T00:35:53.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:53 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T00:35:53.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:53 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T00:35:53.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:53 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T00:35:53.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:53 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T00:35:53.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:53 smithi038 ceph-mon[30945]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T00:35:53.466 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:53 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/2653764777' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T00:35:53.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:53 smithi073 ceph-mon[35254]: pgmap v458: 129 pgs: 129 active+clean; 13 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 716 B/s wr, 7 op/s 2024-09-16T00:35:53.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:53 smithi073 ceph-mon[35254]: mds.? [v2:172.21.15.73:6832/924741942,v1:172.21.15.73:6833/924741942] up:boot 2024-09-16T00:35:53.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:53 smithi073 ceph-mon[35254]: mds.? [v2:172.21.15.122:6837/3452934854,v1:172.21.15.122:6838/3452934854] up:boot 2024-09-16T00:35:53.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:53 smithi073 ceph-mon[35254]: mds.? [v2:172.21.15.122:6836/1692316263,v1:172.21.15.122:6839/1692316263] up:boot 2024-09-16T00:35:53.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:53 smithi073 ceph-mon[35254]: mds.? [v2:172.21.15.73:6834/2379405889,v1:172.21.15.73:6835/2379405889] up:boot 2024-09-16T00:35:53.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:53 smithi073 ceph-mon[35254]: mds.? [v2:172.21.15.73:6836/1772129912,v1:172.21.15.73:6838/1772129912] up:boot 2024-09-16T00:35:53.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:53 smithi073 ceph-mon[35254]: mds.? [v2:172.21.15.73:6837/1775328014,v1:172.21.15.73:6839/1775328014] up:boot 2024-09-16T00:35:53.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:53 smithi073 ceph-mon[35254]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-16T00:35:53.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:53 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T00:35:53.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:53 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T00:35:53.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:53 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T00:35:53.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:53 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T00:35:53.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:53 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T00:35:53.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:53 smithi073 ceph-mon[35254]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T00:35:53.545 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:53 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/2653764777' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T00:35:53.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:53 smithi122 ceph-mon[35412]: pgmap v458: 129 pgs: 129 active+clean; 13 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 716 B/s wr, 7 op/s 2024-09-16T00:35:53.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:53 smithi122 ceph-mon[35412]: mds.? [v2:172.21.15.73:6832/924741942,v1:172.21.15.73:6833/924741942] up:boot 2024-09-16T00:35:53.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:53 smithi122 ceph-mon[35412]: mds.? [v2:172.21.15.122:6837/3452934854,v1:172.21.15.122:6838/3452934854] up:boot 2024-09-16T00:35:53.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:53 smithi122 ceph-mon[35412]: mds.? [v2:172.21.15.122:6836/1692316263,v1:172.21.15.122:6839/1692316263] up:boot 2024-09-16T00:35:53.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:53 smithi122 ceph-mon[35412]: mds.? [v2:172.21.15.73:6834/2379405889,v1:172.21.15.73:6835/2379405889] up:boot 2024-09-16T00:35:53.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:53 smithi122 ceph-mon[35412]: mds.? [v2:172.21.15.73:6836/1772129912,v1:172.21.15.73:6838/1772129912] up:boot 2024-09-16T00:35:53.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:53 smithi122 ceph-mon[35412]: mds.? [v2:172.21.15.73:6837/1775328014,v1:172.21.15.73:6839/1775328014] up:boot 2024-09-16T00:35:53.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:53 smithi122 ceph-mon[35412]: fsmap cephfs:0/3 12 up:standby, 3 failed 2024-09-16T00:35:53.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:53 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-09-16T00:35:53.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:53 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-09-16T00:35:53.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:53 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-09-16T00:35:53.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:53 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-09-16T00:35:53.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:53 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-09-16T00:35:53.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:53 smithi122 ceph-mon[35412]: from='mgr.14150 172.21.15.38:0/1835822858' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-09-16T00:35:53.718 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:53 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/2653764777' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-09-16T00:35:54.159 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-16T00:35:54.409 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:54 smithi038 ceph-mon[30945]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-16T00:35:54.409 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:54 smithi038 ceph-mon[30945]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-16T00:35:54.410 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:54 smithi038 ceph-mon[30945]: Cluster is now healthy 2024-09-16T00:35:54.410 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:54 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/2653764777' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-16T00:35:54.410 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:54 smithi038 ceph-mon[30945]: fsmap 12 up:standby 2024-09-16T00:35:54.421 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:35:54.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:54 smithi073 ceph-mon[35254]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-16T00:35:54.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:54 smithi073 ceph-mon[35254]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-16T00:35:54.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:54 smithi073 ceph-mon[35254]: Cluster is now healthy 2024-09-16T00:35:54.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:54 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/2653764777' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-16T00:35:54.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:54 smithi073 ceph-mon[35254]: fsmap 12 up:standby 2024-09-16T00:35:54.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:54 smithi122 ceph-mon[35412]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-16T00:35:54.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:54 smithi122 ceph-mon[35412]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-16T00:35:54.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:54 smithi122 ceph-mon[35412]: Cluster is now healthy 2024-09-16T00:35:54.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:54 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/2653764777' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-16T00:35:54.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:54 smithi122 ceph-mon[35412]: fsmap 12 up:standby 2024-09-16T00:35:54.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:54.901+0000 7f0876c9d640 1 -- 172.21.15.38:0/440337798 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0870103780 msgr2=0x7f0870107bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:54.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:54.901+0000 7f0876c9d640 1 --2- 172.21.15.38:0/440337798 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0870103780 0x7f0870107bc0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f0864009930 tx=0x7f086402a420 comp rx=0 tx=0).stop 2024-09-16T00:35:54.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:54.901+0000 7f0876c9d640 1 -- 172.21.15.38:0/440337798 shutdown_connections 2024-09-16T00:35:54.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:54.901+0000 7f0876c9d640 1 --2- 172.21.15.38:0/440337798 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0870108300 0x7f087010ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:54.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:54.901+0000 7f0876c9d640 1 --2- 172.21.15.38:0/440337798 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0870103780 0x7f0870107bc0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:54.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:54.901+0000 7f0876c9d640 1 --2- 172.21.15.38:0/440337798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0870102dd0 0x7f08701031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:54.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:54.901+0000 7f0876c9d640 1 -- 172.21.15.38:0/440337798 >> 172.21.15.38:0/440337798 conn(0x7f08700fc7d0 msgr2=0x7f08700febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:54.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:54.901+0000 7f0876c9d640 1 -- 172.21.15.38:0/440337798 shutdown_connections 2024-09-16T00:35:54.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:54.901+0000 7f0876c9d640 1 -- 172.21.15.38:0/440337798 wait complete. 2024-09-16T00:35:54.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:54.901+0000 7f0876c9d640 1 Processor -- start 2024-09-16T00:35:54.916 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:54.901+0000 7f0876c9d640 1 -- start start 2024-09-16T00:35:54.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:54.902+0000 7f0876c9d640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0870102dd0 0x7f0870198500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:54.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:54.902+0000 7f0876c9d640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0870103780 0x7f0870198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:54.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:54.902+0000 7f0876c9d640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0870108300 0x7f087019cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:54.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:54.902+0000 7f0876c9d640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f08701120a0 con 0x7f0870102dd0 2024-09-16T00:35:54.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:54.902+0000 7f0876c9d640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f0870111f20 con 0x7f0870108300 2024-09-16T00:35:54.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:54.902+0000 7f0876c9d640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f0870112220 con 0x7f0870103780 2024-09-16T00:35:54.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:54.902+0000 7f0874a39640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0870102dd0 0x7f0870198500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:54.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:54.902+0000 7f0874a39640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0870102dd0 0x7f0870198500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:54100/0 (socket says 172.21.15.38:54100) 2024-09-16T00:35:54.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:54.902+0000 7f0874a39640 1 -- 172.21.15.38:0/2574423853 learned_addr learned my addr 172.21.15.38:0/2574423853 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:35:54.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:54.902+0000 7f086ffff640 1 --2- 172.21.15.38:0/2574423853 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0870103780 0x7f0870198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:54.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:54.902+0000 7f087523a640 1 --2- 172.21.15.38:0/2574423853 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0870108300 0x7f087019cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:54.917 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:54.902+0000 7f0874a39640 1 -- 172.21.15.38:0/2574423853 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0870103780 msgr2=0x7f0870198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:54.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:54.902+0000 7f0874a39640 1 --2- 172.21.15.38:0/2574423853 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0870103780 0x7f0870198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:54.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:54.902+0000 7f0874a39640 1 -- 172.21.15.38:0/2574423853 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0870108300 msgr2=0x7f087019cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:54.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:54.902+0000 7f0874a39640 1 --2- 172.21.15.38:0/2574423853 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0870108300 0x7f087019cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:54.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:54.902+0000 7f0874a39640 1 -- 172.21.15.38:0/2574423853 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f087019d4b0 con 0x7f0870102dd0 2024-09-16T00:35:54.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:54.902+0000 7f0874a39640 1 --2- 172.21.15.38:0/2574423853 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0870102dd0 0x7f0870198500 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f086000c920 tx=0x7f086000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:54.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:54.902+0000 7f086dffb640 1 -- 172.21.15.38:0/2574423853 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0860007c70 con 0x7f0870102dd0 2024-09-16T00:35:54.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:54.902+0000 7f086dffb640 1 -- 172.21.15.38:0/2574423853 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f0860007e10 con 0x7f0870102dd0 2024-09-16T00:35:54.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:54.902+0000 7f0876c9d640 1 -- 172.21.15.38:0/2574423853 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f087019d7a0 con 0x7f0870102dd0 2024-09-16T00:35:54.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:54.902+0000 7f086dffb640 1 -- 172.21.15.38:0/2574423853 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0860018890 con 0x7f0870102dd0 2024-09-16T00:35:54.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:54.903+0000 7f0876c9d640 1 -- 172.21.15.38:0/2574423853 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08701a5080 con 0x7f0870102dd0 2024-09-16T00:35:54.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:54.904+0000 7f0876c9d640 1 -- 172.21.15.38:0/2574423853 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08701031b0 con 0x7f0870102dd0 2024-09-16T00:35:54.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:54.905+0000 7f086dffb640 1 -- 172.21.15.38:0/2574423853 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f0860018a70 con 0x7f0870102dd0 2024-09-16T00:35:54.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:54.905+0000 7f086dffb640 1 --2- 172.21.15.38:0/2574423853 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f08580a3240 0x7f08580a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:54.918 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:54.906+0000 7f086ffff640 1 --2- 172.21.15.38:0/2574423853 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f08580a3240 0x7f08580a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:54.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:54.906+0000 7f086dffb640 1 -- 172.21.15.38:0/2574423853 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9387+0+0 (secure 0 0 0) 0x7f08600cfee0 con 0x7f0870102dd0 2024-09-16T00:35:54.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:54.906+0000 7f086ffff640 1 --2- 172.21.15.38:0/2574423853 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f08580a3240 0x7f08580a5700 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f086402a940 tx=0x7f0864005a40 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:54.919 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:54.910+0000 7f086dffb640 1 -- 172.21.15.38:0/2574423853 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f0860014030 con 0x7f0870102dd0 2024-09-16T00:35:55.088 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:55.087+0000 7f0876c9d640 1 -- 172.21.15.38:0/2574423853 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f087010a290 con 0x7f0870102dd0 2024-09-16T00:35:55.242 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:55.242+0000 7f086dffb640 1 -- 172.21.15.38:0/2574423853 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v94) ==== 181+0+0 (secure 0 0 0) 0x7f086009c780 con 0x7f0870102dd0 2024-09-16T00:35:55.242 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_metadata' removed 2024-09-16T00:35:55.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:55.245+0000 7f0876c9d640 1 -- 172.21.15.38:0/2574423853 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f08580a3240 msgr2=0x7f08580a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:55.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:55.245+0000 7f0876c9d640 1 --2- 172.21.15.38:0/2574423853 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f08580a3240 0x7f08580a5700 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f086402a940 tx=0x7f0864005a40 comp rx=0 tx=0).stop 2024-09-16T00:35:55.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:55.245+0000 7f0876c9d640 1 -- 172.21.15.38:0/2574423853 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0870102dd0 msgr2=0x7f0870198500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:55.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:55.245+0000 7f0876c9d640 1 --2- 172.21.15.38:0/2574423853 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0870102dd0 0x7f0870198500 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f086000c920 tx=0x7f086000cde0 comp rx=0 tx=0).stop 2024-09-16T00:35:55.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:55.245+0000 7f0876c9d640 1 -- 172.21.15.38:0/2574423853 shutdown_connections 2024-09-16T00:35:55.246 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:55.245+0000 7f0876c9d640 1 --2- 172.21.15.38:0/2574423853 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f08580a3240 0x7f08580a5700 secure :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f086402a940 tx=0x7f0864005a40 comp rx=0 tx=0).stop 2024-09-16T00:35:55.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:55.245+0000 7f0876c9d640 1 --2- 172.21.15.38:0/2574423853 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0870108300 0x7f087019cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:55.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:55.245+0000 7f0876c9d640 1 --2- 172.21.15.38:0/2574423853 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f0870103780 0x7f0870198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:55.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:55.245+0000 7f0876c9d640 1 --2- 172.21.15.38:0/2574423853 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f0870102dd0 0x7f0870198500 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:55.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:55.245+0000 7f0876c9d640 1 -- 172.21.15.38:0/2574423853 >> 172.21.15.38:0/2574423853 conn(0x7f08700fc7d0 msgr2=0x7f08701063b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:55.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:55.246+0000 7f0876c9d640 1 -- 172.21.15.38:0/2574423853 shutdown_connections 2024-09-16T00:35:55.247 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:55.246+0000 7f0876c9d640 1 -- 172.21.15.38:0/2574423853 wait complete. 2024-09-16T00:35:55.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:55 smithi038 ceph-mon[30945]: pgmap v459: 129 pgs: 129 active+clean; 13 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 716 B/s wr, 6 op/s 2024-09-16T00:35:55.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:55 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/2574423853' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:35:55.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:55 smithi073 ceph-mon[35254]: pgmap v459: 129 pgs: 129 active+clean; 13 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 716 B/s wr, 6 op/s 2024-09-16T00:35:55.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:55 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/2574423853' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:35:55.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:55 smithi122 ceph-mon[35412]: pgmap v459: 129 pgs: 129 active+clean; 13 MiB data, 421 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 716 B/s wr, 6 op/s 2024-09-16T00:35:55.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:55 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/2574423853' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:35:56.103 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-16T00:35:56.363 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:35:56.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:56 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/2574423853' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:35:56.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:56 smithi038 ceph-mon[30945]: osdmap e94: 12 total, 12 up, 12 in 2024-09-16T00:35:56.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:56 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/2574423853' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:35:56.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:56 smithi073 ceph-mon[35254]: osdmap e94: 12 total, 12 up, 12 in 2024-09-16T00:35:56.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:56 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/2574423853' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:35:56.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:56 smithi122 ceph-mon[35412]: osdmap e94: 12 total, 12 up, 12 in 2024-09-16T00:35:56.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:56.845+0000 7f9b6f006640 1 -- 172.21.15.38:0/587337514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b68102790 msgr2=0x7f9b681067e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:56.859 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:56.845+0000 7f9b6f006640 1 --2- 172.21.15.38:0/587337514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b68102790 0x7f9b681067e0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f9b54009900 tx=0x7f9b5402a410 comp rx=0 tx=0).stop 2024-09-16T00:35:56.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:56.845+0000 7f9b6f006640 1 -- 172.21.15.38:0/587337514 shutdown_connections 2024-09-16T00:35:56.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:56.845+0000 7f9b6f006640 1 --2- 172.21.15.38:0/587337514 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b68106f10 0x7f9b6810ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:56.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:56.845+0000 7f9b6f006640 1 --2- 172.21.15.38:0/587337514 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b68102790 0x7f9b681067e0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:56.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:56.845+0000 7f9b6f006640 1 --2- 172.21.15.38:0/587337514 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b68101de0 0x7f9b681021c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:56.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:56.845+0000 7f9b6f006640 1 -- 172.21.15.38:0/587337514 >> 172.21.15.38:0/587337514 conn(0x7f9b680fd8c0 msgr2=0x7f9b680ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:56.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:56.845+0000 7f9b6f006640 1 -- 172.21.15.38:0/587337514 shutdown_connections 2024-09-16T00:35:56.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:56.845+0000 7f9b6f006640 1 -- 172.21.15.38:0/587337514 wait complete. 2024-09-16T00:35:56.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:56.846+0000 7f9b6f006640 1 Processor -- start 2024-09-16T00:35:56.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:56.846+0000 7f9b6f006640 1 -- start start 2024-09-16T00:35:56.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:56.846+0000 7f9b6f006640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b68101de0 0x7f9b68198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:56.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:56.846+0000 7f9b6f006640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b68102790 0x7f9b681989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:56.860 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:56.846+0000 7f9b6f006640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b68106f10 0x7f9b6819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:56.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:56.846+0000 7f9b6f006640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7f9b68112000 con 0x7f9b68106f10 2024-09-16T00:35:56.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:56.846+0000 7f9b6f006640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7f9b68111e80 con 0x7f9b68102790 2024-09-16T00:35:56.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:56.846+0000 7f9b6f006640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7f9b68112180 con 0x7f9b68101de0 2024-09-16T00:35:56.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:56.846+0000 7f9b6d5a3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b68106f10 0x7f9b6819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:56.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:56.846+0000 7f9b6d5a3640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b68106f10 0x7f9b6819cd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59934/0 (socket says 172.21.15.38:59934) 2024-09-16T00:35:56.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:56.846+0000 7f9b6d5a3640 1 -- 172.21.15.38:0/1460998947 learned_addr learned my addr 172.21.15.38:0/1460998947 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:35:56.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:56.846+0000 7f9b6cda2640 1 --2- 172.21.15.38:0/1460998947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b68101de0 0x7f9b68198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:56.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:56.846+0000 7f9b67fff640 1 --2- 172.21.15.38:0/1460998947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b68102790 0x7f9b681989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:56.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:56.846+0000 7f9b6d5a3640 1 -- 172.21.15.38:0/1460998947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b68101de0 msgr2=0x7f9b68198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:56.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:56.846+0000 7f9b6d5a3640 1 --2- 172.21.15.38:0/1460998947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b68101de0 0x7f9b68198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:56.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:56.846+0000 7f9b6d5a3640 1 -- 172.21.15.38:0/1460998947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b68102790 msgr2=0x7f9b681989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:56.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:56.847+0000 7f9b6d5a3640 1 --2- 172.21.15.38:0/1460998947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b68102790 0x7f9b681989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:56.861 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:56.847+0000 7f9b6d5a3640 1 -- 172.21.15.38:0/1460998947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b6819d430 con 0x7f9b68106f10 2024-09-16T00:35:56.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:56.847+0000 7f9b6d5a3640 1 --2- 172.21.15.38:0/1460998947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b68106f10 0x7f9b6819cd50 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f9b6000b970 tx=0x7f9b6000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:56.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:56.847+0000 7f9b65ffb640 1 -- 172.21.15.38:0/1460998947 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b60019070 con 0x7f9b68106f10 2024-09-16T00:35:56.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:56.847+0000 7f9b65ffb640 1 -- 172.21.15.38:0/1460998947 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7f9b6000cb40 con 0x7f9b68106f10 2024-09-16T00:35:56.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:56.847+0000 7f9b6f006640 1 -- 172.21.15.38:0/1460998947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b6819d720 con 0x7f9b68106f10 2024-09-16T00:35:56.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:56.847+0000 7f9b65ffb640 1 -- 172.21.15.38:0/1460998947 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b60007650 con 0x7f9b68106f10 2024-09-16T00:35:56.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:56.847+0000 7f9b6f006640 1 -- 172.21.15.38:0/1460998947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b681a5000 con 0x7f9b68106f10 2024-09-16T00:35:56.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:56.848+0000 7f9b6f006640 1 -- 172.21.15.38:0/1460998947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b681021c0 con 0x7f9b68106f10 2024-09-16T00:35:56.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:56.849+0000 7f9b65ffb640 1 -- 172.21.15.38:0/1460998947 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7f9b6001c070 con 0x7f9b68106f10 2024-09-16T00:35:56.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:56.850+0000 7f9b65ffb640 1 --2- 172.21.15.38:0/1460998947 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f9b580a3240 0x7f9b580a5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:56.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:56.850+0000 7f9b6cda2640 1 --2- 172.21.15.38:0/1460998947 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f9b580a3240 0x7f9b580a5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:56.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:56.850+0000 7f9b65ffb640 1 -- 172.21.15.38:0/1460998947 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 8862+0+0 (secure 0 0 0) 0x7f9b600cf170 con 0x7f9b68106f10 2024-09-16T00:35:56.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:56.850+0000 7f9b6cda2640 1 --2- 172.21.15.38:0/1460998947 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f9b580a3240 0x7f9b580a5700 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c0058c0 tx=0x7f9b5c00a250 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:56.862 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:56.854+0000 7f9b65ffb640 1 -- 172.21.15.38:0/1460998947 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7f9b6009b5d0 con 0x7f9b68106f10 2024-09-16T00:35:57.031 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:57.031+0000 7f9b6f006640 1 -- 172.21.15.38:0/1460998947 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f9b6810a120 con 0x7f9b68106f10 2024-09-16T00:35:57.258 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:57.258+0000 7f9b65ffb640 1 -- 172.21.15.38:0/1460998947 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v95) ==== 169+0+0 (secure 0 0 0) 0x7f9b6009f8d0 con 0x7f9b68106f10 2024-09-16T00:35:57.258 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data' removed 2024-09-16T00:35:57.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:57.260+0000 7f9b6f006640 1 -- 172.21.15.38:0/1460998947 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f9b580a3240 msgr2=0x7f9b580a5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:57.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:57.260+0000 7f9b6f006640 1 --2- 172.21.15.38:0/1460998947 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f9b580a3240 0x7f9b580a5700 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9b5c0058c0 tx=0x7f9b5c00a250 comp rx=0 tx=0).stop 2024-09-16T00:35:57.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:57.260+0000 7f9b6f006640 1 -- 172.21.15.38:0/1460998947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b68106f10 msgr2=0x7f9b6819cd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:57.260 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:57.260+0000 7f9b6f006640 1 --2- 172.21.15.38:0/1460998947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b68106f10 0x7f9b6819cd50 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f9b6000b970 tx=0x7f9b6000be30 comp rx=0 tx=0).stop 2024-09-16T00:35:57.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:57.260+0000 7f9b6f006640 1 -- 172.21.15.38:0/1460998947 shutdown_connections 2024-09-16T00:35:57.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:57.260+0000 7f9b6f006640 1 --2- 172.21.15.38:0/1460998947 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7f9b580a3240 0x7f9b580a5700 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:57.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:57.260+0000 7f9b6f006640 1 --2- 172.21.15.38:0/1460998947 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7f9b68106f10 0x7f9b6819cd50 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:57.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:57.260+0000 7f9b6f006640 1 --2- 172.21.15.38:0/1460998947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b68102790 0x7f9b681989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:57.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:57.260+0000 7f9b6f006640 1 --2- 172.21.15.38:0/1460998947 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7f9b68101de0 0x7f9b68198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:57.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:57.260+0000 7f9b6f006640 1 -- 172.21.15.38:0/1460998947 >> 172.21.15.38:0/1460998947 conn(0x7f9b680fd8c0 msgr2=0x7f9b680fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:57.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:57.260+0000 7f9b6f006640 1 -- 172.21.15.38:0/1460998947 shutdown_connections 2024-09-16T00:35:57.261 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:57.260+0000 7f9b6f006640 1 -- 172.21.15.38:0/1460998947 wait complete. 2024-09-16T00:35:57.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:57 smithi038 ceph-mon[30945]: pgmap v461: 97 pgs: 97 active+clean; 577 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:35:57.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:57 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/1460998947' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:35:57.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:57 smithi073 ceph-mon[35254]: pgmap v461: 97 pgs: 97 active+clean; 577 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:35:57.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:57 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/1460998947' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:35:57.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:57 smithi122 ceph-mon[35412]: pgmap v461: 97 pgs: 97 active+clean; 577 KiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:35:57.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:57 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/1460998947' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:35:58.081 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc shell --fsid 983b2072-73c1-11ef-bceb-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-09-16T00:35:58.343 INFO:teuthology.orchestra.run.smithi038.stderr:Inferring config /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/mon.a/config 2024-09-16T00:35:58.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:58 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/1460998947' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:35:58.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:58 smithi038 ceph-mon[30945]: osdmap e95: 12 total, 12 up, 12 in 2024-09-16T00:35:58.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:58 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/1460998947' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:35:58.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:58 smithi073 ceph-mon[35254]: osdmap e95: 12 total, 12 up, 12 in 2024-09-16T00:35:58.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:58 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/1460998947' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:35:58.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:58 smithi122 ceph-mon[35412]: osdmap e95: 12 total, 12 up, 12 in 2024-09-16T00:35:58.852 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:58.829+0000 7fa6e53b5640 1 -- 172.21.15.38:0/241487077 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6e0103740 msgr2=0x7fa6e0107b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:58.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:58.829+0000 7fa6e53b5640 1 --2- 172.21.15.38:0/241487077 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6e0103740 0x7fa6e0107b80 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8009930 tx=0x7fa6c802a420 comp rx=0 tx=0).stop 2024-09-16T00:35:58.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:58.829+0000 7fa6e53b5640 1 -- 172.21.15.38:0/241487077 shutdown_connections 2024-09-16T00:35:58.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:58.829+0000 7fa6e53b5640 1 --2- 172.21.15.38:0/241487077 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6e0108290 0x7fa6e010ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:58.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:58.829+0000 7fa6e53b5640 1 --2- 172.21.15.38:0/241487077 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6e0103740 0x7fa6e0107b80 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:58.853 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:58.830+0000 7fa6e53b5640 1 --2- 172.21.15.38:0/241487077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6e0102d90 0x7fa6e0103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:58.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:58.830+0000 7fa6e53b5640 1 -- 172.21.15.38:0/241487077 >> 172.21.15.38:0/241487077 conn(0x7fa6e00fc7d0 msgr2=0x7fa6e00febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:58.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:58.830+0000 7fa6e53b5640 1 -- 172.21.15.38:0/241487077 shutdown_connections 2024-09-16T00:35:58.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:58.830+0000 7fa6e53b5640 1 -- 172.21.15.38:0/241487077 wait complete. 2024-09-16T00:35:58.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:58.830+0000 7fa6e53b5640 1 Processor -- start 2024-09-16T00:35:58.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:58.830+0000 7fa6e53b5640 1 -- start start 2024-09-16T00:35:58.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:58.831+0000 7fa6e53b5640 1 --2- >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6e0102d90 0x7fa6e0198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:58.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:58.831+0000 7fa6e53b5640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6e0103740 0x7fa6e0198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:58.854 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:58.831+0000 7fa6e53b5640 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6e0108290 0x7fa6e019cde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:58.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:58.831+0000 7fa6e53b5640 1 -- --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e0112050 con 0x7fa6e0103740 2024-09-16T00:35:58.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:58.831+0000 7fa6e53b5640 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e0111ed0 con 0x7fa6e0108290 2024-09-16T00:35:58.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:58.831+0000 7fa6e53b5640 1 -- --> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] -- mon_getmap magic: 0 -- 0x7fa6e01121d0 con 0x7fa6e0102d90 2024-09-16T00:35:58.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:58.831+0000 7fa6d7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6e0103740 0x7fa6e0198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:58.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:58.831+0000 7fa6d7fff640 1 --2- >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6e0103740 0x7fa6e0198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.38:3300/0 says I am v2:172.21.15.38:59952/0 (socket says 172.21.15.38:59952) 2024-09-16T00:35:58.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:58.831+0000 7fa6d7fff640 1 -- 172.21.15.38:0/2242074483 learned_addr learned my addr 172.21.15.38:0/2242074483 (peer_addr_for_me v2:172.21.15.38:0/0) 2024-09-16T00:35:58.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:58.831+0000 7fa6deffd640 1 --2- 172.21.15.38:0/2242074483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6e0102d90 0x7fa6e0198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:58.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:58.831+0000 7fa6df7fe640 1 --2- 172.21.15.38:0/2242074483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6e0108290 0x7fa6e019cde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:58.855 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:58.831+0000 7fa6d7fff640 1 -- 172.21.15.38:0/2242074483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6e0102d90 msgr2=0x7fa6e0198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:58.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:58.831+0000 7fa6d7fff640 1 --2- 172.21.15.38:0/2242074483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6e0102d90 0x7fa6e0198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:58.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:58.831+0000 7fa6d7fff640 1 -- 172.21.15.38:0/2242074483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6e0108290 msgr2=0x7fa6e019cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:58.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:58.831+0000 7fa6d7fff640 1 --2- 172.21.15.38:0/2242074483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6e0108290 0x7fa6e019cde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:58.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:58.831+0000 7fa6d7fff640 1 -- 172.21.15.38:0/2242074483 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6e019d4c0 con 0x7fa6e0103740 2024-09-16T00:35:58.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:58.831+0000 7fa6deffd640 1 --2- 172.21.15.38:0/2242074483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6e0102d90 0x7fa6e0198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-16T00:35:58.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:58.831+0000 7fa6d7fff640 1 --2- 172.21.15.38:0/2242074483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6e0103740 0x7fa6e0198a50 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8005940 tx=0x7fa6c80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:58.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:58.831+0000 7fa6dcff9640 1 -- 172.21.15.38:0/2242074483 <== mon.0 v2:172.21.15.38:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c803f030 con 0x7fa6e0103740 2024-09-16T00:35:58.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:58.832+0000 7fa6dcff9640 1 -- 172.21.15.38:0/2242074483 <== mon.0 v2:172.21.15.38:3300/0 2 ==== config(27 keys) ==== 1140+0+0 (secure 0 0 0) 0x7fa6c8003ce0 con 0x7fa6e0103740 2024-09-16T00:35:58.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:58.832+0000 7fa6e53b5640 1 -- 172.21.15.38:0/2242074483 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6e019d750 con 0x7fa6e0103740 2024-09-16T00:35:58.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:58.832+0000 7fa6dcff9640 1 -- 172.21.15.38:0/2242074483 <== mon.0 v2:172.21.15.38:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6c8035070 con 0x7fa6e0103740 2024-09-16T00:35:58.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:58.832+0000 7fa6e53b5640 1 -- 172.21.15.38:0/2242074483 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6e01a5090 con 0x7fa6e0103740 2024-09-16T00:35:58.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:58.833+0000 7fa6e53b5640 1 -- 172.21.15.38:0/2242074483 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6e0103170 con 0x7fa6e0103740 2024-09-16T00:35:58.856 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:58.834+0000 7fa6dcff9640 1 -- 172.21.15.38:0/2242074483 <== mon.0 v2:172.21.15.38:3300/0 4 ==== mgrmap(e 20) ==== 138355+0+0 (secure 0 0 0) 0x7fa6c8042030 con 0x7fa6e0103740 2024-09-16T00:35:58.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:58.834+0000 7fa6dcff9640 1 --2- 172.21.15.38:0/2242074483 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fa6cc0a32c0 0x7fa6cc0a5780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-16T00:35:58.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:58.834+0000 7fa6deffd640 1 --2- 172.21.15.38:0/2242074483 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fa6cc0a32c0 0x7fa6cc0a5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-16T00:35:58.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:58.835+0000 7fa6dcff9640 1 -- 172.21.15.38:0/2242074483 <== mon.0 v2:172.21.15.38:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 8324+0+0 (secure 0 0 0) 0x7fa6c80eff20 con 0x7fa6e0103740 2024-09-16T00:35:58.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:58.835+0000 7fa6deffd640 1 --2- 172.21.15.38:0/2242074483 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fa6cc0a32c0 0x7fa6cc0a5780 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa6d000abe0 tx=0x7fa6d000a5c0 comp rx=0 tx=0).ready entity=mgr.14150 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-16T00:35:58.857 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:58.838+0000 7fa6dcff9640 1 -- 172.21.15.38:0/2242074483 <== mon.0 v2:172.21.15.38:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188821 (secure 0 0 0) 0x7fa6c80bcbe0 con 0x7fa6e0103740 2024-09-16T00:35:59.027 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:59.026+0000 7fa6e53b5640 1 -- 172.21.15.38:0/2242074483 --> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa6e010a2a0 con 0x7fa6e0103740 2024-09-16T00:35:59.281 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:59.281+0000 7fa6dcff9640 1 -- 172.21.15.38:0/2242074483 <== mon.0 v2:172.21.15.38:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v96) ==== 178+0+0 (secure 0 0 0) 0x7fa6c80bc4e0 con 0x7fa6e0103740 2024-09-16T00:35:59.282 INFO:teuthology.orchestra.run.smithi038.stderr:pool 'cephfs_data_ec' removed 2024-09-16T00:35:59.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:59.284+0000 7fa6e53b5640 1 -- 172.21.15.38:0/2242074483 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fa6cc0a32c0 msgr2=0x7fa6cc0a5780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:59.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:59.284+0000 7fa6e53b5640 1 --2- 172.21.15.38:0/2242074483 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fa6cc0a32c0 0x7fa6cc0a5780 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa6d000abe0 tx=0x7fa6d000a5c0 comp rx=0 tx=0).stop 2024-09-16T00:35:59.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:59.284+0000 7fa6e53b5640 1 -- 172.21.15.38:0/2242074483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6e0103740 msgr2=0x7fa6e0198a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-16T00:35:59.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:59.284+0000 7fa6e53b5640 1 --2- 172.21.15.38:0/2242074483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6e0103740 0x7fa6e0198a50 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fa6c8005940 tx=0x7fa6c80042f0 comp rx=0 tx=0).stop 2024-09-16T00:35:59.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:59.284+0000 7fa6e53b5640 1 -- 172.21.15.38:0/2242074483 shutdown_connections 2024-09-16T00:35:59.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:59.284+0000 7fa6e53b5640 1 --2- 172.21.15.38:0/2242074483 >> [v2:172.21.15.38:6800/1461245815,v1:172.21.15.38:6801/1461245815] conn(0x7fa6cc0a32c0 0x7fa6cc0a5780 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:59.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:59.284+0000 7fa6e53b5640 1 --2- 172.21.15.38:0/2242074483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6e0108290 0x7fa6e019cde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:59.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:59.284+0000 7fa6e53b5640 1 --2- 172.21.15.38:0/2242074483 >> [v2:172.21.15.38:3300/0,v1:172.21.15.38:6789/0] conn(0x7fa6e0103740 0x7fa6e0198a50 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:59.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:59.284+0000 7fa6e53b5640 1 --2- 172.21.15.38:0/2242074483 >> [v2:172.21.15.122:3300/0,v1:172.21.15.122:6789/0] conn(0x7fa6e0102d90 0x7fa6e0198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-16T00:35:59.285 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:59.284+0000 7fa6e53b5640 1 -- 172.21.15.38:0/2242074483 >> 172.21.15.38:0/2242074483 conn(0x7fa6e00fc7d0 msgr2=0x7fa6e0105d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-16T00:35:59.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:59.285+0000 7fa6e53b5640 1 -- 172.21.15.38:0/2242074483 shutdown_connections 2024-09-16T00:35:59.286 INFO:teuthology.orchestra.run.smithi038.stderr:2024-09-16T00:35:59.285+0000 7fa6e53b5640 1 -- 172.21.15.38:0/2242074483 wait complete. 2024-09-16T00:35:59.543 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:59 smithi073 ceph-mon[35254]: pgmap v463: 33 pgs: 33 active+clean; 577 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:35:59.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:35:59 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/2242074483' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:35:59.666 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:59 smithi038 ceph-mon[30945]: pgmap v463: 33 pgs: 33 active+clean; 577 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:35:59.666 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:35:59 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/2242074483' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:35:59.716 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:59 smithi122 ceph-mon[35412]: pgmap v463: 33 pgs: 33 active+clean; 577 KiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-16T00:35:59.717 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:35:59 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/2242074483' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-16T00:36:00.115 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:36:00.144 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:36:00.178 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:36:00.212 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-16T00:36:00.212 DEBUG:teuthology.orchestra.run.smithi038:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-16T00:36:00.229 DEBUG:teuthology.orchestra.run.smithi073:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-16T00:36:00.247 DEBUG:teuthology.orchestra.run.smithi122:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-16T00:36:00.270 INFO:tasks.cephadm:Stopping all daemons... 2024-09-16T00:36:00.271 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-16T00:36:00.271 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-983b2072-73c1-11ef-bceb-c7b262605968@mon.a 2024-09-16T00:36:00.464 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:36:00 smithi038 ceph-mon[30945]: from='client.? 172.21.15.38:0/2242074483' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:36:00.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:36:00 smithi038 ceph-mon[30945]: osdmap e96: 12 total, 12 up, 12 in 2024-09-16T00:36:00.465 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:36:00 smithi038 systemd[1]: Stopping Ceph mon.a for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:36:00.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:36:00 smithi122 ceph-mon[35412]: from='client.? 172.21.15.38:0/2242074483' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:36:00.466 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:36:00 smithi122 ceph-mon[35412]: osdmap e96: 12 total, 12 up, 12 in 2024-09-16T00:36:00.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:36:00 smithi073 ceph-mon[35254]: from='client.? 172.21.15.38:0/2242074483' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-09-16T00:36:00.544 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:36:00 smithi073 ceph-mon[35254]: osdmap e96: 12 total, 12 up, 12 in 2024-09-16T00:36:00.964 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:36:00 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mon-a[30941]: 2024-09-16T00:36:00.605+0000 7fb68741a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:36:00.964 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:36:00 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mon-a[30941]: 2024-09-16T00:36:00.605+0000 7fb68741a640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-16T00:36:00.964 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:36:00 smithi038 podman[93611]: 2024-09-16 00:36:00.718630553 +0000 UTC m=+0.256032566 container died e8fd9f4da33c2aeedbf006f12065ead99db393676b9dd914abc8c6920451f9fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731) 2024-09-16T00:36:01.217 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:36:01 smithi038 podman[93611]: 2024-09-16 00:36:01.172708653 +0000 UTC m=+0.710110708 container cleanup e8fd9f4da33c2aeedbf006f12065ead99db393676b9dd914abc8c6920451f9fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:36:01.217 INFO:journalctl@ceph.mon.a.smithi038.stdout:Sep 16 00:36:01 smithi038 bash[93611]: ceph-983b2072-73c1-11ef-bceb-c7b262605968-mon-a 2024-09-16T00:36:01.672 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@mon.a.service' 2024-09-16T00:36:01.718 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:36:01.718 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-16T00:36:01.718 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-16T00:36:01.718 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-983b2072-73c1-11ef-bceb-c7b262605968@mon.b 2024-09-16T00:36:02.034 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:36:01 smithi073 systemd[1]: Stopping Ceph mon.b for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:36:02.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:36:02 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mon-b[35250]: 2024-09-16T00:36:02.033+0000 7f8188bc8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:36:02.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:36:02 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mon-b[35250]: 2024-09-16T00:36:02.033+0000 7f8188bc8640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-16T00:36:02.294 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:36:02 smithi073 podman[68234]: 2024-09-16 00:36:02.258189469 +0000 UTC m=+0.343729253 container died 98333bd912656779cfeaf10e22ccdaa16709e8286c738323d507af9cc7d2550b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True) 2024-09-16T00:36:02.976 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:36:02 smithi073 podman[68234]: 2024-09-16 00:36:02.676159332 +0000 UTC m=+0.761699117 container cleanup 98333bd912656779cfeaf10e22ccdaa16709e8286c738323d507af9cc7d2550b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mon-b, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:36:02.976 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:36:02 smithi073 bash[68234]: ceph-983b2072-73c1-11ef-bceb-c7b262605968-mon-b 2024-09-16T00:36:03.236 INFO:journalctl@ceph.mon.b.smithi073.stdout:Sep 16 00:36:02 smithi073 podman[68246]: 2024-09-16 00:36:02.976378449 +0000 UTC m=+0.713714193 container remove 98333bd912656779cfeaf10e22ccdaa16709e8286c738323d507af9cc7d2550b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mon-b, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-16T00:36:03.262 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@mon.b.service' 2024-09-16T00:36:03.307 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:36:03.307 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-16T00:36:03.307 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-16T00:36:03.307 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-983b2072-73c1-11ef-bceb-c7b262605968@mon.c 2024-09-16T00:36:03.621 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:36:03 smithi122 systemd[1]: Stopping Ceph mon.c for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:36:03.923 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:36:03 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mon-c[35408]: 2024-09-16T00:36:03.620+0000 7fe70ccc4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:36:03.923 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:36:03 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mon-c[35408]: 2024-09-16T00:36:03.620+0000 7fe70ccc4640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-16T00:36:03.923 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:36:03 smithi122 podman[68347]: 2024-09-16 00:36:03.652203251 +0000 UTC m=+0.140355718 container died ae3c68d8f84e1f7808d41fbac12ef7695511523d18d1529edb70bde76306f494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mon-c, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e) 2024-09-16T00:36:04.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:36:04 smithi122 podman[68347]: 2024-09-16 00:36:04.080145176 +0000 UTC m=+0.568297636 container cleanup ae3c68d8f84e1f7808d41fbac12ef7695511523d18d1529edb70bde76306f494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mon-c, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e) 2024-09-16T00:36:04.217 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:36:04 smithi122 bash[68347]: ceph-983b2072-73c1-11ef-bceb-c7b262605968-mon-c 2024-09-16T00:36:04.569 INFO:journalctl@ceph.mon.c.smithi122.stdout:Sep 16 00:36:04 smithi122 podman[68359]: 2024-09-16 00:36:04.315273136 +0000 UTC m=+0.658342577 container remove ae3c68d8f84e1f7808d41fbac12ef7695511523d18d1529edb70bde76306f494 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mon-c, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-16T00:36:04.574 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@mon.c.service' 2024-09-16T00:36:04.622 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:36:04.622 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-16T00:36:04.622 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-16T00:36:04.622 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-983b2072-73c1-11ef-bceb-c7b262605968@mgr.x 2024-09-16T00:36:04.953 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:36:04 smithi038 systemd[1]: Stopping Ceph mgr.x for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:36:05.215 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:36:04 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x[31172]: 2024-09-16T00:36:04.952+0000 7f1bf1aab640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:36:05.215 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:36:04 smithi038 podman[93743]: 2024-09-16 00:36:04.983707524 +0000 UTC m=+0.158063553 container died d898ac20a9d134ac46b1a5105f46db363aafc064eee96ca783d3eaf498a9fac2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-16T00:36:05.955 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:36:05 smithi038 podman[93743]: 2024-09-16 00:36:05.678988159 +0000 UTC m=+0.853344177 container cleanup d898ac20a9d134ac46b1a5105f46db363aafc064eee96ca783d3eaf498a9fac2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:8921c8ef947807b717db041d85f0761b7914d5fc, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-09-16T00:36:05.956 INFO:journalctl@ceph.mgr.x.smithi038.stdout:Sep 16 00:36:05 smithi038 bash[93743]: ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-x 2024-09-16T00:36:06.084 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@mgr.x.service' 2024-09-16T00:36:06.122 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:36:06.122 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-16T00:36:06.123 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-16T00:36:06.123 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-983b2072-73c1-11ef-bceb-c7b262605968@mgr.y 2024-09-16T00:36:06.461 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Sep 16 00:36:06 smithi073 systemd[1]: Stopping Ceph mgr.y for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:36:06.773 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Sep 16 00:36:06 smithi073 podman[68358]: 2024-09-16 00:36:06.460756554 +0000 UTC m=+0.144360759 container died 33aa75fcc7b01fe2263720841db8a4ebf0904298dc43ebab53cb0be2ffbdd7bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-y, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True) 2024-09-16T00:36:07.043 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Sep 16 00:36:06 smithi073 podman[68358]: 2024-09-16 00:36:06.882109602 +0000 UTC m=+0.565713809 container cleanup 33aa75fcc7b01fe2263720841db8a4ebf0904298dc43ebab53cb0be2ffbdd7bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-y, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:36:07.044 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Sep 16 00:36:06 smithi073 bash[68358]: ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-y 2024-09-16T00:36:07.439 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Sep 16 00:36:07 smithi073 podman[68370]: 2024-09-16 00:36:07.159089987 +0000 UTC m=+0.694311595 container remove 33aa75fcc7b01fe2263720841db8a4ebf0904298dc43ebab53cb0be2ffbdd7bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:36:07.439 INFO:journalctl@ceph.mgr.y.smithi073.stdout:Sep 16 00:36:07 smithi073 systemd[1]: ceph-983b2072-73c1-11ef-bceb-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-09-16T00:36:07.453 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@mgr.y.service' 2024-09-16T00:36:07.488 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:36:07.488 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-16T00:36:07.488 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-16T00:36:07.488 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-983b2072-73c1-11ef-bceb-c7b262605968@mgr.z 2024-09-16T00:36:07.831 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Sep 16 00:36:07 smithi122 systemd[1]: Stopping Ceph mgr.z for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:36:08.127 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Sep 16 00:36:07 smithi122 podman[68484]: 2024-09-16 00:36:07.831395007 +0000 UTC m=+0.132884431 container died b280204398a613416c1a5dc3826ed883f7f2efc8815b61e4c7d14c68224b469e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-z, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True) 2024-09-16T00:36:08.466 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Sep 16 00:36:08 smithi122 podman[68484]: 2024-09-16 00:36:08.219455628 +0000 UTC m=+0.520945051 container cleanup b280204398a613416c1a5dc3826ed883f7f2efc8815b61e4c7d14c68224b469e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-z, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:36:08.466 INFO:journalctl@ceph.mgr.z.smithi122.stdout:Sep 16 00:36:08 smithi122 bash[68484]: ceph-983b2072-73c1-11ef-bceb-c7b262605968-mgr-z 2024-09-16T00:36:08.629 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@mgr.z.service' 2024-09-16T00:36:08.665 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:36:08.665 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-16T00:36:08.666 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-16T00:36:08.666 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-983b2072-73c1-11ef-bceb-c7b262605968@osd.0 2024-09-16T00:36:08.964 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 16 00:36:08 smithi038 systemd[1]: Stopping Ceph osd.0 for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:36:09.465 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 16 00:36:08 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-0[45103]: 2024-09-16T00:36:08.992+0000 7f0f4af28640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:36:09.465 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 16 00:36:08 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-0[45103]: 2024-09-16T00:36:08.992+0000 7f0f4af28640 -1 osd.0 96 *** Got signal Terminated *** 2024-09-16T00:36:09.465 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 16 00:36:08 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-0[45103]: 2024-09-16T00:36:08.992+0000 7f0f4af28640 -1 osd.0 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:36:14.465 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 16 00:36:14 smithi038 podman[93866]: 2024-09-16 00:36:14.324976508 +0000 UTC m=+5.449341593 container died f6234478898f08cc9ae0c5a4208301b871550a4681d79be95ce6fc652e6d853f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0) 2024-09-16T00:36:14.965 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 16 00:36:14 smithi038 podman[93866]: 2024-09-16 00:36:14.868777302 +0000 UTC m=+5.993142390 container cleanup f6234478898f08cc9ae0c5a4208301b871550a4681d79be95ce6fc652e6d853f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True) 2024-09-16T00:36:14.965 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 16 00:36:14 smithi038 bash[93866]: ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-0 2024-09-16T00:36:15.465 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 16 00:36:15 smithi038 podman[93883]: 2024-09-16 00:36:15.09540873 +0000 UTC m=+0.762614064 container remove f6234478898f08cc9ae0c5a4208301b871550a4681d79be95ce6fc652e6d853f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-16T00:36:15.824 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 16 00:36:15 smithi038 podman[93972]: 2024-09-16 00:36:15.454860229 +0000 UTC m=+0.016290588 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:36:15.824 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 16 00:36:15 smithi038 podman[93972]: 2024-09-16 00:36:15.580013704 +0000 UTC m=+0.141444058 container create d1ad760b2452138c24dbbda741893d043a598c91e14c7acd3e20b90d55429246 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e) 2024-09-16T00:36:16.215 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 16 00:36:15 smithi038 podman[93972]: 2024-09-16 00:36:15.824470553 +0000 UTC m=+0.385900915 container init d1ad760b2452138c24dbbda741893d043a598c91e14c7acd3e20b90d55429246 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e) 2024-09-16T00:36:16.215 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 16 00:36:15 smithi038 podman[93972]: 2024-09-16 00:36:15.828070469 +0000 UTC m=+0.389500819 container start d1ad760b2452138c24dbbda741893d043a598c91e14c7acd3e20b90d55429246 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:36:16.215 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 16 00:36:15 smithi038 podman[93972]: 2024-09-16 00:36:15.932869334 +0000 UTC m=+0.494299694 container attach d1ad760b2452138c24dbbda741893d043a598c91e14c7acd3e20b90d55429246 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-16T00:36:16.215 INFO:journalctl@ceph.osd.0.smithi038.stdout:Sep 16 00:36:16 smithi038 podman[93972]: 2024-09-16 00:36:16.03167869 +0000 UTC m=+0.593109038 container died d1ad760b2452138c24dbbda741893d043a598c91e14c7acd3e20b90d55429246 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-0-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True) 2024-09-16T00:36:16.805 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@osd.0.service' 2024-09-16T00:36:16.846 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:36:16.846 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-16T00:36:16.846 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-16T00:36:16.846 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-983b2072-73c1-11ef-bceb-c7b262605968@osd.1 2024-09-16T00:36:17.170 INFO:journalctl@ceph.osd.1.smithi073.stdout:Sep 16 00:36:16 smithi073 systemd[1]: Stopping Ceph osd.1 for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:36:17.543 INFO:journalctl@ceph.osd.1.smithi073.stdout:Sep 16 00:36:17 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-1[39704]: 2024-09-16T00:36:17.169+0000 7f82f1392640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:36:17.544 INFO:journalctl@ceph.osd.1.smithi073.stdout:Sep 16 00:36:17 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-1[39704]: 2024-09-16T00:36:17.169+0000 7f82f1392640 -1 osd.1 96 *** Got signal Terminated *** 2024-09-16T00:36:17.544 INFO:journalctl@ceph.osd.1.smithi073.stdout:Sep 16 00:36:17 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-1[39704]: 2024-09-16T00:36:17.169+0000 7f82f1392640 -1 osd.1 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:36:22.922 INFO:journalctl@ceph.osd.1.smithi073.stdout:Sep 16 00:36:22 smithi073 podman[68496]: 2024-09-16 00:36:22.61338467 +0000 UTC m=+5.572411300 container died 5301a16e10957e03a538f5d1d8e7125c4359c065ca4ae090b0271a055c13ad5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-1, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:36:23.294 INFO:journalctl@ceph.osd.1.smithi073.stdout:Sep 16 00:36:23 smithi073 podman[68496]: 2024-09-16 00:36:23.020976348 +0000 UTC m=+5.980003020 container cleanup 5301a16e10957e03a538f5d1d8e7125c4359c065ca4ae090b0271a055c13ad5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-1, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, GIT_CLEAN=True) 2024-09-16T00:36:23.294 INFO:journalctl@ceph.osd.1.smithi073.stdout:Sep 16 00:36:23 smithi073 bash[68496]: ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-1 2024-09-16T00:36:23.610 INFO:journalctl@ceph.osd.1.smithi073.stdout:Sep 16 00:36:23 smithi073 podman[68512]: 2024-09-16 00:36:23.305838263 +0000 UTC m=+0.681185770 container remove 5301a16e10957e03a538f5d1d8e7125c4359c065ca4ae090b0271a055c13ad5f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-1, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-16T00:36:23.926 INFO:journalctl@ceph.osd.1.smithi073.stdout:Sep 16 00:36:23 smithi073 podman[68603]: 2024-09-16 00:36:23.513507611 +0000 UTC m=+0.015967764 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:36:23.926 INFO:journalctl@ceph.osd.1.smithi073.stdout:Sep 16 00:36:23 smithi073 podman[68603]: 2024-09-16 00:36:23.64015188 +0000 UTC m=+0.142612021 container create 09cdba3eed185f222579f0419f4b3fc3267a01f1165bfc06e3a8b82c6f730df7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731) 2024-09-16T00:36:24.294 INFO:journalctl@ceph.osd.1.smithi073.stdout:Sep 16 00:36:23 smithi073 podman[68603]: 2024-09-16 00:36:23.92669714 +0000 UTC m=+0.429157282 container init 09cdba3eed185f222579f0419f4b3fc3267a01f1165bfc06e3a8b82c6f730df7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:36:24.294 INFO:journalctl@ceph.osd.1.smithi073.stdout:Sep 16 00:36:23 smithi073 podman[68603]: 2024-09-16 00:36:23.930536673 +0000 UTC m=+0.432996818 container start 09cdba3eed185f222579f0419f4b3fc3267a01f1165bfc06e3a8b82c6f730df7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:36:24.294 INFO:journalctl@ceph.osd.1.smithi073.stdout:Sep 16 00:36:24 smithi073 podman[68603]: 2024-09-16 00:36:24.035359986 +0000 UTC m=+0.537820143 container attach 09cdba3eed185f222579f0419f4b3fc3267a01f1165bfc06e3a8b82c6f730df7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-16T00:36:24.294 INFO:journalctl@ceph.osd.1.smithi073.stdout:Sep 16 00:36:24 smithi073 podman[68603]: 2024-09-16 00:36:24.121155715 +0000 UTC m=+0.623615857 container died 09cdba3eed185f222579f0419f4b3fc3267a01f1165bfc06e3a8b82c6f730df7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-1-deactivate, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2) 2024-09-16T00:36:24.821 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@osd.1.service' 2024-09-16T00:36:24.860 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:36:24.860 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-16T00:36:24.860 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-16T00:36:24.861 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-983b2072-73c1-11ef-bceb-c7b262605968@osd.2 2024-09-16T00:36:25.174 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 16 00:36:24 smithi122 systemd[1]: Stopping Ceph osd.2 for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:36:25.466 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 16 00:36:25 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-2[40385]: 2024-09-16T00:36:25.173+0000 7f2252b3d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:36:25.466 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 16 00:36:25 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-2[40385]: 2024-09-16T00:36:25.173+0000 7f2252b3d640 -1 osd.2 96 *** Got signal Terminated *** 2024-09-16T00:36:25.467 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 16 00:36:25 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-2[40385]: 2024-09-16T00:36:25.173+0000 7f2252b3d640 -1 osd.2 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:36:30.858 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 16 00:36:30 smithi122 podman[68608]: 2024-09-16 00:36:30.533452979 +0000 UTC m=+5.480369657 container died 1783ff12716c6df8b622cb406bbff53fb50fcc636d2826e8d6b0f6e9b8e8e4dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-16T00:36:31.175 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 16 00:36:30 smithi122 podman[68608]: 2024-09-16 00:36:30.957372252 +0000 UTC m=+5.904288930 container cleanup 1783ff12716c6df8b622cb406bbff53fb50fcc636d2826e8d6b0f6e9b8e8e4dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-2, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731) 2024-09-16T00:36:31.175 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 16 00:36:30 smithi122 bash[68608]: ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-2 2024-09-16T00:36:31.466 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 16 00:36:31 smithi122 podman[68624]: 2024-09-16 00:36:31.175603436 +0000 UTC m=+0.642598633 container remove 1783ff12716c6df8b622cb406bbff53fb50fcc636d2826e8d6b0f6e9b8e8e4dd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:36:31.845 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 16 00:36:31 smithi122 podman[68712]: 2024-09-16 00:36:31.480320198 +0000 UTC m=+0.015662153 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:36:31.846 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 16 00:36:31 smithi122 podman[68712]: 2024-09-16 00:36:31.593377948 +0000 UTC m=+0.128719945 container create b394740a76e125291d9c24169046ca1eebf15543c4085819560dacb6216bde14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-16T00:36:32.216 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 16 00:36:31 smithi122 podman[68712]: 2024-09-16 00:36:31.8460505 +0000 UTC m=+0.381392456 container init b394740a76e125291d9c24169046ca1eebf15543c4085819560dacb6216bde14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, io.buildah.version=1.37.2) 2024-09-16T00:36:32.217 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 16 00:36:31 smithi122 podman[68712]: 2024-09-16 00:36:31.849607706 +0000 UTC m=+0.384949667 container start b394740a76e125291d9c24169046ca1eebf15543c4085819560dacb6216bde14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0) 2024-09-16T00:36:32.217 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 16 00:36:31 smithi122 podman[68712]: 2024-09-16 00:36:31.945959597 +0000 UTC m=+0.481301572 container attach b394740a76e125291d9c24169046ca1eebf15543c4085819560dacb6216bde14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-16T00:36:32.217 INFO:journalctl@ceph.osd.2.smithi122.stdout:Sep 16 00:36:32 smithi122 podman[68712]: 2024-09-16 00:36:32.042547656 +0000 UTC m=+0.577889612 container died b394740a76e125291d9c24169046ca1eebf15543c4085819560dacb6216bde14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-2-deactivate, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-16T00:36:32.826 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@osd.2.service' 2024-09-16T00:36:32.864 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:36:32.864 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-16T00:36:32.864 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-16T00:36:32.865 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-983b2072-73c1-11ef-bceb-c7b262605968@osd.3 2024-09-16T00:36:33.191 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 16 00:36:32 smithi038 systemd[1]: Stopping Ceph osd.3 for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:36:33.465 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 16 00:36:33 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-3[51710]: 2024-09-16T00:36:33.190+0000 7f6ea251e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:36:33.465 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 16 00:36:33 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-3[51710]: 2024-09-16T00:36:33.190+0000 7f6ea251e640 -1 osd.3 96 *** Got signal Terminated *** 2024-09-16T00:36:33.465 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 16 00:36:33 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-3[51710]: 2024-09-16T00:36:33.190+0000 7f6ea251e640 -1 osd.3 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:36:35.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:35 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:35.340+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:36:35.967 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:35 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5[47134]: 2024-09-16T00:36:35.469+0000 7fbb42b3d640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.686043+0000 front 2024-09-16T00:36:10.686005+0000 (oldest deadline 2024-09-16T00:36:34.785937+0000) 2024-09-16T00:36:35.967 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:35 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:35.719+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:36:36.717 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:36 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5[47134]: 2024-09-16T00:36:36.461+0000 7fbb42b3d640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.686043+0000 front 2024-09-16T00:36:10.686005+0000 (oldest deadline 2024-09-16T00:36:34.785937+0000) 2024-09-16T00:36:36.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:36:36 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:36:36.699+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:36:36.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:36 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:36.365+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:36:36.965 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:36 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:36.486+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:36:37.216 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:36 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:36.717+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:36:37.695 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:37 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:37.380+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:36:37.695 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:37 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5[47134]: 2024-09-16T00:36:37.430+0000 7fbb42b3d640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.686043+0000 front 2024-09-16T00:36:10.686005+0000 (oldest deadline 2024-09-16T00:36:34.785937+0000) 2024-09-16T00:36:37.714 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 16 00:36:37 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-3[51710]: 2024-09-16T00:36:37.237+0000 7f6e9ab2c640 -1 osd.3 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:11.923884+0000 front 2024-09-16T00:36:11.923917+0000 (oldest deadline 2024-09-16T00:36:36.624048+0000) 2024-09-16T00:36:37.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:37 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:37.481+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:36:37.966 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:36:37 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:36:37.711+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:36:37.966 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:37 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:37.695+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:36:38.664 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:38 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5[47134]: 2024-09-16T00:36:38.395+0000 7fbb42b3d640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.686043+0000 front 2024-09-16T00:36:10.686005+0000 (oldest deadline 2024-09-16T00:36:34.785937+0000) 2024-09-16T00:36:38.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:38 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:38.384+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:36:38.965 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 16 00:36:38 smithi038 podman[94088]: 2024-09-16 00:36:38.694524718 +0000 UTC m=+5.632391012 container died 4b92370e832517c42861d91fe43a75f6e83fdde37cf8ef5c8c35f802fe392d9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e) 2024-09-16T00:36:38.965 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:38 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:38.478+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:36:38.966 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:38 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:38.741+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:36:38.966 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:36:38 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:36:38.663+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:36:39.456 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 16 00:36:39 smithi038 podman[94088]: 2024-09-16 00:36:39.207559292 +0000 UTC m=+6.145425645 container cleanup 4b92370e832517c42861d91fe43a75f6e83fdde37cf8ef5c8c35f802fe392d9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-3, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:36:39.456 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 16 00:36:39 smithi038 bash[94088]: ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-3 2024-09-16T00:36:39.672 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:39 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5[47134]: 2024-09-16T00:36:39.353+0000 7fbb42b3d640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.686043+0000 front 2024-09-16T00:36:10.686005+0000 (oldest deadline 2024-09-16T00:36:34.785937+0000) 2024-09-16T00:36:39.714 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:39 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:39.455+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:36:39.715 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 16 00:36:39 smithi038 podman[94115]: 2024-09-16 00:36:39.487098123 +0000 UTC m=+0.790933326 container remove 4b92370e832517c42861d91fe43a75f6e83fdde37cf8ef5c8c35f802fe392d9d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-3, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731) 2024-09-16T00:36:39.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:39 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:39.418+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:36:39.969 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:39 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:39.711+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:36:39.969 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:36:39 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:36:39.671+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:36:40.087 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 16 00:36:39 smithi038 podman[94204]: 2024-09-16 00:36:39.711846312 +0000 UTC m=+0.016233319 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:36:40.088 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 16 00:36:39 smithi038 podman[94204]: 2024-09-16 00:36:39.835543163 +0000 UTC m=+0.139930156 container create 8303dc91180280328f1fcee82f684ed4babb3063e329de46f8f2325455e89ece (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-16T00:36:40.088 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 16 00:36:40 smithi038 podman[94204]: 2024-09-16 00:36:40.088209037 +0000 UTC m=+0.392596037 container init 8303dc91180280328f1fcee82f684ed4babb3063e329de46f8f2325455e89ece (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-16T00:36:40.439 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 16 00:36:40 smithi038 podman[94204]: 2024-09-16 00:36:40.091977804 +0000 UTC m=+0.396364797 container start 8303dc91180280328f1fcee82f684ed4babb3063e329de46f8f2325455e89ece (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-16T00:36:40.440 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 16 00:36:40 smithi038 podman[94204]: 2024-09-16 00:36:40.188156299 +0000 UTC m=+0.492543299 container attach 8303dc91180280328f1fcee82f684ed4babb3063e329de46f8f2325455e89ece (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-3-deactivate, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:36:40.440 INFO:journalctl@ceph.osd.3.smithi038.stdout:Sep 16 00:36:40 smithi038 podman[94204]: 2024-09-16 00:36:40.290643542 +0000 UTC m=+0.595030536 container died 8303dc91180280328f1fcee82f684ed4babb3063e329de46f8f2325455e89ece (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True) 2024-09-16T00:36:40.650 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:40 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5[47134]: 2024-09-16T00:36:40.305+0000 7fbb42b3d640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.686043+0000 front 2024-09-16T00:36:10.686005+0000 (oldest deadline 2024-09-16T00:36:34.785937+0000) 2024-09-16T00:36:40.714 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:40 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:40.438+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:36:40.793 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:40 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:40.447+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:36:40.966 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:40 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:40.696+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:36:40.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:36:40 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:36:40.650+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:36:41.082 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@osd.3.service' 2024-09-16T00:36:41.130 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:36:41.130 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-16T00:36:41.130 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-16T00:36:41.131 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-983b2072-73c1-11ef-bceb-c7b262605968@osd.4 2024-09-16T00:36:41.430 INFO:journalctl@ceph.osd.4.smithi073.stdout:Sep 16 00:36:41 smithi073 systemd[1]: Stopping Ceph osd.4 for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:36:41.687 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:41 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5[47134]: 2024-09-16T00:36:41.354+0000 7fbb42b3d640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.686043+0000 front 2024-09-16T00:36:10.686005+0000 (oldest deadline 2024-09-16T00:36:34.785937+0000) 2024-09-16T00:36:41.793 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:41 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:41.429+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:36:41.794 INFO:journalctl@ceph.osd.4.smithi073.stdout:Sep 16 00:36:41 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-4[46579]: 2024-09-16T00:36:41.467+0000 7f14dbb1f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:36:41.794 INFO:journalctl@ceph.osd.4.smithi073.stdout:Sep 16 00:36:41 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-4[46579]: 2024-09-16T00:36:41.467+0000 7f14dbb1f640 -1 osd.4 96 *** Got signal Terminated *** 2024-09-16T00:36:41.794 INFO:journalctl@ceph.osd.4.smithi073.stdout:Sep 16 00:36:41 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-4[46579]: 2024-09-16T00:36:41.467+0000 7f14dbb1f640 -1 osd.4 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:36:41.965 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:41 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:41.464+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:36:41.966 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:41 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:41.723+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:36:41.966 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:36:41 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:36:41.687+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:36:42.652 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:42 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5[47134]: 2024-09-16T00:36:42.362+0000 7fbb42b3d640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.686043+0000 front 2024-09-16T00:36:10.686005+0000 (oldest deadline 2024-09-16T00:36:34.785937+0000) 2024-09-16T00:36:42.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:42 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:42.447+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:36:42.965 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:42 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:42.470+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:36:42.966 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:36:42 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:36:42.651+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:36:42.966 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:42 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:42.769+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:36:43.632 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:43 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5[47134]: 2024-09-16T00:36:43.363+0000 7fbb42b3d640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.686043+0000 front 2024-09-16T00:36:10.686005+0000 (oldest deadline 2024-09-16T00:36:34.785937+0000) 2024-09-16T00:36:43.714 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:43 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:43.448+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:36:43.771 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:43 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:43.410+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:36:43.966 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:43 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:43.801+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:36:43.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:36:43 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:36:43.632+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:36:44.044 INFO:journalctl@ceph.osd.4.smithi073.stdout:Sep 16 00:36:43 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-4[46579]: 2024-09-16T00:36:43.771+0000 7f14d412d640 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.173030+0000 front 2024-09-16T00:36:21.172992+0000 (oldest deadline 2024-09-16T00:36:43.473135+0000) 2024-09-16T00:36:44.404 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:36:44 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:36:44.055+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:19.296273+0000 front 2024-09-16T00:36:19.296363+0000 (oldest deadline 2024-09-16T00:36:43.297018+0000) 2024-09-16T00:36:44.598 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:44 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5[47134]: 2024-09-16T00:36:44.410+0000 7fbb42b3d640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.686043+0000 front 2024-09-16T00:36:10.686005+0000 (oldest deadline 2024-09-16T00:36:34.785937+0000) 2024-09-16T00:36:44.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:44 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:44.432+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:36:44.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:44 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:44.432+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:36:44.715 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:36:44 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6[58162]: 2024-09-16T00:36:44.307+0000 7fdd1d85e640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.032461+0000 front 2024-09-16T00:36:21.032293+0000 (oldest deadline 2024-09-16T00:36:43.932140+0000) 2024-09-16T00:36:44.785 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:44 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:44.403+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:36:44.850 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:36:44 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:36:44.598+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:36:45.043 INFO:journalctl@ceph.osd.4.smithi073.stdout:Sep 16 00:36:44 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-4[46579]: 2024-09-16T00:36:44.784+0000 7f14d412d640 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.173030+0000 front 2024-09-16T00:36:21.172992+0000 (oldest deadline 2024-09-16T00:36:43.473135+0000) 2024-09-16T00:36:45.216 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:44 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:44.850+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:36:45.217 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:44 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:44.850+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:20.095457+0000 front 2024-09-16T00:36:20.095507+0000 (oldest deadline 2024-09-16T00:36:44.795398+0000) 2024-09-16T00:36:45.431 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:36:45 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:36:45.066+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:19.296273+0000 front 2024-09-16T00:36:19.296363+0000 (oldest deadline 2024-09-16T00:36:43.297018+0000) 2024-09-16T00:36:45.715 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:36:45 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6[58162]: 2024-09-16T00:36:45.316+0000 7fdd1d85e640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.032461+0000 front 2024-09-16T00:36:21.032293+0000 (oldest deadline 2024-09-16T00:36:43.932140+0000) 2024-09-16T00:36:45.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:45 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:45.387+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:36:45.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:45 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:45.387+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:36:45.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:36:45 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:36:45.620+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:36:45.717 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:45 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5[47134]: 2024-09-16T00:36:45.429+0000 7fbb42b3d640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.686043+0000 front 2024-09-16T00:36:10.686005+0000 (oldest deadline 2024-09-16T00:36:34.785937+0000) 2024-09-16T00:36:45.717 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:45 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5[47134]: 2024-09-16T00:36:45.429+0000 7fbb42b3d640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:18.687826+0000 front 2024-09-16T00:36:18.687856+0000 (oldest deadline 2024-09-16T00:36:44.587757+0000) 2024-09-16T00:36:45.793 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:45 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:45.430+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:36:46.216 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:45 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:45.870+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:36:46.216 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:45 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:45.870+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:20.095457+0000 front 2024-09-16T00:36:20.095507+0000 (oldest deadline 2024-09-16T00:36:44.795398+0000) 2024-09-16T00:36:46.294 INFO:journalctl@ceph.osd.4.smithi073.stdout:Sep 16 00:36:45 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-4[46579]: 2024-09-16T00:36:45.809+0000 7f14d412d640 -1 osd.4 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.173030+0000 front 2024-09-16T00:36:21.172992+0000 (oldest deadline 2024-09-16T00:36:43.473135+0000) 2024-09-16T00:36:46.294 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:36:46 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:36:46.038+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:19.296273+0000 front 2024-09-16T00:36:19.296363+0000 (oldest deadline 2024-09-16T00:36:43.297018+0000) 2024-09-16T00:36:46.715 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:36:46 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6[58162]: 2024-09-16T00:36:46.326+0000 7fdd1d85e640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.032461+0000 front 2024-09-16T00:36:21.032293+0000 (oldest deadline 2024-09-16T00:36:43.932140+0000) 2024-09-16T00:36:46.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:46 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:46.363+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:36:46.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:46 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:46.363+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:36:46.716 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:36:46 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:36:46.642+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:36:46.716 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:46 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5[47134]: 2024-09-16T00:36:46.464+0000 7fbb42b3d640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.686043+0000 front 2024-09-16T00:36:10.686005+0000 (oldest deadline 2024-09-16T00:36:34.785937+0000) 2024-09-16T00:36:46.717 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:46 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5[47134]: 2024-09-16T00:36:46.464+0000 7fbb42b3d640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:18.687826+0000 front 2024-09-16T00:36:18.687856+0000 (oldest deadline 2024-09-16T00:36:44.587757+0000) 2024-09-16T00:36:46.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:46 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:46.463+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:36:46.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:46 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:46.463+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:36:47.193 INFO:journalctl@ceph.osd.4.smithi073.stdout:Sep 16 00:36:46 smithi073 podman[68729]: 2024-09-16 00:36:46.878545036 +0000 UTC m=+5.551514585 container died 2918aab07d64e7b3f38a24f1e24bbabf6aca796358caa05d1bdf2c3813d86de8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-4, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:36:47.193 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:36:47 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:36:47.057+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:19.296273+0000 front 2024-09-16T00:36:19.296363+0000 (oldest deadline 2024-09-16T00:36:43.297018+0000) 2024-09-16T00:36:47.216 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:46 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:46.851+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:36:47.216 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:46 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:46.851+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:20.095457+0000 front 2024-09-16T00:36:20.095507+0000 (oldest deadline 2024-09-16T00:36:44.795398+0000) 2024-09-16T00:36:47.544 INFO:journalctl@ceph.osd.4.smithi073.stdout:Sep 16 00:36:47 smithi073 podman[68729]: 2024-09-16 00:36:47.301224048 +0000 UTC m=+5.974193595 container cleanup 2918aab07d64e7b3f38a24f1e24bbabf6aca796358caa05d1bdf2c3813d86de8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-4, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-16T00:36:47.544 INFO:journalctl@ceph.osd.4.smithi073.stdout:Sep 16 00:36:47 smithi073 bash[68729]: ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-4 2024-09-16T00:36:47.544 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:47 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:47.421+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:36:47.544 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:47 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:47.421+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:36:47.715 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:36:47 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6[58162]: 2024-09-16T00:36:47.283+0000 7fdd1d85e640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.032461+0000 front 2024-09-16T00:36:21.032293+0000 (oldest deadline 2024-09-16T00:36:43.932140+0000) 2024-09-16T00:36:47.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:47 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:47.377+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:36:47.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:47 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:47.377+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:36:47.716 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:36:47 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:36:47.640+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:36:47.716 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:47 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5[47134]: 2024-09-16T00:36:47.417+0000 7fbb42b3d640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.686043+0000 front 2024-09-16T00:36:10.686005+0000 (oldest deadline 2024-09-16T00:36:34.785937+0000) 2024-09-16T00:36:47.716 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:47 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5[47134]: 2024-09-16T00:36:47.417+0000 7fbb42b3d640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:18.687826+0000 front 2024-09-16T00:36:18.687856+0000 (oldest deadline 2024-09-16T00:36:44.587757+0000) 2024-09-16T00:36:47.962 INFO:journalctl@ceph.osd.4.smithi073.stdout:Sep 16 00:36:47 smithi073 podman[68744]: 2024-09-16 00:36:47.560969295 +0000 UTC m=+0.680937977 container remove 2918aab07d64e7b3f38a24f1e24bbabf6aca796358caa05d1bdf2c3813d86de8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:36:48.217 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:47 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:47.890+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:36:48.217 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:47 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:47.890+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:20.095457+0000 front 2024-09-16T00:36:20.095507+0000 (oldest deadline 2024-09-16T00:36:44.795398+0000) 2024-09-16T00:36:48.256 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:36:48 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:36:48.059+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:19.296273+0000 front 2024-09-16T00:36:19.296363+0000 (oldest deadline 2024-09-16T00:36:43.297018+0000) 2024-09-16T00:36:48.256 INFO:journalctl@ceph.osd.4.smithi073.stdout:Sep 16 00:36:47 smithi073 podman[68838]: 2024-09-16 00:36:47.865439287 +0000 UTC m=+0.015775624 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:36:48.257 INFO:journalctl@ceph.osd.4.smithi073.stdout:Sep 16 00:36:47 smithi073 podman[68838]: 2024-09-16 00:36:47.986969213 +0000 UTC m=+0.137305540 container create c07b7c6bc2af7ccc7067b9f463d1562f132efcf3663dedfb286200a096b0a889 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True) 2024-09-16T00:36:48.257 INFO:journalctl@ceph.osd.4.smithi073.stdout:Sep 16 00:36:48 smithi073 podman[68838]: 2024-09-16 00:36:48.256704009 +0000 UTC m=+0.407040337 container init c07b7c6bc2af7ccc7067b9f463d1562f132efcf3663dedfb286200a096b0a889 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:36:48.544 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:48 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:48.386+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:36:48.544 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:48 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:48.386+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:36:48.544 INFO:journalctl@ceph.osd.4.smithi073.stdout:Sep 16 00:36:48 smithi073 podman[68838]: 2024-09-16 00:36:48.260072473 +0000 UTC m=+0.410408809 container start c07b7c6bc2af7ccc7067b9f463d1562f132efcf3663dedfb286200a096b0a889 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-16T00:36:48.544 INFO:journalctl@ceph.osd.4.smithi073.stdout:Sep 16 00:36:48 smithi073 podman[68838]: 2024-09-16 00:36:48.356831763 +0000 UTC m=+0.507168099 container attach c07b7c6bc2af7ccc7067b9f463d1562f132efcf3663dedfb286200a096b0a889 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-16T00:36:48.544 INFO:journalctl@ceph.osd.4.smithi073.stdout:Sep 16 00:36:48 smithi073 podman[68838]: 2024-09-16 00:36:48.452686622 +0000 UTC m=+0.603022962 container died c07b7c6bc2af7ccc7067b9f463d1562f132efcf3663dedfb286200a096b0a889 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:36:48.714 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:36:48 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6[58162]: 2024-09-16T00:36:48.251+0000 7fdd1d85e640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.032461+0000 front 2024-09-16T00:36:21.032293+0000 (oldest deadline 2024-09-16T00:36:43.932140+0000) 2024-09-16T00:36:48.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:48 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:48.342+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:36:48.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:48 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:48.342+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:36:48.716 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:36:48 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:36:48.662+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:36:48.716 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:48 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5[47134]: 2024-09-16T00:36:48.420+0000 7fbb42b3d640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.686043+0000 front 2024-09-16T00:36:10.686005+0000 (oldest deadline 2024-09-16T00:36:34.785937+0000) 2024-09-16T00:36:48.716 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:48 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5[47134]: 2024-09-16T00:36:48.420+0000 7fbb42b3d640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:18.687826+0000 front 2024-09-16T00:36:18.687856+0000 (oldest deadline 2024-09-16T00:36:44.587757+0000) 2024-09-16T00:36:49.128 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@osd.4.service' 2024-09-16T00:36:49.166 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:36:49.166 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-16T00:36:49.166 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-16T00:36:49.166 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-983b2072-73c1-11ef-bceb-c7b262605968@osd.5 2024-09-16T00:36:49.193 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:48 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:48.842+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:36:49.193 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:48 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:48.842+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:20.095457+0000 front 2024-09-16T00:36:20.095507+0000 (oldest deadline 2024-09-16T00:36:44.795398+0000) 2024-09-16T00:36:49.423 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:36:49 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:36:49.051+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:19.296273+0000 front 2024-09-16T00:36:19.296363+0000 (oldest deadline 2024-09-16T00:36:43.297018+0000) 2024-09-16T00:36:49.444 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:49 smithi122 systemd[1]: Stopping Ceph osd.5 for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:36:49.698 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:49 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5[47134]: 2024-09-16T00:36:49.444+0000 7fbb42b3d640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.686043+0000 front 2024-09-16T00:36:10.686005+0000 (oldest deadline 2024-09-16T00:36:34.785937+0000) 2024-09-16T00:36:49.698 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:49 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5[47134]: 2024-09-16T00:36:49.444+0000 7fbb42b3d640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:18.687826+0000 front 2024-09-16T00:36:18.687856+0000 (oldest deadline 2024-09-16T00:36:44.587757+0000) 2024-09-16T00:36:49.698 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:49 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5[47134]: 2024-09-16T00:36:49.495+0000 7fbb46527640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:36:49.698 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:49 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5[47134]: 2024-09-16T00:36:49.495+0000 7fbb46527640 -1 osd.5 96 *** Got signal Terminated *** 2024-09-16T00:36:49.698 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:49 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5[47134]: 2024-09-16T00:36:49.495+0000 7fbb46527640 -1 osd.5 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:36:49.714 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:36:49 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6[58162]: 2024-09-16T00:36:49.251+0000 7fdd1d85e640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.032461+0000 front 2024-09-16T00:36:21.032293+0000 (oldest deadline 2024-09-16T00:36:43.932140+0000) 2024-09-16T00:36:49.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:49 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:49.303+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:36:49.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:49 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:49.303+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:36:49.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:49 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:49.423+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:36:49.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:49 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:49.423+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:36:49.966 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:36:49 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:36:49.698+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:36:49.967 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:49 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:49.878+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:36:49.967 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:49 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:49.878+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:20.095457+0000 front 2024-09-16T00:36:20.095507+0000 (oldest deadline 2024-09-16T00:36:44.795398+0000) 2024-09-16T00:36:50.374 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:36:50 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:36:50.085+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:19.296273+0000 front 2024-09-16T00:36:19.296363+0000 (oldest deadline 2024-09-16T00:36:43.297018+0000) 2024-09-16T00:36:50.715 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:36:50 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6[58162]: 2024-09-16T00:36:50.229+0000 7fdd1d85e640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.032461+0000 front 2024-09-16T00:36:21.032293+0000 (oldest deadline 2024-09-16T00:36:43.932140+0000) 2024-09-16T00:36:50.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:50 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:50.312+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:36:50.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:50 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:50.312+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:36:50.793 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:50 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:50.374+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:36:50.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:50 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:50.374+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:36:50.923 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:36:50 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:36:50.682+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:36:50.923 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:50 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5[47134]: 2024-09-16T00:36:50.492+0000 7fbb42b3d640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.686043+0000 front 2024-09-16T00:36:10.686005+0000 (oldest deadline 2024-09-16T00:36:34.785937+0000) 2024-09-16T00:36:50.923 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:50 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5[47134]: 2024-09-16T00:36:50.492+0000 7fbb42b3d640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:18.687826+0000 front 2024-09-16T00:36:18.687856+0000 (oldest deadline 2024-09-16T00:36:44.587757+0000) 2024-09-16T00:36:51.216 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:50 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:50.923+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:36:51.217 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:50 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:50.923+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:20.095457+0000 front 2024-09-16T00:36:20.095507+0000 (oldest deadline 2024-09-16T00:36:44.795398+0000) 2024-09-16T00:36:51.421 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:36:51 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:36:51.050+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:19.296273+0000 front 2024-09-16T00:36:19.296363+0000 (oldest deadline 2024-09-16T00:36:43.297018+0000) 2024-09-16T00:36:51.715 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:36:51 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6[58162]: 2024-09-16T00:36:51.276+0000 7fdd1d85e640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.032461+0000 front 2024-09-16T00:36:21.032293+0000 (oldest deadline 2024-09-16T00:36:43.932140+0000) 2024-09-16T00:36:51.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:51 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:51.346+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:36:51.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:51 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:51.346+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:36:51.716 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:51 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5[47134]: 2024-09-16T00:36:51.448+0000 7fbb42b3d640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.686043+0000 front 2024-09-16T00:36:10.686005+0000 (oldest deadline 2024-09-16T00:36:34.785937+0000) 2024-09-16T00:36:51.716 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:51 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5[47134]: 2024-09-16T00:36:51.448+0000 7fbb42b3d640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:18.687826+0000 front 2024-09-16T00:36:18.687856+0000 (oldest deadline 2024-09-16T00:36:44.587757+0000) 2024-09-16T00:36:51.716 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:36:51 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:36:51.652+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:36:51.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:51 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:51.421+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:36:51.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:51 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:51.421+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:36:52.216 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:51 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:51.957+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:36:52.217 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:51 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:51.957+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:20.095457+0000 front 2024-09-16T00:36:20.095507+0000 (oldest deadline 2024-09-16T00:36:44.795398+0000) 2024-09-16T00:36:52.217 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:51 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:51.957+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.696613+0000 front 2024-09-16T00:36:27.696680+0000 (oldest deadline 2024-09-16T00:36:51.196771+0000) 2024-09-16T00:36:52.373 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:36:52 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:36:52.053+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:19.296273+0000 front 2024-09-16T00:36:19.296363+0000 (oldest deadline 2024-09-16T00:36:43.297018+0000) 2024-09-16T00:36:52.694 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:52 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5[47134]: 2024-09-16T00:36:52.441+0000 7fbb42b3d640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.686043+0000 front 2024-09-16T00:36:10.686005+0000 (oldest deadline 2024-09-16T00:36:34.785937+0000) 2024-09-16T00:36:52.694 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:52 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5[47134]: 2024-09-16T00:36:52.441+0000 7fbb42b3d640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:18.687826+0000 front 2024-09-16T00:36:18.687856+0000 (oldest deadline 2024-09-16T00:36:44.587757+0000) 2024-09-16T00:36:52.717 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:36:52 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6[58162]: 2024-09-16T00:36:52.316+0000 7fdd1d85e640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.032461+0000 front 2024-09-16T00:36:21.032293+0000 (oldest deadline 2024-09-16T00:36:43.932140+0000) 2024-09-16T00:36:52.717 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:36:52 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6[58162]: 2024-09-16T00:36:52.316+0000 7fdd1d85e640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.134162+0000 front 2024-09-16T00:36:29.133955+0000 (oldest deadline 2024-09-16T00:36:52.033997+0000) 2024-09-16T00:36:52.717 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:52 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:52.336+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:36:52.717 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:52 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:52.336+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:36:52.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:52 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:52.372+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:36:52.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:52 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:52.372+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:36:52.966 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:36:52 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:36:52.693+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:36:53.293 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:36:53 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:36:53.013+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:19.296273+0000 front 2024-09-16T00:36:19.296363+0000 (oldest deadline 2024-09-16T00:36:43.297018+0000) 2024-09-16T00:36:53.294 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:36:53 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:36:53.013+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.997974+0000 front 2024-09-16T00:36:27.998173+0000 (oldest deadline 2024-09-16T00:36:52.697943+0000) 2024-09-16T00:36:53.422 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:52 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:52.984+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:36:53.422 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:52 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:52.984+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:20.095457+0000 front 2024-09-16T00:36:20.095507+0000 (oldest deadline 2024-09-16T00:36:44.795398+0000) 2024-09-16T00:36:53.422 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:52 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:52.984+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.696613+0000 front 2024-09-16T00:36:27.696680+0000 (oldest deadline 2024-09-16T00:36:51.196771+0000) 2024-09-16T00:36:53.715 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:36:53 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6[58162]: 2024-09-16T00:36:53.350+0000 7fdd1d85e640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.032461+0000 front 2024-09-16T00:36:21.032293+0000 (oldest deadline 2024-09-16T00:36:43.932140+0000) 2024-09-16T00:36:53.715 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:36:53 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6[58162]: 2024-09-16T00:36:53.350+0000 7fdd1d85e640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.134162+0000 front 2024-09-16T00:36:29.133955+0000 (oldest deadline 2024-09-16T00:36:52.033997+0000) 2024-09-16T00:36:53.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:53 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:53.317+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:36:53.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:53 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:53.317+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:36:53.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:53 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:53.317+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:30.072124+0000 front 2024-09-16T00:36:30.072271+0000 (oldest deadline 2024-09-16T00:36:52.371981+0000) 2024-09-16T00:36:53.716 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:53 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5[47134]: 2024-09-16T00:36:53.422+0000 7fbb42b3d640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.686043+0000 front 2024-09-16T00:36:10.686005+0000 (oldest deadline 2024-09-16T00:36:34.785937+0000) 2024-09-16T00:36:53.716 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:53 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5[47134]: 2024-09-16T00:36:53.422+0000 7fbb42b3d640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:18.687826+0000 front 2024-09-16T00:36:18.687856+0000 (oldest deadline 2024-09-16T00:36:44.587757+0000) 2024-09-16T00:36:53.716 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:36:53 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:36:53.659+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:36:53.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:36:53 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:36:53.659+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:36:53.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:53 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:53.374+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:36:53.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:53 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:53.374+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:36:53.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:53 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:53.374+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:36:54.294 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:36:53 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:36:53.966+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:19.296273+0000 front 2024-09-16T00:36:19.296363+0000 (oldest deadline 2024-09-16T00:36:43.297018+0000) 2024-09-16T00:36:54.294 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:36:53 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:36:53.966+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.997974+0000 front 2024-09-16T00:36:27.998173+0000 (oldest deadline 2024-09-16T00:36:52.697943+0000) 2024-09-16T00:36:54.430 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:53 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:53.985+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:36:54.430 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:53 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:53.985+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:20.095457+0000 front 2024-09-16T00:36:20.095507+0000 (oldest deadline 2024-09-16T00:36:44.795398+0000) 2024-09-16T00:36:54.430 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:53 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:53.985+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.696613+0000 front 2024-09-16T00:36:27.696680+0000 (oldest deadline 2024-09-16T00:36:51.196771+0000) 2024-09-16T00:36:54.688 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:54 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5[47134]: 2024-09-16T00:36:54.430+0000 7fbb42b3d640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.686043+0000 front 2024-09-16T00:36:10.686005+0000 (oldest deadline 2024-09-16T00:36:34.785937+0000) 2024-09-16T00:36:54.688 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:54 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5[47134]: 2024-09-16T00:36:54.430+0000 7fbb42b3d640 -1 osd.5 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:18.687826+0000 front 2024-09-16T00:36:18.687856+0000 (oldest deadline 2024-09-16T00:36:44.587757+0000) 2024-09-16T00:36:54.714 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:36:54 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6[58162]: 2024-09-16T00:36:54.337+0000 7fdd1d85e640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.032461+0000 front 2024-09-16T00:36:21.032293+0000 (oldest deadline 2024-09-16T00:36:43.932140+0000) 2024-09-16T00:36:54.715 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:36:54 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6[58162]: 2024-09-16T00:36:54.337+0000 7fdd1d85e640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.134162+0000 front 2024-09-16T00:36:29.133955+0000 (oldest deadline 2024-09-16T00:36:52.033997+0000) 2024-09-16T00:36:54.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:54 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:54.311+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:36:54.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:54 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:54.311+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:36:54.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:54 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:54.311+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:30.072124+0000 front 2024-09-16T00:36:30.072271+0000 (oldest deadline 2024-09-16T00:36:52.371981+0000) 2024-09-16T00:36:54.793 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:54 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:54.374+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:36:54.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:54 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:54.374+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:36:54.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:54 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:54.374+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:36:54.964 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:36:54 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:36:54.687+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:36:54.964 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:36:54 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:36:54.687+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:36:55.216 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:54 smithi122 podman[68827]: 2024-09-16 00:36:54.964477789 +0000 UTC m=+5.595939160 container died fea16b1b88fe008b0b0620078cbae39db51f94a1446470d75061ef7f66a9dcd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:36:55.217 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:55 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:55.002+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:36:55.217 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:55 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:55.002+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:20.095457+0000 front 2024-09-16T00:36:20.095507+0000 (oldest deadline 2024-09-16T00:36:44.795398+0000) 2024-09-16T00:36:55.217 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:55 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:55.002+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.696613+0000 front 2024-09-16T00:36:27.696680+0000 (oldest deadline 2024-09-16T00:36:51.196771+0000) 2024-09-16T00:36:55.294 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:36:54 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:36:54.961+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:19.296273+0000 front 2024-09-16T00:36:19.296363+0000 (oldest deadline 2024-09-16T00:36:43.297018+0000) 2024-09-16T00:36:55.294 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:36:54 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:36:54.961+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.997974+0000 front 2024-09-16T00:36:27.998173+0000 (oldest deadline 2024-09-16T00:36:52.697943+0000) 2024-09-16T00:36:55.715 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:36:55 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6[58162]: 2024-09-16T00:36:55.343+0000 7fdd1d85e640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.032461+0000 front 2024-09-16T00:36:21.032293+0000 (oldest deadline 2024-09-16T00:36:43.932140+0000) 2024-09-16T00:36:55.715 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:36:55 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6[58162]: 2024-09-16T00:36:55.343+0000 7fdd1d85e640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.134162+0000 front 2024-09-16T00:36:29.133955+0000 (oldest deadline 2024-09-16T00:36:52.033997+0000) 2024-09-16T00:36:55.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:55 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:55.359+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:36:55.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:55 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:55.359+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:36:55.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:55 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:55.359+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:30.072124+0000 front 2024-09-16T00:36:30.072271+0000 (oldest deadline 2024-09-16T00:36:52.371981+0000) 2024-09-16T00:36:55.716 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:55 smithi122 podman[68827]: 2024-09-16 00:36:55.27863725 +0000 UTC m=+5.910098616 container cleanup fea16b1b88fe008b0b0620078cbae39db51f94a1446470d75061ef7f66a9dcd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, RELEASE=squid-8921c8e) 2024-09-16T00:36:55.716 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:55 smithi122 bash[68827]: ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5 2024-09-16T00:36:55.716 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:55 smithi122 podman[68844]: 2024-09-16 00:36:55.513793057 +0000 UTC m=+0.546981064 container remove fea16b1b88fe008b0b0620078cbae39db51f94a1446470d75061ef7f66a9dcd0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5, ceph=True, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux ) 2024-09-16T00:36:55.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:55 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:55.369+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:36:55.795 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:55 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:55.369+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:36:55.795 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:55 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:55.369+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:36:56.030 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:55 smithi122 podman[68931]: 2024-09-16 00:36:55.846392753 +0000 UTC m=+0.015680254 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:36:56.031 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:55 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:55.968+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:36:56.031 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:55 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:55.968+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:20.095457+0000 front 2024-09-16T00:36:20.095507+0000 (oldest deadline 2024-09-16T00:36:44.795398+0000) 2024-09-16T00:36:56.031 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:55 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:55.968+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.696613+0000 front 2024-09-16T00:36:27.696680+0000 (oldest deadline 2024-09-16T00:36:51.196771+0000) 2024-09-16T00:36:56.031 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:36:55 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:36:55.720+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:36:56.031 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:36:55 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:36:55.720+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:36:56.291 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:56 smithi122 podman[68931]: 2024-09-16 00:36:56.031343062 +0000 UTC m=+0.200630563 container create 713ac4faf3fc4cd360649461705f7b85597e223238158a4b8205817a2fcc27d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS) 2024-09-16T00:36:56.292 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:56 smithi122 podman[68931]: 2024-09-16 00:36:56.292186525 +0000 UTC m=+0.461474026 container init 713ac4faf3fc4cd360649461705f7b85597e223238158a4b8205817a2fcc27d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-16T00:36:56.293 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:36:55 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:36:55.975+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:19.296273+0000 front 2024-09-16T00:36:19.296363+0000 (oldest deadline 2024-09-16T00:36:43.297018+0000) 2024-09-16T00:36:56.294 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:36:55 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:36:55.975+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.997974+0000 front 2024-09-16T00:36:27.998173+0000 (oldest deadline 2024-09-16T00:36:52.697943+0000) 2024-09-16T00:36:56.674 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:56 smithi122 podman[68931]: 2024-09-16 00:36:56.295574128 +0000 UTC m=+0.464861630 container start 713ac4faf3fc4cd360649461705f7b85597e223238158a4b8205817a2fcc27d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-16T00:36:56.674 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:56 smithi122 podman[68931]: 2024-09-16 00:36:56.400722107 +0000 UTC m=+0.570009611 container attach 713ac4faf3fc4cd360649461705f7b85597e223238158a4b8205817a2fcc27d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-16T00:36:56.674 INFO:journalctl@ceph.osd.5.smithi122.stdout:Sep 16 00:36:56 smithi122 podman[68931]: 2024-09-16 00:36:56.481526818 +0000 UTC m=+0.650814322 container died 713ac4faf3fc4cd360649461705f7b85597e223238158a4b8205817a2fcc27d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=) 2024-09-16T00:36:56.674 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:36:56 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:36:56.672+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:36:56.674 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:36:56 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:36:56.672+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:36:56.715 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:36:56 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6[58162]: 2024-09-16T00:36:56.391+0000 7fdd1d85e640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.032461+0000 front 2024-09-16T00:36:21.032293+0000 (oldest deadline 2024-09-16T00:36:43.932140+0000) 2024-09-16T00:36:56.715 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:36:56 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6[58162]: 2024-09-16T00:36:56.391+0000 7fdd1d85e640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.134162+0000 front 2024-09-16T00:36:29.133955+0000 (oldest deadline 2024-09-16T00:36:52.033997+0000) 2024-09-16T00:36:56.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:56 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:56.370+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:36:56.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:56 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:56.370+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:36:56.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:56 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:56.370+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:30.072124+0000 front 2024-09-16T00:36:30.072271+0000 (oldest deadline 2024-09-16T00:36:52.371981+0000) 2024-09-16T00:36:56.793 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:56 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:56.338+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:36:56.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:56 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:56.338+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:36:56.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:56 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:56.338+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:36:57.171 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@osd.5.service' 2024-09-16T00:36:57.254 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:36:57.254 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-16T00:36:57.254 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-16T00:36:57.255 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-983b2072-73c1-11ef-bceb-c7b262605968@osd.6 2024-09-16T00:36:57.293 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:36:56 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:36:56.935+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:19.296273+0000 front 2024-09-16T00:36:19.296363+0000 (oldest deadline 2024-09-16T00:36:43.297018+0000) 2024-09-16T00:36:57.294 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:36:56 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:36:56.935+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.997974+0000 front 2024-09-16T00:36:27.998173+0000 (oldest deadline 2024-09-16T00:36:52.697943+0000) 2024-09-16T00:36:57.466 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:57 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:57.011+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:36:57.466 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:57 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:57.011+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:20.095457+0000 front 2024-09-16T00:36:20.095507+0000 (oldest deadline 2024-09-16T00:36:44.795398+0000) 2024-09-16T00:36:57.467 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:57 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:57.011+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.696613+0000 front 2024-09-16T00:36:27.696680+0000 (oldest deadline 2024-09-16T00:36:51.196771+0000) 2024-09-16T00:36:57.579 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:57 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:57.327+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:36:57.579 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:57 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:57.327+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:36:57.579 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:57 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:57.327+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:30.072124+0000 front 2024-09-16T00:36:30.072271+0000 (oldest deadline 2024-09-16T00:36:52.371981+0000) 2024-09-16T00:36:57.579 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:36:57 smithi038 systemd[1]: Stopping Ceph osd.6 for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:36:57.579 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:36:57 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6[58162]: 2024-09-16T00:36:57.365+0000 7fdd1d85e640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.032461+0000 front 2024-09-16T00:36:21.032293+0000 (oldest deadline 2024-09-16T00:36:43.932140+0000) 2024-09-16T00:36:57.580 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:36:57 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6[58162]: 2024-09-16T00:36:57.365+0000 7fdd1d85e640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.134162+0000 front 2024-09-16T00:36:29.133955+0000 (oldest deadline 2024-09-16T00:36:52.033997+0000) 2024-09-16T00:36:57.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:57 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:57.368+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:36:57.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:57 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:57.368+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:36:57.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:57 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:57.368+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:36:57.964 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:36:57 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6[58162]: 2024-09-16T00:36:57.578+0000 7fdd25250640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:36:57.965 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:36:57 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6[58162]: 2024-09-16T00:36:57.578+0000 7fdd25250640 -1 osd.6 96 *** Got signal Terminated *** 2024-09-16T00:36:57.965 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:36:57 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6[58162]: 2024-09-16T00:36:57.578+0000 7fdd25250640 -1 osd.6 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:36:57.966 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:36:57 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:36:57.704+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:36:57.966 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:36:57 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:36:57.704+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:36:58.294 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:36:57 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:36:57.898+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:19.296273+0000 front 2024-09-16T00:36:19.296363+0000 (oldest deadline 2024-09-16T00:36:43.297018+0000) 2024-09-16T00:36:58.294 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:36:57 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:36:57.898+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.997974+0000 front 2024-09-16T00:36:27.998173+0000 (oldest deadline 2024-09-16T00:36:52.697943+0000) 2024-09-16T00:36:58.467 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:58 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:58.000+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:36:58.467 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:58 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:58.000+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:20.095457+0000 front 2024-09-16T00:36:20.095507+0000 (oldest deadline 2024-09-16T00:36:44.795398+0000) 2024-09-16T00:36:58.467 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:58 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:58.000+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.696613+0000 front 2024-09-16T00:36:27.696680+0000 (oldest deadline 2024-09-16T00:36:51.196771+0000) 2024-09-16T00:36:58.715 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:36:58 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6[58162]: 2024-09-16T00:36:58.351+0000 7fdd1d85e640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.032461+0000 front 2024-09-16T00:36:21.032293+0000 (oldest deadline 2024-09-16T00:36:43.932140+0000) 2024-09-16T00:36:58.715 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:36:58 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6[58162]: 2024-09-16T00:36:58.351+0000 7fdd1d85e640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.134162+0000 front 2024-09-16T00:36:29.133955+0000 (oldest deadline 2024-09-16T00:36:52.033997+0000) 2024-09-16T00:36:58.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:58 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:58.312+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:36:58.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:58 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:58.312+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:36:58.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:58 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:58.312+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:30.072124+0000 front 2024-09-16T00:36:30.072271+0000 (oldest deadline 2024-09-16T00:36:52.371981+0000) 2024-09-16T00:36:58.793 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:58 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:58.344+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:36:58.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:58 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:58.344+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:36:58.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:58 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:58.344+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:36:59.216 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:36:58 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:36:58.740+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:36:59.217 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:36:58 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:36:58.740+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:36:59.217 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:58 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:58.951+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:36:59.217 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:58 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:58.951+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:20.095457+0000 front 2024-09-16T00:36:20.095507+0000 (oldest deadline 2024-09-16T00:36:44.795398+0000) 2024-09-16T00:36:59.217 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:58 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:58.951+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.696613+0000 front 2024-09-16T00:36:27.696680+0000 (oldest deadline 2024-09-16T00:36:51.196771+0000) 2024-09-16T00:36:59.293 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:36:58 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:36:58.858+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:19.296273+0000 front 2024-09-16T00:36:19.296363+0000 (oldest deadline 2024-09-16T00:36:43.297018+0000) 2024-09-16T00:36:59.294 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:36:58 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:36:58.858+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.997974+0000 front 2024-09-16T00:36:27.998173+0000 (oldest deadline 2024-09-16T00:36:52.697943+0000) 2024-09-16T00:36:59.715 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:36:59 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6[58162]: 2024-09-16T00:36:59.341+0000 7fdd1d85e640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.032461+0000 front 2024-09-16T00:36:21.032293+0000 (oldest deadline 2024-09-16T00:36:43.932140+0000) 2024-09-16T00:36:59.715 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:36:59 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6[58162]: 2024-09-16T00:36:59.341+0000 7fdd1d85e640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.134162+0000 front 2024-09-16T00:36:29.133955+0000 (oldest deadline 2024-09-16T00:36:52.033997+0000) 2024-09-16T00:36:59.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:59 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:59.290+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:36:59.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:59 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:59.290+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:36:59.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:36:59 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:36:59.290+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:30.072124+0000 front 2024-09-16T00:36:30.072271+0000 (oldest deadline 2024-09-16T00:36:52.371981+0000) 2024-09-16T00:36:59.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:59 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:59.304+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:36:59.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:59 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:59.304+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:36:59.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:36:59 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:36:59.304+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:37:00.216 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:36:59 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:36:59.731+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:00.217 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:36:59 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:36:59.731+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:00.217 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:59 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:59.918+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:37:00.217 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:59 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:59.918+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:20.095457+0000 front 2024-09-16T00:36:20.095507+0000 (oldest deadline 2024-09-16T00:36:44.795398+0000) 2024-09-16T00:37:00.217 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:36:59 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:36:59.918+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.696613+0000 front 2024-09-16T00:36:27.696680+0000 (oldest deadline 2024-09-16T00:36:51.196771+0000) 2024-09-16T00:37:00.294 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:36:59 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:36:59.814+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:19.296273+0000 front 2024-09-16T00:36:19.296363+0000 (oldest deadline 2024-09-16T00:36:43.297018+0000) 2024-09-16T00:37:00.294 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:36:59 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:36:59.814+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.997974+0000 front 2024-09-16T00:36:27.998173+0000 (oldest deadline 2024-09-16T00:36:52.697943+0000) 2024-09-16T00:37:00.715 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:37:00 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6[58162]: 2024-09-16T00:37:00.362+0000 7fdd1d85e640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.032461+0000 front 2024-09-16T00:36:21.032293+0000 (oldest deadline 2024-09-16T00:36:43.932140+0000) 2024-09-16T00:37:00.715 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:37:00 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6[58162]: 2024-09-16T00:37:00.362+0000 7fdd1d85e640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.134162+0000 front 2024-09-16T00:36:29.133955+0000 (oldest deadline 2024-09-16T00:36:52.033997+0000) 2024-09-16T00:37:00.715 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:37:00 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6[58162]: 2024-09-16T00:37:00.362+0000 7fdd1d85e640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:35.435412+0000 front 2024-09-16T00:36:35.435357+0000 (oldest deadline 2024-09-16T00:37:00.135361+0000) 2024-09-16T00:37:00.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:00 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:00.241+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:37:00.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:00 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:00.241+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:37:00.716 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:00 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:00.241+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:30.072124+0000 front 2024-09-16T00:36:30.072271+0000 (oldest deadline 2024-09-16T00:36:52.371981+0000) 2024-09-16T00:37:00.716 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:00 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:00.241+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:33.472747+0000 front 2024-09-16T00:36:33.472711+0000 (oldest deadline 2024-09-16T00:36:59.372794+0000) 2024-09-16T00:37:00.792 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:00 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:00.350+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:37:00.792 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:00 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:00.350+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:37:00.792 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:00 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:00.350+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:37:00.792 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:00 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:00.350+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.627413+0000 front 2024-09-16T00:36:37.627341+0000 (oldest deadline 2024-09-16T00:36:59.327248+0000) 2024-09-16T00:37:00.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:00 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:00.711+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:00.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:00 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:00.711+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:00.968 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:00 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:00.915+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:37:00.968 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:00 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:00.915+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:20.095457+0000 front 2024-09-16T00:36:20.095507+0000 (oldest deadline 2024-09-16T00:36:44.795398+0000) 2024-09-16T00:37:00.968 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:00 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:00.915+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.696613+0000 front 2024-09-16T00:36:27.696680+0000 (oldest deadline 2024-09-16T00:36:51.196771+0000) 2024-09-16T00:37:01.044 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:00 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:37:00.792+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:19.296273+0000 front 2024-09-16T00:36:19.296363+0000 (oldest deadline 2024-09-16T00:36:43.297018+0000) 2024-09-16T00:37:01.044 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:00 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:37:00.792+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.997974+0000 front 2024-09-16T00:36:27.998173+0000 (oldest deadline 2024-09-16T00:36:52.697943+0000) 2024-09-16T00:37:01.044 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:00 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:37:00.792+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.800293+0000 front 2024-09-16T00:36:37.800202+0000 (oldest deadline 2024-09-16T00:37:00.000246+0000) 2024-09-16T00:37:01.465 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:37:01 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6[58162]: 2024-09-16T00:37:01.367+0000 7fdd1d85e640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.032461+0000 front 2024-09-16T00:36:21.032293+0000 (oldest deadline 2024-09-16T00:36:43.932140+0000) 2024-09-16T00:37:01.465 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:37:01 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6[58162]: 2024-09-16T00:37:01.367+0000 7fdd1d85e640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.134162+0000 front 2024-09-16T00:36:29.133955+0000 (oldest deadline 2024-09-16T00:36:52.033997+0000) 2024-09-16T00:37:01.465 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:37:01 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6[58162]: 2024-09-16T00:37:01.367+0000 7fdd1d85e640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:35.435412+0000 front 2024-09-16T00:36:35.435357+0000 (oldest deadline 2024-09-16T00:37:00.135361+0000) 2024-09-16T00:37:01.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:01 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:01.201+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:37:01.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:01 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:01.201+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:37:01.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:01 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:01.201+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:30.072124+0000 front 2024-09-16T00:36:30.072271+0000 (oldest deadline 2024-09-16T00:36:52.371981+0000) 2024-09-16T00:37:01.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:01 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:01.201+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:33.472747+0000 front 2024-09-16T00:36:33.472711+0000 (oldest deadline 2024-09-16T00:36:59.372794+0000) 2024-09-16T00:37:01.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:01 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:01.360+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:37:01.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:01 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:01.360+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:37:01.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:01 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:01.360+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:37:01.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:01 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:01.360+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.627413+0000 front 2024-09-16T00:36:37.627341+0000 (oldest deadline 2024-09-16T00:36:59.327248+0000) 2024-09-16T00:37:02.217 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:01 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:01.760+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:02.217 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:01 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:01.760+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:02.217 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:01 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:01.912+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:37:02.217 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:01 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:01.912+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:20.095457+0000 front 2024-09-16T00:36:20.095507+0000 (oldest deadline 2024-09-16T00:36:44.795398+0000) 2024-09-16T00:37:02.217 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:01 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:01.912+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.696613+0000 front 2024-09-16T00:36:27.696680+0000 (oldest deadline 2024-09-16T00:36:51.196771+0000) 2024-09-16T00:37:02.217 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:01 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:01.912+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.898832+0000 front 2024-09-16T00:36:36.898816+0000 (oldest deadline 2024-09-16T00:37:00.998799+0000) 2024-09-16T00:37:02.293 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:01 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:37:01.811+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:19.296273+0000 front 2024-09-16T00:36:19.296363+0000 (oldest deadline 2024-09-16T00:36:43.297018+0000) 2024-09-16T00:37:02.294 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:01 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:37:01.811+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.997974+0000 front 2024-09-16T00:36:27.998173+0000 (oldest deadline 2024-09-16T00:36:52.697943+0000) 2024-09-16T00:37:02.294 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:01 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:37:01.811+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.800293+0000 front 2024-09-16T00:36:37.800202+0000 (oldest deadline 2024-09-16T00:37:00.000246+0000) 2024-09-16T00:37:02.715 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:37:02 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6[58162]: 2024-09-16T00:37:02.352+0000 7fdd1d85e640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.032461+0000 front 2024-09-16T00:36:21.032293+0000 (oldest deadline 2024-09-16T00:36:43.932140+0000) 2024-09-16T00:37:02.715 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:37:02 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6[58162]: 2024-09-16T00:37:02.352+0000 7fdd1d85e640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.134162+0000 front 2024-09-16T00:36:29.133955+0000 (oldest deadline 2024-09-16T00:36:52.033997+0000) 2024-09-16T00:37:02.715 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:37:02 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6[58162]: 2024-09-16T00:37:02.352+0000 7fdd1d85e640 -1 osd.6 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:35.435412+0000 front 2024-09-16T00:36:35.435357+0000 (oldest deadline 2024-09-16T00:37:00.135361+0000) 2024-09-16T00:37:02.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:02 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:02.238+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:37:02.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:02 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:02.238+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:37:02.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:02 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:02.238+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:30.072124+0000 front 2024-09-16T00:36:30.072271+0000 (oldest deadline 2024-09-16T00:36:52.371981+0000) 2024-09-16T00:37:02.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:02 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:02.238+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:33.472747+0000 front 2024-09-16T00:36:33.472711+0000 (oldest deadline 2024-09-16T00:36:59.372794+0000) 2024-09-16T00:37:02.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:02 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:02.370+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:37:02.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:02 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:02.370+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:37:02.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:02 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:02.370+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:37:02.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:02 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:02.370+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.627413+0000 front 2024-09-16T00:36:37.627341+0000 (oldest deadline 2024-09-16T00:36:59.327248+0000) 2024-09-16T00:37:03.118 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:02 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:02.955+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:37:03.119 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:02 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:02.955+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:20.095457+0000 front 2024-09-16T00:36:20.095507+0000 (oldest deadline 2024-09-16T00:36:44.795398+0000) 2024-09-16T00:37:03.119 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:02 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:02.955+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.696613+0000 front 2024-09-16T00:36:27.696680+0000 (oldest deadline 2024-09-16T00:36:51.196771+0000) 2024-09-16T00:37:03.119 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:02 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:02.955+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.898832+0000 front 2024-09-16T00:36:36.898816+0000 (oldest deadline 2024-09-16T00:37:00.998799+0000) 2024-09-16T00:37:03.119 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:02 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:02.787+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:03.119 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:02 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:02.787+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:03.119 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:02 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:02.787+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.354234+0000 front 2024-09-16T00:36:36.354282+0000 (oldest deadline 2024-09-16T00:37:02.254386+0000) 2024-09-16T00:37:03.294 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:02 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:37:02.845+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:19.296273+0000 front 2024-09-16T00:36:19.296363+0000 (oldest deadline 2024-09-16T00:36:43.297018+0000) 2024-09-16T00:37:03.294 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:02 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:37:02.845+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.997974+0000 front 2024-09-16T00:36:27.998173+0000 (oldest deadline 2024-09-16T00:36:52.697943+0000) 2024-09-16T00:37:03.294 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:02 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:37:02.845+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.800293+0000 front 2024-09-16T00:36:37.800202+0000 (oldest deadline 2024-09-16T00:37:00.000246+0000) 2024-09-16T00:37:03.345 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:37:03 smithi038 podman[94306]: 2024-09-16 00:37:03.043137733 +0000 UTC m=+5.589206327 container died 4b65e29809916884d8caa46fe932659b444f0d2fdd63604db6f963f4bf1ce020 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:37:03.346 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:03 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:03.191+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:37:03.346 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:03 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:03.191+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:37:03.346 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:03 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:03.191+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:30.072124+0000 front 2024-09-16T00:36:30.072271+0000 (oldest deadline 2024-09-16T00:36:52.371981+0000) 2024-09-16T00:37:03.346 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:03 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:03.191+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:33.472747+0000 front 2024-09-16T00:36:33.472711+0000 (oldest deadline 2024-09-16T00:36:59.372794+0000) 2024-09-16T00:37:03.715 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:37:03 smithi038 podman[94306]: 2024-09-16 00:37:03.537012785 +0000 UTC m=+6.083081415 container cleanup 4b65e29809916884d8caa46fe932659b444f0d2fdd63604db6f963f4bf1ce020 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-09-16T00:37:03.715 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:37:03 smithi038 bash[94306]: ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6 2024-09-16T00:37:03.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:03 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:03.356+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:37:03.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:03 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:03.356+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:37:03.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:03 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:03.356+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:37:03.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:03 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:03.356+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.627413+0000 front 2024-09-16T00:36:37.627341+0000 (oldest deadline 2024-09-16T00:36:59.327248+0000) 2024-09-16T00:37:04.156 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:37:03 smithi038 podman[94326]: 2024-09-16 00:37:03.788343519 +0000 UTC m=+0.746239133 container remove 4b65e29809916884d8caa46fe932659b444f0d2fdd63604db6f963f4bf1ce020 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2) 2024-09-16T00:37:04.216 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:03 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:03.991+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:37:04.217 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:03 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:03.991+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:20.095457+0000 front 2024-09-16T00:36:20.095507+0000 (oldest deadline 2024-09-16T00:36:44.795398+0000) 2024-09-16T00:37:04.217 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:03 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:03.991+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.696613+0000 front 2024-09-16T00:36:27.696680+0000 (oldest deadline 2024-09-16T00:36:51.196771+0000) 2024-09-16T00:37:04.217 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:03 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:03.991+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.898832+0000 front 2024-09-16T00:36:36.898816+0000 (oldest deadline 2024-09-16T00:37:00.998799+0000) 2024-09-16T00:37:04.217 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:03 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:03.754+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:04.217 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:03 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:03.754+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:04.217 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:03 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:03.754+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.354234+0000 front 2024-09-16T00:36:36.354282+0000 (oldest deadline 2024-09-16T00:37:02.254386+0000) 2024-09-16T00:37:04.293 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:03 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:37:03.832+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:19.296273+0000 front 2024-09-16T00:36:19.296363+0000 (oldest deadline 2024-09-16T00:36:43.297018+0000) 2024-09-16T00:37:04.294 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:03 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:37:03.832+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.997974+0000 front 2024-09-16T00:36:27.998173+0000 (oldest deadline 2024-09-16T00:36:52.697943+0000) 2024-09-16T00:37:04.294 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:03 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:37:03.832+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.800293+0000 front 2024-09-16T00:36:37.800202+0000 (oldest deadline 2024-09-16T00:37:00.000246+0000) 2024-09-16T00:37:04.408 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:04 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:04.212+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:37:04.408 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:04 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:04.212+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:37:04.408 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:04 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:04.212+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:30.072124+0000 front 2024-09-16T00:36:30.072271+0000 (oldest deadline 2024-09-16T00:36:52.371981+0000) 2024-09-16T00:37:04.408 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:04 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:04.212+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:33.472747+0000 front 2024-09-16T00:36:33.472711+0000 (oldest deadline 2024-09-16T00:36:59.372794+0000) 2024-09-16T00:37:04.408 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:37:04 smithi038 podman[94415]: 2024-09-16 00:37:04.059856783 +0000 UTC m=+0.016205444 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:37:04.408 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:37:04 smithi038 podman[94415]: 2024-09-16 00:37:04.172548655 +0000 UTC m=+0.128897349 container create 260a39a314c1190cbc8ed6690e16c8c3d54b9181987eee90a224c35e69f3bffe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True) 2024-09-16T00:37:04.408 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:37:04 smithi038 podman[94415]: 2024-09-16 00:37:04.408395313 +0000 UTC m=+0.364743965 container init 260a39a314c1190cbc8ed6690e16c8c3d54b9181987eee90a224c35e69f3bffe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=) 2024-09-16T00:37:04.715 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:37:04 smithi038 podman[94415]: 2024-09-16 00:37:04.41216494 +0000 UTC m=+0.368513593 container start 260a39a314c1190cbc8ed6690e16c8c3d54b9181987eee90a224c35e69f3bffe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-16T00:37:04.715 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:37:04 smithi038 podman[94415]: 2024-09-16 00:37:04.500100887 +0000 UTC m=+0.456449553 container attach 260a39a314c1190cbc8ed6690e16c8c3d54b9181987eee90a224c35e69f3bffe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-16T00:37:04.715 INFO:journalctl@ceph.osd.6.smithi038.stdout:Sep 16 00:37:04 smithi038 podman[94415]: 2024-09-16 00:37:04.602803657 +0000 UTC m=+0.559152310 container died 260a39a314c1190cbc8ed6690e16c8c3d54b9181987eee90a224c35e69f3bffe (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-16T00:37:04.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:04 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:04.382+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:37:04.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:04 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:04.382+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:37:04.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:04 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:04.382+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:37:04.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:04 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:04.382+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.627413+0000 front 2024-09-16T00:36:37.627341+0000 (oldest deadline 2024-09-16T00:36:59.327248+0000) 2024-09-16T00:37:05.215 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:05 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:05.183+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:37:05.215 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:05 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:05.183+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:37:05.215 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:05 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:05.183+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:30.072124+0000 front 2024-09-16T00:36:30.072271+0000 (oldest deadline 2024-09-16T00:36:52.371981+0000) 2024-09-16T00:37:05.215 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:05 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:05.183+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:33.472747+0000 front 2024-09-16T00:36:33.472711+0000 (oldest deadline 2024-09-16T00:36:59.372794+0000) 2024-09-16T00:37:05.216 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:04 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:04.753+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:05.216 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:04 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:04.753+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:05.216 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:04 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:04.753+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.354234+0000 front 2024-09-16T00:36:36.354282+0000 (oldest deadline 2024-09-16T00:37:02.254386+0000) 2024-09-16T00:37:05.217 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:04 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:04.994+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:37:05.217 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:04 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:04.995+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:20.095457+0000 front 2024-09-16T00:36:20.095507+0000 (oldest deadline 2024-09-16T00:36:44.795398+0000) 2024-09-16T00:37:05.217 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:04 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:04.995+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.696613+0000 front 2024-09-16T00:36:27.696680+0000 (oldest deadline 2024-09-16T00:36:51.196771+0000) 2024-09-16T00:37:05.217 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:04 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:04.995+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.898832+0000 front 2024-09-16T00:36:36.898816+0000 (oldest deadline 2024-09-16T00:37:00.998799+0000) 2024-09-16T00:37:05.294 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:04 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:37:04.870+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:19.296273+0000 front 2024-09-16T00:36:19.296363+0000 (oldest deadline 2024-09-16T00:36:43.297018+0000) 2024-09-16T00:37:05.294 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:04 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:37:04.870+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.997974+0000 front 2024-09-16T00:36:27.998173+0000 (oldest deadline 2024-09-16T00:36:52.697943+0000) 2024-09-16T00:37:05.294 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:04 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:37:04.870+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.800293+0000 front 2024-09-16T00:36:37.800202+0000 (oldest deadline 2024-09-16T00:37:00.000246+0000) 2024-09-16T00:37:05.377 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@osd.6.service' 2024-09-16T00:37:05.423 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:37:05.423 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-16T00:37:05.423 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-16T00:37:05.423 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-983b2072-73c1-11ef-bceb-c7b262605968@osd.7 2024-09-16T00:37:05.746 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:05 smithi073 systemd[1]: Stopping Ceph osd.7 for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:37:05.747 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:05 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:05.411+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:37:05.747 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:05 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:05.411+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:37:05.747 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:05 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:05.411+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:37:05.747 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:05 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:05.411+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.627413+0000 front 2024-09-16T00:36:37.627341+0000 (oldest deadline 2024-09-16T00:36:59.327248+0000) 2024-09-16T00:37:05.966 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:05 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:05.714+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:05.966 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:05 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:05.714+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:05.966 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:05 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:05.714+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.354234+0000 front 2024-09-16T00:36:36.354282+0000 (oldest deadline 2024-09-16T00:37:02.254386+0000) 2024-09-16T00:37:06.044 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:05 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:37:05.745+0000 7f519f14c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:37:06.044 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:05 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:37:05.745+0000 7f519f14c640 -1 osd.7 96 *** Got signal Terminated *** 2024-09-16T00:37:06.044 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:05 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:37:05.745+0000 7f519f14c640 -1 osd.7 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:37:06.044 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:05 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:37:05.900+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:19.296273+0000 front 2024-09-16T00:36:19.296363+0000 (oldest deadline 2024-09-16T00:36:43.297018+0000) 2024-09-16T00:37:06.044 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:05 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:37:05.900+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.997974+0000 front 2024-09-16T00:36:27.998173+0000 (oldest deadline 2024-09-16T00:36:52.697943+0000) 2024-09-16T00:37:06.044 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:05 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:37:05.900+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.800293+0000 front 2024-09-16T00:36:37.800202+0000 (oldest deadline 2024-09-16T00:37:00.000246+0000) 2024-09-16T00:37:06.464 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:06 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:06.196+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:37:06.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:06 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:06.196+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:37:06.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:06 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:06.196+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:30.072124+0000 front 2024-09-16T00:36:30.072271+0000 (oldest deadline 2024-09-16T00:36:52.371981+0000) 2024-09-16T00:37:06.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:06 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:06.196+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:33.472747+0000 front 2024-09-16T00:36:33.472711+0000 (oldest deadline 2024-09-16T00:36:59.372794+0000) 2024-09-16T00:37:06.466 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:06 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:06.022+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:37:06.466 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:06 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:06.022+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:20.095457+0000 front 2024-09-16T00:36:20.095507+0000 (oldest deadline 2024-09-16T00:36:44.795398+0000) 2024-09-16T00:37:06.467 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:06 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:06.022+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.696613+0000 front 2024-09-16T00:36:27.696680+0000 (oldest deadline 2024-09-16T00:36:51.196771+0000) 2024-09-16T00:37:06.467 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:06 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:06.022+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.898832+0000 front 2024-09-16T00:36:36.898816+0000 (oldest deadline 2024-09-16T00:37:00.998799+0000) 2024-09-16T00:37:06.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:06 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:06.426+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:37:06.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:06 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:06.426+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:37:06.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:06 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:06.426+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:37:06.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:06 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:06.426+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.627413+0000 front 2024-09-16T00:36:37.627341+0000 (oldest deadline 2024-09-16T00:36:59.327248+0000) 2024-09-16T00:37:06.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:06 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:06.666+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:06.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:06 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:06.666+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:06.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:06 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:06.666+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.354234+0000 front 2024-09-16T00:36:36.354282+0000 (oldest deadline 2024-09-16T00:37:02.254386+0000) 2024-09-16T00:37:07.294 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:06 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:37:06.915+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:19.296273+0000 front 2024-09-16T00:36:19.296363+0000 (oldest deadline 2024-09-16T00:36:43.297018+0000) 2024-09-16T00:37:07.294 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:06 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:37:06.915+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.997974+0000 front 2024-09-16T00:36:27.998173+0000 (oldest deadline 2024-09-16T00:36:52.697943+0000) 2024-09-16T00:37:07.294 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:06 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:37:06.915+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.800293+0000 front 2024-09-16T00:36:37.800202+0000 (oldest deadline 2024-09-16T00:37:00.000246+0000) 2024-09-16T00:37:07.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:07 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:07.210+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:37:07.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:07 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:07.210+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:37:07.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:07 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:07.210+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:30.072124+0000 front 2024-09-16T00:36:30.072271+0000 (oldest deadline 2024-09-16T00:36:52.371981+0000) 2024-09-16T00:37:07.466 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:07 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:07.210+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:33.472747+0000 front 2024-09-16T00:36:33.472711+0000 (oldest deadline 2024-09-16T00:36:59.372794+0000) 2024-09-16T00:37:07.466 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:07 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:07.034+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:37:07.466 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:07 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:07.034+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:20.095457+0000 front 2024-09-16T00:36:20.095507+0000 (oldest deadline 2024-09-16T00:36:44.795398+0000) 2024-09-16T00:37:07.466 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:07 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:07.034+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.696613+0000 front 2024-09-16T00:36:27.696680+0000 (oldest deadline 2024-09-16T00:36:51.196771+0000) 2024-09-16T00:37:07.467 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:07 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:07.034+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.898832+0000 front 2024-09-16T00:36:36.898816+0000 (oldest deadline 2024-09-16T00:37:00.998799+0000) 2024-09-16T00:37:07.569 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:07 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:07.396+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:37:07.569 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:07 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:07.396+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:37:07.569 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:07 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:07.396+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:37:07.569 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:07 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:07.396+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.627413+0000 front 2024-09-16T00:36:37.627341+0000 (oldest deadline 2024-09-16T00:36:59.327248+0000) 2024-09-16T00:37:07.966 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:07 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:07.702+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:07.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:07 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:07.702+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:07.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:07 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:07.702+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.354234+0000 front 2024-09-16T00:36:36.354282+0000 (oldest deadline 2024-09-16T00:37:02.254386+0000) 2024-09-16T00:37:08.293 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:07 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:37:07.948+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:19.296273+0000 front 2024-09-16T00:36:19.296363+0000 (oldest deadline 2024-09-16T00:36:43.297018+0000) 2024-09-16T00:37:08.294 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:07 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:37:07.948+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.997974+0000 front 2024-09-16T00:36:27.998173+0000 (oldest deadline 2024-09-16T00:36:52.697943+0000) 2024-09-16T00:37:08.294 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:07 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:37:07.948+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.800293+0000 front 2024-09-16T00:36:37.800202+0000 (oldest deadline 2024-09-16T00:37:00.000246+0000) 2024-09-16T00:37:08.468 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:08 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:08.017+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:37:08.469 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:08 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:08.017+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:20.095457+0000 front 2024-09-16T00:36:20.095507+0000 (oldest deadline 2024-09-16T00:36:44.795398+0000) 2024-09-16T00:37:08.469 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:08 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:08.017+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.696613+0000 front 2024-09-16T00:36:27.696680+0000 (oldest deadline 2024-09-16T00:36:51.196771+0000) 2024-09-16T00:37:08.469 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:08 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:08.017+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.898832+0000 front 2024-09-16T00:36:36.898816+0000 (oldest deadline 2024-09-16T00:37:00.998799+0000) 2024-09-16T00:37:08.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:08 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:08.231+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:37:08.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:08 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:08.231+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:37:08.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:08 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:08.231+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:30.072124+0000 front 2024-09-16T00:36:30.072271+0000 (oldest deadline 2024-09-16T00:36:52.371981+0000) 2024-09-16T00:37:08.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:08 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:08.231+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:33.472747+0000 front 2024-09-16T00:36:33.472711+0000 (oldest deadline 2024-09-16T00:36:59.372794+0000) 2024-09-16T00:37:08.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:08 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:08.375+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:37:08.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:08 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:08.375+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:37:08.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:08 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:08.375+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:37:08.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:08 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:08.375+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.627413+0000 front 2024-09-16T00:36:37.627341+0000 (oldest deadline 2024-09-16T00:36:59.327248+0000) 2024-09-16T00:37:08.966 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:08 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:08.671+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:08.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:08 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:08.671+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:08.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:08 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:08.671+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.354234+0000 front 2024-09-16T00:36:36.354282+0000 (oldest deadline 2024-09-16T00:37:02.254386+0000) 2024-09-16T00:37:08.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:08 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:08.671+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:43.955213+0000 front 2024-09-16T00:36:43.955262+0000 (oldest deadline 2024-09-16T00:37:07.956010+0000) 2024-09-16T00:37:09.294 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:08 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:37:08.904+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:19.296273+0000 front 2024-09-16T00:36:19.296363+0000 (oldest deadline 2024-09-16T00:36:43.297018+0000) 2024-09-16T00:37:09.294 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:08 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:37:08.904+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.997974+0000 front 2024-09-16T00:36:27.998173+0000 (oldest deadline 2024-09-16T00:36:52.697943+0000) 2024-09-16T00:37:09.294 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:08 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:37:08.904+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.800293+0000 front 2024-09-16T00:36:37.800202+0000 (oldest deadline 2024-09-16T00:37:00.000246+0000) 2024-09-16T00:37:09.294 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:08 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:37:08.904+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.201632+0000 front 2024-09-16T00:36:45.201611+0000 (oldest deadline 2024-09-16T00:37:08.701660+0000) 2024-09-16T00:37:09.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:09 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:09.185+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:37:09.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:09 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:09.185+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:37:09.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:09 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:09.185+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:30.072124+0000 front 2024-09-16T00:36:30.072271+0000 (oldest deadline 2024-09-16T00:36:52.371981+0000) 2024-09-16T00:37:09.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:09 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:09.185+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:33.472747+0000 front 2024-09-16T00:36:33.472711+0000 (oldest deadline 2024-09-16T00:36:59.372794+0000) 2024-09-16T00:37:09.466 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:09 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:09.005+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:37:09.467 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:09 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:09.005+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:20.095457+0000 front 2024-09-16T00:36:20.095507+0000 (oldest deadline 2024-09-16T00:36:44.795398+0000) 2024-09-16T00:37:09.467 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:09 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:09.005+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.696613+0000 front 2024-09-16T00:36:27.696680+0000 (oldest deadline 2024-09-16T00:36:51.196771+0000) 2024-09-16T00:37:09.467 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:09 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:09.005+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.898832+0000 front 2024-09-16T00:36:36.898816+0000 (oldest deadline 2024-09-16T00:37:00.998799+0000) 2024-09-16T00:37:09.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:09 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:09.348+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:37:09.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:09 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:09.348+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:37:09.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:09 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:09.348+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:37:09.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:09 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:09.348+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.627413+0000 front 2024-09-16T00:36:37.627341+0000 (oldest deadline 2024-09-16T00:36:59.327248+0000) 2024-09-16T00:37:09.966 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:09 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:09.634+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:09.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:09 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:09.634+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:09.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:09 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:09.634+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.354234+0000 front 2024-09-16T00:36:36.354282+0000 (oldest deadline 2024-09-16T00:37:02.254386+0000) 2024-09-16T00:37:09.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:09 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:09.634+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:43.955213+0000 front 2024-09-16T00:36:43.955262+0000 (oldest deadline 2024-09-16T00:37:07.956010+0000) 2024-09-16T00:37:10.293 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:09 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:37:09.912+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:19.296273+0000 front 2024-09-16T00:36:19.296363+0000 (oldest deadline 2024-09-16T00:36:43.297018+0000) 2024-09-16T00:37:10.294 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:09 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:37:09.912+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.997974+0000 front 2024-09-16T00:36:27.998173+0000 (oldest deadline 2024-09-16T00:36:52.697943+0000) 2024-09-16T00:37:10.294 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:09 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:37:09.912+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.800293+0000 front 2024-09-16T00:36:37.800202+0000 (oldest deadline 2024-09-16T00:37:00.000246+0000) 2024-09-16T00:37:10.294 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:09 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7[53235]: 2024-09-16T00:37:09.912+0000 7f519b762640 -1 osd.7 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.201632+0000 front 2024-09-16T00:36:45.201611+0000 (oldest deadline 2024-09-16T00:37:08.701660+0000) 2024-09-16T00:37:10.466 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:09 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:09.968+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:37:10.467 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:09 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:09.968+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:20.095457+0000 front 2024-09-16T00:36:20.095507+0000 (oldest deadline 2024-09-16T00:36:44.795398+0000) 2024-09-16T00:37:10.467 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:09 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:09.968+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.696613+0000 front 2024-09-16T00:36:27.696680+0000 (oldest deadline 2024-09-16T00:36:51.196771+0000) 2024-09-16T00:37:10.467 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:09 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:09.968+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.898832+0000 front 2024-09-16T00:36:36.898816+0000 (oldest deadline 2024-09-16T00:37:00.998799+0000) 2024-09-16T00:37:10.716 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:10 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:10.217+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:37:10.716 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:10 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:10.217+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:37:10.716 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:10 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:10.217+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:30.072124+0000 front 2024-09-16T00:36:30.072271+0000 (oldest deadline 2024-09-16T00:36:52.371981+0000) 2024-09-16T00:37:10.716 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:10 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:10.217+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:33.472747+0000 front 2024-09-16T00:36:33.472711+0000 (oldest deadline 2024-09-16T00:36:59.372794+0000) 2024-09-16T00:37:10.717 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:10 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:10.217+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.273657+0000 front 2024-09-16T00:36:45.273851+0000 (oldest deadline 2024-09-16T00:37:09.373684+0000) 2024-09-16T00:37:10.793 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:10 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:10.322+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:37:10.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:10 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:10.322+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:37:10.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:10 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:10.322+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:37:10.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:10 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:10.322+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.627413+0000 front 2024-09-16T00:36:37.627341+0000 (oldest deadline 2024-09-16T00:36:59.327248+0000) 2024-09-16T00:37:10.966 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:10 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:10.598+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:10.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:10 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:10.598+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:10.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:10 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:10.598+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.354234+0000 front 2024-09-16T00:36:36.354282+0000 (oldest deadline 2024-09-16T00:37:02.254386+0000) 2024-09-16T00:37:10.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:10 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:10.598+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:43.955213+0000 front 2024-09-16T00:36:43.955262+0000 (oldest deadline 2024-09-16T00:37:07.956010+0000) 2024-09-16T00:37:11.458 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:11 smithi073 podman[68941]: 2024-09-16 00:37:11.152715245 +0000 UTC m=+5.529986596 container died e35b8391b6db4be8bbd780a7150ef32b542d33a293f1f303126cd116205eb253 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-16T00:37:11.592 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:11 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:11.281+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:37:11.592 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:11 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:11.281+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:37:11.592 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:11 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:11.281+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:37:11.593 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:11 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:11.281+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.627413+0000 front 2024-09-16T00:36:37.627341+0000 (oldest deadline 2024-09-16T00:36:59.327248+0000) 2024-09-16T00:37:11.593 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:11 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:11.281+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.029004+0000 front 2024-09-16T00:36:45.028940+0000 (oldest deadline 2024-09-16T00:37:10.929146+0000) 2024-09-16T00:37:11.593 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:11 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:11.006+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:37:11.594 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:11 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:11.006+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:20.095457+0000 front 2024-09-16T00:36:20.095507+0000 (oldest deadline 2024-09-16T00:36:44.795398+0000) 2024-09-16T00:37:11.594 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:11 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:11.006+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.696613+0000 front 2024-09-16T00:36:27.696680+0000 (oldest deadline 2024-09-16T00:36:51.196771+0000) 2024-09-16T00:37:11.594 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:11 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:11.006+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.898832+0000 front 2024-09-16T00:36:36.898816+0000 (oldest deadline 2024-09-16T00:37:00.998799+0000) 2024-09-16T00:37:11.594 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:11 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:11.180+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:37:11.594 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:11 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:11.180+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:37:11.594 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:11 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:11.180+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:30.072124+0000 front 2024-09-16T00:36:30.072271+0000 (oldest deadline 2024-09-16T00:36:52.371981+0000) 2024-09-16T00:37:11.595 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:11 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:11.180+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:33.472747+0000 front 2024-09-16T00:36:33.472711+0000 (oldest deadline 2024-09-16T00:36:59.372794+0000) 2024-09-16T00:37:11.595 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:11 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:11.180+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.273657+0000 front 2024-09-16T00:36:45.273851+0000 (oldest deadline 2024-09-16T00:37:09.373684+0000) 2024-09-16T00:37:11.790 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:11 smithi073 podman[68941]: 2024-09-16 00:37:11.547404424 +0000 UTC m=+5.924675772 container cleanup e35b8391b6db4be8bbd780a7150ef32b542d33a293f1f303126cd116205eb253 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, RELEASE=squid-8921c8e, ceph=True, io.buildah.version=1.37.2) 2024-09-16T00:37:11.790 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:11 smithi073 bash[68941]: ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7 2024-09-16T00:37:11.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:11 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:11.565+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:11.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:11 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:11.565+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:11.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:11 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:11.565+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.354234+0000 front 2024-09-16T00:36:36.354282+0000 (oldest deadline 2024-09-16T00:37:02.254386+0000) 2024-09-16T00:37:11.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:11 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:11.565+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:43.955213+0000 front 2024-09-16T00:36:43.955262+0000 (oldest deadline 2024-09-16T00:37:07.956010+0000) 2024-09-16T00:37:12.043 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:11 smithi073 podman[68971]: 2024-09-16 00:37:11.790254957 +0000 UTC m=+0.633545091 container remove e35b8391b6db4be8bbd780a7150ef32b542d33a293f1f303126cd116205eb253 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:37:12.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:12 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:12.161+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:37:12.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:12 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:12.161+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:37:12.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:12 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:12.161+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:30.072124+0000 front 2024-09-16T00:36:30.072271+0000 (oldest deadline 2024-09-16T00:36:52.371981+0000) 2024-09-16T00:37:12.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:12 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:12.161+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:33.472747+0000 front 2024-09-16T00:36:33.472711+0000 (oldest deadline 2024-09-16T00:36:59.372794+0000) 2024-09-16T00:37:12.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:12 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:12.161+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.273657+0000 front 2024-09-16T00:36:45.273851+0000 (oldest deadline 2024-09-16T00:37:09.373684+0000) 2024-09-16T00:37:12.466 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:11 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:11.985+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:37:12.466 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:11 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:11.985+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:20.095457+0000 front 2024-09-16T00:36:20.095507+0000 (oldest deadline 2024-09-16T00:36:44.795398+0000) 2024-09-16T00:37:12.466 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:11 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:11.985+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.696613+0000 front 2024-09-16T00:36:27.696680+0000 (oldest deadline 2024-09-16T00:36:51.196771+0000) 2024-09-16T00:37:12.466 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:11 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:11.985+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.898832+0000 front 2024-09-16T00:36:36.898816+0000 (oldest deadline 2024-09-16T00:37:00.998799+0000) 2024-09-16T00:37:12.484 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:12 smithi073 podman[69056]: 2024-09-16 00:37:12.112632852 +0000 UTC m=+0.015687794 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:37:12.485 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:12 smithi073 podman[69056]: 2024-09-16 00:37:12.224608785 +0000 UTC m=+0.127663786 container create 53e5747b302d95dc43e0458f36ea5353c62dbe456b917313aff70f97e3947209 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7-deactivate, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-16T00:37:12.485 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:12 smithi073 podman[69056]: 2024-09-16 00:37:12.485357798 +0000 UTC m=+0.388412732 container init 53e5747b302d95dc43e0458f36ea5353c62dbe456b917313aff70f97e3947209 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-16T00:37:12.485 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:12 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:12.249+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:37:12.485 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:12 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:12.249+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:37:12.485 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:12 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:12.249+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:37:12.485 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:12 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:12.249+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.627413+0000 front 2024-09-16T00:36:37.627341+0000 (oldest deadline 2024-09-16T00:36:59.327248+0000) 2024-09-16T00:37:12.485 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:12 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:12.249+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.029004+0000 front 2024-09-16T00:36:45.028940+0000 (oldest deadline 2024-09-16T00:37:10.929146+0000) 2024-09-16T00:37:12.794 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:12 smithi073 podman[69056]: 2024-09-16 00:37:12.488746375 +0000 UTC m=+0.391801311 container start 53e5747b302d95dc43e0458f36ea5353c62dbe456b917313aff70f97e3947209 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-16T00:37:12.794 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:12 smithi073 podman[69056]: 2024-09-16 00:37:12.577050986 +0000 UTC m=+0.480105930 container attach 53e5747b302d95dc43e0458f36ea5353c62dbe456b917313aff70f97e3947209 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:37:12.794 INFO:journalctl@ceph.osd.7.smithi073.stdout:Sep 16 00:37:12 smithi073 podman[69056]: 2024-09-16 00:37:12.679090016 +0000 UTC m=+0.582144962 container died 53e5747b302d95dc43e0458f36ea5353c62dbe456b917313aff70f97e3947209 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-7-deactivate, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:37:12.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:12 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:12.588+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:12.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:12 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:12.588+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:12.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:12 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:12.588+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.354234+0000 front 2024-09-16T00:36:36.354282+0000 (oldest deadline 2024-09-16T00:37:02.254386+0000) 2024-09-16T00:37:12.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:12 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:12.588+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:43.955213+0000 front 2024-09-16T00:36:43.955262+0000 (oldest deadline 2024-09-16T00:37:07.956010+0000) 2024-09-16T00:37:13.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:13 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:13.125+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:37:13.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:13 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:13.125+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:37:13.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:13 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:13.125+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:30.072124+0000 front 2024-09-16T00:36:30.072271+0000 (oldest deadline 2024-09-16T00:36:52.371981+0000) 2024-09-16T00:37:13.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:13 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:13.125+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:33.472747+0000 front 2024-09-16T00:36:33.472711+0000 (oldest deadline 2024-09-16T00:36:59.372794+0000) 2024-09-16T00:37:13.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:13 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:13.125+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.273657+0000 front 2024-09-16T00:36:45.273851+0000 (oldest deadline 2024-09-16T00:37:09.373684+0000) 2024-09-16T00:37:13.466 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:13 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:13.007+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:37:13.466 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:13 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:13.007+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:20.095457+0000 front 2024-09-16T00:36:20.095507+0000 (oldest deadline 2024-09-16T00:36:44.795398+0000) 2024-09-16T00:37:13.466 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:13 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:13.007+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.696613+0000 front 2024-09-16T00:36:27.696680+0000 (oldest deadline 2024-09-16T00:36:51.196771+0000) 2024-09-16T00:37:13.467 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:13 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:13.007+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.898832+0000 front 2024-09-16T00:36:36.898816+0000 (oldest deadline 2024-09-16T00:37:00.998799+0000) 2024-09-16T00:37:13.480 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:13 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:13.240+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:37:13.481 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:13 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:13.240+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:37:13.481 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:13 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:13.240+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:37:13.481 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:13 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:13.240+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.627413+0000 front 2024-09-16T00:36:37.627341+0000 (oldest deadline 2024-09-16T00:36:59.327248+0000) 2024-09-16T00:37:13.481 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:13 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:13.240+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.029004+0000 front 2024-09-16T00:36:45.028940+0000 (oldest deadline 2024-09-16T00:37:10.929146+0000) 2024-09-16T00:37:13.500 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@osd.7.service' 2024-09-16T00:37:13.543 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:37:13.543 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-16T00:37:13.543 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-16T00:37:13.543 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-983b2072-73c1-11ef-bceb-c7b262605968@osd.8 2024-09-16T00:37:13.842 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:13 smithi122 systemd[1]: Stopping Ceph osd.8 for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:37:13.842 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:13 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:13.572+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:13.842 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:13 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:13.572+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:13.842 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:13 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:13.572+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.354234+0000 front 2024-09-16T00:36:36.354282+0000 (oldest deadline 2024-09-16T00:37:02.254386+0000) 2024-09-16T00:37:13.843 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:13 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:13.572+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:43.955213+0000 front 2024-09-16T00:36:43.955262+0000 (oldest deadline 2024-09-16T00:37:07.956010+0000) 2024-09-16T00:37:14.216 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:13 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:13.849+0000 7ff618166640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:37:14.217 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:13 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:13.849+0000 7ff618166640 -1 osd.8 96 *** Got signal Terminated *** 2024-09-16T00:37:14.217 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:13 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:13.849+0000 7ff618166640 -1 osd.8 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:37:14.217 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:13 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:13.990+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:37:14.217 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:13 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:13.990+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:20.095457+0000 front 2024-09-16T00:36:20.095507+0000 (oldest deadline 2024-09-16T00:36:44.795398+0000) 2024-09-16T00:37:14.217 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:13 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:13.990+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.696613+0000 front 2024-09-16T00:36:27.696680+0000 (oldest deadline 2024-09-16T00:36:51.196771+0000) 2024-09-16T00:37:14.217 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:13 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:13.990+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.898832+0000 front 2024-09-16T00:36:36.898816+0000 (oldest deadline 2024-09-16T00:37:00.998799+0000) 2024-09-16T00:37:14.464 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:14 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:14.134+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:37:14.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:14 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:14.134+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:37:14.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:14 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:14.134+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:30.072124+0000 front 2024-09-16T00:36:30.072271+0000 (oldest deadline 2024-09-16T00:36:52.371981+0000) 2024-09-16T00:37:14.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:14 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:14.134+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:33.472747+0000 front 2024-09-16T00:36:33.472711+0000 (oldest deadline 2024-09-16T00:36:59.372794+0000) 2024-09-16T00:37:14.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:14 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:14.134+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.273657+0000 front 2024-09-16T00:36:45.273851+0000 (oldest deadline 2024-09-16T00:37:09.373684+0000) 2024-09-16T00:37:14.544 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:14 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:14.275+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:37:14.544 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:14 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:14.275+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:37:14.544 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:14 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:14.275+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:37:14.544 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:14 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:14.275+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.627413+0000 front 2024-09-16T00:36:37.627341+0000 (oldest deadline 2024-09-16T00:36:59.327248+0000) 2024-09-16T00:37:14.544 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:14 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:14.275+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.029004+0000 front 2024-09-16T00:36:45.028940+0000 (oldest deadline 2024-09-16T00:37:10.929146+0000) 2024-09-16T00:37:14.716 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:14 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:14.548+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:14.716 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:14 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:14.548+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:14.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:14 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:14.548+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.354234+0000 front 2024-09-16T00:36:36.354282+0000 (oldest deadline 2024-09-16T00:37:02.254386+0000) 2024-09-16T00:37:14.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:14 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:14.548+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:43.955213+0000 front 2024-09-16T00:36:43.955262+0000 (oldest deadline 2024-09-16T00:37:07.956010+0000) 2024-09-16T00:37:15.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:15 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:15.179+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:37:15.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:15 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:15.179+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:37:15.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:15 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:15.179+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:30.072124+0000 front 2024-09-16T00:36:30.072271+0000 (oldest deadline 2024-09-16T00:36:52.371981+0000) 2024-09-16T00:37:15.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:15 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:15.179+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:33.472747+0000 front 2024-09-16T00:36:33.472711+0000 (oldest deadline 2024-09-16T00:36:59.372794+0000) 2024-09-16T00:37:15.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:15 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:15.179+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.273657+0000 front 2024-09-16T00:36:45.273851+0000 (oldest deadline 2024-09-16T00:37:09.373684+0000) 2024-09-16T00:37:15.466 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:15 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:15.024+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:37:15.466 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:15 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:15.024+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:20.095457+0000 front 2024-09-16T00:36:20.095507+0000 (oldest deadline 2024-09-16T00:36:44.795398+0000) 2024-09-16T00:37:15.467 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:15 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:15.024+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.696613+0000 front 2024-09-16T00:36:27.696680+0000 (oldest deadline 2024-09-16T00:36:51.196771+0000) 2024-09-16T00:37:15.467 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:15 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:15.024+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.898832+0000 front 2024-09-16T00:36:36.898816+0000 (oldest deadline 2024-09-16T00:37:00.998799+0000) 2024-09-16T00:37:15.467 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:15 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:15.024+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:52.100494+0000 front 2024-09-16T00:36:52.100511+0000 (oldest deadline 2024-09-16T00:37:15.000623+0000) 2024-09-16T00:37:15.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:15 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:15.325+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:37:15.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:15 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:15.325+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:37:15.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:15 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:15.325+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:37:15.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:15 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:15.325+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.627413+0000 front 2024-09-16T00:36:37.627341+0000 (oldest deadline 2024-09-16T00:36:59.327248+0000) 2024-09-16T00:37:15.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:15 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:15.325+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.029004+0000 front 2024-09-16T00:36:45.028940+0000 (oldest deadline 2024-09-16T00:37:10.929146+0000) 2024-09-16T00:37:15.966 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:15 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:15.588+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:15.966 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:15 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:15.588+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:15.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:15 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:15.588+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.354234+0000 front 2024-09-16T00:36:36.354282+0000 (oldest deadline 2024-09-16T00:37:02.254386+0000) 2024-09-16T00:37:15.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:15 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:15.588+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:43.955213+0000 front 2024-09-16T00:36:43.955262+0000 (oldest deadline 2024-09-16T00:37:07.956010+0000) 2024-09-16T00:37:16.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:16 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:16.175+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:37:16.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:16 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:16.175+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:37:16.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:16 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:16.175+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:30.072124+0000 front 2024-09-16T00:36:30.072271+0000 (oldest deadline 2024-09-16T00:36:52.371981+0000) 2024-09-16T00:37:16.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:16 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:16.175+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:33.472747+0000 front 2024-09-16T00:36:33.472711+0000 (oldest deadline 2024-09-16T00:36:59.372794+0000) 2024-09-16T00:37:16.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:16 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:16.175+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.273657+0000 front 2024-09-16T00:36:45.273851+0000 (oldest deadline 2024-09-16T00:37:09.373684+0000) 2024-09-16T00:37:16.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:16 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:16.175+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:49.374196+0000 front 2024-09-16T00:36:49.374075+0000 (oldest deadline 2024-09-16T00:37:15.274062+0000) 2024-09-16T00:37:16.466 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:15 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:15.976+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:37:16.466 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:15 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:15.976+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:20.095457+0000 front 2024-09-16T00:36:20.095507+0000 (oldest deadline 2024-09-16T00:36:44.795398+0000) 2024-09-16T00:37:16.466 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:15 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:15.976+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.696613+0000 front 2024-09-16T00:36:27.696680+0000 (oldest deadline 2024-09-16T00:36:51.196771+0000) 2024-09-16T00:37:16.467 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:15 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:15.976+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.898832+0000 front 2024-09-16T00:36:36.898816+0000 (oldest deadline 2024-09-16T00:37:00.998799+0000) 2024-09-16T00:37:16.467 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:15 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:15.976+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:52.100494+0000 front 2024-09-16T00:36:52.100511+0000 (oldest deadline 2024-09-16T00:37:15.000623+0000) 2024-09-16T00:37:16.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:16 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:16.321+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:37:16.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:16 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:16.321+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:37:16.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:16 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:16.321+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:37:16.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:16 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:16.321+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.627413+0000 front 2024-09-16T00:36:37.627341+0000 (oldest deadline 2024-09-16T00:36:59.327248+0000) 2024-09-16T00:37:16.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:16 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:16.321+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.029004+0000 front 2024-09-16T00:36:45.028940+0000 (oldest deadline 2024-09-16T00:37:10.929146+0000) 2024-09-16T00:37:16.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:16 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:16.321+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:50.929671+0000 front 2024-09-16T00:36:50.929733+0000 (oldest deadline 2024-09-16T00:37:15.629600+0000) 2024-09-16T00:37:16.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:16 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:16.572+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:16.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:16 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:16.572+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:16.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:16 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:16.572+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.354234+0000 front 2024-09-16T00:36:36.354282+0000 (oldest deadline 2024-09-16T00:37:02.254386+0000) 2024-09-16T00:37:16.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:16 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:16.572+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:43.955213+0000 front 2024-09-16T00:36:43.955262+0000 (oldest deadline 2024-09-16T00:37:07.956010+0000) 2024-09-16T00:37:17.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:17 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:17.203+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:37:17.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:17 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:17.203+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:37:17.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:17 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:17.203+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:30.072124+0000 front 2024-09-16T00:36:30.072271+0000 (oldest deadline 2024-09-16T00:36:52.371981+0000) 2024-09-16T00:37:17.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:17 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:17.203+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:33.472747+0000 front 2024-09-16T00:36:33.472711+0000 (oldest deadline 2024-09-16T00:36:59.372794+0000) 2024-09-16T00:37:17.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:17 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:17.203+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.273657+0000 front 2024-09-16T00:36:45.273851+0000 (oldest deadline 2024-09-16T00:37:09.373684+0000) 2024-09-16T00:37:17.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:17 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:17.203+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:49.374196+0000 front 2024-09-16T00:36:49.374075+0000 (oldest deadline 2024-09-16T00:37:15.274062+0000) 2024-09-16T00:37:17.466 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:16 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:16.992+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:37:17.466 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:16 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:16.992+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:20.095457+0000 front 2024-09-16T00:36:20.095507+0000 (oldest deadline 2024-09-16T00:36:44.795398+0000) 2024-09-16T00:37:17.467 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:16 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:16.992+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.696613+0000 front 2024-09-16T00:36:27.696680+0000 (oldest deadline 2024-09-16T00:36:51.196771+0000) 2024-09-16T00:37:17.467 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:16 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:16.992+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.898832+0000 front 2024-09-16T00:36:36.898816+0000 (oldest deadline 2024-09-16T00:37:00.998799+0000) 2024-09-16T00:37:17.467 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:16 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:16.992+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:52.100494+0000 front 2024-09-16T00:36:52.100511+0000 (oldest deadline 2024-09-16T00:37:15.000623+0000) 2024-09-16T00:37:17.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:17 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:17.368+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:37:17.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:17 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:17.368+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:37:17.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:17 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:17.368+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:37:17.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:17 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:17.368+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.627413+0000 front 2024-09-16T00:36:37.627341+0000 (oldest deadline 2024-09-16T00:36:59.327248+0000) 2024-09-16T00:37:17.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:17 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:17.368+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.029004+0000 front 2024-09-16T00:36:45.028940+0000 (oldest deadline 2024-09-16T00:37:10.929146+0000) 2024-09-16T00:37:17.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:17 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:17.368+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:50.929671+0000 front 2024-09-16T00:36:50.929733+0000 (oldest deadline 2024-09-16T00:37:15.629600+0000) 2024-09-16T00:37:17.966 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:17 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:17.572+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:17.966 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:17 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:17.572+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:17.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:17 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:17.572+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.354234+0000 front 2024-09-16T00:36:36.354282+0000 (oldest deadline 2024-09-16T00:37:02.254386+0000) 2024-09-16T00:37:17.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:17 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:17.572+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:43.955213+0000 front 2024-09-16T00:36:43.955262+0000 (oldest deadline 2024-09-16T00:37:07.956010+0000) 2024-09-16T00:37:18.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:18 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:18.199+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:37:18.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:18 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:18.199+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:37:18.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:18 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:18.199+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:30.072124+0000 front 2024-09-16T00:36:30.072271+0000 (oldest deadline 2024-09-16T00:36:52.371981+0000) 2024-09-16T00:37:18.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:18 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:18.199+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:33.472747+0000 front 2024-09-16T00:36:33.472711+0000 (oldest deadline 2024-09-16T00:36:59.372794+0000) 2024-09-16T00:37:18.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:18 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:18.199+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.273657+0000 front 2024-09-16T00:36:45.273851+0000 (oldest deadline 2024-09-16T00:37:09.373684+0000) 2024-09-16T00:37:18.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:18 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:18.199+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:49.374196+0000 front 2024-09-16T00:36:49.374075+0000 (oldest deadline 2024-09-16T00:37:15.274062+0000) 2024-09-16T00:37:18.466 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:17 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:17.998+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:12.593887+0000 front 2024-09-16T00:36:12.593745+0000 (oldest deadline 2024-09-16T00:36:35.493848+0000) 2024-09-16T00:37:18.466 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:17 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:17.998+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:20.095457+0000 front 2024-09-16T00:36:20.095507+0000 (oldest deadline 2024-09-16T00:36:44.795398+0000) 2024-09-16T00:37:18.466 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:17 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:17.998+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.696613+0000 front 2024-09-16T00:36:27.696680+0000 (oldest deadline 2024-09-16T00:36:51.196771+0000) 2024-09-16T00:37:18.466 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:17 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:17.998+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.898832+0000 front 2024-09-16T00:36:36.898816+0000 (oldest deadline 2024-09-16T00:37:00.998799+0000) 2024-09-16T00:37:18.467 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:17 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8[53774]: 2024-09-16T00:37:17.998+0000 7ff610774640 -1 osd.8 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:52.100494+0000 front 2024-09-16T00:36:52.100511+0000 (oldest deadline 2024-09-16T00:37:15.000623+0000) 2024-09-16T00:37:18.793 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:18 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:18.372+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:37:18.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:18 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:18.372+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:37:18.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:18 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:18.372+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:37:18.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:18 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:18.372+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.627413+0000 front 2024-09-16T00:36:37.627341+0000 (oldest deadline 2024-09-16T00:36:59.327248+0000) 2024-09-16T00:37:18.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:18 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:18.372+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.029004+0000 front 2024-09-16T00:36:45.028940+0000 (oldest deadline 2024-09-16T00:37:10.929146+0000) 2024-09-16T00:37:18.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:18 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:18.372+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:50.929671+0000 front 2024-09-16T00:36:50.929733+0000 (oldest deadline 2024-09-16T00:37:15.629600+0000) 2024-09-16T00:37:18.966 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:18 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:18.529+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:18.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:18 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:18.529+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:18.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:18 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:18.529+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.354234+0000 front 2024-09-16T00:36:36.354282+0000 (oldest deadline 2024-09-16T00:37:02.254386+0000) 2024-09-16T00:37:18.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:18 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:18.529+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:43.955213+0000 front 2024-09-16T00:36:43.955262+0000 (oldest deadline 2024-09-16T00:37:07.956010+0000) 2024-09-16T00:37:18.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:18 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:18.529+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:54.357203+0000 front 2024-09-16T00:36:54.357260+0000 (oldest deadline 2024-09-16T00:37:17.857349+0000) 2024-09-16T00:37:19.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:19 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:19.153+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:37:19.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:19 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:19.153+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:37:19.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:19 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:19.153+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:30.072124+0000 front 2024-09-16T00:36:30.072271+0000 (oldest deadline 2024-09-16T00:36:52.371981+0000) 2024-09-16T00:37:19.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:19 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:19.153+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:33.472747+0000 front 2024-09-16T00:36:33.472711+0000 (oldest deadline 2024-09-16T00:36:59.372794+0000) 2024-09-16T00:37:19.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:19 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:19.153+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.273657+0000 front 2024-09-16T00:36:45.273851+0000 (oldest deadline 2024-09-16T00:37:09.373684+0000) 2024-09-16T00:37:19.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:19 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:19.153+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:49.374196+0000 front 2024-09-16T00:36:49.374075+0000 (oldest deadline 2024-09-16T00:37:15.274062+0000) 2024-09-16T00:37:19.503 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:19 smithi122 podman[69046]: 2024-09-16 00:37:19.252545759 +0000 UTC m=+5.506636192 container died 731b66bd4090e752bcb1840cab5c071497a38ea7e4d2ea6c86c8c41c78e65cd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-16T00:37:19.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:19 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:19.409+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:37:19.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:19 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:19.409+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:37:19.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:19 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:19.409+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:37:19.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:19 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:19.409+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.627413+0000 front 2024-09-16T00:36:37.627341+0000 (oldest deadline 2024-09-16T00:36:59.327248+0000) 2024-09-16T00:37:19.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:19 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:19.409+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.029004+0000 front 2024-09-16T00:36:45.028940+0000 (oldest deadline 2024-09-16T00:37:10.929146+0000) 2024-09-16T00:37:19.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:19 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:19.409+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:50.929671+0000 front 2024-09-16T00:36:50.929733+0000 (oldest deadline 2024-09-16T00:37:15.629600+0000) 2024-09-16T00:37:19.909 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:19 smithi122 podman[69046]: 2024-09-16 00:37:19.690971813 +0000 UTC m=+5.945062254 container cleanup 731b66bd4090e752bcb1840cab5c071497a38ea7e4d2ea6c86c8c41c78e65cd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-09-16T00:37:19.909 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:19 smithi122 bash[69046]: ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8 2024-09-16T00:37:19.909 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:19 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:19.502+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:19.909 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:19 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:19.502+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:19.909 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:19 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:19.502+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.354234+0000 front 2024-09-16T00:36:36.354282+0000 (oldest deadline 2024-09-16T00:37:02.254386+0000) 2024-09-16T00:37:19.909 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:19 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:19.502+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:43.955213+0000 front 2024-09-16T00:36:43.955262+0000 (oldest deadline 2024-09-16T00:37:07.956010+0000) 2024-09-16T00:37:19.909 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:19 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:19.502+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:54.357203+0000 front 2024-09-16T00:36:54.357260+0000 (oldest deadline 2024-09-16T00:37:17.857349+0000) 2024-09-16T00:37:20.216 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:19 smithi122 podman[69061]: 2024-09-16 00:37:19.908788325 +0000 UTC m=+0.650119710 container remove 731b66bd4090e752bcb1840cab5c071497a38ea7e4d2ea6c86c8c41c78e65cd3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-16T00:37:20.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:20 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:20.111+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:37:20.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:20 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:20.111+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:37:20.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:20 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:20.111+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:30.072124+0000 front 2024-09-16T00:36:30.072271+0000 (oldest deadline 2024-09-16T00:36:52.371981+0000) 2024-09-16T00:37:20.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:20 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:20.111+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:33.472747+0000 front 2024-09-16T00:36:33.472711+0000 (oldest deadline 2024-09-16T00:36:59.372794+0000) 2024-09-16T00:37:20.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:20 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:20.111+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.273657+0000 front 2024-09-16T00:36:45.273851+0000 (oldest deadline 2024-09-16T00:37:09.373684+0000) 2024-09-16T00:37:20.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:20 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:20.111+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:49.374196+0000 front 2024-09-16T00:36:49.374075+0000 (oldest deadline 2024-09-16T00:37:15.274062+0000) 2024-09-16T00:37:20.603 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:20 smithi122 podman[69152]: 2024-09-16 00:37:20.219445621 +0000 UTC m=+0.015970818 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:37:20.604 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:20 smithi122 podman[69152]: 2024-09-16 00:37:20.343310685 +0000 UTC m=+0.139835867 container create 9d1ee33f49f612d389eddb0acaa0d5b637700e6198413847515640c15437be17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:37:20.604 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:20 smithi122 podman[69152]: 2024-09-16 00:37:20.604115644 +0000 UTC m=+0.400640836 container init 9d1ee33f49f612d389eddb0acaa0d5b637700e6198413847515640c15437be17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-16T00:37:20.604 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:20 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:20.534+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:20.604 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:20 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:20.534+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:20.604 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:20 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:20.534+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.354234+0000 front 2024-09-16T00:36:36.354282+0000 (oldest deadline 2024-09-16T00:37:02.254386+0000) 2024-09-16T00:37:20.604 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:20 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:20.534+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:43.955213+0000 front 2024-09-16T00:36:43.955262+0000 (oldest deadline 2024-09-16T00:37:07.956010+0000) 2024-09-16T00:37:20.604 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:20 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:20.534+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:54.357203+0000 front 2024-09-16T00:36:54.357260+0000 (oldest deadline 2024-09-16T00:37:17.857349+0000) 2024-09-16T00:37:20.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:20 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:20.421+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:37:20.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:20 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:20.421+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:37:20.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:20 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:20.421+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:37:20.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:20 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:20.421+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.627413+0000 front 2024-09-16T00:36:37.627341+0000 (oldest deadline 2024-09-16T00:36:59.327248+0000) 2024-09-16T00:37:20.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:20 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:20.421+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.029004+0000 front 2024-09-16T00:36:45.028940+0000 (oldest deadline 2024-09-16T00:37:10.929146+0000) 2024-09-16T00:37:20.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:20 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:20.421+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:50.929671+0000 front 2024-09-16T00:36:50.929733+0000 (oldest deadline 2024-09-16T00:37:15.629600+0000) 2024-09-16T00:37:20.967 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:20 smithi122 podman[69152]: 2024-09-16 00:37:20.607702211 +0000 UTC m=+0.404227396 container start 9d1ee33f49f612d389eddb0acaa0d5b637700e6198413847515640c15437be17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-16T00:37:20.967 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:20 smithi122 podman[69152]: 2024-09-16 00:37:20.720787199 +0000 UTC m=+0.517312382 container attach 9d1ee33f49f612d389eddb0acaa0d5b637700e6198413847515640c15437be17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-16T00:37:20.967 INFO:journalctl@ceph.osd.8.smithi122.stdout:Sep 16 00:37:20 smithi122 podman[69152]: 2024-09-16 00:37:20.803199276 +0000 UTC m=+0.599724469 container died 9d1ee33f49f612d389eddb0acaa0d5b637700e6198413847515640c15437be17 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-16T00:37:21.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:21 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:21.070+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:37:21.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:21 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:21.070+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:37:21.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:21 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:21.070+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:30.072124+0000 front 2024-09-16T00:36:30.072271+0000 (oldest deadline 2024-09-16T00:36:52.371981+0000) 2024-09-16T00:37:21.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:21 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:21.070+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:33.472747+0000 front 2024-09-16T00:36:33.472711+0000 (oldest deadline 2024-09-16T00:36:59.372794+0000) 2024-09-16T00:37:21.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:21 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:21.070+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.273657+0000 front 2024-09-16T00:36:45.273851+0000 (oldest deadline 2024-09-16T00:37:09.373684+0000) 2024-09-16T00:37:21.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:21 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:21.070+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:49.374196+0000 front 2024-09-16T00:36:49.374075+0000 (oldest deadline 2024-09-16T00:37:15.274062+0000) 2024-09-16T00:37:21.477 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@osd.8.service' 2024-09-16T00:37:21.519 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:37:21.519 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-16T00:37:21.519 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-16T00:37:21.519 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-983b2072-73c1-11ef-bceb-c7b262605968@osd.9 2024-09-16T00:37:21.716 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:21 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:21.512+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:21.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:21 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:21.512+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:21.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:21 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:21.512+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.354234+0000 front 2024-09-16T00:36:36.354282+0000 (oldest deadline 2024-09-16T00:37:02.254386+0000) 2024-09-16T00:37:21.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:21 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:21.512+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:43.955213+0000 front 2024-09-16T00:36:43.955262+0000 (oldest deadline 2024-09-16T00:37:07.956010+0000) 2024-09-16T00:37:21.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:21 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:21.512+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:54.357203+0000 front 2024-09-16T00:36:54.357260+0000 (oldest deadline 2024-09-16T00:37:17.857349+0000) 2024-09-16T00:37:21.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:21 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:21.424+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:37:21.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:21 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:21.424+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:37:21.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:21 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:21.424+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:37:21.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:21 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:21.424+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.627413+0000 front 2024-09-16T00:36:37.627341+0000 (oldest deadline 2024-09-16T00:36:59.327248+0000) 2024-09-16T00:37:21.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:21 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:21.424+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.029004+0000 front 2024-09-16T00:36:45.028940+0000 (oldest deadline 2024-09-16T00:37:10.929146+0000) 2024-09-16T00:37:21.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:21 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:21.424+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:50.929671+0000 front 2024-09-16T00:36:50.929733+0000 (oldest deadline 2024-09-16T00:37:15.629600+0000) 2024-09-16T00:37:21.857 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:21 smithi038 systemd[1]: Stopping Ceph osd.9 for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:37:21.857 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:21 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:21.857+0000 7f306df18640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:37:21.857 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:21 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:21.857+0000 7f306df18640 -1 osd.9 96 *** Got signal Terminated *** 2024-09-16T00:37:22.215 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:21 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:21.857+0000 7f306df18640 -1 osd.9 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:37:22.215 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:22 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:22.033+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:37:22.215 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:22 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:22.033+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:37:22.215 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:22 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:22.033+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:30.072124+0000 front 2024-09-16T00:36:30.072271+0000 (oldest deadline 2024-09-16T00:36:52.371981+0000) 2024-09-16T00:37:22.215 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:22 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:22.033+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:33.472747+0000 front 2024-09-16T00:36:33.472711+0000 (oldest deadline 2024-09-16T00:36:59.372794+0000) 2024-09-16T00:37:22.215 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:22 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:22.033+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.273657+0000 front 2024-09-16T00:36:45.273851+0000 (oldest deadline 2024-09-16T00:37:09.373684+0000) 2024-09-16T00:37:22.215 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:22 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:22.033+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:49.374196+0000 front 2024-09-16T00:36:49.374075+0000 (oldest deadline 2024-09-16T00:37:15.274062+0000) 2024-09-16T00:37:22.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:22 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:22.399+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:37:22.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:22 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:22.399+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:37:22.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:22 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:22.399+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:37:22.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:22 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:22.399+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.627413+0000 front 2024-09-16T00:36:37.627341+0000 (oldest deadline 2024-09-16T00:36:59.327248+0000) 2024-09-16T00:37:22.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:22 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:22.399+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.029004+0000 front 2024-09-16T00:36:45.028940+0000 (oldest deadline 2024-09-16T00:37:10.929146+0000) 2024-09-16T00:37:22.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:22 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:22.399+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:50.929671+0000 front 2024-09-16T00:36:50.929733+0000 (oldest deadline 2024-09-16T00:37:15.629600+0000) 2024-09-16T00:37:22.966 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:22 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:22.559+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:22.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:22 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:22.559+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:22.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:22 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:22.559+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.354234+0000 front 2024-09-16T00:36:36.354282+0000 (oldest deadline 2024-09-16T00:37:02.254386+0000) 2024-09-16T00:37:22.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:22 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:22.559+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:43.955213+0000 front 2024-09-16T00:36:43.955262+0000 (oldest deadline 2024-09-16T00:37:07.956010+0000) 2024-09-16T00:37:22.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:22 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:22.559+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:54.357203+0000 front 2024-09-16T00:36:54.357260+0000 (oldest deadline 2024-09-16T00:37:17.857349+0000) 2024-09-16T00:37:23.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:23 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:23.028+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:37:23.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:23 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:23.028+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:37:23.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:23 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:23.028+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:30.072124+0000 front 2024-09-16T00:36:30.072271+0000 (oldest deadline 2024-09-16T00:36:52.371981+0000) 2024-09-16T00:37:23.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:23 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:23.028+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:33.472747+0000 front 2024-09-16T00:36:33.472711+0000 (oldest deadline 2024-09-16T00:36:59.372794+0000) 2024-09-16T00:37:23.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:23 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:23.028+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.273657+0000 front 2024-09-16T00:36:45.273851+0000 (oldest deadline 2024-09-16T00:37:09.373684+0000) 2024-09-16T00:37:23.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:23 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:23.028+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:49.374196+0000 front 2024-09-16T00:36:49.374075+0000 (oldest deadline 2024-09-16T00:37:15.274062+0000) 2024-09-16T00:37:23.514 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:23 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:23.513+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:23.515 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:23 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:23.513+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:23.515 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:23 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:23.513+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.354234+0000 front 2024-09-16T00:36:36.354282+0000 (oldest deadline 2024-09-16T00:37:02.254386+0000) 2024-09-16T00:37:23.515 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:23 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:23.513+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:43.955213+0000 front 2024-09-16T00:36:43.955262+0000 (oldest deadline 2024-09-16T00:37:07.956010+0000) 2024-09-16T00:37:23.515 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:23 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:23.513+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:54.357203+0000 front 2024-09-16T00:36:54.357260+0000 (oldest deadline 2024-09-16T00:37:17.857349+0000) 2024-09-16T00:37:23.515 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:23 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:23.513+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-16T00:36:57.858011+0000 front 2024-09-16T00:36:57.858024+0000 (oldest deadline 2024-09-16T00:37:23.157915+0000) 2024-09-16T00:37:23.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:23 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:23.415+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:37:23.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:23 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:23.415+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:37:23.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:23 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:23.415+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:37:23.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:23 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:23.415+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.627413+0000 front 2024-09-16T00:36:37.627341+0000 (oldest deadline 2024-09-16T00:36:59.327248+0000) 2024-09-16T00:37:23.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:23 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:23.415+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.029004+0000 front 2024-09-16T00:36:45.028940+0000 (oldest deadline 2024-09-16T00:37:10.929146+0000) 2024-09-16T00:37:23.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:23 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:23.415+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:50.929671+0000 front 2024-09-16T00:36:50.929733+0000 (oldest deadline 2024-09-16T00:37:15.629600+0000) 2024-09-16T00:37:24.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:24 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:24.073+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:37:24.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:24 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:24.073+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:37:24.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:24 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:24.073+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:30.072124+0000 front 2024-09-16T00:36:30.072271+0000 (oldest deadline 2024-09-16T00:36:52.371981+0000) 2024-09-16T00:37:24.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:24 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:24.073+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:33.472747+0000 front 2024-09-16T00:36:33.472711+0000 (oldest deadline 2024-09-16T00:36:59.372794+0000) 2024-09-16T00:37:24.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:24 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:24.073+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.273657+0000 front 2024-09-16T00:36:45.273851+0000 (oldest deadline 2024-09-16T00:37:09.373684+0000) 2024-09-16T00:37:24.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:24 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:24.073+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:49.374196+0000 front 2024-09-16T00:36:49.374075+0000 (oldest deadline 2024-09-16T00:37:15.274062+0000) 2024-09-16T00:37:24.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:24 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:24.387+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:37:24.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:24 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:24.387+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:37:24.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:24 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:24.387+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:37:24.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:24 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:24.387+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.627413+0000 front 2024-09-16T00:36:37.627341+0000 (oldest deadline 2024-09-16T00:36:59.327248+0000) 2024-09-16T00:37:24.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:24 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:24.387+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.029004+0000 front 2024-09-16T00:36:45.028940+0000 (oldest deadline 2024-09-16T00:37:10.929146+0000) 2024-09-16T00:37:24.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:24 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:24.387+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:50.929671+0000 front 2024-09-16T00:36:50.929733+0000 (oldest deadline 2024-09-16T00:37:15.629600+0000) 2024-09-16T00:37:24.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:24 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:24.387+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-16T00:36:58.530860+0000 front 2024-09-16T00:36:58.530947+0000 (oldest deadline 2024-09-16T00:37:23.830577+0000) 2024-09-16T00:37:24.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:24 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:24.547+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:24.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:24 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:24.547+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:24.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:24 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:24.547+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.354234+0000 front 2024-09-16T00:36:36.354282+0000 (oldest deadline 2024-09-16T00:37:02.254386+0000) 2024-09-16T00:37:24.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:24 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:24.547+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:43.955213+0000 front 2024-09-16T00:36:43.955262+0000 (oldest deadline 2024-09-16T00:37:07.956010+0000) 2024-09-16T00:37:24.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:24 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:24.547+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:54.357203+0000 front 2024-09-16T00:36:54.357260+0000 (oldest deadline 2024-09-16T00:37:17.857349+0000) 2024-09-16T00:37:24.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:24 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:24.547+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-16T00:36:57.858011+0000 front 2024-09-16T00:36:57.858024+0000 (oldest deadline 2024-09-16T00:37:23.157915+0000) 2024-09-16T00:37:25.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:25 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:25.084+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:37:25.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:25 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:25.084+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:37:25.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:25 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:25.084+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:30.072124+0000 front 2024-09-16T00:36:30.072271+0000 (oldest deadline 2024-09-16T00:36:52.371981+0000) 2024-09-16T00:37:25.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:25 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:25.084+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:33.472747+0000 front 2024-09-16T00:36:33.472711+0000 (oldest deadline 2024-09-16T00:36:59.372794+0000) 2024-09-16T00:37:25.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:25 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:25.084+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.273657+0000 front 2024-09-16T00:36:45.273851+0000 (oldest deadline 2024-09-16T00:37:09.373684+0000) 2024-09-16T00:37:25.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:25 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:25.084+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:49.374196+0000 front 2024-09-16T00:36:49.374075+0000 (oldest deadline 2024-09-16T00:37:15.274062+0000) 2024-09-16T00:37:25.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:25 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:25.385+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:37:25.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:25 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:25.385+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:37:25.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:25 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:25.385+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:37:25.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:25 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:25.385+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.627413+0000 front 2024-09-16T00:36:37.627341+0000 (oldest deadline 2024-09-16T00:36:59.327248+0000) 2024-09-16T00:37:25.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:25 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:25.385+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.029004+0000 front 2024-09-16T00:36:45.028940+0000 (oldest deadline 2024-09-16T00:37:10.929146+0000) 2024-09-16T00:37:25.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:25 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:25.385+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:50.929671+0000 front 2024-09-16T00:36:50.929733+0000 (oldest deadline 2024-09-16T00:37:15.629600+0000) 2024-09-16T00:37:25.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:25 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:25.385+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-16T00:36:58.530860+0000 front 2024-09-16T00:36:58.530947+0000 (oldest deadline 2024-09-16T00:37:23.830577+0000) 2024-09-16T00:37:25.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:25 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:25.502+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:25.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:25 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:25.502+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:25.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:25 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:25.502+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.354234+0000 front 2024-09-16T00:36:36.354282+0000 (oldest deadline 2024-09-16T00:37:02.254386+0000) 2024-09-16T00:37:25.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:25 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:25.502+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:43.955213+0000 front 2024-09-16T00:36:43.955262+0000 (oldest deadline 2024-09-16T00:37:07.956010+0000) 2024-09-16T00:37:25.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:25 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:25.502+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:54.357203+0000 front 2024-09-16T00:36:54.357260+0000 (oldest deadline 2024-09-16T00:37:17.857349+0000) 2024-09-16T00:37:25.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:25 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:25.502+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-16T00:36:57.858011+0000 front 2024-09-16T00:36:57.858024+0000 (oldest deadline 2024-09-16T00:37:23.157915+0000) 2024-09-16T00:37:26.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:26 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:26.039+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.467752+0000 front 2024-09-16T00:36:10.467785+0000 (oldest deadline 2024-09-16T00:36:35.767889+0000) 2024-09-16T00:37:26.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:26 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:26.039+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:21.470032+0000 front 2024-09-16T00:36:21.469911+0000 (oldest deadline 2024-09-16T00:36:43.769596+0000) 2024-09-16T00:37:26.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:26 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:26.039+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:30.072124+0000 front 2024-09-16T00:36:30.072271+0000 (oldest deadline 2024-09-16T00:36:52.371981+0000) 2024-09-16T00:37:26.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:26 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:26.039+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:33.472747+0000 front 2024-09-16T00:36:33.472711+0000 (oldest deadline 2024-09-16T00:36:59.372794+0000) 2024-09-16T00:37:26.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:26 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:26.039+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.273657+0000 front 2024-09-16T00:36:45.273851+0000 (oldest deadline 2024-09-16T00:37:09.373684+0000) 2024-09-16T00:37:26.465 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:26 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9[64683]: 2024-09-16T00:37:26.039+0000 7f306a52e640 -1 osd.9 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:49.374196+0000 front 2024-09-16T00:36:49.374075+0000 (oldest deadline 2024-09-16T00:37:15.274062+0000) 2024-09-16T00:37:26.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:26 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:26.465+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:26.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:26 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:26.465+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:26.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:26 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:26.465+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.354234+0000 front 2024-09-16T00:36:36.354282+0000 (oldest deadline 2024-09-16T00:37:02.254386+0000) 2024-09-16T00:37:26.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:26 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:26.465+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:43.955213+0000 front 2024-09-16T00:36:43.955262+0000 (oldest deadline 2024-09-16T00:37:07.956010+0000) 2024-09-16T00:37:26.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:26 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:26.465+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:54.357203+0000 front 2024-09-16T00:36:54.357260+0000 (oldest deadline 2024-09-16T00:37:17.857349+0000) 2024-09-16T00:37:26.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:26 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:26.465+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-16T00:36:57.858011+0000 front 2024-09-16T00:36:57.858024+0000 (oldest deadline 2024-09-16T00:37:23.157915+0000) 2024-09-16T00:37:26.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:26 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:26.347+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:37:26.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:26 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:26.347+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:37:26.795 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:26 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:26.347+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:37:26.795 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:26 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:26.347+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.627413+0000 front 2024-09-16T00:36:37.627341+0000 (oldest deadline 2024-09-16T00:36:59.327248+0000) 2024-09-16T00:37:26.795 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:26 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:26.347+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.029004+0000 front 2024-09-16T00:36:45.028940+0000 (oldest deadline 2024-09-16T00:37:10.929146+0000) 2024-09-16T00:37:26.795 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:26 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:26.347+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:50.929671+0000 front 2024-09-16T00:36:50.929733+0000 (oldest deadline 2024-09-16T00:37:15.629600+0000) 2024-09-16T00:37:26.795 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:26 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:26.347+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-16T00:36:58.530860+0000 front 2024-09-16T00:36:58.530947+0000 (oldest deadline 2024-09-16T00:37:23.830577+0000) 2024-09-16T00:37:27.457 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:27 smithi038 podman[94532]: 2024-09-16 00:37:27.19052229 +0000 UTC m=+5.459502481 container died fcdd85ecf9a6eaa244d944907f55e4f04329dbb82555df9f532f969d6b66e89d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9, ceph=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731) 2024-09-16T00:37:27.714 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:27 smithi038 podman[94532]: 2024-09-16 00:37:27.623266286 +0000 UTC m=+5.892246467 container cleanup fcdd85ecf9a6eaa244d944907f55e4f04329dbb82555df9f532f969d6b66e89d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:37:27.715 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:27 smithi038 bash[94532]: ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9 2024-09-16T00:37:27.716 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:27 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:27.421+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:27.716 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:27 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:27.421+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:27.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:27 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:27.421+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.354234+0000 front 2024-09-16T00:36:36.354282+0000 (oldest deadline 2024-09-16T00:37:02.254386+0000) 2024-09-16T00:37:27.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:27 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:27.421+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:43.955213+0000 front 2024-09-16T00:36:43.955262+0000 (oldest deadline 2024-09-16T00:37:07.956010+0000) 2024-09-16T00:37:27.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:27 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:27.421+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:54.357203+0000 front 2024-09-16T00:36:54.357260+0000 (oldest deadline 2024-09-16T00:37:17.857349+0000) 2024-09-16T00:37:27.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:27 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:27.421+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-16T00:36:57.858011+0000 front 2024-09-16T00:36:57.858024+0000 (oldest deadline 2024-09-16T00:37:23.157915+0000) 2024-09-16T00:37:27.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:27 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:27.397+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:37:27.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:27 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:27.397+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:37:27.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:27 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:27.397+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:37:27.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:27 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:27.397+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.627413+0000 front 2024-09-16T00:36:37.627341+0000 (oldest deadline 2024-09-16T00:36:59.327248+0000) 2024-09-16T00:37:27.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:27 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:27.397+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.029004+0000 front 2024-09-16T00:36:45.028940+0000 (oldest deadline 2024-09-16T00:37:10.929146+0000) 2024-09-16T00:37:27.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:27 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:27.397+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:50.929671+0000 front 2024-09-16T00:36:50.929733+0000 (oldest deadline 2024-09-16T00:37:15.629600+0000) 2024-09-16T00:37:27.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:27 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:27.397+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-16T00:36:58.530860+0000 front 2024-09-16T00:36:58.530947+0000 (oldest deadline 2024-09-16T00:37:23.830577+0000) 2024-09-16T00:37:28.215 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:27 smithi038 podman[94549]: 2024-09-16 00:37:27.832944513 +0000 UTC m=+0.642899758 container remove fcdd85ecf9a6eaa244d944907f55e4f04329dbb82555df9f532f969d6b66e89d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-16T00:37:28.552 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:28 smithi038 podman[94638]: 2024-09-16 00:37:28.1471522 +0000 UTC m=+0.016446416 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:37:28.552 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:28 smithi038 podman[94638]: 2024-09-16 00:37:28.250389767 +0000 UTC m=+0.119684013 container create 3c731d0d3771e377090710f7ba8e72914efe0862d88e75c5f0068066faf16649 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True) 2024-09-16T00:37:28.552 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:28 smithi038 podman[94638]: 2024-09-16 00:37:28.469254138 +0000 UTC m=+0.338548339 container init 3c731d0d3771e377090710f7ba8e72914efe0862d88e75c5f0068066faf16649 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-16T00:37:28.552 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:28 smithi038 podman[94638]: 2024-09-16 00:37:28.472718072 +0000 UTC m=+0.342012273 container start 3c731d0d3771e377090710f7ba8e72914efe0862d88e75c5f0068066faf16649 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9-deactivate, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-16T00:37:28.553 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:28 smithi038 podman[94638]: 2024-09-16 00:37:28.552514741 +0000 UTC m=+0.421808942 container attach 3c731d0d3771e377090710f7ba8e72914efe0862d88e75c5f0068066faf16649 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9-deactivate, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e) 2024-09-16T00:37:28.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:28 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:28.427+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:28.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:28 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:28.427+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:28.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:28 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:28.427+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.354234+0000 front 2024-09-16T00:36:36.354282+0000 (oldest deadline 2024-09-16T00:37:02.254386+0000) 2024-09-16T00:37:28.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:28 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:28.427+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:43.955213+0000 front 2024-09-16T00:36:43.955262+0000 (oldest deadline 2024-09-16T00:37:07.956010+0000) 2024-09-16T00:37:28.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:28 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:28.427+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:54.357203+0000 front 2024-09-16T00:36:54.357260+0000 (oldest deadline 2024-09-16T00:37:17.857349+0000) 2024-09-16T00:37:28.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:28 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:28.427+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-16T00:36:57.858011+0000 front 2024-09-16T00:36:57.858024+0000 (oldest deadline 2024-09-16T00:37:23.157915+0000) 2024-09-16T00:37:28.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:28 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:28.431+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:37:28.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:28 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:28.431+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:37:28.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:28 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:28.431+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:37:28.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:28 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:28.431+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.627413+0000 front 2024-09-16T00:36:37.627341+0000 (oldest deadline 2024-09-16T00:36:59.327248+0000) 2024-09-16T00:37:28.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:28 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:28.431+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.029004+0000 front 2024-09-16T00:36:45.028940+0000 (oldest deadline 2024-09-16T00:37:10.929146+0000) 2024-09-16T00:37:28.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:28 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:28.431+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:50.929671+0000 front 2024-09-16T00:36:50.929733+0000 (oldest deadline 2024-09-16T00:37:15.629600+0000) 2024-09-16T00:37:28.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:28 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:28.431+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-16T00:36:58.530860+0000 front 2024-09-16T00:36:58.530947+0000 (oldest deadline 2024-09-16T00:37:23.830577+0000) 2024-09-16T00:37:28.964 INFO:journalctl@ceph.osd.9.smithi038.stdout:Sep 16 00:37:28 smithi038 podman[94638]: 2024-09-16 00:37:28.677769641 +0000 UTC m=+0.547063856 container died 3c731d0d3771e377090710f7ba8e72914efe0862d88e75c5f0068066faf16649 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-9-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS) 2024-09-16T00:37:29.384 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@osd.9.service' 2024-09-16T00:37:29.426 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:37:29.426 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-16T00:37:29.426 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-16T00:37:29.426 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-983b2072-73c1-11ef-bceb-c7b262605968@osd.10 2024-09-16T00:37:29.716 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:29 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:29.456+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:29.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:29 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:29.456+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:29.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:29 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:29.456+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.354234+0000 front 2024-09-16T00:36:36.354282+0000 (oldest deadline 2024-09-16T00:37:02.254386+0000) 2024-09-16T00:37:29.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:29 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:29.456+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:43.955213+0000 front 2024-09-16T00:36:43.955262+0000 (oldest deadline 2024-09-16T00:37:07.956010+0000) 2024-09-16T00:37:29.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:29 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:29.456+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:54.357203+0000 front 2024-09-16T00:36:54.357260+0000 (oldest deadline 2024-09-16T00:37:17.857349+0000) 2024-09-16T00:37:29.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:29 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:29.456+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-16T00:36:57.858011+0000 front 2024-09-16T00:36:57.858024+0000 (oldest deadline 2024-09-16T00:37:23.157915+0000) 2024-09-16T00:37:29.759 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:29 smithi073 systemd[1]: Stopping Ceph osd.10 for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:37:29.760 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:29 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:29.479+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:37:29.760 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:29 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:29.479+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:37:29.760 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:29 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:29.479+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:37:29.760 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:29 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:29.479+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.627413+0000 front 2024-09-16T00:36:37.627341+0000 (oldest deadline 2024-09-16T00:36:59.327248+0000) 2024-09-16T00:37:29.760 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:29 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:29.479+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.029004+0000 front 2024-09-16T00:36:45.028940+0000 (oldest deadline 2024-09-16T00:37:10.929146+0000) 2024-09-16T00:37:29.760 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:29 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:29.479+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:50.929671+0000 front 2024-09-16T00:36:50.929733+0000 (oldest deadline 2024-09-16T00:37:15.629600+0000) 2024-09-16T00:37:29.760 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:29 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:29.479+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-16T00:36:58.530860+0000 front 2024-09-16T00:36:58.530947+0000 (oldest deadline 2024-09-16T00:37:23.830577+0000) 2024-09-16T00:37:29.760 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:29 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:29.759+0000 7fb12a8a2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:37:29.760 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:29 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:29.759+0000 7fb12a8a2640 -1 osd.10 96 *** Got signal Terminated *** 2024-09-16T00:37:29.760 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:29 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:29.759+0000 7fb12a8a2640 -1 osd.10 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:37:30.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:30 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:30.465+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:37:30.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:30 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:30.465+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:37:30.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:30 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:30.465+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:37:30.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:30 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:30.465+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.627413+0000 front 2024-09-16T00:36:37.627341+0000 (oldest deadline 2024-09-16T00:36:59.327248+0000) 2024-09-16T00:37:30.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:30 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:30.465+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.029004+0000 front 2024-09-16T00:36:45.028940+0000 (oldest deadline 2024-09-16T00:37:10.929146+0000) 2024-09-16T00:37:30.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:30 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:30.465+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:50.929671+0000 front 2024-09-16T00:36:50.929733+0000 (oldest deadline 2024-09-16T00:37:15.629600+0000) 2024-09-16T00:37:30.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:30 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:30.465+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-16T00:36:58.530860+0000 front 2024-09-16T00:36:58.530947+0000 (oldest deadline 2024-09-16T00:37:23.830577+0000) 2024-09-16T00:37:30.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:30 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:30.470+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:30.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:30 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:30.470+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:30.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:30 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:30.470+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.354234+0000 front 2024-09-16T00:36:36.354282+0000 (oldest deadline 2024-09-16T00:37:02.254386+0000) 2024-09-16T00:37:30.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:30 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:30.470+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:43.955213+0000 front 2024-09-16T00:36:43.955262+0000 (oldest deadline 2024-09-16T00:37:07.956010+0000) 2024-09-16T00:37:30.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:30 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:30.470+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:54.357203+0000 front 2024-09-16T00:36:54.357260+0000 (oldest deadline 2024-09-16T00:37:17.857349+0000) 2024-09-16T00:37:30.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:30 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:30.470+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-16T00:36:57.858011+0000 front 2024-09-16T00:36:57.858024+0000 (oldest deadline 2024-09-16T00:37:23.157915+0000) 2024-09-16T00:37:31.716 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:31 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:31.464+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:31.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:31 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:31.464+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:31.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:31 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:31.464+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.354234+0000 front 2024-09-16T00:36:36.354282+0000 (oldest deadline 2024-09-16T00:37:02.254386+0000) 2024-09-16T00:37:31.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:31 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:31.464+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:43.955213+0000 front 2024-09-16T00:36:43.955262+0000 (oldest deadline 2024-09-16T00:37:07.956010+0000) 2024-09-16T00:37:31.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:31 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:31.464+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:54.357203+0000 front 2024-09-16T00:36:54.357260+0000 (oldest deadline 2024-09-16T00:37:17.857349+0000) 2024-09-16T00:37:31.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:31 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:31.464+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-16T00:36:57.858011+0000 front 2024-09-16T00:36:57.858024+0000 (oldest deadline 2024-09-16T00:37:23.157915+0000) 2024-09-16T00:37:31.793 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:31 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:31.442+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:37:31.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:31 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:31.442+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:37:31.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:31 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:31.442+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:37:31.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:31 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:31.442+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.627413+0000 front 2024-09-16T00:36:37.627341+0000 (oldest deadline 2024-09-16T00:36:59.327248+0000) 2024-09-16T00:37:31.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:31 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:31.442+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.029004+0000 front 2024-09-16T00:36:45.028940+0000 (oldest deadline 2024-09-16T00:37:10.929146+0000) 2024-09-16T00:37:31.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:31 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:31.442+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:50.929671+0000 front 2024-09-16T00:36:50.929733+0000 (oldest deadline 2024-09-16T00:37:15.629600+0000) 2024-09-16T00:37:31.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:31 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:31.442+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-16T00:36:58.530860+0000 front 2024-09-16T00:36:58.530947+0000 (oldest deadline 2024-09-16T00:37:23.830577+0000) 2024-09-16T00:37:32.759 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:32 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:32.480+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:32.759 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:32 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:32.480+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:32.759 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:32 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:32.480+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.354234+0000 front 2024-09-16T00:36:36.354282+0000 (oldest deadline 2024-09-16T00:37:02.254386+0000) 2024-09-16T00:37:32.759 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:32 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:32.480+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:43.955213+0000 front 2024-09-16T00:36:43.955262+0000 (oldest deadline 2024-09-16T00:37:07.956010+0000) 2024-09-16T00:37:32.759 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:32 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:32.480+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:54.357203+0000 front 2024-09-16T00:36:54.357260+0000 (oldest deadline 2024-09-16T00:37:17.857349+0000) 2024-09-16T00:37:32.759 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:32 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:32.480+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-16T00:36:57.858011+0000 front 2024-09-16T00:36:57.858024+0000 (oldest deadline 2024-09-16T00:37:23.157915+0000) 2024-09-16T00:37:32.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:32 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:32.431+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:37:32.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:32 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:32.431+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:37:32.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:32 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:32.431+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:37:32.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:32 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:32.431+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.627413+0000 front 2024-09-16T00:36:37.627341+0000 (oldest deadline 2024-09-16T00:36:59.327248+0000) 2024-09-16T00:37:32.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:32 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:32.431+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.029004+0000 front 2024-09-16T00:36:45.028940+0000 (oldest deadline 2024-09-16T00:37:10.929146+0000) 2024-09-16T00:37:32.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:32 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:32.431+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:50.929671+0000 front 2024-09-16T00:36:50.929733+0000 (oldest deadline 2024-09-16T00:37:15.629600+0000) 2024-09-16T00:37:32.795 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:32 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:32.431+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-16T00:36:58.530860+0000 front 2024-09-16T00:36:58.530947+0000 (oldest deadline 2024-09-16T00:37:23.830577+0000) 2024-09-16T00:37:33.482 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:33 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:33.444+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:33.482 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:33 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:33.444+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:33.482 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:33 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:33.444+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.354234+0000 front 2024-09-16T00:36:36.354282+0000 (oldest deadline 2024-09-16T00:37:02.254386+0000) 2024-09-16T00:37:33.482 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:33 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:33.444+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:43.955213+0000 front 2024-09-16T00:36:43.955262+0000 (oldest deadline 2024-09-16T00:37:07.956010+0000) 2024-09-16T00:37:33.482 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:33 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:33.444+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:54.357203+0000 front 2024-09-16T00:36:54.357260+0000 (oldest deadline 2024-09-16T00:37:17.857349+0000) 2024-09-16T00:37:33.482 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:33 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:33.444+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-16T00:36:57.858011+0000 front 2024-09-16T00:36:57.858024+0000 (oldest deadline 2024-09-16T00:37:23.157915+0000) 2024-09-16T00:37:33.482 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:33 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:33.444+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6820 osd.7 since back 2024-09-16T00:37:09.058834+0000 front 2024-09-16T00:37:09.058905+0000 (oldest deadline 2024-09-16T00:37:32.558718+0000) 2024-09-16T00:37:33.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:33 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:33.420+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:37:33.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:33 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:33.420+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:37:33.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:33 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:33.420+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:37:33.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:33 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:33.420+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.627413+0000 front 2024-09-16T00:36:37.627341+0000 (oldest deadline 2024-09-16T00:36:59.327248+0000) 2024-09-16T00:37:33.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:33 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:33.420+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.029004+0000 front 2024-09-16T00:36:45.028940+0000 (oldest deadline 2024-09-16T00:37:10.929146+0000) 2024-09-16T00:37:33.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:33 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:33.420+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:50.929671+0000 front 2024-09-16T00:36:50.929733+0000 (oldest deadline 2024-09-16T00:37:15.629600+0000) 2024-09-16T00:37:33.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:33 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:33.420+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-16T00:36:58.530860+0000 front 2024-09-16T00:36:58.530947+0000 (oldest deadline 2024-09-16T00:37:23.830577+0000) 2024-09-16T00:37:34.716 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:34 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:34.437+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:34.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:34 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:34.437+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:34.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:34 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:34.437+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.354234+0000 front 2024-09-16T00:36:36.354282+0000 (oldest deadline 2024-09-16T00:37:02.254386+0000) 2024-09-16T00:37:34.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:34 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:34.437+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:43.955213+0000 front 2024-09-16T00:36:43.955262+0000 (oldest deadline 2024-09-16T00:37:07.956010+0000) 2024-09-16T00:37:34.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:34 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:34.437+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:54.357203+0000 front 2024-09-16T00:36:54.357260+0000 (oldest deadline 2024-09-16T00:37:17.857349+0000) 2024-09-16T00:37:34.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:34 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:34.437+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-16T00:36:57.858011+0000 front 2024-09-16T00:36:57.858024+0000 (oldest deadline 2024-09-16T00:37:23.157915+0000) 2024-09-16T00:37:34.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:34 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:34.437+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6820 osd.7 since back 2024-09-16T00:37:09.058834+0000 front 2024-09-16T00:37:09.058905+0000 (oldest deadline 2024-09-16T00:37:32.558718+0000) 2024-09-16T00:37:34.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:34 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:34.426+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:08.821015+0000 front 2024-09-16T00:36:08.820995+0000 (oldest deadline 2024-09-16T00:36:34.621650+0000) 2024-09-16T00:37:34.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:34 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:34.426+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6804 osd.1 since back 2024-09-16T00:36:22.123826+0000 front 2024-09-16T00:36:22.123748+0000 (oldest deadline 2024-09-16T00:36:46.223940+0000) 2024-09-16T00:37:34.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:34 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:34.426+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:27.225620+0000 front 2024-09-16T00:36:27.225438+0000 (oldest deadline 2024-09-16T00:36:53.125390+0000) 2024-09-16T00:37:34.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:34 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:34.426+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:37.627413+0000 front 2024-09-16T00:36:37.627341+0000 (oldest deadline 2024-09-16T00:36:59.327248+0000) 2024-09-16T00:37:34.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:34 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:34.426+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:45.029004+0000 front 2024-09-16T00:36:45.028940+0000 (oldest deadline 2024-09-16T00:37:10.929146+0000) 2024-09-16T00:37:34.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:34 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:34.426+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:50.929671+0000 front 2024-09-16T00:36:50.929733+0000 (oldest deadline 2024-09-16T00:37:15.629600+0000) 2024-09-16T00:37:34.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:34 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10[60196]: 2024-09-16T00:37:34.426+0000 7fb122eb0640 -1 osd.10 96 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-16T00:36:58.530860+0000 front 2024-09-16T00:36:58.530947+0000 (oldest deadline 2024-09-16T00:37:23.830577+0000) 2024-09-16T00:37:35.534 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:35 smithi073 podman[69158]: 2024-09-16 00:37:35.238167459 +0000 UTC m=+5.613925184 container died 8b803c705ab4c9e72545553195ed1abc6fa6f60f2e3bc638609dced54c40effb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-09-16T00:37:35.716 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:35 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:35.412+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:35.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:35 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:35.412+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:35.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:35 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:35.412+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.354234+0000 front 2024-09-16T00:36:36.354282+0000 (oldest deadline 2024-09-16T00:37:02.254386+0000) 2024-09-16T00:37:35.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:35 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:35.412+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:43.955213+0000 front 2024-09-16T00:36:43.955262+0000 (oldest deadline 2024-09-16T00:37:07.956010+0000) 2024-09-16T00:37:35.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:35 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:35.412+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:54.357203+0000 front 2024-09-16T00:36:54.357260+0000 (oldest deadline 2024-09-16T00:37:17.857349+0000) 2024-09-16T00:37:35.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:35 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:35.412+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-16T00:36:57.858011+0000 front 2024-09-16T00:36:57.858024+0000 (oldest deadline 2024-09-16T00:37:23.157915+0000) 2024-09-16T00:37:35.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:35 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:35.412+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6820 osd.7 since back 2024-09-16T00:37:09.058834+0000 front 2024-09-16T00:37:09.058905+0000 (oldest deadline 2024-09-16T00:37:32.558718+0000) 2024-09-16T00:37:35.793 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:35 smithi073 podman[69158]: 2024-09-16 00:37:35.634153271 +0000 UTC m=+6.009910997 container cleanup 8b803c705ab4c9e72545553195ed1abc6fa6f60f2e3bc638609dced54c40effb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, ceph=True, io.buildah.version=1.37.2) 2024-09-16T00:37:35.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:35 smithi073 bash[69158]: ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10 2024-09-16T00:37:36.250 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:35 smithi073 podman[69175]: 2024-09-16 00:37:35.89364614 +0000 UTC m=+0.647298175 container remove 8b803c705ab4c9e72545553195ed1abc6fa6f60f2e3bc638609dced54c40effb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-16T00:37:36.394 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:36 smithi073 podman[69264]: 2024-09-16 00:37:36.152932731 +0000 UTC m=+0.015825360 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:37:36.394 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:36 smithi073 podman[69264]: 2024-09-16 00:37:36.269453406 +0000 UTC m=+0.132346068 container create 9961f55a2c6c4eab6064551cfca9e56cd5df075aa3e30c01aa30d6222a500f34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10-deactivate, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-16T00:37:36.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:36 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:36.399+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:36.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:36 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:36.399+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:36.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:36 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:36.399+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.354234+0000 front 2024-09-16T00:36:36.354282+0000 (oldest deadline 2024-09-16T00:37:02.254386+0000) 2024-09-16T00:37:36.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:36 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:36.399+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:43.955213+0000 front 2024-09-16T00:36:43.955262+0000 (oldest deadline 2024-09-16T00:37:07.956010+0000) 2024-09-16T00:37:36.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:36 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:36.399+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:54.357203+0000 front 2024-09-16T00:36:54.357260+0000 (oldest deadline 2024-09-16T00:37:17.857349+0000) 2024-09-16T00:37:36.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:36 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:36.399+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-16T00:36:57.858011+0000 front 2024-09-16T00:36:57.858024+0000 (oldest deadline 2024-09-16T00:37:23.157915+0000) 2024-09-16T00:37:36.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:36 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:36.399+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6820 osd.7 since back 2024-09-16T00:37:09.058834+0000 front 2024-09-16T00:37:09.058905+0000 (oldest deadline 2024-09-16T00:37:32.558718+0000) 2024-09-16T00:37:36.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:36 smithi073 podman[69264]: 2024-09-16 00:37:36.513663123 +0000 UTC m=+0.376555740 container init 9961f55a2c6c4eab6064551cfca9e56cd5df075aa3e30c01aa30d6222a500f34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731) 2024-09-16T00:37:36.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:36 smithi073 podman[69264]: 2024-09-16 00:37:36.516879488 +0000 UTC m=+0.379772105 container start 9961f55a2c6c4eab6064551cfca9e56cd5df075aa3e30c01aa30d6222a500f34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-16T00:37:36.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:36 smithi073 podman[69264]: 2024-09-16 00:37:36.59682084 +0000 UTC m=+0.459713471 container attach 9961f55a2c6c4eab6064551cfca9e56cd5df075aa3e30c01aa30d6222a500f34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:37:36.794 INFO:journalctl@ceph.osd.10.smithi073.stdout:Sep 16 00:37:36 smithi073 podman[69264]: 2024-09-16 00:37:36.715880645 +0000 UTC m=+0.578773273 container died 9961f55a2c6c4eab6064551cfca9e56cd5df075aa3e30c01aa30d6222a500f34 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e) 2024-09-16T00:37:37.299 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@osd.10.service' 2024-09-16T00:37:37.348 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:37:37.348 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-16T00:37:37.348 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-16T00:37:37.348 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-983b2072-73c1-11ef-bceb-c7b262605968@osd.11 2024-09-16T00:37:37.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:37 smithi122 systemd[1]: Stopping Ceph osd.11 for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:37:37.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:37 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:37.439+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:37.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:37 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:37.439+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:37.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:37 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:37.439+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.354234+0000 front 2024-09-16T00:36:36.354282+0000 (oldest deadline 2024-09-16T00:37:02.254386+0000) 2024-09-16T00:37:37.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:37 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:37.439+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:43.955213+0000 front 2024-09-16T00:36:43.955262+0000 (oldest deadline 2024-09-16T00:37:07.956010+0000) 2024-09-16T00:37:37.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:37 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:37.439+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:54.357203+0000 front 2024-09-16T00:36:54.357260+0000 (oldest deadline 2024-09-16T00:37:17.857349+0000) 2024-09-16T00:37:37.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:37 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:37.439+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-16T00:36:57.858011+0000 front 2024-09-16T00:36:57.858024+0000 (oldest deadline 2024-09-16T00:37:23.157915+0000) 2024-09-16T00:37:37.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:37 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:37.439+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6820 osd.7 since back 2024-09-16T00:37:09.058834+0000 front 2024-09-16T00:37:09.058905+0000 (oldest deadline 2024-09-16T00:37:32.558718+0000) 2024-09-16T00:37:38.217 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:37 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:37.726+0000 7ffb3a12a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:37:38.217 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:37 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:37.726+0000 7ffb3a12a640 -1 osd.11 96 *** Got signal Terminated *** 2024-09-16T00:37:38.217 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:37 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:37.726+0000 7ffb3a12a640 -1 osd.11 96 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-16T00:37:38.716 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:38 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:38.439+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:38.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:38 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:38.439+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:38.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:38 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:38.439+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.354234+0000 front 2024-09-16T00:36:36.354282+0000 (oldest deadline 2024-09-16T00:37:02.254386+0000) 2024-09-16T00:37:38.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:38 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:38.439+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:43.955213+0000 front 2024-09-16T00:36:43.955262+0000 (oldest deadline 2024-09-16T00:37:07.956010+0000) 2024-09-16T00:37:38.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:38 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:38.439+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:54.357203+0000 front 2024-09-16T00:36:54.357260+0000 (oldest deadline 2024-09-16T00:37:17.857349+0000) 2024-09-16T00:37:38.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:38 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:38.439+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-16T00:36:57.858011+0000 front 2024-09-16T00:36:57.858024+0000 (oldest deadline 2024-09-16T00:37:23.157915+0000) 2024-09-16T00:37:38.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:38 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:38.439+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6820 osd.7 since back 2024-09-16T00:37:09.058834+0000 front 2024-09-16T00:37:09.058905+0000 (oldest deadline 2024-09-16T00:37:32.558718+0000) 2024-09-16T00:37:39.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:39 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:39.438+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:39.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:39 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:39.438+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:39.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:39 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:39.438+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.354234+0000 front 2024-09-16T00:36:36.354282+0000 (oldest deadline 2024-09-16T00:37:02.254386+0000) 2024-09-16T00:37:39.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:39 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:39.438+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:43.955213+0000 front 2024-09-16T00:36:43.955262+0000 (oldest deadline 2024-09-16T00:37:07.956010+0000) 2024-09-16T00:37:39.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:39 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:39.438+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:54.357203+0000 front 2024-09-16T00:36:54.357260+0000 (oldest deadline 2024-09-16T00:37:17.857349+0000) 2024-09-16T00:37:39.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:39 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:39.438+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-16T00:36:57.858011+0000 front 2024-09-16T00:36:57.858024+0000 (oldest deadline 2024-09-16T00:37:23.157915+0000) 2024-09-16T00:37:39.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:39 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:39.438+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6820 osd.7 since back 2024-09-16T00:37:09.058834+0000 front 2024-09-16T00:37:09.058905+0000 (oldest deadline 2024-09-16T00:37:32.558718+0000) 2024-09-16T00:37:40.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:40 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:40.411+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:40.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:40 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:40.411+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:40.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:40 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:40.411+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.354234+0000 front 2024-09-16T00:36:36.354282+0000 (oldest deadline 2024-09-16T00:37:02.254386+0000) 2024-09-16T00:37:40.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:40 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:40.411+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:43.955213+0000 front 2024-09-16T00:36:43.955262+0000 (oldest deadline 2024-09-16T00:37:07.956010+0000) 2024-09-16T00:37:40.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:40 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:40.411+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:54.357203+0000 front 2024-09-16T00:36:54.357260+0000 (oldest deadline 2024-09-16T00:37:17.857349+0000) 2024-09-16T00:37:40.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:40 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:40.411+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-16T00:36:57.858011+0000 front 2024-09-16T00:36:57.858024+0000 (oldest deadline 2024-09-16T00:37:23.157915+0000) 2024-09-16T00:37:40.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:40 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:40.411+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6820 osd.7 since back 2024-09-16T00:37:09.058834+0000 front 2024-09-16T00:37:09.058905+0000 (oldest deadline 2024-09-16T00:37:32.558718+0000) 2024-09-16T00:37:40.718 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:40 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:40.411+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6820 osd.8 since back 2024-09-16T00:37:13.659557+0000 front 2024-09-16T00:37:13.659571+0000 (oldest deadline 2024-09-16T00:37:39.559579+0000) 2024-09-16T00:37:41.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:41 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:41.441+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:41.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:41 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:41.441+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:41.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:41 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:41.441+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.354234+0000 front 2024-09-16T00:36:36.354282+0000 (oldest deadline 2024-09-16T00:37:02.254386+0000) 2024-09-16T00:37:41.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:41 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:41.441+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:43.955213+0000 front 2024-09-16T00:36:43.955262+0000 (oldest deadline 2024-09-16T00:37:07.956010+0000) 2024-09-16T00:37:41.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:41 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:41.441+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:54.357203+0000 front 2024-09-16T00:36:54.357260+0000 (oldest deadline 2024-09-16T00:37:17.857349+0000) 2024-09-16T00:37:41.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:41 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:41.441+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-16T00:36:57.858011+0000 front 2024-09-16T00:36:57.858024+0000 (oldest deadline 2024-09-16T00:37:23.157915+0000) 2024-09-16T00:37:41.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:41 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:41.441+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6820 osd.7 since back 2024-09-16T00:37:09.058834+0000 front 2024-09-16T00:37:09.058905+0000 (oldest deadline 2024-09-16T00:37:32.558718+0000) 2024-09-16T00:37:41.718 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:41 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:41.441+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6820 osd.8 since back 2024-09-16T00:37:13.659557+0000 front 2024-09-16T00:37:13.659571+0000 (oldest deadline 2024-09-16T00:37:39.559579+0000) 2024-09-16T00:37:42.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:42 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:42.409+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6806 osd.0 since back 2024-09-16T00:36:10.750276+0000 front 2024-09-16T00:36:10.750329+0000 (oldest deadline 2024-09-16T00:36:36.650334+0000) 2024-09-16T00:37:42.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:42 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:42.409+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6804 osd.2 since back 2024-09-16T00:36:29.353548+0000 front 2024-09-16T00:36:29.353640+0000 (oldest deadline 2024-09-16T00:36:53.453554+0000) 2024-09-16T00:37:42.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:42 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:42.409+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6814 osd.3 since back 2024-09-16T00:36:36.354234+0000 front 2024-09-16T00:36:36.354282+0000 (oldest deadline 2024-09-16T00:37:02.254386+0000) 2024-09-16T00:37:42.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:42 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:42.409+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6812 osd.4 since back 2024-09-16T00:36:43.955213+0000 front 2024-09-16T00:36:43.955262+0000 (oldest deadline 2024-09-16T00:37:07.956010+0000) 2024-09-16T00:37:42.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:42 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:42.409+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6812 osd.5 since back 2024-09-16T00:36:54.357203+0000 front 2024-09-16T00:36:54.357260+0000 (oldest deadline 2024-09-16T00:37:17.857349+0000) 2024-09-16T00:37:42.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:42 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:42.409+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.38:6822 osd.6 since back 2024-09-16T00:36:57.858011+0000 front 2024-09-16T00:36:57.858024+0000 (oldest deadline 2024-09-16T00:37:23.157915+0000) 2024-09-16T00:37:42.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:42 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:42.409+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.73:6820 osd.7 since back 2024-09-16T00:37:09.058834+0000 front 2024-09-16T00:37:09.058905+0000 (oldest deadline 2024-09-16T00:37:32.558718+0000) 2024-09-16T00:37:42.717 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:42 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11[60296]: 2024-09-16T00:37:42.409+0000 7ffb32738640 -1 osd.11 96 heartbeat_check: no reply from 172.21.15.122:6820 osd.8 since back 2024-09-16T00:37:13.659557+0000 front 2024-09-16T00:37:13.659571+0000 (oldest deadline 2024-09-16T00:37:39.559579+0000) 2024-09-16T00:37:43.561 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:43 smithi122 podman[69280]: 2024-09-16 00:37:43.264884977 +0000 UTC m=+5.711236758 container died ea2f03571f35c9a81fbb8bf321a25e4876bf3c3a5e814e490435e86dc4c824d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=squid-8921c8e, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-16T00:37:43.943 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:43 smithi122 podman[69280]: 2024-09-16 00:37:43.660533819 +0000 UTC m=+6.106885599 container cleanup ea2f03571f35c9a81fbb8bf321a25e4876bf3c3a5e814e490435e86dc4c824d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-16T00:37:43.943 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:43 smithi122 bash[69280]: ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11 2024-09-16T00:37:44.216 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:43 smithi122 podman[69297]: 2024-09-16 00:37:43.961556327 +0000 UTC m=+0.689090418 container remove ea2f03571f35c9a81fbb8bf321a25e4876bf3c3a5e814e490435e86dc4c824d1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-16T00:37:44.531 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:44 smithi122 podman[69384]: 2024-09-16 00:37:44.172440687 +0000 UTC m=+0.015802857 image pull f1455bd0219c0836c14d3880561c736ab957883ab2384892956003bb36edc4ee quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80 2024-09-16T00:37:44.531 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:44 smithi122 podman[69384]: 2024-09-16 00:37:44.270554747 +0000 UTC m=+0.113916904 container create eb34ed5d85daa0b0a4420c7d1c0b43772915cea100e874d4d82fc6057d26c958 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-16T00:37:44.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:44 smithi122 podman[69384]: 2024-09-16 00:37:44.531564533 +0000 UTC m=+0.374926690 container init eb34ed5d85daa0b0a4420c7d1c0b43772915cea100e874d4d82fc6057d26c958 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11-deactivate, ceph=True, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-16T00:37:44.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:44 smithi122 podman[69384]: 2024-09-16 00:37:44.534966953 +0000 UTC m=+0.378329111 container start eb34ed5d85daa0b0a4420c7d1c0b43772915cea100e874d4d82fc6057d26c958 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11-deactivate, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e) 2024-09-16T00:37:44.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:44 smithi122 podman[69384]: 2024-09-16 00:37:44.614801126 +0000 UTC m=+0.458163297 container attach eb34ed5d85daa0b0a4420c7d1c0b43772915cea100e874d4d82fc6057d26c958 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:37:44.967 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:44 smithi122 podman[69384]: 2024-09-16 00:37:44.722830031 +0000 UTC m=+0.566192208 container died eb34ed5d85daa0b0a4420c7d1c0b43772915cea100e874d4d82fc6057d26c958 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-16T00:37:45.276 INFO:journalctl@ceph.osd.11.smithi122.stdout:Sep 16 00:37:45 smithi122 podman[69407]: 2024-09-16 00:37:45.259806067 +0000 UTC m=+0.531327418 container remove eb34ed5d85daa0b0a4420c7d1c0b43772915cea100e874d4d82fc6057d26c958 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=squid-8921c8e, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-16T00:37:45.282 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@osd.11.service' 2024-09-16T00:37:45.319 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:37:45.319 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-16T00:37:45.320 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-16T00:37:45.320 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-983b2072-73c1-11ef-bceb-c7b262605968@mds.a 2024-09-16T00:37:45.625 INFO:journalctl@ceph.mds.a.smithi038.stdout:Sep 16 00:37:45 smithi038 systemd[1]: Stopping Ceph mds.a for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:37:45.904 INFO:journalctl@ceph.mds.a.smithi038.stdout:Sep 16 00:37:45 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-a[71681]: 2024-09-16T00:37:45.624+0000 7f56db7cd640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:37:45.904 INFO:journalctl@ceph.mds.a.smithi038.stdout:Sep 16 00:37:45 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-a[71681]: 2024-09-16T00:37:45.624+0000 7f56db7cd640 -1 mds.a *** got signal Terminated *** 2024-09-16T00:37:49.965 INFO:journalctl@ceph.mds.a.smithi038.stdout:Sep 16 00:37:49 smithi038 podman[94741]: 2024-09-16 00:37:49.631357586 +0000 UTC m=+4.111283677 container died 12df256a6ebe95dc07b2ce65d194765d0282f0b365e32732cd01a9b4e29df6ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-a, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-16T00:37:50.465 INFO:journalctl@ceph.mds.a.smithi038.stdout:Sep 16 00:37:50 smithi038 podman[94741]: 2024-09-16 00:37:50.224560862 +0000 UTC m=+4.704486936 container cleanup 12df256a6ebe95dc07b2ce65d194765d0282f0b365e32732cd01a9b4e29df6ec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-a, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0) 2024-09-16T00:37:50.465 INFO:journalctl@ceph.mds.a.smithi038.stdout:Sep 16 00:37:50 smithi038 bash[94741]: ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-a 2024-09-16T00:37:50.668 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@mds.a.service' 2024-09-16T00:37:50.706 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:37:50.706 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-16T00:37:50.706 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-16T00:37:50.706 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-983b2072-73c1-11ef-bceb-c7b262605968@mds.d 2024-09-16T00:37:51.038 INFO:journalctl@ceph.mds.d.smithi038.stdout:Sep 16 00:37:50 smithi038 systemd[1]: Stopping Ceph mds.d for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:37:51.465 INFO:journalctl@ceph.mds.d.smithi038.stdout:Sep 16 00:37:51 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-d[70392]: 2024-09-16T00:37:51.037+0000 7f43166a5640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:37:51.465 INFO:journalctl@ceph.mds.d.smithi038.stdout:Sep 16 00:37:51 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-d[70392]: 2024-09-16T00:37:51.037+0000 7f43166a5640 -1 mds.d *** got signal Terminated *** 2024-09-16T00:37:54.313 INFO:journalctl@ceph.mds.d.smithi038.stdout:Sep 16 00:37:54 smithi038 podman[94876]: 2024-09-16 00:37:54.046206354 +0000 UTC m=+3.105650980 container died 250b747ee3944c6339781f21ed89d9a1b8a919c643c5b12883c639cbb64b7c1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-d, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux ) 2024-09-16T00:37:54.714 INFO:journalctl@ceph.mds.d.smithi038.stdout:Sep 16 00:37:54 smithi038 podman[94876]: 2024-09-16 00:37:54.495403673 +0000 UTC m=+3.554848296 container cleanup 250b747ee3944c6339781f21ed89d9a1b8a919c643c5b12883c639cbb64b7c1c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-d, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-8921c8e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-16T00:37:54.715 INFO:journalctl@ceph.mds.d.smithi038.stdout:Sep 16 00:37:54 smithi038 bash[94876]: ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-d 2024-09-16T00:37:54.980 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@mds.d.service' 2024-09-16T00:37:55.015 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:37:55.015 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-16T00:37:55.015 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-16T00:37:55.015 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-983b2072-73c1-11ef-bceb-c7b262605968@mds.g 2024-09-16T00:37:55.341 INFO:journalctl@ceph.mds.g.smithi038.stdout:Sep 16 00:37:55 smithi038 systemd[1]: Stopping Ceph mds.g for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:37:55.715 INFO:journalctl@ceph.mds.g.smithi038.stdout:Sep 16 00:37:55 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-g[71267]: 2024-09-16T00:37:55.339+0000 7ff03099e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:37:55.715 INFO:journalctl@ceph.mds.g.smithi038.stdout:Sep 16 00:37:55 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-g[71267]: 2024-09-16T00:37:55.339+0000 7ff03099e640 -1 mds.g *** got signal Terminated *** 2024-09-16T00:37:59.291 INFO:journalctl@ceph.mds.g.smithi038.stdout:Sep 16 00:37:59 smithi038 podman[94996]: 2024-09-16 00:37:59.014304218 +0000 UTC m=+3.776768191 container died f715123d24f86a1a5a2c158c78f63895e741b6600aae82c2260603d6af7ef4f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-g, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731) 2024-09-16T00:37:59.650 INFO:journalctl@ceph.mds.g.smithi038.stdout:Sep 16 00:37:59 smithi038 podman[94996]: 2024-09-16 00:37:59.457581779 +0000 UTC m=+4.220045791 container cleanup f715123d24f86a1a5a2c158c78f63895e741b6600aae82c2260603d6af7ef4f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-g, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-16T00:37:59.651 INFO:journalctl@ceph.mds.g.smithi038.stdout:Sep 16 00:37:59 smithi038 bash[94996]: ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-g 2024-09-16T00:37:59.898 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@mds.g.service' 2024-09-16T00:37:59.920 INFO:journalctl@ceph.mds.g.smithi038.stdout:Sep 16 00:37:59 smithi038 podman[95009]: 2024-09-16 00:37:59.650804432 +0000 UTC m=+0.628227961 container remove f715123d24f86a1a5a2c158c78f63895e741b6600aae82c2260603d6af7ef4f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-g, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-16T00:37:59.921 INFO:journalctl@ceph.mds.g.smithi038.stdout:Sep 16 00:37:59 smithi038 systemd[1]: ceph-983b2072-73c1-11ef-bceb-c7b262605968@mds.g.service: Deactivated successfully. 2024-09-16T00:37:59.921 INFO:journalctl@ceph.mds.g.smithi038.stdout:Sep 16 00:37:59 smithi038 systemd[1]: Stopped Ceph mds.g for 983b2072-73c1-11ef-bceb-c7b262605968. 2024-09-16T00:37:59.933 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:37:59.933 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-16T00:37:59.933 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-16T00:37:59.933 DEBUG:teuthology.orchestra.run.smithi038:> sudo systemctl stop ceph-983b2072-73c1-11ef-bceb-c7b262605968@mds.j 2024-09-16T00:38:00.215 INFO:journalctl@ceph.mds.j.smithi038.stdout:Sep 16 00:38:00 smithi038 systemd[1]: Stopping Ceph mds.j for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:38:00.714 INFO:journalctl@ceph.mds.j.smithi038.stdout:Sep 16 00:38:00 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-j[70011]: 2024-09-16T00:38:00.406+0000 7f7345db7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:38:00.715 INFO:journalctl@ceph.mds.j.smithi038.stdout:Sep 16 00:38:00 smithi038 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-j[70011]: 2024-09-16T00:38:00.406+0000 7f7345db7640 -1 mds.j *** got signal Terminated *** 2024-09-16T00:38:02.679 INFO:journalctl@ceph.mds.j.smithi038.stdout:Sep 16 00:38:02 smithi038 podman[95116]: 2024-09-16 00:38:02.387756963 +0000 UTC m=+2.221855703 container died 7b4923415ebd94682272d866fd5e96ce210ddb0fb29942f00919ac817d0f4a14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-j, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-16T00:38:02.964 INFO:journalctl@ceph.mds.j.smithi038.stdout:Sep 16 00:38:02 smithi038 podman[95116]: 2024-09-16 00:38:02.837007371 +0000 UTC m=+2.671106111 container cleanup 7b4923415ebd94682272d866fd5e96ce210ddb0fb29942f00919ac817d0f4a14 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-j, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-16T00:38:02.965 INFO:journalctl@ceph.mds.j.smithi038.stdout:Sep 16 00:38:02 smithi038 bash[95116]: ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-j 2024-09-16T00:38:03.270 DEBUG:teuthology.orchestra.run.smithi038:> sudo pkill -f 'journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@mds.j.service' 2024-09-16T00:38:03.303 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:38:03.303 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-16T00:38:03.304 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-16T00:38:03.304 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-983b2072-73c1-11ef-bceb-c7b262605968@mds.b 2024-09-16T00:38:03.601 INFO:journalctl@ceph.mds.b.smithi073.stdout:Sep 16 00:38:03 smithi073 systemd[1]: Stopping Ceph mds.b for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:38:04.043 INFO:journalctl@ceph.mds.b.smithi073.stdout:Sep 16 00:38:03 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-b[65406]: 2024-09-16T00:38:03.600+0000 7f2db40ad640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:38:04.044 INFO:journalctl@ceph.mds.b.smithi073.stdout:Sep 16 00:38:03 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-b[65406]: 2024-09-16T00:38:03.600+0000 7f2db40ad640 -1 mds.b *** got signal Terminated *** 2024-09-16T00:38:07.543 INFO:journalctl@ceph.mds.b.smithi073.stdout:Sep 16 00:38:07 smithi073 podman[69380]: 2024-09-16 00:38:07.291535975 +0000 UTC m=+3.795192808 container died 47c89ec1eb8f9059ea6d20628eb4a2e3151d2d628100947b440b903a74fbb6f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-b, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:38:07.942 INFO:journalctl@ceph.mds.b.smithi073.stdout:Sep 16 00:38:07 smithi073 podman[69380]: 2024-09-16 00:38:07.64186642 +0000 UTC m=+4.145523260 container cleanup 47c89ec1eb8f9059ea6d20628eb4a2e3151d2d628100947b440b903a74fbb6f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-b, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-16T00:38:07.942 INFO:journalctl@ceph.mds.b.smithi073.stdout:Sep 16 00:38:07 smithi073 bash[69380]: ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-b 2024-09-16T00:38:08.123 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@mds.b.service' 2024-09-16T00:38:08.158 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:38:08.158 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-16T00:38:08.158 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-16T00:38:08.158 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-983b2072-73c1-11ef-bceb-c7b262605968@mds.e 2024-09-16T00:38:08.479 INFO:journalctl@ceph.mds.e.smithi073.stdout:Sep 16 00:38:08 smithi073 systemd[1]: Stopping Ceph mds.e for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:38:08.793 INFO:journalctl@ceph.mds.e.smithi073.stdout:Sep 16 00:38:08 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-e[64789]: 2024-09-16T00:38:08.479+0000 7fa39df7f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:38:08.794 INFO:journalctl@ceph.mds.e.smithi073.stdout:Sep 16 00:38:08 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-e[64789]: 2024-09-16T00:38:08.479+0000 7fa39df7f640 -1 mds.e *** got signal Terminated *** 2024-09-16T00:38:11.572 INFO:journalctl@ceph.mds.e.smithi073.stdout:Sep 16 00:38:11 smithi073 podman[69501]: 2024-09-16 00:38:11.298077596 +0000 UTC m=+2.922972081 container died 2355052d8723af358dcef72db200312946eb484e326c3a1c8d43d3d223f5d6c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-e, RELEASE=squid-8921c8e, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:38:11.947 INFO:journalctl@ceph.mds.e.smithi073.stdout:Sep 16 00:38:11 smithi073 podman[69501]: 2024-09-16 00:38:11.662648209 +0000 UTC m=+3.287542694 container cleanup 2355052d8723af358dcef72db200312946eb484e326c3a1c8d43d3d223f5d6c0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-e, CEPH_POINT_RELEASE=, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e) 2024-09-16T00:38:11.947 INFO:journalctl@ceph.mds.e.smithi073.stdout:Sep 16 00:38:11 smithi073 bash[69501]: ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-e 2024-09-16T00:38:12.097 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@mds.e.service' 2024-09-16T00:38:12.130 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:38:12.130 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-16T00:38:12.130 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-16T00:38:12.130 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-983b2072-73c1-11ef-bceb-c7b262605968@mds.h 2024-09-16T00:38:12.544 INFO:journalctl@ceph.mds.h.smithi073.stdout:Sep 16 00:38:12 smithi073 systemd[1]: Stopping Ceph mds.h for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:38:12.544 INFO:journalctl@ceph.mds.h.smithi073.stdout:Sep 16 00:38:12 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-h[65090]: 2024-09-16T00:38:12.449+0000 7fc257b0d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:38:12.544 INFO:journalctl@ceph.mds.h.smithi073.stdout:Sep 16 00:38:12 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-h[65090]: 2024-09-16T00:38:12.449+0000 7fc257b0d640 -1 mds.h *** got signal Terminated *** 2024-09-16T00:38:15.544 INFO:journalctl@ceph.mds.h.smithi073.stdout:Sep 16 00:38:15 smithi073 podman[69623]: 2024-09-16 00:38:15.288276635 +0000 UTC m=+2.945104103 container died 90fff75c7a54a1d4b3b0583f874b3928fc90642bf4139efa0ba28c9a08024160 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-h, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:38:15.925 INFO:journalctl@ceph.mds.h.smithi073.stdout:Sep 16 00:38:15 smithi073 podman[69623]: 2024-09-16 00:38:15.649605757 +0000 UTC m=+3.306433232 container cleanup 90fff75c7a54a1d4b3b0583f874b3928fc90642bf4139efa0ba28c9a08024160 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-h, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, RELEASE=squid-8921c8e, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-16T00:38:15.926 INFO:journalctl@ceph.mds.h.smithi073.stdout:Sep 16 00:38:15 smithi073 bash[69623]: ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-h 2024-09-16T00:38:16.066 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@mds.h.service' 2024-09-16T00:38:16.099 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:38:16.099 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-16T00:38:16.099 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-16T00:38:16.099 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ceph-983b2072-73c1-11ef-bceb-c7b262605968@mds.k 2024-09-16T00:38:16.544 INFO:journalctl@ceph.mds.k.smithi073.stdout:Sep 16 00:38:16 smithi073 systemd[1]: Stopping Ceph mds.k for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:38:16.544 INFO:journalctl@ceph.mds.k.smithi073.stdout:Sep 16 00:38:16 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-k[64476]: 2024-09-16T00:38:16.420+0000 7f75476e0640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:38:16.544 INFO:journalctl@ceph.mds.k.smithi073.stdout:Sep 16 00:38:16 smithi073 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-k[64476]: 2024-09-16T00:38:16.420+0000 7f75476e0640 -1 mds.k *** got signal Terminated *** 2024-09-16T00:38:19.544 INFO:journalctl@ceph.mds.k.smithi073.stdout:Sep 16 00:38:19 smithi073 podman[69745]: 2024-09-16 00:38:19.292744382 +0000 UTC m=+2.978281964 container died 9d7f1d729e1f44e39ca303e70ac14989d8c45059f975b5279d21e78f1db663c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-k, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:38:20.029 INFO:journalctl@ceph.mds.k.smithi073.stdout:Sep 16 00:38:19 smithi073 podman[69745]: 2024-09-16 00:38:19.803862134 +0000 UTC m=+3.489399715 container cleanup 9d7f1d729e1f44e39ca303e70ac14989d8c45059f975b5279d21e78f1db663c8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-k, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-16T00:38:20.029 INFO:journalctl@ceph.mds.k.smithi073.stdout:Sep 16 00:38:19 smithi073 bash[69745]: ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-k 2024-09-16T00:38:20.145 DEBUG:teuthology.orchestra.run.smithi073:> sudo pkill -f 'journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@mds.k.service' 2024-09-16T00:38:20.176 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:38:20.177 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-16T00:38:20.177 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-16T00:38:20.177 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-983b2072-73c1-11ef-bceb-c7b262605968@mds.c 2024-09-16T00:38:20.467 INFO:journalctl@ceph.mds.c.smithi122.stdout:Sep 16 00:38:20 smithi122 systemd[1]: Stopping Ceph mds.c for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:38:20.966 INFO:journalctl@ceph.mds.c.smithi122.stdout:Sep 16 00:38:20 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-c[65133]: 2024-09-16T00:38:20.467+0000 7f2f1f4a8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:38:20.966 INFO:journalctl@ceph.mds.c.smithi122.stdout:Sep 16 00:38:20 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-c[65133]: 2024-09-16T00:38:20.467+0000 7f2f1f4a8640 -1 mds.c *** got signal Terminated *** 2024-09-16T00:38:23.552 INFO:journalctl@ceph.mds.c.smithi122.stdout:Sep 16 00:38:23 smithi122 podman[69486]: 2024-09-16 00:38:23.285917036 +0000 UTC m=+2.917876869 container died 6b38590c30494560b06fd03d28dd3aa06a1ddaca02ccb02fea17cf3b9ccabffb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-16T00:38:23.919 INFO:journalctl@ceph.mds.c.smithi122.stdout:Sep 16 00:38:23 smithi122 podman[69486]: 2024-09-16 00:38:23.643134001 +0000 UTC m=+3.275093844 container cleanup 6b38590c30494560b06fd03d28dd3aa06a1ddaca02ccb02fea17cf3b9ccabffb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-c, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-8921c8e, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-16T00:38:23.920 INFO:journalctl@ceph.mds.c.smithi122.stdout:Sep 16 00:38:23 smithi122 bash[69486]: ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-c 2024-09-16T00:38:24.037 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@mds.c.service' 2024-09-16T00:38:24.073 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:38:24.074 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-16T00:38:24.074 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-16T00:38:24.074 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-983b2072-73c1-11ef-bceb-c7b262605968@mds.f 2024-09-16T00:38:24.216 INFO:journalctl@ceph.mds.f.smithi122.stdout:Sep 16 00:38:24 smithi122 systemd[1]: Stopping Ceph mds.f for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:38:24.716 INFO:journalctl@ceph.mds.f.smithi122.stdout:Sep 16 00:38:24 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-f[64832]: 2024-09-16T00:38:24.429+0000 7f89c9211640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:38:24.717 INFO:journalctl@ceph.mds.f.smithi122.stdout:Sep 16 00:38:24 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-f[64832]: 2024-09-16T00:38:24.429+0000 7f89c9211640 -1 mds.f *** got signal Terminated *** 2024-09-16T00:38:27.716 INFO:journalctl@ceph.mds.f.smithi122.stdout:Sep 16 00:38:27 smithi122 podman[69621]: 2024-09-16 00:38:27.317619082 +0000 UTC m=+3.022367534 container died f3f7a5fa1fd3ffa4272ffedb935f0b5ffc4371cdd1bde927eb6795cbfdce2787 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-f, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=squid-8921c8e, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-16T00:38:28.048 INFO:journalctl@ceph.mds.f.smithi122.stdout:Sep 16 00:38:27 smithi122 podman[69621]: 2024-09-16 00:38:27.822162659 +0000 UTC m=+3.526911120 container cleanup f3f7a5fa1fd3ffa4272ffedb935f0b5ffc4371cdd1bde927eb6795cbfdce2787 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-f, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-8921c8e, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965) 2024-09-16T00:38:28.048 INFO:journalctl@ceph.mds.f.smithi122.stdout:Sep 16 00:38:27 smithi122 bash[69621]: ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-f 2024-09-16T00:38:28.198 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@mds.f.service' 2024-09-16T00:38:28.230 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:38:28.231 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-16T00:38:28.231 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-16T00:38:28.231 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-983b2072-73c1-11ef-bceb-c7b262605968@mds.i 2024-09-16T00:38:28.716 INFO:journalctl@ceph.mds.i.smithi122.stdout:Sep 16 00:38:28 smithi122 systemd[1]: Stopping Ceph mds.i for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:38:28.717 INFO:journalctl@ceph.mds.i.smithi122.stdout:Sep 16 00:38:28 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-i[65444]: 2024-09-16T00:38:28.559+0000 7f133d3f9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:38:28.717 INFO:journalctl@ceph.mds.i.smithi122.stdout:Sep 16 00:38:28 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-i[65444]: 2024-09-16T00:38:28.559+0000 7f133d3f9640 -1 mds.i *** got signal Terminated *** 2024-09-16T00:38:31.634 INFO:journalctl@ceph.mds.i.smithi122.stdout:Sep 16 00:38:31 smithi122 podman[69743]: 2024-09-16 00:38:31.285433607 +0000 UTC m=+2.837555103 container died 0fa3ecb62b42108c88b40164811d26f0409088f6f44e3999605bca13452e2581 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-i, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD) 2024-09-16T00:38:31.927 INFO:journalctl@ceph.mds.i.smithi122.stdout:Sep 16 00:38:31 smithi122 podman[69743]: 2024-09-16 00:38:31.700905235 +0000 UTC m=+3.253026714 container cleanup 0fa3ecb62b42108c88b40164811d26f0409088f6f44e3999605bca13452e2581 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-i, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:38:31.927 INFO:journalctl@ceph.mds.i.smithi122.stdout:Sep 16 00:38:31 smithi122 bash[69743]: ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-i 2024-09-16T00:38:32.143 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@mds.i.service' 2024-09-16T00:38:32.176 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:38:32.176 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-16T00:38:32.176 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-16T00:38:32.177 DEBUG:teuthology.orchestra.run.smithi122:> sudo systemctl stop ceph-983b2072-73c1-11ef-bceb-c7b262605968@mds.l 2024-09-16T00:38:32.466 INFO:journalctl@ceph.mds.l.smithi122.stdout:Sep 16 00:38:32 smithi122 systemd[1]: Stopping Ceph mds.l for 983b2072-73c1-11ef-bceb-c7b262605968... 2024-09-16T00:38:32.966 INFO:journalctl@ceph.mds.l.smithi122.stdout:Sep 16 00:38:32 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-l[64528]: 2024-09-16T00:38:32.506+0000 7f77b8922640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-16T00:38:32.966 INFO:journalctl@ceph.mds.l.smithi122.stdout:Sep 16 00:38:32 smithi122 ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-l[64528]: 2024-09-16T00:38:32.506+0000 7f77b8922640 -1 mds.l *** got signal Terminated *** 2024-09-16T00:38:36.505 INFO:journalctl@ceph.mds.l.smithi122.stdout:Sep 16 00:38:36 smithi122 podman[69862]: 2024-09-16 00:38:36.226261426 +0000 UTC m=+3.835808608 container died 4a9bfdbf120dafb8779524c6a70d47fb37fba3e155678431853b619e7a655a99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-l, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-16T00:38:36.847 INFO:journalctl@ceph.mds.l.smithi122.stdout:Sep 16 00:38:36 smithi122 podman[69862]: 2024-09-16 00:38:36.604790472 +0000 UTC m=+4.214337689 container cleanup 4a9bfdbf120dafb8779524c6a70d47fb37fba3e155678431853b619e7a655a99 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6b5c73953776cdbbe2c2650e2c9813ab5ac3fdead855fa68a3941507184c3a80, name=ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-l, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=009c5a8162e3d9e92e49bb850b92bd3cd406d965, org.label-schema.schema-version=1.0, RELEASE=squid-8921c8e, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-16T00:38:36.847 INFO:journalctl@ceph.mds.l.smithi122.stdout:Sep 16 00:38:36 smithi122 bash[69862]: ceph-983b2072-73c1-11ef-bceb-c7b262605968-mds-l 2024-09-16T00:38:36.979 DEBUG:teuthology.orchestra.run.smithi122:> sudo pkill -f 'journalctl -f -n 0 -u ceph-983b2072-73c1-11ef-bceb-c7b262605968@mds.l.service' 2024-09-16T00:38:37.010 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-16T00:38:37.010 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-16T00:38:37.010 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 983b2072-73c1-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-16T00:38:37.224 INFO:teuthology.orchestra.run.smithi038.stdout:Deleting cluster with fsid: 983b2072-73c1-11ef-bceb-c7b262605968 2024-09-16T00:38:39.949 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 983b2072-73c1-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-16T00:38:40.151 INFO:teuthology.orchestra.run.smithi073.stdout:Deleting cluster with fsid: 983b2072-73c1-11ef-bceb-c7b262605968 2024-09-16T00:38:42.622 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 983b2072-73c1-11ef-bceb-c7b262605968 --force --keep-logs 2024-09-16T00:38:42.821 INFO:teuthology.orchestra.run.smithi122.stdout:Deleting cluster with fsid: 983b2072-73c1-11ef-bceb-c7b262605968 2024-09-16T00:38:45.179 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:38:45.213 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:38:45.246 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-16T00:38:45.275 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-16T00:38:45.277 DEBUG:teuthology.misc:Transferring archived files from smithi038:/var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906392/remote/smithi038/crash 2024-09-16T00:38:45.277 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar c -f - -C /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/crash -- . 2024-09-16T00:38:45.318 INFO:teuthology.orchestra.run.smithi038.stderr:tar: /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-16T00:38:45.318 INFO:teuthology.orchestra.run.smithi038.stderr:tar: Error is not recoverable: exiting now 2024-09-16T00:38:45.320 DEBUG:teuthology.misc:Transferring archived files from smithi073:/var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906392/remote/smithi073/crash 2024-09-16T00:38:45.321 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar c -f - -C /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/crash -- . 2024-09-16T00:38:45.435 INFO:teuthology.orchestra.run.smithi073.stderr:tar: /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-16T00:38:45.435 INFO:teuthology.orchestra.run.smithi073.stderr:tar: Error is not recoverable: exiting now 2024-09-16T00:38:45.438 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906392/remote/smithi122/crash 2024-09-16T00:38:45.439 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/crash -- . 2024-09-16T00:38:45.473 INFO:teuthology.orchestra.run.smithi122.stderr:tar: /var/lib/ceph/983b2072-73c1-11ef-bceb-c7b262605968/crash: Cannot open: No such file or directory 2024-09-16T00:38:45.473 INFO:teuthology.orchestra.run.smithi122.stderr:tar: Error is not recoverable: exiting now 2024-09-16T00:38:45.473 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-16T00:38:45.474 DEBUG:teuthology.orchestra.run.smithi038:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-16T00:38:45.518 INFO:tasks.cephadm:Compressing logs... 2024-09-16T00:38:45.518 DEBUG:teuthology.orchestra.run.smithi038:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T00:38:45.561 DEBUG:teuthology.orchestra.run.smithi073:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T00:38:45.564 DEBUG:teuthology.orchestra.run.smithi122:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-16T00:38:45.586 INFO:teuthology.orchestra.run.smithi038.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-09-16T00:38:45.586 INFO:teuthology.orchestra.run.smithi038.stderr:: No such file or directory 2024-09-16T00:38:45.586 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-osd.0.log 2024-09-16T00:38:45.588 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-osd.9.log 2024-09-16T00:38:45.589 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/ceph-client.admin.log: /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-16T00:38:45.589 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-16T00:38:45.589 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mds.a.log 2024-09-16T00:38:45.589 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-16T00:38:45.589 INFO:teuthology.orchestra.run.smithi038.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-09-16T00:38:45.589 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mon.a.log 2024-09-16T00:38:45.591 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mds.g.log 2024-09-16T00:38:45.591 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-osd.6.log 2024-09-16T00:38:45.591 INFO:teuthology.orchestra.run.smithi038.stderr: 82.4% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-16T00:38:45.598 INFO:teuthology.orchestra.run.smithi122.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-16T00:38:45.598 INFO:teuthology.orchestra.run.smithi122.stderr:: No such file or directory 2024-09-16T00:38:45.599 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-16T00:38:45.599 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mds.i.log 2024-09-16T00:38:45.599 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-volume.log: /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mon.c.log 2024-09-16T00:38:45.599 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-16T00:38:45.599 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-osd.11.log 2024-09-16T00:38:45.599 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mon.c.log: /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-osd.2.log 2024-09-16T00:38:45.599 INFO:teuthology.orchestra.run.smithi122.stderr: 82.2% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-16T00:38:45.599 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-osd.5.log 2024-09-16T00:38:45.599 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph.log 2024-09-16T00:38:45.599 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mds.l.log 2024-09-16T00:38:45.599 INFO:teuthology.orchestra.run.smithi122.stderr: 90.3% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-16T00:38:45.599 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-osd.8.log 2024-09-16T00:38:45.600 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mds.l.log: 86.2% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-16T00:38:45.600 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mgr.z.log 2024-09-16T00:38:45.600 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-osd.8.log: 94.9% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mds.l.log.gz 2024-09-16T00:38:45.600 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mds.c.log 2024-09-16T00:38:45.602 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mds.g.log: 90.1% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-16T00:38:45.602 INFO:teuthology.orchestra.run.smithi038.stderr: 94.6% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mds.a.log.gz 2024-09-16T00:38:45.603 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph.log 2024-09-16T00:38:45.603 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mds.d.log 2024-09-16T00:38:45.603 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mgr.x.log 2024-09-16T00:38:45.603 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mds.d.log: 94.7% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mds.g.log.gz 2024-09-16T00:38:45.603 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mds.j.log 2024-09-16T00:38:45.603 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mgr.x.log: 86.2% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-16T00:38:45.603 INFO:teuthology.orchestra.run.smithi038.stderr: 94.7% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mds.d.log.gz 2024-09-16T00:38:45.604 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mgr.z.log: 92.3% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mgr.z.log.gz 2024-09-16T00:38:45.605 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mds.f.log 2024-09-16T00:38:45.605 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mds.c.log: 90.8% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-16T00:38:45.608 INFO:teuthology.orchestra.run.smithi038.stderr: 90.9% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-volume.log.gz 2024-09-16T00:38:45.608 INFO:teuthology.orchestra.run.smithi038.stderr:gzip -5 --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-osd.3.log 2024-09-16T00:38:45.608 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-16T00:38:45.609 INFO:teuthology.orchestra.run.smithi038.stderr:/var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 94.7% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mds.j.log.gz 2024-09-16T00:38:45.610 INFO:teuthology.orchestra.run.smithi073.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-osd.10.log 2024-09-16T00:38:45.610 INFO:teuthology.orchestra.run.smithi073.stderr:: No such file or directory 2024-09-16T00:38:45.610 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-osd.1.log 2024-09-16T00:38:45.610 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5/var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-osd.10.log: --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-osd.7.log 2024-09-16T00:38:45.610 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-volume.log 2024-09-16T00:38:45.610 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mds.b.log 2024-09-16T00:38:45.611 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mds.k.log 2024-09-16T00:38:45.611 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-osd.4.log 2024-09-16T00:38:45.611 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mds.b.log: /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph.audit.log 2024-09-16T00:38:45.611 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph.cephadm.log 2024-09-16T00:38:45.611 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph.audit.log: 90.3% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph.audit.log.gz 2024-09-16T00:38:45.611 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mgr.y.log 2024-09-16T00:38:45.611 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph.cephadm.log: 82.3% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph.cephadm.log.gz 2024-09-16T00:38:45.611 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph.log 2024-09-16T00:38:45.611 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mgr.y.log: 92.2% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mgr.y.log.gz 2024-09-16T00:38:45.611 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mds.h.log 2024-09-16T00:38:45.611 INFO:teuthology.orchestra.run.smithi073.stderr: 90.9% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-volume.log.gz/var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph.log: 2024-09-16T00:38:45.611 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mon.b.log 2024-09-16T00:38:45.611 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mds.h.log: 86.1% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph.log.gz 2024-09-16T00:38:45.611 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mds.e.log 2024-09-16T00:38:45.613 INFO:teuthology.orchestra.run.smithi122.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-16T00:38:45.613 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mds.f.log: 94.7% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mds.f.log.gz 2024-09-16T00:38:45.613 INFO:teuthology.orchestra.run.smithi122.stderr:/var/log/ceph/cephadm.log: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-16T00:38:45.614 INFO:teuthology.orchestra.run.smithi038.stderr: 93.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-16T00:38:45.619 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-16T00:38:45.630 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mds.e.log: 93.9% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mds.k.log.gz 2024-09-16T00:38:45.632 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-16T00:38:45.637 INFO:teuthology.orchestra.run.smithi122.stderr: 94.0% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mds.c.log.gz 2024-09-16T00:38:45.671 INFO:teuthology.orchestra.run.smithi073.stderr: 92.9% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mds.h.log.gz 2024-09-16T00:38:45.680 INFO:teuthology.orchestra.run.smithi073.stderr: 93.3% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mds.e.log.gz 2024-09-16T00:38:45.743 INFO:teuthology.orchestra.run.smithi038.stderr: 90.1% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mgr.x.log.gz 2024-09-16T00:38:45.791 INFO:teuthology.orchestra.run.smithi122.stderr: 91.7% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mon.c.log.gz 2024-09-16T00:38:45.917 INFO:teuthology.orchestra.run.smithi073.stderr: 91.8% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mon.b.log.gz 2024-09-16T00:38:46.020 INFO:teuthology.orchestra.run.smithi122.stderr: 95.8% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mds.i.log.gz 2024-09-16T00:38:46.212 INFO:teuthology.orchestra.run.smithi038.stderr: 90.6% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mon.a.log.gz 2024-09-16T00:38:47.165 INFO:teuthology.orchestra.run.smithi038.stderr: 92.2% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-osd.9.log.gz 2024-09-16T00:38:47.210 INFO:teuthology.orchestra.run.smithi122.stderr: 92.9% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-osd.11.log.gz 2024-09-16T00:38:47.281 INFO:teuthology.orchestra.run.smithi073.stderr: 92.5% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-osd.7.log.gz 2024-09-16T00:38:47.485 INFO:teuthology.orchestra.run.smithi073.stderr: 92.3% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-osd.1.log.gz 2024-09-16T00:38:47.556 INFO:teuthology.orchestra.run.smithi073.stderr: 92.5% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-osd.10.log.gz 2024-09-16T00:38:47.578 INFO:teuthology.orchestra.run.smithi038.stderr: 92.6% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-osd.3.log.gz 2024-09-16T00:38:47.600 INFO:teuthology.orchestra.run.smithi038.stderr: 92.8% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-osd.0.log.gz 2024-09-16T00:38:47.729 INFO:teuthology.orchestra.run.smithi038.stderr: 92.6% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-osd.6.log.gz 2024-09-16T00:38:47.731 INFO:teuthology.orchestra.run.smithi038.stderr: 2024-09-16T00:38:47.732 INFO:teuthology.orchestra.run.smithi038.stderr:real 0m2.156s 2024-09-16T00:38:47.732 INFO:teuthology.orchestra.run.smithi038.stderr:user 0m8.239s 2024-09-16T00:38:47.732 INFO:teuthology.orchestra.run.smithi038.stderr:sys 0m0.298s 2024-09-16T00:38:47.765 INFO:teuthology.orchestra.run.smithi122.stderr: 93.0% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-osd.8.log.gz 2024-09-16T00:38:47.885 INFO:teuthology.orchestra.run.smithi073.stderr: 92.9% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-osd.4.log.gz 2024-09-16T00:38:48.148 INFO:teuthology.orchestra.run.smithi122.stderr: 93.5% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-osd.2.log.gz 2024-09-16T00:38:49.553 INFO:teuthology.orchestra.run.smithi073.stderr: 93.5% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-mds.b.log.gz 2024-09-16T00:38:49.555 INFO:teuthology.orchestra.run.smithi073.stderr: 2024-09-16T00:38:49.555 INFO:teuthology.orchestra.run.smithi073.stderr:real 0m3.977s 2024-09-16T00:38:49.556 INFO:teuthology.orchestra.run.smithi073.stderr:user 0m11.887s 2024-09-16T00:38:49.556 INFO:teuthology.orchestra.run.smithi073.stderr:sys 0m0.409s 2024-09-16T00:38:50.655 INFO:teuthology.orchestra.run.smithi122.stderr: 95.0% -- replaced with /var/log/ceph/983b2072-73c1-11ef-bceb-c7b262605968/ceph-osd.5.log.gz 2024-09-16T00:38:50.657 INFO:teuthology.orchestra.run.smithi122.stderr: 2024-09-16T00:38:50.657 INFO:teuthology.orchestra.run.smithi122.stderr:real 0m5.079s 2024-09-16T00:38:50.657 INFO:teuthology.orchestra.run.smithi122.stderr:user 0m11.688s 2024-09-16T00:38:50.657 INFO:teuthology.orchestra.run.smithi122.stderr:sys 0m0.436s 2024-09-16T00:38:50.657 INFO:tasks.cephadm:Archiving logs... 2024-09-16T00:38:50.658 DEBUG:teuthology.misc:Transferring archived files from smithi038:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906392/remote/smithi038/log 2024-09-16T00:38:50.659 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T00:38:51.922 DEBUG:teuthology.misc:Transferring archived files from smithi073:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906392/remote/smithi073/log 2024-09-16T00:38:51.923 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T00:38:53.659 DEBUG:teuthology.misc:Transferring archived files from smithi122:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906392/remote/smithi122/log 2024-09-16T00:38:53.660 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-16T00:38:55.402 INFO:tasks.cephadm:Removing cluster... 2024-09-16T00:38:55.403 DEBUG:teuthology.orchestra.run.smithi038:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 983b2072-73c1-11ef-bceb-c7b262605968 --force 2024-09-16T00:38:55.616 INFO:teuthology.orchestra.run.smithi038.stdout:Deleting cluster with fsid: 983b2072-73c1-11ef-bceb-c7b262605968 2024-09-16T00:38:55.895 DEBUG:teuthology.orchestra.run.smithi073:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 983b2072-73c1-11ef-bceb-c7b262605968 --force 2024-09-16T00:38:56.095 INFO:teuthology.orchestra.run.smithi073.stdout:Deleting cluster with fsid: 983b2072-73c1-11ef-bceb-c7b262605968 2024-09-16T00:38:56.375 DEBUG:teuthology.orchestra.run.smithi122:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 983b2072-73c1-11ef-bceb-c7b262605968 --force 2024-09-16T00:38:56.574 INFO:teuthology.orchestra.run.smithi122.stdout:Deleting cluster with fsid: 983b2072-73c1-11ef-bceb-c7b262605968 2024-09-16T00:38:57.022 INFO:tasks.cephadm:Removing cephadm ... 2024-09-16T00:38:57.022 DEBUG:teuthology.orchestra.run.smithi038:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-16T00:38:57.042 DEBUG:teuthology.orchestra.run.smithi073:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-16T00:38:57.060 DEBUG:teuthology.orchestra.run.smithi122:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-16T00:38:57.076 INFO:tasks.cephadm:Teardown complete 2024-09-16T00:38:57.076 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-16T00:38:57.089 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-16T00:38:57.089 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T00:38:57.092 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T00:38:57.102 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-16T00:38:57.210 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-16T00:38:57.220 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-16T00:38:57.220 DEBUG:teuthology.orchestra.run.smithi038:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T00:38:57.223 DEBUG:teuthology.orchestra.run.smithi073:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T00:38:57.226 DEBUG:teuthology.orchestra.run.smithi122:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-16T00:38:57.238 INFO:teuthology.orchestra.run.smithi038.stderr:bash: line 1: ntpq: command not found 2024-09-16T00:38:57.239 INFO:teuthology.orchestra.run.smithi073.stderr:bash: line 1: ntpq: command not found 2024-09-16T00:38:57.240 INFO:teuthology.orchestra.run.smithi122.stderr:bash: line 1: ntpq: command not found 2024-09-16T00:38:57.244 INFO:teuthology.orchestra.run.smithi073.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T00:38:57.244 INFO:teuthology.orchestra.run.smithi073.stdout:=============================================================================== 2024-09-16T00:38:57.245 INFO:teuthology.orchestra.run.smithi073.stdout:^* hv01.front.sepia.ceph.com 3 6 377 61 -75us[ -77us] +/- 45ms 2024-09-16T00:38:57.245 INFO:teuthology.orchestra.run.smithi073.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 59 -1182us[-1182us] +/- 63ms 2024-09-16T00:38:57.245 INFO:teuthology.orchestra.run.smithi073.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 61 +1368us[+1368us] +/- 63ms 2024-09-16T00:38:57.245 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:38:57.246 INFO:teuthology.orchestra.run.smithi122.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T00:38:57.246 INFO:teuthology.orchestra.run.smithi122.stdout:=============================================================================== 2024-09-16T00:38:57.246 INFO:teuthology.orchestra.run.smithi122.stdout:^* hv01.front.sepia.ceph.com 3 6 377 63 -91us[ -91us] +/- 45ms 2024-09-16T00:38:57.247 INFO:teuthology.orchestra.run.smithi122.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 60 -1198us[-1198us] +/- 63ms 2024-09-16T00:38:57.247 INFO:teuthology.orchestra.run.smithi122.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 62 +1340us[+1340us] +/- 63ms 2024-09-16T00:38:57.247 INFO:teuthology.orchestra.run.smithi122.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:38:57.247 INFO:teuthology.orchestra.run.smithi038.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-16T00:38:57.247 INFO:teuthology.orchestra.run.smithi038.stdout:=============================================================================== 2024-09-16T00:38:57.247 INFO:teuthology.orchestra.run.smithi038.stdout:^* hv01.front.sepia.ceph.com 3 6 377 63 -96us[ -111us] +/- 45ms 2024-09-16T00:38:57.247 INFO:teuthology.orchestra.run.smithi038.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 62 -1197us[-1197us] +/- 63ms 2024-09-16T00:38:57.247 INFO:teuthology.orchestra.run.smithi038.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 62 +1350us[+1350us] +/- 63ms 2024-09-16T00:38:57.247 INFO:teuthology.orchestra.run.smithi038.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-16T00:38:57.247 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-16T00:38:57.257 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-16T00:38:57.257 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-16T00:38:57.273 DEBUG:teuthology.orchestra.run.smithi038:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T00:38:57.375 DEBUG:teuthology.orchestra.run.smithi073:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T00:38:57.446 DEBUG:teuthology.orchestra.run.smithi122:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-16T00:38:57.512 DEBUG:teuthology.orchestra.run.smithi038:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T00:38:57.544 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:38:57.544 DEBUG:teuthology.orchestra.run.smithi073:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T00:38:57.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:38:57.574 DEBUG:teuthology.orchestra.run.smithi122:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-09-16T00:38:57.601 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:38:57.601 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-16T00:38:57.611 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-16T00:38:57.623 INFO:teuthology.task.internal:Duration was 1410.785209 seconds 2024-09-16T00:38:57.624 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-16T00:38:57.638 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-16T00:38:57.638 DEBUG:teuthology.orchestra.run.smithi038:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T00:38:57.641 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T00:38:57.643 DEBUG:teuthology.orchestra.run.smithi122:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-16T00:38:57.681 INFO:teuthology.orchestra.run.smithi073.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T00:38:57.682 INFO:teuthology.orchestra.run.smithi122.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T00:38:57.682 INFO:teuthology.orchestra.run.smithi038.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-16T00:38:58.083 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-16T00:38:58.083 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi038.front.sepia.ceph.com 2024-09-16T00:38:58.083 DEBUG:teuthology.orchestra.run.smithi038:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T00:38:58.107 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi073.front.sepia.ceph.com 2024-09-16T00:38:58.107 DEBUG:teuthology.orchestra.run.smithi073:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T00:38:58.147 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi122.front.sepia.ceph.com 2024-09-16T00:38:58.147 DEBUG:teuthology.orchestra.run.smithi122:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-16T00:38:58.169 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-16T00:38:58.170 DEBUG:teuthology.orchestra.run.smithi038:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T00:38:58.173 DEBUG:teuthology.orchestra.run.smithi073:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T00:38:58.190 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-16T00:38:58.236 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-16T00:38:58.236 DEBUG:teuthology.orchestra.run.smithi038:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T00:38:58.243 DEBUG:teuthology.orchestra.run.smithi073:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T00:38:58.261 DEBUG:teuthology.orchestra.run.smithi122:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-16T00:38:58.955 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-16T00:38:58.967 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-16T00:38:58.967 DEBUG:teuthology.orchestra.run.smithi038:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T00:38:58.993 DEBUG:teuthology.orchestra.run.smithi073:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T00:38:59.019 DEBUG:teuthology.orchestra.run.smithi122:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-16T00:38:59.045 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-16T00:38:59.056 DEBUG:teuthology.orchestra.run.smithi038:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T00:38:59.058 DEBUG:teuthology.orchestra.run.smithi073:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T00:38:59.061 DEBUG:teuthology.orchestra.run.smithi122:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-16T00:38:59.080 INFO:teuthology.orchestra.run.smithi038.stdout:kernel.core_pattern = core 2024-09-16T00:38:59.083 INFO:teuthology.orchestra.run.smithi073.stdout:kernel.core_pattern = core 2024-09-16T00:38:59.109 INFO:teuthology.orchestra.run.smithi122.stdout:kernel.core_pattern = core 2024-09-16T00:38:59.134 DEBUG:teuthology.orchestra.run.smithi038:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T00:38:59.149 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:38:59.150 DEBUG:teuthology.orchestra.run.smithi073:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T00:38:59.164 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:38:59.165 DEBUG:teuthology.orchestra.run.smithi122:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-16T00:38:59.189 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-16T00:38:59.190 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-16T00:38:59.200 INFO:teuthology.task.internal:Transferring archived files... 2024-09-16T00:38:59.201 DEBUG:teuthology.misc:Transferring archived files from smithi038:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906392/remote/smithi038 2024-09-16T00:38:59.201 DEBUG:teuthology.orchestra.run.smithi038:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T00:38:59.254 DEBUG:teuthology.misc:Transferring archived files from smithi073:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906392/remote/smithi073 2024-09-16T00:38:59.254 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T00:38:59.291 DEBUG:teuthology.misc:Transferring archived files from smithi122:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-09-15_21:24:03-fs-squid-distro-default-smithi/7906392/remote/smithi122 2024-09-16T00:38:59.291 DEBUG:teuthology.orchestra.run.smithi122:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-16T00:38:59.328 INFO:teuthology.task.internal:Removing archive directory... 2024-09-16T00:38:59.328 DEBUG:teuthology.orchestra.run.smithi038:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T00:38:59.331 DEBUG:teuthology.orchestra.run.smithi073:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T00:38:59.333 DEBUG:teuthology.orchestra.run.smithi122:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-16T00:38:59.377 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-16T00:38:59.389 INFO:teuthology.task.internal:Not uploading archives. 2024-09-16T00:38:59.389 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-16T00:38:59.401 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-16T00:38:59.401 DEBUG:teuthology.orchestra.run.smithi038:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T00:38:59.404 DEBUG:teuthology.orchestra.run.smithi073:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T00:38:59.406 DEBUG:teuthology.orchestra.run.smithi122:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-16T00:38:59.417 INFO:teuthology.orchestra.run.smithi038.stdout: 262169 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 00:38 /home/ubuntu/cephtest 2024-09-16T00:38:59.418 INFO:teuthology.orchestra.run.smithi073.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 00:38 /home/ubuntu/cephtest 2024-09-16T00:38:59.432 INFO:teuthology.orchestra.run.smithi122.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 16 00:38 /home/ubuntu/cephtest 2024-09-16T00:38:59.433 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-16T00:38:59.447 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-16T00:38:59.477 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/no 4-flush/no 5-quiesce/with-quiesce 6-workunit/direct_io}} duration: 1410.7852091789246 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-09-16T00:38:59.477 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-16T00:38:59.556 INFO:teuthology.run:pass